AMD Metal, rewritten.
Custom attention, matrix multiplication and wave64 paths turn corrupted or crawling inference into fast, correct output on RDNA and GCN hardware.
Native intelligence · AMD accelerated
Your Mac.
Unleashed.
Run modern open models entirely on the Intel Mac you already own. Native Metal acceleration, zero cloud, and no per-token bill.
macOS 14+ · Intel + AMD GPU · Free and open source
Speed ceiling · lightweight assistantLlama 3.2 1BQ4_K_M · Dense
Prompt2994 t/s
Generate250 t/s
01 / Why ToshLLM
Built for the forgotten Macs
The hardware was never the problem.
Private by architecture.
Your prompts, files and conversations stay on your Mac. No account, no telemetry dependency, no cloud round trip.
Native from first click.
A focused SwiftUI experience with model discovery, hardware-aware recommendations, vision, image generation and a local API, all in one app.
One app · every local workflow
Complex machinery.
Calm interface.
ToshLLM reads your hardware, recommends what fits, and exposes the performance controls when you want them.
Native SwiftUI
100% local
Open performance index
Numbers you can reproduce.
Compare models, GPUs and configurations. Every verified result carries the exact context you need to run it again. Community submissions make the map better.
Open the index#ModelGPUGen.
01
Llama 3.2 1BQ4_K_M · Dense
RX 6700 XT12 GB
250 t/s02
Qwen3 4BQ4_K_M · Dense
RX 6700 XT12 GB
94 t/s03
gpt-oss 20BQ4_K_M · MoE
RX 6700 XT12 GB
93 t/s04
Gemma 3 4BQ4_K_M · Dense
RX 6700 XT12 GB
90 t/sCommunity supported
Keep ToshLLM
independent.
Free, open source, and built for the Mac AMD community. Sponsoring gives the project more time to keep moving forward.
Your Mac is still capable
Stop renting intelligence.
Download ToshLLM and put your AMD GPU back to work.
Get the latest beta