~/guides/run-llm-in-browser
Run an LLM in your browser
A language model can run inside a web page on your own graphics chip, with no install and no account. Try one below, then see what it takes: download sizes, measured speeds, browser support and the limits.
Try it here
Mentria's engine runs the model in this tab. The first run downloads the model and saves it in your browser. After that it starts in seconds and works offline.
Your prompt is processed on this device, not on a server. See Privacy.
How it works
WebGPU gives a web page your graphics chip. Browsers now let pages run their own programs on the GPU, the same chip games use. Mentria's engine, written from scratch for WebGPU, runs every step of the model as small GPU programs.
The model is compressed to fit. Each of its numbers is stored in 4 bits, or about 1 bit for the 27B, instead of the usual 16. That's how a model with 0.8 billion numbers becomes a 490 MB download.
It writes one token at a time. Your prompt is split into tokens, pieces of words, and the model predicts the next token again and again on your GPU, at the speeds measured below.
What you need
- A browser with WebGPU. Most current browsers have it; the table below shows where.
- Room in memory. The model and its working memory have to fit in your device's graphics memory. On phones and Macs that memory is shared with everything else that's open.
- Disk space for the download. The browser keeps the model until you clear the site's data or it needs the space.
- A few minutes the first time on a slow connection.
| Browser | Where it works today |
|---|---|
| Chrome, Edge and other Chromium browsers | On by default on Windows, macOS and ChromeOS (since version 113), on most Android 12+ phones (since 121), and on Linux with some GPUs (since 144). |
| Safari | On by default in Safari 26 on macOS, iOS and iPadOS. |
| Firefox | On by default on Windows (since 141) and on Apple-silicon Macs (since 145 on macOS 26, and 147 on earlier macOS). Not yet on Linux or Android. |
From the WebGPU group's implementation status page.
Which model your device runs
| Model | Download | Where Mentria uses it |
|---|---|---|
| Qwen3.5 0.8B | 490 MB | Every device with WebGPU, including phones. |
| Qwen3.5 2B | 1.4 GB | Devices with at least 8 GB of memory. |
| Qwen3.5 4B | 2.8 GB | Computers that can hold it, and some flagship Android phones. |
| Bonsai 27B | 3.8 GB | Computers with a dedicated graphics card, and Apple-silicon Macs. |
Mentria starts with the smallest model and offers a bigger one when your device can take it. You can switch in AI Chat. All four can also look at pictures.
How fast it runs
| Model | Starts in | First word | Speed |
|---|---|---|---|
| Qwen3.5 0.8B | 2.5 s | 0.1 s | 157 tokens/s |
| Qwen3.5 2B | 4.1 s | 0.2 s | 62 tokens/s |
| Qwen3.5 4B | 7.4 s | 0.4 s | 25 tokens/s |
| Bonsai 27B | 9.0 s | 1.2 s | 39 tokens/s |
A token is about three-quarters of a word, so 25 tokens a second is faster than most people read. The 27B beats the 4B here because the engine runs it with settings tuned for Apple GPUs, which the smaller models don't use yet.
Measured with Mentria's benchmark on an Apple M4 Pro with 24 GB of memory, in Chrome 154, October 2026. Starts in: loading a model that is already saved. First word and speed: medians of three runs of a 120-token answer. Your device will differ, and the demo above shows its own numbers.
The limits
- The first download is big. From 490 MB to 3.8 GB. On mobile data that matters, so Mentria asks first on slow or metered connections.
- Phones have the least room. Phone browsers close a tab that uses too much memory, so phones get the smaller models.
- Small models make more mistakes. They're good at explaining, rewriting and summarizing, and much weaker than large cloud models on facts and reasoning. Check anything that matters.
- They read a few pages at a time. A browser model works with a limited context, so very long documents need to be split up.
- It works your GPU hard. Long answers use battery and warm the device, the way a game does.
- The browser can clear it. If your disk runs low, the browser may delete the saved model, and the next run downloads it again.
Use it on Mentria
- AI Chat: chat with the model, show it pictures and pick a bigger model.
- Annotate Image: get a description, the text or tags for a picture.
- Motivational Quote: a small model tuned to write original quotes.
- Privacy: what stays on your device and what uses the network.
Updated 9 October 2026.