WebLLM is an open-source JavaScript engine that runs large language models directly inside a browser tab, using WebGPU for hardware acceleration instead of a server API call. It's built by the MLC AI team — the group behind MLC-LLM — and ships as the @mlc-ai/web-llm package on npm under an Apache-2.0 license. Nothing you type leaves the device: the model downloads once, caches locally, and every answer after that runs offline.
Short answer: WebLLM (
@mlc-ai/web-llmon npm, Apache-2.0, 18,800+ GitHub stars) runs models like Llama 3.2, Qwen2.5, and Phi-3.5 in a WebGPU-enabled browser tab with no server involved. Install it, callCreateMLCEngine()with a model ID, and it downloads once, caches locally, and answers offline afterward. It's free, but GPU memory hungry — an 8B model needs roughly 5 GB of VRAM.
In my testing, I installed the package straight from the published docs rather than trusting a landing page: npm install @mlc-ai/web-llm pulled about 14 MB into node_modules on a single dependency (loglevel), with a 6.5 MB minified index.js at the core — and that's before a single model weight downloads. I also opened the official chat demo at chat.webllm.ai and checked WebGPU adapter detection myself in a headless Chromium build with no real GPU attached. The adapter object came back present anyway, running on a software fallback. That's the first thing worth knowing before you build on this: WebLLM will happily initialize on hardware that can't run it well, and it won't warn you until inference crawls.
What you'll need
You need three things: a WebGPU-capable browser, enough GPU memory for whichever model you pick, and Node.js if you're building rather than just chatting. WebGPU ships by default in Chrome and Edge 113+ on Windows, macOS, and ChromeOS, plus Chrome 121+ on Android 12 devices with a Qualcomm or ARM GPU. Firefox added it in version 141 on Windows and version 145 on Apple Silicon Macs running macOS Tahoe; Safari supports it on macOS Tahoe 26, iOS 26, and iPadOS 26, per Chrome’s own WebGPU rollout notes. If you just want to chat, skip the install entirely and open chat.webllm.ai — same engine, prebuilt, nothing to configure. For a real project, you'll want npm install @mlc-ai/web-llm and a model small enough for your device's VRAM, which is the part most people get wrong first.
Step-by-step: running WebLLM in your browser
1. Install the package
Run npm install @mlc-ai/web-llm inside a JavaScript or TypeScript project. In my install this added one dependency and about 14 MB total on disk — a genuinely small footprint, since the model weights themselves aren't part of the npm package at all.
2. Pick a model that actually fits your GPU
Version 0.2.84 ships 135 prebuilt model variants, covering Llama 3, 3.1, and 3.2, Qwen2.5, Phi-3.5, Gemma 2 and 3, Mistral, DeepSeek-R1-Distill, TinyLlama, and SmolLM2, all pulled from Hugging Face under the mlc-ai org. Each entry in the shipped config lists a vram_required_MB figure. I checked a handful directly in the installed package: Llama-3.2-1B-Instruct-q4f16_1 needs about 879 MB, Qwen2.5-0.5B-Instruct-q4f16_1 about 945 MB, and Llama-3.1-8B-Instruct-q4f16_1 closer to 5 GB. Match the model to the GPU you actually have, not the name you recognize.
3. Create the engine and send a message
“`ts import { CreateMLCEngine } from "@mlc-ai/web-llm";
const engine = await CreateMLCEngine("Llama-3.2-1B-Instruct-q4f16_1-MLC"); const reply = await engine.chat.completions.create({ messages: [{ role: "user", content: "Explain WebGPU in one sentence." }], }); console.log(reply.choices[0].message.content); “`
The chat.completions.create call mirrors the OpenAI SDK on purpose, so most existing OpenAI client code drops in with only the import changed.
4. Expect a real first-load wait, then instant loads after
The first run downloads and compiles the model — a few seconds for a small model on a fast connection, a couple of minutes for an 8B model. WebLLM caches the result across whichever of its four backends you configure (Cache API, IndexedDB, OPFS, or Cross-Origin Storage), so every load after that is near-instant and works with no network connection at all.
5. Move inference off the main thread before you ship anything
For a real app, CreateWebWorkerMLCEngine runs inference in a Web Worker so it doesn't freeze your UI while a response streams in, and CreateServiceWorkerMLCEngine keeps a model resident across tabs and reloads — worth it for a Chrome extension or a PWA that shouldn't re-download multi-gigabyte weights every time someone opens a new tab.
Example prompts you can copy
These are test prompts worth running once your engine is wired up, to confirm streaming, structured output, and context handling actually work before you build a UI around them:
- Confirm streaming works:
for await (const chunk of await engine.chat.completions.create({ messages, stream: true }))— print each chunk as it arrives and watch for a smooth token-by-token response instead of one long pause. - Test JSON mode: "Return a JSON object with keys
titleandsummaryfor this paragraph: [paste text]" — WebLLM supports schema-constrained JSON output, so this is the fastest way to check your grammar/schema config is actually being enforced. - Check context handling: "Here's a 500-word document: [paste]. What's the third sentence, verbatim?" — a quick test of whether your chosen model's context window (some prebuilt configs cap it at 1k tokens) is large enough for your use case.
- Sanity-check a small model: "Write a haiku about browser-based AI." — good first call after loading any new model ID, since it's short enough to catch a broken load fast.
Common mistakes to avoid
The mistake I made first was assuming a present navigator.gpu object means WebGPU will actually perform well — it doesn't. My headless Chromium test returned an adapter with zero real GPU behind it, and WebLLM will start loading a model on that adapter without complaint; you find out it's unusable only once generation crawls at a token every few seconds. Second, people pick an 8B model because it's the one they've heard of, then wonder why it won't load on a laptop with 4 GB of VRAM — check the vram_required_MB field before you pick a model ID, not after. Third, the quantization suffix isn't cosmetic: q4f16_1 and q4f32_1 versions of the same model carry different memory footprints and slightly different output quality, and swapping one for the other mid-project without re-testing is asking for a regression. Fourth, cached models are scoped per browser origin, not shared globally — a user who visits two different sites built on WebLLM downloads the same multi-gigabyte model twice, which is worth designing around if you're building anything beyond a single demo page. Fifth, running inference on the main thread and being surprised the page freezes; that's what CreateWebWorkerMLCEngine exists to prevent, and it should be the default for anything shipped past a prototype.
Picking a model: real VRAM requirements
I pulled these directly from the vram_required_MB values shipped in the installed @mlc-ai/web-llm v0.2.84 config, not from marketing copy:
| Model (q4f16_1 quant) | VRAM required | Good fit for |
|---|---|---|
| Llama-3.2-1B-Instruct | ~879 MB | Integrated GPUs, older laptops |
| Qwen2.5-0.5B-Instruct | ~945 MB | Fast, low-memory devices |
| gemma-2-2b-it | ~1,895 MB | Mid-range laptops with 4 GB+ VRAM |
| Phi-3.5-mini-instruct | ~3,672 MB | Dedicated GPUs, 6 GB+ VRAM |
| Llama-3.1-8B-Instruct | ~5,001 MB | Desktop GPUs, 8 GB+ VRAM |
Treat these as floors, not targets — the browser tab, the OS, and anything else competing for GPU memory all eat into what's actually available at runtime.
Tools that make this easier
If you're deciding which model family to standardize on before wiring up WebLLM, best AI models compares the underlying model families WebLLM ships, including how Llama and Qwen stack up outside the browser. For the coding-assistant side of the same workflow, best AI tool for code covers what's worth pairing with a local, in-browser model versus a hosted one. WebLLM sits in a growing category of browser-native AI APIs — WebMCP is the adjacent standard for letting a page expose actions to an AI agent, and it's worth understanding both if you're building agentic browser features. If you'd rather test a model in an isolated environment before shipping it client-side, Docker sandboxes for AI agents covers a disposable setup for that. Serving your own docs or data to an AI client alongside a WebLLM-powered feature is a related problem — serving Markdown to AI agents with Accept headers covers the read side of it. For a hosted alternative when a user's device can't handle local inference, how to use Claude AI is a reasonable fallback path. And for how I verify claims like the VRAM numbers above, see how we test AI tools.
My take
WebLLM does what it says: a real LLM, running in a browser tab, with no round trip to a server once the weights are cached. The npm package is small, the API mirrors the OpenAI SDK closely enough that migrating existing code is genuinely easy, and 135 prebuilt models is a wide enough menu that most projects will find something that fits. The catch isn't the software — it's that GPU memory is the real budget here, and the library won't stop you from picking a model your hardware can't run. If you're building something where privacy or offline access matters more than raw model quality, WebLLM is worth building on. If you need an 8B-class model to work on every visitor's device regardless of what GPU they own, plan a server-side fallback; not every laptop clears 5 GB of free VRAM.
Frequently Asked Questions
Is WebLLM free to use?
Yes. It's Apache-2.0 licensed and free to install and run — there's no subscription or API fee, since inference happens on the visitor's own device instead of a paid server call.
How long does it take to get a model running?
Installing the package and writing the CreateMLCEngine call took me a few minutes. The first real wait is the model download itself: a few seconds for a small model like Qwen2.5-0.5B on a fast connection, up to a couple of minutes for an 8B model. Every load after that is near-instant, since the weights are cached in the browser.
What's the easiest way to try WebLLM without writing code?
Open chat.webllm.ai in a WebGPU-capable browser and pick a model from the dropdown — it's the same engine as the npm package, with nothing to install.
Which models work on a laptop without a dedicated GPU?
Stick to the smallest end of the lineup — Qwen2.5-0.5B (~945 MB) or Llama-3.2-1B (~879 MB) both fit inside what an integrated GPU typically has available. An 8B model needing roughly 5 GB of VRAM is unrealistic on most integrated graphics.
Does WebLLM send my prompts to a server?
No. Inference runs entirely client-side through WebGPU and WebAssembly. The only network traffic is the one-time model download from Hugging Face; after that, the page can run fully offline.