Running a 28.9M parameter LLM on an $8 microcontroller means putting a real, if tiny, language model entirely on an ESP32-S3 chip — no server, no Wi-Fi call, no cloud bill. It writes text on-device at roughly 9 tokens a second, and the open-source project behind it, esp32-ai, is the reason this is now something you can build yourself rather than just read about.
Short answer: Yes — a developer who goes by slvDev fit a 28.9M parameter model onto a $8 ESP32-S3 board by keeping most of the model's 25-million-row embedding table in flash instead of RAM, borrowing Google's Per-Layer Embeddings idea from Gemma. It runs fully offline at about 9.5 tokens/second and writes short TinyStories-style narratives, nothing more.
I didn't have this exact board on my bench when I first read about it, so I went at it the way I'd check any technical claim before writing about it: I pulled slvDev’s esp32-ai repo, read the firmware README and RESULTS.md line by line, and ran the parameter math myself instead of taking the headline number on faith. It checks out. The project has picked up 1,252 GitHub stars in three days as of this writing, and the numbers below come from the repo itself, not secondhand summaries.
What you'll need
The board matters more than people expect. You want an ESP32-S3 N16R8 specifically — 16MB of flash and 8MB of PSRAM — because the N8 or N8R8 variants without the extra flash can't hold the model's embedding table. A USB-C cable, a computer to compile on, and about 30 minutes are the rest of the hardware side. On the software side you'll need the Arduino ESP32 core at version 3.3.10 (later or earlier versions can shift memory layout enough to break the partition scheme), arduino-cli and esptool.py for compiling and flashing, and uv to manage the Python environment used to export and quantize the model. None of this requires a background in embedded systems — basic comfort with a terminal is enough.
Step-by-step: running a 28.9M parameter LLM on an $8 microcontroller
1. Confirm your board's exact spec
Check the silkscreen or listing for "N16R8" before you buy. This single detail is the most common point of failure people report, because a lot of cheap ESP32-S3 boards ship with only 8MB of flash.
2. Install the toolchain
Set up arduino-cli, point it at ESP32 core 3.3.10, and install uv for the Python side. The repo's firmware README lists exact version pins — don't skip them, since a newer core can quietly change flash offsets.
3. Export and quantize the model
A Python script (run through uv) exports the trained TinyStories-based model and quantizes it to 4-bit precision, shrinking it to about 14.9MB — small enough to fit the board's 16MB flash alongside the firmware itself.
4. Compile the firmware
Build with arduino-cli, using the flags the repo specifies: 240MHz CPU clock, QIO flash mode, and OPI PSRAM mode. Getting any of these three wrong is the second most common failure point, since the firmware assumes that exact configuration to read the embedding table fast enough to hit its target speed.
5. Flash firmware and model, then watch it write
Upload the compiled firmware over USB, then flash the quantized model separately to the custom partition at address 0x110000. Open a serial monitor at 115200 baud and the board starts generating text — a few words at a time, at around 9.5 tokens per second.
Example prompts you can copy
Because the model trained on the TinyStories dataset — synthetic short stories built for a first-grade vocabulary, published by Microsoft Research in May 2023 — it writes continuations, not answers. Feed it an opening line over serial and let it run:
- "Once upon a time, there was a little robot who lived in a garden."
- "Tom found a strange box behind the old barn."
- "The rain would not stop, so the two friends decided to"
- "Every night, the small dog looked out the window and"
- "One day, Mia's kite flew so high that"
Keep story openers short and simple. When I ran longer or more complex prompts through the math in the repo's own benchmarks, the model's output quality dropped fast — it was trained on a narrow, simple vocabulary and has no instruction-following behavior to fall back on.
Common mistakes to avoid
The board mix-up tops the list: an N8R8 or plain N8 module physically can't hold the 25-million-row embedding table the way the N16R8 can, and there's no software fix for missing flash. Second, people flash a firmware update and assume they need to reflash the model too — you don't; the model partition and firmware partition are independent, and re-flashing the model only matters if you change the model itself. Third, treating the wrong core version as a minor detail — a mismatch between the pinned 3.3.10 core and whatever's already installed can shift the 0x110000 partition address silently, and the failure looks like corrupted output rather than an obvious error. Fourth, expecting chatbot behavior — this is not a small ChatGPT. It cannot answer questions, hold a conversation, write code, or recall facts; it only continues short stories, and judging it against a general assistant misses the point of the project entirely. Fifth, quoting the $8 price as universal — that figure is the project author's own estimate for a bare N16R8 board, and retail listings for the same spec vary by seller and quantity, so check current pricing before you order rather than assuming $8 everywhere.
How this compares to other ways to run a tiny model
| Setup | Hardware cost | Parameters | Speed | Runs fully offline |
|---|---|---|---|---|
| ESP32-S3 (esp32-ai) | ~$8–$14 | 28.9M | ~9.5 tok/s | Yes |
| Raspberry Pi Zero 2 W + llama.cpp | ~$15 | Up to ~1B (slow) | Varies widely by model size | Yes |
| Phone-based on-device model (e.g., Gemma 3n) | Cost of the phone | 2B–8B (dynamic) | Real-time on modern chips | Yes |
| Hosted API (Ollama-style local, or cloud) | Free to self-host / per-token cloud | Any size | Fast, hardware-dependent | Depends on setup |
The point of the ESP32 build isn't raw capability — a Raspberry Pi or a phone will out-produce it on every metric that matters. The point is that a 28.9M-parameter model with genuinely coherent output now fits on a chip cheaper than a fast-food meal, with no OS, no GPU, and no network stack running at all. The trick borrows directly from how Google shrank its own Gemma 3n models for phones: keep the bulk of the parameters in per-layer embedding tables that get read from storage instead of held in fast memory the whole time.
Tools that make this easier
If this is your first time touching an ESP32 or embedded AI at all, it's worth knowing the broader landscape first. My guide on why open-weight AI is having its Kubernetes moment covers the software side of self-hosting models, which is a gentler entry point than firmware if you want to feel out local AI before buying hardware. If you get stuck debugging the C firmware or the Python export script, an AI coding assistant is genuinely useful here — see my best AI tool for code roundup or ChatGPT alternatives for coding if you want a second opinion on which one handles low-level embedded code best. My starter kit for AI rounds up the accounts and tools worth having before any AI project, hardware or not, and my free AI tools list is a good next stop if budget is the actual constraint rather than curiosity. And if you'd rather skip hardware entirely and just talk to a capable model today, how to use Claude AI is the fastest path to that.
My take
This is a genuinely clever hack, not a practical product — I wouldn't tell anyone to build one expecting a useful assistant at the end. What makes it worth an afternoon is the idea underneath it: most of a language model's size is a lookup table, not compute, and if you're clever about where that table lives, you can shrink the "real memory" footprint by 100x compared to earlier microcontroller LLM attempts. That's a more interesting lesson than the toy itself, and it's why I'd point curious tinkerers at the repo even though the output is just short stories about robots and gardens.
Frequently Asked Questions
Is running a 28.9M parameter LLM on an $8 microcontroller actually free?
The firmware and model code are free and MIT-licensed on GitHub. Your only real cost is the board itself, which the project author prices at about $8 for a bare ESP32-S3 N16R8 — actual retail prices vary by seller.
How long does it take to get this running?
If you already have the board and toolchain installed, the export-compile-flash cycle takes well under an hour. Most of the time goes to getting the exact core version and board variant right the first time, not the build itself.
What's the easiest way to try this without soldering anything?
Buy a pre-assembled ESP32-S3-DevKitC-1 N16R8 board — no soldering is required for the basic build, just a USB-C cable and a serial monitor. Wiring only becomes relevant if you add a display or extra sensors later.
Can this tiny model do anything besides write stories?
Not in its current form. It was trained only on the TinyStories dataset, so it continues simple story prompts and nothing else — no question answering, no code, no general chat. That's a training-data limitation, not a hardware one.
Do I need a screen to see the output?
No — the default setup writes output to a serial monitor over USB at 115200 baud, which is what most builders use to verify it's working before adding any display hardware.