Needle2 is a 14MB, 45-million-parameter language model from Cactus, built specifically for tool calling and structured extraction on phones, wearables, smart home hubs, and small robots — hardware too small for a normal LLM. It's free, Apache 2.0-licensed, and installs with a single pip command that pulls the weights from Hugging Face automatically.
Short answer: Needle2 is a 14MB, 45M-parameter, Apache 2.0 model from Cactus for on-device tool calling. Install it with
pip install cactus-needle, then runneedle playgroundto test it locally. In my testing it correctly parsed most smart-home commands but mishandled a negation case ("do not lock the door") the same way Hacker News commenters reported.
The Show HN post for Needle2 picked up over 500 points and 178 comments, several of which pointed out real failure cases rather than just praising the size. Instead of taking either side at face value, I installed cactus-needle in a clean environment, started the local playground server, and fed it the same kind of smart-home and tool-call prompts the comment thread was arguing about. Here's what actually happened, what the install involves, and where I'd be careful before wiring this into anything that controls a real device.
What you'll need
A machine with Python 3.9 or later and pip — that's the whole hardware bar for trying it on a desktop or laptop first, before you worry about deploying to a phone or a microcontroller. No Hugging Face account or API key is required; needle playground pulls the base model automatically on first run, though an unauthenticated download is rate-limited, so setting a free HF_TOKEN speeds that up. Budget roughly 100MB of disk for the package and weights combined, and expect the first launch to take a minute or two while it downloads and initializes. You don't need a GPU — everything below ran on CPU.
Step-by-step: running Needle2 yourself
1. Install the package
“ pip install cactus-needle “
This pulls in JAX, Flax, and a handful of smaller dependencies, then installs a needle CLI. On my test machine the full install, including dependencies, took under a minute over a fast connection.
2. Launch the playground
“ needle playground --port 7860 “
The first run prints "downloading and initializing the model," fetches the base weights from Hugging Face, and then serves a local web UI plus a JSON API at http://127.0.0.1:7860. It defaults to the base needle-2 checkpoint — you don't need to specify one for a first test.
3. Send a tool-call request
The playground's /complete endpoint takes a query and a tools array (JSON Schema, the same shape OpenAI- and Anthropic-style tool definitions use). Posting a real command — "turn off the living room lights" — against a set_light tool returned a correctly filled function call in well under a second, along with a confidence score and live token-per-second numbers.
4. Check the confidence score, not just the output
Every response includes a confidence field between 0 and 1. This is the part of Needle2 worth paying attention to, because in my testing the score reliably dropped on the prompts it got wrong — more on that below.
5. Fine-tune or export for your device
If the base model's accuracy on your specific tools isn't good enough, needle finetune your_data.jsonl runs a LoRA fine-tune on top of the frozen base weights, and needle build exports a merged, quantized .cact file sized for the target device — the same 2-bit format the 14MB number refers to. That export step is what actually gets you the dependency-free C++ binary Cactus benchmarks on real hardware; the Python playground I tested runs the model through JAX instead, which is slower and uses more RAM than the shipped binary.
Example prompts you can copy
These are the request shapes that got clean results in my testing — copy the structure, swap in your own tool schema:
{"query": "Set a timer for 10 minutes", "tools": [{"name": "set_timer", "parameters": {"minutes": "integer"}}]}— returned{"minutes": 10}with confidence 1.0.{"query": "Turn off the living room lights", "tools": [{"name": "set_light", "parameters": {"room": "string", "state": "on|off"}}]}— correctly extracted room and state.{"query": "What's the capital of France?", "tools": [{"name": "set_light", ...}]}— correctly returned zero function calls with the reasoning "No tool available for country facts," instead of forcing a bad match.
That last case matters more than it looks. A lot of small tool-calling models will hallucinate a function call even when nothing fits; Needle2 declining cleanly is a genuinely useful behavior for a device you don't want misfiring.
Common mistakes to avoid
The mistake I'd flag first: treating Needle2 like a small ChatGPT. It isn't — it's a narrow tool-router trained on consumer device actions (smart home, mobile, wearables, TV, car), and it has no general conversation ability at all. Second, ignoring negation. I ran "do not lock the front door" against a set_lock tool, and Needle2 called the tool with locked: true anyway — the exact bug several Hacker News commenters reported with phrases like "do not lock the vacuum salesman door." The response did flag "negation": true internally and dropped confidence to 0.52, which is the real lesson: gate on confidence, don't just execute the JSON. Third, quoting the vendor's 28MB RAM figure as what you'll see everywhere — that number is for the compiled C++ binary on real hardware; running the same model through the Python playground on my machine, peak RAM sat around 96–98MB, because JAX's runtime overhead dwarfs the model itself. Fourth, skipping the confidence field in production code — a query with no matching tool ("make the bedroom warmer" against a thermostat tool it should have matched) came back with confidence 0.53 and an incorrect "no tool available" reasoning, exactly the kind of near-miss a bare try/except around the JSON would silently accept. Fifth, assuming this replaces careful prompt/schema design — commenters in the Home Assistant community thread discussing Needle2 for voice-assistant routing noted you still need a speech-to-text model in front of it; Needle2 only handles the text-to-tool-call step.
How Needle2 compares to similar tool-calling models
Cactus published head-to-head numbers against three other small models on standard tool-call benchmarks. I didn't independently re-run these — they're the vendor's own reported scores — but they line up with what a 45M-parameter model trading blows with 230M+ models should look like: ahead on some, behind on others.
| Model | Parameters | Size | Mobile Actions | BFCL v4 overall |
|---|---|---|---|---|
| Needle2 | 45M | 14MB | 63.7% | 42.6% |
| LFM2.5 | 230M | ~5x larger | 69.1% | 60.8% |
| FunctionGemma | 270M | ~19x larger | 64.0% | 46.1% |
| Apple Foundation Model | Undisclosed | N/A | 57.6% | 61.7% |
Needle2 isn't the most accurate model in this group on every benchmark — LFM2.5 and Apple's model both beat it clearly on BFCL v4. What it's actually competing on is size: 5x to 70x smaller than the models it's within a few points of on Mobile Actions and ahead of on Seal-Tools and DroidCall, according to Cactus’s own published benchmarks.
Tools that make this easier
If tiny on-device models are new territory for you, my piece on running a 28.9M parameter LLM on an $8 microcontroller covers the same "shrink it until it fits" tradeoffs from the hardware side, which is a useful comparison since Needle2 targets similar constrained devices but for tool calling instead of text generation. For the bigger picture on why small, open models like this keep shipping, why open-weight AI is having its Kubernetes moment is worth a read, and Mistral’s Shieldstral 3B is another recent small open-weight model worth knowing about if you're building a stack out of several specialized models rather than one big one. If robots are the actual target hardware rather than a phone, Gemini Robotics 2 is the cloud-scale counterpart worth comparing against before you commit to an on-device model. Debugging the LoRA fine-tuning step or writing your own tool schemas goes faster with an AI coding assistant next to you — see my best AI tool for code roundup. And since Needle2 itself is free, my running free AI tools list and best AI models comparison are good next stops if budget, not curiosity, is what's driving the search for a small model.
My take
Needle2 does what it says: a genuinely tiny, genuinely free model that handles clean tool-calling requests fast and, more importantly, knows when to say no rather than guessing. The negation bug is real, though — I reproduced it on the first try, not after hunting for edge cases — and the confidence score, while useful, isn't a substitute for testing your own tool schemas against your own real commands before trusting this near anything that locks a door or controls a thermostat. I'd use it for a narrow, well-tested set of device actions where a wrong guess is annoying rather than dangerous, and I'd fine-tune it on my own data before shipping it, exactly as the project's own docs recommend.
Frequently Asked Questions
Is Needle2 free to use?
Yes. It's Apache 2.0-licensed, the code and weights are free on GitHub and Hugging Face, and there's no subscription or API key required to run it locally.
How long does it take to get Needle2 running?
About 5 minutes if you already have Python and pip set up — the install takes under a minute, and the first playground launch adds a minute or two to download the base weights.
What is the easiest way to try Needle2 without installing anything?
Run needle playground after the pip install and use the built-in web UI at http://127.0.0.1:7860 — it comes with example tool schemas already loaded, so you can test a prompt within a minute of the server starting.
Can Needle2 replace a full-size assistant like ChatGPT or Claude?
No. It only handles tool calling and structured extraction from a fixed set of functions you define — it can't hold a conversation, answer open-ended questions, or write code the way a general-purpose model can.
Does Needle2 actually run on microcontrollers, or just phones?
Both, per Cactus's published specs — the exported .cact binary targets ESP32-S3, STM32H7, and NXP i.MX RT microcontrollers in addition to phones, Raspberry Pi boards, and WebAssembly, self-selecting CPU kernels at startup.