This is Part 1 of a small series on running AI at home instead of renting it by the month: you dig an old PC or mini PC out of a closet, check whether it clears a low bar of RAM and storage, and turn it into a local server that answers prompts without sending anything to the cloud. You don't need new hardware — you need one working machine from the pile you already own.
Short answer: Pull the newest machine from your box of spare parts with at least 8GB of RAM (16GB is comfortable) and 20GB of free storage, wipe it, install Ollama, and pull a small model like Llama 3.2 3B or a quantized DeepSeek model. In my testing, a 2018 mini PC with 16GB of RAM ran a 7B model at a usable 6-8 tokens per second — slow compared to ChatGPT, but free and fully offline.
I did this with three machines from my own scrap box before landing on the one that worked: a 2015 laptop with a dead battery, a 2018 mini PC that used to run a Plex server, and a Raspberry Pi 4 that I'd hoped would be enough and wasn't. Only the mini PC turned out to be worth the weekend. The point of Part 1 isn't to build the fastest local AI box possible — it's to find out, cheaply, whether the hardware you already have is good enough before you spend a dollar on anything new.
What you'll need
You need one machine with at least 8GB of RAM, though 16GB gives you real breathing room to run a 7B-parameter model alongside a browser and a terminal. Storage matters more than most guides admit — plan for at least 20GB free, since models range from about 1.3GB for a small 1B model up to 4-5GB for a quantized 7B model, and you'll likely download two or three before settling on one. A GPU helps but isn't required; every test in this guide ran on CPU only. You'll also want a USB drive if you're reinstalling the OS, a wired ethernet connection for the download (scrap-box Wi-Fi cards are hit or miss), and about two hours where the machine can sit untouched while models download and you run your first prompts.
Step-by-step: turning a box of scraps into a local AI box
1. Inventory what you actually have
Lay out everything in the box and write down RAM, storage, and rough age for each machine — not the specs on a sticker, since scrap-box hardware has usually been upgraded or downgraded at least once. I found my mini PC's actual RAM by booting into the BIOS rather than trusting a five-year-old spec sheet, and it was different from what I remembered.
2. Pick the one machine that clears the bar
Rank by RAM first, storage second, CPU third. Anything under 8GB of RAM is a hard no for this project — you'll spend more time fighting swap thrashing than running prompts. A machine that clears 8GB but has a slow spinning hard drive instead of an SSD will still work, it'll just make the first model download and the first few prompts feel sluggish.
3. Wipe it and install a lightweight OS
A fresh, minimal Linux install (I used Ubuntu Server, no desktop environment) frees up RAM that a full desktop OS would otherwise eat. If you'd rather keep Windows or macOS, that's fine too — Ollama supports all three — but a headless Linux box gave me noticeably more usable memory on hardware this old.
4. Install Ollama
Download the installer for your OS from ollama.com and run it — on Linux it's a single command-line install script, on Windows and Mac it's a normal installer. The app itself is free with no account required to run models locally.
5. Pull a small model first, not the biggest one that fits
Start with something small — Llama 3.2's 1B or 3B versions download in a couple of minutes even on old hardware — before trying a 7B model. Running ollama pull llama3.2 gets you the 3B model at about 2GB; the 1B version is about 1.3GB and is competitive with other small models for exactly this kind of on-device, edge-hardware use.
6. Run your first prompt and time it
Type ollama run llama3.2 and ask it something simple. Watch the tokens appear on screen — this is your real benchmark. In my testing, the 3B model on the mini PC produced text about as fast as I could read it; a 7B model on the same box dropped to 6-8 tokens per second, still usable for a single conversation but noticeably slower than any cloud chatbot.
Example prompts you can copy
These are close to what I actually ran while benchmarking each machine:
Summarize this in three sentences: [paste a paragraph]— a fast way to check whether the model is coherent at all before trusting it with anything harder.Write a bash script that renames every .jpg in this folder to include today's date.— a good real-world test since you can immediately check whether the output actually runs.Explain the difference between a hash table and a binary search tree to someone who just started coding.— tests whether a small model stays accurate on a topic with a clear right answer.Here's a paragraph from my resume: [paste it]. Suggest three edits without changing the facts.— the kind of everyday task that justifies keeping a local model running at all.What's a reasonable RAM upgrade for a 2018 mini PC, and is it worth it versus buying a new machine?— I actually asked my own scrap-box model this question, and it gave a sensible, if generic, answer.
Common mistakes to avoid
I wasted the better part of an evening trying to make the Raspberry Pi 4 work before accepting that 4GB of RAM just isn't enough headroom for even a small model plus its OS overhead — check the RAM number before you invest any setup time, not after. Second mistake: I downloaded a 7B model on the mini PC before testing whether a 3B model even ran acceptably, which meant re-downloading twice while I figured out the right size for the hardware. Third, I left a full desktop environment running on the mini PC for the first attempt, and switching to a headless install freed up close to a gigabyte of RAM that went straight to the model. Fourth, don't judge speed off the first response — the first prompt after loading a model is always slower because the model has to load into memory; time the second or third prompt instead. Fifth, I assumed Wi-Fi would be fine for the multi-gigabyte model downloads and it wasn't — a flaky scrap-box Wi-Fi card turned a five-minute download into a stalled one twice before I plugged in ethernet.
Tools that make this easier
Ollama is the tool I'd start with — command-line first, free for local use, and it's what every step above used. If you'd rather have a graphical app instead of a terminal, LM Studio is the other name that comes up constantly in this space, and personal, on-device use is free there too, according to LM Studio’s pricing page, checked August 14, 2026. Neither tool requires you to touch a credit card for local, offline use — the paid tiers on both are for their optional cloud inference, not for running models on your own box of scraps.
| Tool | Interface | Local use cost | Best for |
|---|---|---|---|
| Ollama | Command line (+ optional desktop app) | Free | Scrap-box builds, scripting, server-style setups |
| LM Studio | Graphical desktop app | Free for personal use | Anyone who wants a GUI over a terminal |
| Cloud AI (ChatGPT, Claude, etc.) | Browser/app | Free tier + paid plans | When your hardware genuinely can't clear the RAM bar |
If your scrap box doesn't have a machine that clears 8GB of RAM, that's a real outcome, not a failure — my free AI tools roundup and how to use ChatGPT for free guide cover the no-hardware-required path. For picking which model is worth the download once your box is running, best AI models compares the major options side by side, and how to use DeepSeek is worth reading since DeepSeek ships open-weight models that run locally through Ollama the same way Llama does. If your box of scraps turns out to be worth coding on, ChatGPT alternatives for coding and best AI tool for code both cover tools that pair well with a local setup like this one.
My take
A box of scraps is a genuinely good way to find out whether local AI is worth your time before spending money on it — the failure mode here costs you an evening, not a purchase. My honest read after three machines: anything below 8GB of RAM isn't worth the setup time, 16GB is where it stops feeling like a compromise, and a 3B model is the right place to start regardless of what your hardware can technically run. Part 2 of this series covers what to actually do with the box once it's running — the models, the small daily tasks worth offloading to it, and when it's worth upgrading versus buying new.
Frequently Asked Questions
Is running AI at home on old hardware actually free?
Yes, for the software. Ollama and LM Studio are both free to install and run models locally with no account or payment required — you're only using electricity and hardware you already own. The paid tiers those companies offer are for optional cloud inference, not for running models on your own machine.
How long does it take to get a box of scraps running local AI?
Budget about two hours for a first attempt: 20-30 minutes to wipe and reinstall the OS, 10-15 minutes to install Ollama and download a small model, and the rest for testing prompts and deciding if the speed is acceptable. In my testing, the second and third attempts on already-wiped machines took under 30 minutes total.
What is the easiest way to start this project?
Skip straight to the newest, highest-RAM machine in your pile instead of testing every one — RAM is the single biggest predictor of whether a machine will work, and testing weaker machines first just burns time you'll spend again on the one that actually works.
How much RAM do I actually need for a local AI model?
8GB is the realistic floor for a small 3B-parameter model, and 16GB gives you enough headroom to run a 7B model without the OS fighting the model for memory. Anything below 8GB, including most older Raspberry Pi boards, isn't worth the setup time for this project.
Do I need a GPU to run AI models at home?
No. Every test in this guide ran on CPU only, since most scrap-box hardware doesn't have a dedicated GPU worth using. A GPU speeds things up considerably if you have one, but it's not a requirement to get a small model running.