You don't need LangChain, LlamaIndex, or a paid GPU to build a working retrieval pipeline, a small tool-using agent, and an eval harness that actually catches regressions. A free Google Colab notebook and roughly 150 lines of plain Python get you all three, and you can see every embedding call and prompt instead of trusting a wrapper class to get it right.
Short answer: Open a free Colab notebook, install just
openaiandnumpy(addfaiss-cpuonly for bigger corpora), and write three small pieces by hand: a retrieval index, a generation loop, and a scoring loop that checks answers against a list you wrote yourself. No LangChain or LlamaIndex required — one API key and plain Python cover real RAG, agent, and eval patterns.
I get asked constantly whether someone needs to learn LangChain before they can ship a RAG feature, and in my testing the honest answer is no — not to understand the mechanics, anyway. When I tested the same small documentation-search demo two ways, once with LlamaIndex and once with a hand-rolled version in Colab, the framework-free notebook took longer to write the first time and was much faster to debug the second time, because every failure pointed straight at my own code instead of three layers of abstraction.
What you'll need
A free Google account is the only hard requirement — Colab runs in the browser, so there's nothing to install locally. You'll also want an API key from whichever model you're testing; I used an OpenAI key for embeddings and chat in this walkthrough, but the same three-piece structure works if you swap in Claude or another provider for the generation step. Budget 15 to 20 minutes for the first pass. You don't need a GPU runtime — the API calls do the heavy lifting, so the default CPU runtime is fine. For a demo of a few hundred chunks, plain numpy handles retrieval; only reach for faiss-cpu once your corpus grows past a few thousand chunks and a linear scan gets slow.
Step-by-step: framework-free RAG, agents, and evals on Colab
1. Open a blank notebook and skip the GPU runtime
Go to colab.research.google.com and start a new notebook. Leave the runtime on CPU unless you're also testing local embedding models — for API-based RAG, a GPU buys you nothing. One limit worth planning around: according to Google’s Colaboratory FAQ, free-tier notebooks "can run for at most 12 hours, depending on availability and your usage patterns." Save your index to a file before that window closes, or you'll rebuild it from scratch and pay for the embeddings twice.
2. Install two packages, not a framework
!pip install openai numpy is enough to start. No LangChain, no LlamaIndex, no agent SDK. That's the whole point of doing this framework-free — you can read every line that touches the model.
“ !pip install openai numpy “
3. Write the retrieval index by hand
Split your source text into roughly 200-word chunks with a bit of overlap so an answer that straddles a chunk boundary doesn't get lost, call the embeddings endpoint on each chunk, and stack the resulting vectors into a numpy array. Checked against OpenAI's own pricing page, text-embedding-3-small runs $0.02 per 1M tokens and the larger text-embedding-3-large model runs $0.13 per 1M tokens — for a few hundred chunks, the whole index costs a fraction of a cent.
“`python import numpy as np from openai import OpenAI client = OpenAI()
def embed(texts): r = client.embeddings.create(model="text-embedding-3-small", input=texts) return np.array([d.embedding for d in r.data])
index = embed(chunks) # chunks: list[str] “`
4. Write the retrieval-plus-generation loop
Embed the incoming question with the same model, score it against your index with cosine similarity, pull the top few chunks, and drop them into a plain chat completion prompt. That's a working RAG loop in under 20 lines — no retriever class, no vector-store adapter.
5. Add a small agent loop, still without a framework
A minimal agent is just a while loop: the model asks to call a function, your Python code runs it, and you feed the result back in the next message. Cap the loop at a fixed number of turns — five or six is plenty for a demo — so a confused model can't loop forever on your API bill.
6. Write a framework-free eval harness
Keep a list of question-and-expected-answer pairs as a plain list of dictionaries, loop the RAG pipeline over each one, and score with exact-match or embedding similarity against the expected answer. In my testing, this simple pass-rate script caught two prompt regressions in an afternoon that manual spot-checking had missed for a week, because nobody was re-running all ten questions by hand every time the prompt changed.
Example prompts you can copy
These are prompts for asking a coding assistant to write each piece above, not prompts for the RAG app itself:
- "Write a Python function that splits this text into ~200-word chunks with 20-word overlap, no external chunking library."
- "Given a numpy array of embeddings and one query vector, write the cosine-similarity top-k search by hand — no faiss."
- "Write a 15-line function-calling loop against the OpenAI API where the model can call a
get_weather(city)Python function and see the result before it answers." - "Turn this list of 10 question-and-answer pairs into a Python eval script that scores exact match, prints a pass rate, and lists which questions failed."
- "Add file persistence so my embeddings index survives a Colab runtime restart instead of re-embedding everything."
Each one asks for a specific, small piece of code — that's what keeps the assistant's output short enough to actually read before you run it.
Framework-free vs. a RAG framework
Both approaches solve the same problem. Here's where they actually differ once you're past the first hour:
| Framework-free (raw API + numpy) | LangChain / LlamaIndex | |
|---|---|---|
| First-hour setup | Two pip installs, you write ~150 lines |
One install, but many classes to learn first |
| Debugging a bad answer | You read your own traceback | Often three layers of abstraction deep |
| What you actually learn | Every embedding call and prompt, explicitly | The library's API surface |
| Lines of code for basic RAG | ~60 | ~15, hidden behind the retriever class |
| Best for | Learning the mechanics, small demos, evals you must audit | Production apps with many data connectors and integrations |
If you're trying to understand what RAG actually does, write it by hand once. If you're shipping a connector to a dozen data sources next quarter, a framework earns its keep.
Common mistakes to avoid
The one I see most: pasting an API key straight into a Colab cell instead of using the notebook's secrets manager, then sharing the notebook link with the key still baked into cell output. Second, not persisting the embeddings index before the session times out — you rebuild it from scratch and pay for the same embeddings twice. Third, skipping chunk overlap entirely, so an answer that spans two chunks never surfaces in retrieval no matter how good the query is. Fourth, writing the agent loop with no turn cap; a model that gets confused about whether it already called a function will happily call it another twenty times on your account. Fifth, and this is the one that bit me directly: treating "it worked when I tried it once" as a passing eval. When I tested a prompt tweak without running the eval script, it looked fine in a quick manual check and quietly broke two of the ten questions — I only caught it because the harness re-ran all ten automatically.
Tools that make this easier
If you'd rather skip maintaining your own notebook, Google AI Studio offers a similar free sandbox with Gemini's function-calling built in, which is worth comparing against a hand-rolled agent loop. For testing the generation step against a different model family, how to use Claude AI and how to use DeepSeek for coding both cover the setup on the provider side. If you want a broader view of which assistant writes the cleanest Python for this kind of exercise, my ChatGPT alternatives for coding roundup compares several head-to-head. Once your agent loop needs more structure than a plain while loop, how to use ChatGPT agents covers the task-framing patterns that keep a longer-running agent on track. And if your source material starts as messy PDFs or lecture notes rather than clean text, how to use NotebookLM is a faster way to get organized notes before you start chunking anything.
My take
Writing RAG, an agent loop, and an eval harness by hand in one Colab notebook took me about half a day the first time, and I'd do it again before reaching for a framework on anything I was trying to actually understand. The three pieces are genuinely small — a retrieval index, a loop, and a scoring script — and seeing them without a wrapper class in between is what made debugging fast instead of frustrating. I still reach for a framework on production work with a dozen data connectors. For learning or for a demo you can defend in a code review, framework-free wins.
Frequently Asked Questions
Is it free to build RAG, agents, and evals on Colab?
Yes, for a small demo. Colab's free tier costs nothing, and the only real expense is API usage — embedding a few hundred chunks with text-embedding-3-small costs a fraction of a cent at OpenAI's current $0.02-per-1M-token rate, and a handful of chat completions for testing adds only a little more.
How long does it take to set up a framework-free RAG notebook?
Around 15 to 20 minutes for a first working version once you have an API key — a chunking function, an embedding call, a cosine-similarity search, and a chat completion prompt. Adding the agent loop and eval harness on top takes another hour or so if you're writing them from scratch rather than copying working code.
What is the easiest way to do this?
Start with retrieval only. Get chunking, embedding, and a similarity search working and confirm it returns the right chunk for a few test questions before you add generation on top, and add the agent loop and eval harness only after the retrieval piece works on its own.
Do I need LangChain or LlamaIndex to learn RAG?
No. The core mechanics — chunk, embed, search, generate — fit in about 60 lines of plain Python, and writing them yourself once is the fastest way to actually understand what a framework is doing for you later.
What happens when my Colab session times out?
Anything in memory disappears, including your embeddings index, since free-tier sessions run for at most 12 hours depending on availability and usage. Save your index to a file (or mount Google Drive and save there) before you close the tab, and load it back in at the top of your next session instead of re-embedding everything.