Terminal-Bench-Science is a benchmark that drops AI agents into a terminal and asks them to complete real computational workflows pulled from working scientists' own research — not textbook trivia, but the kind of messy data-wrangling, simulation, and analysis tasks a lab actually runs. In its first public release, the best-performing model solved less than a third of the tasks.
Short answer: Terminal-Bench-Science (TB-Science) is a Stanford-led benchmark, built on the Harbor framework, that scores AI agents on 70 real scientific computing tasks across five domains, verified with deterministic pytest checks instead of human judgment. As of the v0.1 leaderboard, Claude Opus 5 leads at 30.0% resolution — well below the 50–80% frontier models hit on general coding benchmarks.
Last updated: August 28, 2026 · By Vishal Swami, Founder & Lead AI Reviewer, AISagely
I cover AI benchmarks regularly, and most of them measure the same thing: can a model write and debug code. Terminal-Bench-Science measures something narrower and, honestly, more useful — can an agent survive an actual research pipeline, the kind with a malformed FASTA file, a half-documented physics simulation, or a dataset that needs cleaning before any analysis is possible. In my testing of the setup process, the install step alone surfaced a real compatibility issue worth knowing about before you try to run this yourself.
What you'll need
To run Terminal-Bench-Science yourself, you need Python 3.12 or newer — the harbor package refuses to install on 3.11, which is still the default on a lot of servers and dev containers, so check this first. You'll also want a sandbox backend account, either Modal or Daytona, since tasks run in isolated containers rather than on your own machine. If you're testing an agent rather than just the oracle baseline, you need API access to that model (Anthropic, OpenAI, or whatever you're evaluating) and familiarity with reading pytest output, since that's how every task is scored. None of this requires a GPU on your end — the compute happens in the sandbox.
Step-by-step: running Terminal-Bench-Science
1. Confirm your Python version
Run python3 --version before anything else. When I tried installing on a Python 3.11 environment, pip rejected every release of the harbor package with a "Requires-Python >=3.12" error and offered nothing to fall back to. Upgrade first; don't waste time debugging an install that was never going to work.
2. Install Harbor
With a supported Python, install the framework with uv tool install "harbor[modal,daytona]" (the maintainers' recommended path) or pip install "harbor[modal,daytona]". The [modal,daytona] extras pull in both supported sandbox backends so you can pick one later.
3. Set up a sandbox backend
Create an account with Modal or Daytona and export the credentials Harbor expects. Tasks run inside these containers — Terminal-Bench-Science doesn't touch your local filesystem, which matters given some tasks run genuine simulation and analysis code. If you want the general background on why agent frameworks isolate work this way, see our piece on disposable, isolated sandboxes for AI agents.
4. Run the oracle baseline first
Before pointing a real model at the suite, run the reference solutions to confirm your environment is wired up correctly:
“ harbor run -d terminal-bench-science/terminal-bench-science@latest \ -k 5 \ --agent oracle \ --n-concurrent 32 \ --env modal “
If the oracle doesn't clear its own tasks, the problem is your setup, not the model you're about to test.
5. Point it at the agent you want to score
Swap the oracle flag for a real agent and model. This is the command the benchmark's own maintainers use to test Claude Code against Claude Opus 5:
“ harbor run -d terminal-bench-science/terminal-bench-science@latest \ --agent claude-code \ --model anthropic/claude-opus-5 \ --ak reasoning_effort=max \ --n-concurrent 32 \ --env modal “
6. Read the resolution rate, not just the pass/fail count
Harbor reports a resolution rate with a confidence interval (for example, "30% ±3.2"), because task difficulty varies enough across 70 tasks that a raw pass count without the interval overstates precision. Compare that number against the current leaderboard rather than a single prior run of your own.
Example commands you can copy
These cover the situations you'll actually hit — testing your own agent, limiting scope while you debug, and checking a single domain:
- Oracle sanity check before any real run:
harbor run -d terminal-bench-science/terminal-bench-science@latest -k 5 --agent oracle --n-concurrent 32 --env modal - Full run against a specific model:
harbor run -d terminal-bench-science/terminal-bench-science@latest --agent claude-code --model anthropic/claude-opus-5 --ak reasoning_effort=max --n-concurrent 32 --env modal - Lower concurrency for a smaller Modal/Daytona plan: drop
--n-concurrent 32to--n-concurrent 8— slower, but avoids sandbox quota errors on a free-tier account. - If you're proposing a new task rather than running the suite, the maintainers ask you to route it through their task proposal form first, not a direct pull request — it saves you building a task that fails review.
Common mistakes to avoid
The one that cost me the most time was skipping the Python version check — 3.11 looks close enough to 3.12 that the error message is easy to misread as a dependency conflict instead of an interpreter problem. Second, people run the full 70-task suite against an expensive model on their first attempt instead of the oracle baseline; if your sandbox credentials or network egress are misconfigured, you'll burn API spend finding that out the hard way. Third, don't compare a Terminal-Bench-Science score against a general coding benchmark score as if they're the same scale — Claude Opus 5's 30.0% here sits nowhere near the 50–80% range frontier models reach on standard software-engineering benchmarks, and that gap is the entire point of the test, not a flaw in it. Fourth, if you're contributing a task, don't skip the "agent trial run" step in the review pipeline — tasks that turn out to be solvable by a shortcut instead of the intended method get bounced back, and it's faster to catch that yourself before a senior reviewer does.
The v0.1 leaderboard
Nine frontier models have public scores against the 70-task v0.1 release as of this writing. All of them run through Harbor's containerized, pytest-verified scoring — no human graders, no partial credit for "close enough."
| Rank | Model | Agent | Resolution rate | Result date |
|---|---|---|---|---|
| 1 | Claude Opus 5 | Claude Code | 30.0% ±3.2 | Jul 24, 2026 |
| 2 | GPT-5.6 Sol | Codex | 22.4% ±2.9 | Jul 9, 2026 |
| 3 | Claude Fable 5 | Claude Code | 21.4% ±2.8 | Jun 9, 2026 |
| 4 | Claude Opus 4.8 | Claude Code | 10.5% ±2.1 | May 28, 2026 |
| 5 | GPT-5.6 Terra | Codex | 8.6% ±1.9 | Jul 9, 2026 |
| 6 | GLM 5.3 | Claude Code | 8.1% ±1.9 | Aug 14, 2026 |
| 7 | Kimi K3 | Claude Code | 7.1% ±1.8 | Jul 16, 2026 |
| 8 | Grok 4.6 | Grok Build | 7.1% ±1.8 | Aug 12, 2026 |
| 9 | GPT-5.6 Luna | Codex | 3.3% ±1.2 | Jul 9, 2026 |
The jump from Opus 4.8 (10.5%) to Opus 5 (30.0%) is the single biggest generational gain on the board, which lines up with our own testing of Claude Opus 5 on other agentic tasks — it's the model where the gap between "handles a scripted demo" and "survives an open-ended workflow" narrowed the most. Still, a 30% ceiling on the best model available means two out of three tasks fail even with the strongest agent, which is a very different story than the near-saturated scores you see on general benchmarks that have plateaued.
What the tasks actually cover
Terminal-Bench-Science's 70 v0.1 tasks split across five domains: life sciences (19 tasks), physical sciences (17), mathematical sciences (17), earth sciences (8), and engineering sciences (9). Each one is sourced from a real research workflow contributed by a domain scientist, not written by the benchmark team from a textbook. According to Snorkel AI’s project page, the review pipeline runs contributed tasks through LLM screening, a human reviewer, a senior reviewer, a pull-request review, static checks, and an agent trial run confirming the task is solvable without a shortcut — eight gates before a task ships. New tasks target a 10–20% solve rate at release, deliberately hard enough to leave headroom as models improve.
The project is led by Stanford's Steven Dillmann, Sanmi Koyejo, and Ludwig Schmidt, hosted with the Laude Institute, and backed in part by Snorkel AI’s Open Benchmarks Grants program. It builds on the original Terminal-Bench, which the team notes has already been adopted internally by Anthropic, OpenAI, and Google DeepMind for measuring coding-agent progress — TB-Science is the same idea aimed at the harder problem of real lab work. A v0.2 release targeting 100+ tasks is planned for October 5, 2026, and the repository is Apache 2.0 licensed on GitHub.
Tools that make this easier
If you're evaluating agents for your own workflows rather than just reading the leaderboard, start by getting comfortable with the model that currently tops this benchmark — our full Claude Opus 5 review covers its pricing and where it actually earns that lead. Since every TB-Science run happens inside a container, it's worth understanding the sandboxing approach generally before you wire up your own tasks; our guide to disposable AI agent sandboxes covers the tradeoffs between Modal-style and Docker-style isolation. If you're choosing between models to test in the first place, our AI tool ratings and best AI models roundups track more than just benchmark scores, and Claude vs. ChatGPT is the more practical head-to-head if you're deciding which vendor's agent to build on before you ever touch a scientific workload.
My take on what a 30% score actually means
A benchmark where the best model in the world still fails 70% of the time is, in a strange way, more useful than one where every frontier model clusters at 90%-plus — it still has room to tell you something. What struck me most isn't the leaderboard order, which mostly tracks each vendor's general release cadence, it's the size of the drop from general coding benchmarks. A model that reaches 70-80% on standard software-engineering tests losing two-thirds of its attempts on real lab workflows is a concrete, measurable reminder that "can write working code" and "can run actual science" are not the same skill. If you're building or buying an agent for research use, treat this leaderboard as the more honest number.
Frequently Asked Questions
Terminal-Bench-Science: is it free?
Yes. The benchmark, its tasks, and the Harbor evaluation framework are open source under an Apache 2.0 license on GitHub. Running it costs whatever you spend on sandbox compute (Modal or Daytona) and on API calls to whatever model you're testing — the benchmark itself has no fee.
How long does it take to run Terminal-Bench-Science?
A full 70-task run at --n-concurrent 32 typically finishes in well under an hour on the maintainers' setup, though your actual time depends on your sandbox tier's concurrency limits and how long your chosen agent takes per task. Budget more time for your first run, since confirming the oracle baseline and fixing environment issues (like the Python version requirement) usually takes longer than the run itself.
What is the easiest way to check where a model ranks?
Skip running it yourself and check the public leaderboard first — nine frontier models already have published scores there. Only run the suite locally if you need to test a model or agent configuration that isn't already listed.
How is Terminal-Bench-Science different from the original Terminal-Bench?
The original Terminal-Bench measures general coding and software-engineering tasks in a terminal environment, and frontier models already score 50-80% on it. Terminal-Bench-Science applies the same terminal-agent format to real scientific research workflows instead, and the much lower top score (30.0%) shows the two are measuring different levels of difficulty, not the same skill.
Can I contribute my own research workflow as a task?
Yes. Domain scientists can propose a task through the task proposal rubric on GitHub, and merged tasks earn the contributor co-authorship on the Terminal-Bench-Science paper. Expect an eight-stage review pipeline, including an agent trial run to confirm the task can't be solved with a shortcut.