Progress AI Observability is a platform from Progress Telerik that traces every step an AI agent takes — prompts, tool calls, retrieval steps, and model output — so you can see why an agent failed instead of just knowing that it did. It launched on Product Hunt today, August 7, 2026, after an earlier private early-access period, with a free tier that needs no credit card.
Short answer: Progress AI Observability is a tracing and evaluation platform for AI agents built in .NET, Python, or JavaScript. It logs every model call, tool use, and retrieval step, flags hallucinations with LLM-as-a-judge scoring, and tracks token cost per agent. The free plan covers 10,000 units with 7-day retention; paid plans start at $29/month.
Last updated August 7, 2026.
When I worked through Progress AI Observability's sign-up flow, its SDK docs, and its published pricing page today, I wanted the actual setup path and real numbers, not a vendor deck — this is a same-day launch, so there's no independent usage history to lean on yet. Here's what the platform actually sets up to do, what each tier costs, and what I'd want to see holding up under real traffic before trusting it on a production agent.
What you'll need
You need an agent already running somewhere — Progress AI Observability instruments existing code, it doesn't build the agent for you. It currently supports three languages: .NET (C#), Python, and JavaScript/TypeScript, with direct integrations for Semantic Kernel, LangChain, LlamaIndex, AutoGen, and Microsoft Agent Framework. Model support spans Azure OpenAI, OpenAI, and Anthropic, plus open-weight models like Llama, Mistral, Mixtral, Falcon, and Gemma if you're self-hosting. You don't need a credit card to start — the free plan signs up without one — but you do need a package manager for your language (pip, npm, or dotnet) and enough access to the agent's code to drop in a few lines of instrumentation.
Step-by-step: setting up Progress AI Observability
1. Sign up for the free plan
Create an account through Progress AI Observability's sign-up page. No payment method is required to get started, and the free tier is meant to be enough to evaluate the product on a real agent, not just a demo.
2. Install the SDK for your language
Pull in the package that matches your stack: pip install progress-observability for Python, npm install progress-observability for JavaScript or TypeScript, or dotnet add package Progress.Observability.Instrumentation for .NET. Each SDK wraps the model and tool calls your agent already makes rather than requiring a separate proxy service in front of your traffic.
3. Add your app name and API key
Grab the API key from your dashboard and set it alongside an app name in your instrumentation config. This is the step that ties traces back to a specific agent or workflow in the dashboard, which matters once you're running more than one agent through the same account.
4. Run your agent and check the Trace Explorer
Trigger a real run and open the Trace Explorer. Per Progress's own product page, the platform targets under 5 minutes from install to first trace — a claim I can't confirm firsthand on a launch-day product, but the SDK-only setup path (no proxy, no reverse tunnel) makes it plausible for a single agent.
5. Turn on LLM-as-a-judge evaluations
Configure a judge model to score outputs against criteria you define — factual grounding, tone, whether the agent used the right tool. This is what catches an answer that reads as confident but isn't backed by the retrieved context, which plain uptime monitoring won't flag.
6. Set cost and anomaly alerts
Attach token and cost tracking to specific agents, models, or workflows, and set thresholds for anomaly alerts so an expensive retry loop pages someone instead of quietly running up a bill overnight.
7. Pick a plan once you know your trace volume
Watch your unit consumption on the free tier for a week or two before committing to a paid plan — the jump from Free's 10,000 units to Starter's 200,000 is large, and you want real numbers, not a guess, before you're paying for headroom you don't need.
Example commands and judge prompts you can copy
A few starting points close to what the docs walk through:
- Python install:
pip install progress-observability - JavaScript install:
npm install progress-observability - .NET install:
dotnet add package Progress.Observability.Instrumentation - Judge criterion: "Score this response 0–10 on whether every claim is directly supported by the retrieved context. Flag anything the agent stated with confidence but didn't retrieve."
- Judge criterion: "Check whether the agent selected the correct tool for this request. If it skipped a tool it should have used, or used one it didn't need, mark it as a routing failure."
- Anomaly alert framing: "Alert if any single agent run exceeds 3x the median token count for that workflow in the last 7 days."
Judge prompts work best narrow and specific — a criterion like "is this good" gives the judge model nothing consistent to score against, the same problem a vague human rubric has.
Common mistakes to avoid
The mistake I'd flag first for anyone trying this today: treating a launch-day product's own performance claims — the "85% faster root cause analysis" and "3x faster time to resolution" figures on Progress's product page — as proven rather than vendor-reported, since there's no independent case study to check them against yet. Second, skipping the judge-criteria step and relying only on trace visibility; seeing every step an agent took doesn't tell you whether the final answer was actually correct, that's what the evaluation layer is for. Third, underestimating unit consumption on the free plan — 10,000 units with 7-day retention sounds generous until a single busy agent with multi-step tool chains burns through it faster than a simple chatbot would. Fourth, instrumenting only the happy path during testing and never triggering a deliberate failure, so you don't find out what a broken trace actually looks like until it happens in production. Fifth, forgetting that retention is capped even on paid tiers below Enterprise — 30 days on Starter and 60 on Pro — so if you need to investigate an incident from two months back, you need the top tier or your own export pipeline.
Progress AI Observability vs. LangWatch at a glance
Both are agent-tracing platforms with a free entry tier, but they come from different starting points — Progress ships as part of a broader Telerik developer toolchain, LangWatch is a standalone observability product.
| Progress AI Observability | LangWatch | |
|---|---|---|
| Free tier | 10,000 units, 7-day retention, no card | 50,000 events/month, 14-day retention, no card |
| Entry paid plan | $29/month (200,000 units, 30-day retention) | ~$29/seat/month (200,000 events included) |
| Languages | .NET, Python, JavaScript/TypeScript | Any language via OpenTelemetry |
| Evaluation | Built-in LLM-as-a-judge scoring | Available via integrations |
| Best for | .NET or Microsoft-stack agent teams | OpenTelemetry-native teams, Claude Code users |
I confirmed Progress's tiers on its AI Observability Platform page and cross-checked the launch details on Product Hunt, both on August 7, 2026. LangWatch's numbers come from my own testing in Claude Code usage tracking by LangWatch. Check both vendors' live pricing pages before budgeting — plans in this category change often, and Progress's is brand new.
Tools that make this easier
If cost visibility across your whole AI stack matters more than agent-level tracing specifically, AI Spend Console by Rippling approaches the same cost-control problem from the HR and finance side rather than the engineering side. For teams building the agents this platform would monitor, AI coding assistant: a beginner’s guide and my best AI tool for code roundup cover the tools most likely to be generating the traces you'd want visibility into. If you're evaluating observability platforms against each other more broadly, my AI tool reviews hub rounds up first-hand write-ups across categories, and best AI models is worth a look if the models behind your agent are as much a question mark as the monitoring layer.
My take
The mechanism here — SDK-level tracing plus an LLM judge, priced by unit volume with a genuinely free tier — is a sound answer to a real problem: traditional uptime monitoring can't tell you why an agent chose the wrong tool or answered confidently with nothing behind it. What I can't verify on launch day is whether the Trace Explorer holds up under a busy multi-agent workload the way the marketing claims suggest, or whether 10,000 free units go as far in practice as they look on paper. If you're already in the .NET or Microsoft Agent Framework ecosystem, the native integration is a real advantage over bolting on a third-party tool. If you're OpenTelemetry-native or already invested in LangChain tooling, I'd trial both this and LangWatch side by side before picking one, since the free tiers make that comparison cheap.
Frequently Asked Questions
Is Progress AI Observability free?
Yes, for a starting volume. The Free plan covers 10,000 trace units with 7-day retention and doesn't require a credit card. Heavier or longer-running agents will outgrow it quickly, at which point Starter ($29/month, 200,000 units, 30-day retention) is the next step up.
How long does it take to set up Progress AI Observability?
Installing the SDK and adding your API key takes a few minutes for a single agent, matching the platform's own under-5-minutes-to-first-trace claim for the SDK-only setup path. Configuring judge criteria and anomaly thresholds well enough to trust them takes longer — budget a couple of real testing sessions before you rely on the alerts.
What is the easiest way to get started with Progress AI Observability?
Sign up for the free plan, install the SDK for whichever language your agent is already written in, and instrument one agent before trying to cover your whole stack. Watching real unit consumption for a week tells you far more about which paid tier you need than reading the pricing page alone.
Does Progress AI Observability work outside the .NET ecosystem?
Yes. It supports Python and JavaScript/TypeScript in addition to .NET, with integrations for LangChain, LlamaIndex, AutoGen, and Microsoft Agent Framework. The native Microsoft-stack tie-in is strongest for .NET teams, but the SDK approach works the same way across all three supported languages.
How does Progress AI Observability compare to LangWatch?
Both offer a genuinely free entry tier and per-unit pricing beyond it. Progress adds built-in LLM-as-a-judge evaluation and leans toward .NET and Microsoft-stack teams; LangWatch is OpenTelemetry-native and works with any language that can export traces, including a dedicated Claude Code integration.