Reliability testing for AI agents

Prove your AI agents actually work.

Statistical regression runs, semantic trace diffs, and audit-ready evidence packs — so you can prove your agents hold up, not just hope they do. Built in Europe.

Request a demo → See how it works

EU data residency · Your traces never leave the EU

tracefold — zsh

Illustrative run · seeded demo agent

The problem

Agents don't fail like software does.

Deterministic tools were built for deterministic code. Agents are probabilistic — and that breaks the assumptions every existing test suite is built on.

Same input, different output.

An agent that passed yesterday fails today — with identical code. Traditional test tools give a binary answer to a probabilistic system.

Multi-step failures stay invisible.

The agent completes the task but took a wrong, costly path. Nobody sees it until a customer does.

Model updates regress flows silently.

A provider ships a new model version; 3 of your 40 workflows quietly break.

You can't ship what you can't measure.

How it works

Capture. Generate. Run. Prove.

Four steps, one SDK. Reliability testing that fits the way your agents already run.

Capture 01

Wrap your agent in a thin SDK. Tracefold records semantic traces: tool calls, LLM inputs and outputs, DOM accessibility state, MCP messages.

Generate 02

Tracefold builds regression suites from real observed behaviour — diffing intent, not pixels.

Run 03

tracefold run --repeat N with variance budgets. A statistical pass/fail verdict, not a flaky binary one.

Prove 04

Export signed, versioned evidence packs (PDF + JSON-LD): what was tested, how, and with what result.

import { trace } from "@tracefold/sdk";

// wrap the agent you already ship
const agent = trace(myAgent, {
  name: "invoice-extractor",
  capture: ["tools", "llm", "dom"],
});

await agent.run(task);

$ tracefold run invoice-extractor --repeat 20
  ✓✓✓✓✓✗✓✓✓✓ ✓✓✓✓✓✓✗✓✗✓
  17/20 passed · variance budget exceeded
  → 3 runs diverged at step 4 (tool: submit_form)

Illustrative — API surface may change.

Ahead of the AI Act

Ready before December 2027.

The EU AI Act's technical-documentation obligations for high-risk systems apply from 2 December 2027. Teams that wait will retrofit evidence under pressure. Tracefold generates Annex IV-aligned test documentation as a by-product of testing you already run.

Today — prepare 2 Dec 2027

Annex IV-aligned, not certified. Tracefold produces documentation structured to the Act's technical-file requirements; conformity assessment remains the provider's responsibility.

EU data residencyDesigned to run on EU cloud infrastructure, under EU jurisdiction.

Signed & versioned evidenceEvery pack is timestamped and tamper-evident.

Annex IV-aligned exportsPDF + JSON-LD, mapped to the technical file.

Storyboard

A full run — trace, verdict, evidence.

One agent, one task, twenty runs — the behaviour you couldn't see before, and the proof you can hand to an auditor.

semantic trace diff01

step 1 navigate ✓

step 2 read_invoice ✓

step 3 map_fields ✓

step 4 submit_form ✗ diverged

step 5 confirm — skipped

Intent, not pixels

See the exact step and tool call where behaviour diverged.

statistical verdict02

17/20

VARIANCE BUDGET EXCEEDED

A verdict, not a coin flip

Repeat-N runs and variance budgets replace the flaky yes/no.

evidence pack03

Evidence PackSIGNED

Suiteinvoice-extractor

Runs20 · 17 passed

Versionv0.4 · sha 8f2a…

ExportedPDF · JSON-LD

Proof on demand

One command exports the paperwork — signed and versioned.

Illustrative — product in development.

Want to see a full run live? Request a demo and we'll walk you through it.

Request a demo →

Who it's for

For the teams who have to be sure.

AI product teams

Browser, voice and tool-calling agents in production — shipped with confidence, not crossed fingers.

Agencies & integrators

Prove delivered agents work — per-client projects, parallel runs, evidence the client can keep.

Regulated SMEs

Insurers and fintechs adopting agents with evidence from day one — the trust barrier, lowered.

Early access

Become a design partner.

We're looking for a small group of teams that ship agents for real. Design partners get direct access to the founding team and shape the evidence-pack format — in exchange for honest feedback.

Direct line to the founder — no sales funnel.
Shape the roadmap and the evidence format.
Your traces stay in the EU — always.

Thanks — your message is ready to send. We'll be in touch shortly.