Thinking is free. Trusting is not.
AI output is cheap. Verification is the new constraint.
Evidence & implications
Bosch rolled generative tooling out to 12,000 engineers, who self-reported feeling about 20% more productive. Then Bosch’s Head of AI Governance, Jochen Kokemueller, pointed to a controlled trial of experienced developers that found the opposite: they were 19% slower with AI assistance, and believed they were 20% faster. A 40-point gap between feeling and measuring. Code now ships in an hour and then waits three days for review, and Kokemueller’s read is that the constraint on AI adoption is no longer how fast machines generate, it’s how fast humans can verify. His fix borrows from algorithmic trading: machines checking machines, with a named human accountable at the end.
Meta’s Rohit Patel, Director at Meta Superintelligence Labs, described the engineering version: agent output is nondeterministic, so you need evals, a judge model scoring real outputs against curated golden answers, before you can tell whether your agent is getting better or worse.
Collect 500–1,000 real task examples with actual responses. Build “golden” reference answers through multiple review passes, set a higher bar than any single human answer. Optionally write task-specific rubrics for partial credit.
Prompt a judge model with the question, the golden answer, and the response being scored. Optionally fine-tune the judge for your domain. The judge becomes the quantitative compass for everything you ship.
A single trackable score, as simple as share correct, or a weighted rubric for complex tasks. This is what you watch over time to know whether your agent is improving or drifting.
What this is. The three-step pipeline Meta uses to put a number on agent quality — because without one, every release is a vibe check. Source: Meta AI research; Rohit Patel’s talk.
“Thinking is free. Trusting is not.”
Jochen Kokemueller · Head of AI Governance, BoschPharma taught me verification before AI made it fashionable. Nothing ships without a claim being checked against the source. The interesting question isn’t whether to bolt an AI verification step onto a workflow; it’s whether the domain knowledge behind that check is what actually makes it valuable. That’s the moat. Not the model.