Writing
Technical notes on reliable AI systems.
Essays, architecture notes, and implementation memos on evaluation, observability, governance, and production infrastructure for AI agents.
Essays
-
Retry, Fallback, or Escalate? Evaluating Recovery Decisions in AI Agent Orchestration
A practical framework for evaluating whether an AI agent workflow chooses the right recovery path after a failed step.
-
When Should an Agent Hand Off? Evaluating Handoff Decisions in Multi-Agent Workflows
A practical framework for evaluating whether multi-agent systems hand work to the right agent, at the right time, with the right context.
-
The Right Tool at the Wrong Time: Evaluating Tool-Call Order in AI Agent Workflows
Why multi-step AI agents need ordering checks, not just tool-selection and argument validation.
-
When the Tool Is Right but the Arguments Are Wrong: Evaluating Tool-Call Arguments in AI Agents
A practical framework for evaluating whether an AI agent passed the right argument payload to the right tool.
-
From Golden Traces to Release Gates: Building an Agent Regression Harness, Part 2
A practical implementation sketch for turning golden traces into CI-friendly regression tests for tool-using AI agents.
-
Final Answers Are Not Enough: Golden Trace Regression Testing for Tool-Using AI Agents, Part 1
Why multi-step AI agents need regression tests that check the execution path, not just the final response.