agent reliability vs capability
AWS bets enterprise agentic AI is gated on defect rates, not model capability DeepLearning.AITL;DW
- AWS believes defect rate improvements, not frontier model advances, will unlock enterprise agentic AI adoption and value creation.
- Low-frequency, low-consequence defects represent the key opportunity; high-consequence defects require expert-only fixes and severely limit scale.
- Hydro (Rust framework) enables agents to build correct-by-construction distributed systems, addressing models' weakness in concurrency and failure reasoning.
- Cedar policy language uses formal reasoning to make authorization correct-by-construction, reducing defects in critical control systems.
- Auto-formalization converts natural language specifications into mathematically precise Cedar or Lean code through interactive conversation with customers.
- Deterministic agent steering via pre- and post-conditions on tool calls balances model creativity with mathematically precise behavioral constraints.
- Benchmarks should measure failure severity and customer impact, not just failure density; replace metrics like pass@10.
- Industry needs end-to-end evaluation including operational properties—performance, cost, durability, availability—not just raw model capability.
- AWS open-sourced Trusted Remote Execution, constraining agent-built cloud operation scripts with formal Cedar policies in production.
- Investing in failure understanding and mitigation is as important as improving best-case performance; culture must treat worst days seriously.
TL;DW
- AWS believes defect rate improvements, not frontier model advances, will unlock enterprise agentic AI adoption and value creation.
- Low-frequency, low-consequence defects represent the key opportunity; high-consequence defects require expert-only fixes and severely limit scale.
- Hydro (Rust framework) enables agents to build correct-by-construction distributed systems, addressing models' weakness in concurrency and failure reasoning.
- Cedar policy language uses formal reasoning to make authorization correct-by-construction, reducing defects in critical control systems.
- Auto-formalization converts natural language specifications into mathematically precise Cedar or Lean code through interactive conversation with customers.
- Deterministic agent steering via pre- and post-conditions on tool calls balances model creativity with mathematically precise behavioral constraints.
- Benchmarks should measure failure severity and customer impact, not just failure density; replace metrics like pass@10.
- Industry needs end-to-end evaluation including operational properties—performance, cost, durability, availability—not just raw model capability.
- AWS open-sourced Trusted Remote Execution, constraining agent-built cloud operation scripts with formal Cedar policies in production.
- Investing in failure understanding and mitigation is as important as improving best-case performance; culture must treat worst days seriously.
Marc Brooker maps agent failures into four quadrants by frequency and severity, argues only low-frequency, low-consequence errors have real enterprise TAM, and outlines AWS investments in correct-by-construction frameworks (Hydro, Cedar), automated reasoning, and deterministic agent steering to get there.