AWS believes defect rate improvements, not frontier model advances, will unlock enterprise agentic AI adoption and value creation.
Low-frequency, low-consequence defects represent the key opportunity; high-consequence defects require expert-only fixes and severely limit scale.
Hydro (Rust framework) enables agents to build correct-by-construction distributed systems, addressing models' weakness in concurrency and failure reasoning.
Cedar policy language uses formal reasoning to make authorization correct-by-construction, reducing defects in critical control systems.
Auto-formalization converts natural language specifications into mathematically precise Cedar or Lean code through interactive conversation with customers.
Deterministic agent steering via pre- and post-conditions on tool calls balances model creativity with mathematically precise behavioral constraints.
Benchmarks should measure failure severity and customer impact, not just failure density; replace metrics like pass@10.
Industry needs end-to-end evaluation including operational properties—performance, cost, durability, availability—not just raw model capability.
AWS open-sourced Trusted Remote Execution, constraining agent-built cloud operation scripts with formal Cedar policies in production.
Investing in failure understanding and mitigation is as important as improving best-case performance; culture must treat worst days seriously.
Marc Brooker maps agent failures into four quadrants by frequency and severity, argues only low-frequency, low-consequence errors have real enterprise TAM, and outlines AWS investments in correct-by-construction frameworks (Hydro, Cedar), automated reasoning, and deterministic agent steering to get there.
K-shaped future of software: top arm (systems thinking, problem framing) accelerates 100x; bottom arm (language syntax, specific coding knowledge) commoditizes via AI agents.
Intent velocity—speed from idea to production—is the core metric, not lines of code. Measure business outcomes, not throughput.
Winners operate agents in parallel autonomously (nights, during meetings); set intent once, unlock multiple agents to run simultaneously.
GPU-to-GPU instruction translation (Rosetta) took 48 hours with AI vs. 4-5 years and 200-300 engineers traditionally; now shipping in production.
Llama CPP runtime now moves tensors between CPU/GPU/NPU with zero-cost overhead; enables full silicon utilization on laptops.
AMD built world's fastest open-source tokenizer (200K lines, one engineer); becomes pre-training data for future models—compounding flywheel.
Agents now handle continuous monitoring: auto-recreate bugs, file PRs, fix code, validate tests, commit if CI passes—no human involvement needed.
Frame problems clearly and guide AI agents; no longer limited by coding capacity but by ability to think forward at first principles.
AI transitions happen in weeks/months, not years—speed and adaptability now essential; leaders must upskill on agentic AI while helping teams transition.
Anush Elangovan details four AMD projects where agentic AI collapsed multi-year cycles: a Rosetta-like GPU instruction translator built in 48 hours, an autonomous performance optimizer, seamless CPU-GPU-NPU tensor movement, and a high-speed tokenizer. The competitive shift moves from syntax knowledge to systems thinking and intent velocity.
Spotify embeds user vectors into LLM token space as 'soft tokens' to inject personalization into generative recommendations without retraining on 750M+ users.
Semantic IDs compress content embeddings (tracks, episodes) into 4-6 tokens hierarchically, enabling LLMs to auto-regressively predict next items like words.
User taste profile feature exposes what Spotify knows about you and accepts text edits to dynamically update the generative model's understanding in real time.
Moving from siloed multi-stage ranking pipelines (candidate generation → rankers) to unified transformer backbone supporting steerable, natural-language-driven recommendations.
Spotify jointly embeds users, tracks, and episodes in same vector space, visualizable as a hypersphere where proximity reveals taste neighborhoods and unexplored regions.
Combining Spotify's knowledge (user/content vectors) with open-weight LLM world knowledge via fine-tuning and semantic ID domain adaptation improves steerability and explainability.
Transformer-based sequential user models replace older autoencoder approach, treating listening history as context like prompts in language models for better personalization.
Spotify's AI Foundation team replaces multi-stage collaborative filtering with a generative model built on three components: transformer-based user embeddings, semantic IDs that compress content into hierarchical tokens, and a soft tokenization layer that projects user state into LLM embedding space. Deployed for podcast recommendations; rolling out via the Taste Profile feature.
UK government created 'insurgent unit' at Number 10 with high political backing, market-rate pay, and autonomous hiring to deploy AI engineers across departments—bypassing traditional civil service constraints.
0.7-0.8% selection rate for fellows using custom technical hiring process; exclusively recruit outsiders from labs, big tech, and research institutes to prevent institutional lock-in.
7.25 million NHS waiting lists, 350,000 court case backlog, only 20% of planning applications decided on time; AI could deliver £40 billion annual productivity gains across UK government.
First forward-deployed engineers embedded in Number 10 policy teams—observing workflows, co-designing solutions, moving from idea to implementation in weeks instead of months.
Extract tool, built with DeepMind on Gemini, digitizes planning applications including handwritten and hand-drawn maps; rolling out to every local authority to address planning delays affecting economic growth.
Cabinet Office avoided £1.5 million lawyer contract by embedding one engineer for two weeks to automate UK statute book analysis—plus created reusable, updatable tool.
Just AI spin-out deploys fellows into prisons and criminal justice system as forward-deployed engineers working with parole officers to reduce drug smuggling and improve operational efficiency.
Policy simulation tool lets decision-makers test impact of policy choices (e.g., universal credit changes on household finances) before implementation at faster pace than traditional analysis.
Recruitment pitch explicitly seeks 'missionaries not mercenaries'—ambitious technologists from Y Combinator, academia, and industry willing to take pay cuts for high-impact public service work.
Scaling strategy focuses on making insurgent model become 'business as usual' and developing horizontal solutions (transcription, call center automation) applicable across 400,000-person civil service.
Britain's No. 10 Data Science Team runs a market-rate fellowship recruiting from labs, big tech, and YC founders—never career civil servants—and embeds them directly in departments. Early deployments include an Extract platform built with DeepMind to automate planning applications, with spin-offs now placing engineers inside prisons and scaling across 400K public-sector workers.
Generator-evaluator pattern (inspired by GANs) splits agent responsibilities into separate context windows: one builds, one critiques using Playwright/browser automation, creating adversarial pressure that improves quality.
Claude Opus 4.6 can run continuously for 12+ hours with single-session compaction instead of resetting context, eliminating need for fresh sessions between tasks.
Models are bad self-evaluators due to sycophancy; a separate harsh critic LLM is more tractable to tune than making builders self-critical.
Planner-generator-evaluator loop with contractual handoffs works better than single agents: planner sets high-level direction, generator and evaluator negotiate testable acceptance criteria before building.
Grading subjective quality (design taste, originality) is possible with detailed rubrics and few-shot examples; Anthropic weights design/originality heavily to prevent AI-slop aesthetics.
Multi-hour full-stack apps (6 hours, ~$200) now achievable with generator-evaluator harness; same prompt in solo loop produces non-functional features (e.g., games with unresponsive controls).
Context rot and context length anxiety (model rushing near window end) were critical problems in earlier models; Opus 4.6 handles coherence much better, reducing need for architectural workarounds.
Read traces by hand, not just evals: identify where model judgment diverges from yours, then tune prompts. Automated trace analysis is a secondary pass; human reading is primary debugging loop.
Harnesses co-evolve with models: as frontier improves, simplify harness (drop sprint decomposition, reduce evaluator cadence), then test to verify. Strip out scaffolding only after confirming models handle it.
File system state (not context windows) for long-running agents: use JSON files for learnings, timestamped logs, git commits; lets future models or humans pick up work without re-inventing history.
Ash Prabaker and Andrew Wilson detail three failure modes for long-horizon agents—context limits, poor planning, and self-evaluation bias—and show how a GAN-inspired generator-evaluator pattern with Playwright-driven rubric testing enables 5-6+ hour runs. Concrete example: a retro game maker that solo single-session runs failed to complete.
