Block's 500-engineer org adopted AI coding agents (Goose, Claude) rapidly, reaching 90% usage by mid-2025, but CEO saw no faster feature shipping—exposing adoption vs. impact gap.
Defined five-stage maturity model for agentic engineering: stage zero (no AI use) through stage five (engineers delegate complete tasks, agents ship without handholding).
Handpicked 50 AI champions from critical repos across Square, Cash App, Afterpay, mobile/backend/data teams—avoided top-down mandate, used trusted engineers to model peer adoption.
Made repos AI-friendly with context rules files, agent MD docs, repeatable workflows, and AI code reviewers—this standardized approach scaled patterns across monorepos, mobile, web differently.
Enabled work delegation from Slack, Jira, Linear, GitHub: engineers can ask agent to find bugs, propose fixes, implement solutions—one bug went from identification to PR in five minutes.
After three months: AI-authored code up 69%, reported time savings +37%, automated PRs increased 21x; teams completed three-sprint backlogs in one week, needed to pull in more work.
Built orchestrator and 25,000-repo company world model so agents understand relationships between services/products—enables cross-codebase task delegation without stopping to ask engineers.
Reached stage five autonomy where non-engineers (via Slack bot) could request features or bug fixes without GitHub access; agents produced shippable results independently.
Code review bottleneck at scale: tripling/quadrupling PR volume overwhelmed human reviewers; deployed AI code reviewer (Codeex) with autofix loop to pre-clean PRs before human review.
Layoffs followed autonomous org success, raising ethical questions: does enabling employees' best work while eliminating jobs represent true progress, or misaligned incentives?
Angie Jones details Block's five-stage maturity model, a 50-person AI champions program that seeded AI-friendly repos with context files and rules, and the cloud workstations plus 25,000-repo world model built to support parallel agent work. Three months in: AI-authored code up 69%, PRs automated 21x, time savings up 37%.
Capability overhang: AI models are more capable than how we actually deploy them in the real world, especially visible in agentic coding where 100,000 Microsoft developers still face slow adoption.
Models improve fastest with closed feedback loops; coding has the tightest loop (compile checks, test generation, expert feedback), while particle physics experiments have none, limiting capability growth.
Organizational constraints—regulated environments, inaccessible infrastructure, legacy systems, human psychology—slow deployment far more than model capability, creating massive plumbing work opportunities.
Just because AI can generate activity at 10x speed doesn't mean that activity is valuable; must measure actual user value and business impact, not just output volume.
Building autonomous systems requires authentic trustworthiness mechanisms, not just beautiful UX or passing tests, before users will delegate critical work to AI agents.
Scaling alone won't solve deployment gaps; requires technical work, organizational change, infrastructure modernization, and addressing human resistance to new workflows.
Even in AI's strongest domain (coding), deployment doesn't follow automatically from capability improvements; expect uneven progress across different problem domains.
Scott identifies organizational friction, regulatory barriers, legacy infrastructure, and trust deficits as harder constraints than model capability. Code generation advances fastest because compilation and testing provide tight feedback loops; domains like particle physics lack those loops and will lag far behind.
Productivity gains from AI have plateaued at 10–15%, despite better models; time savings don't automatically convert to productivity without rethinking workflows.
AI excels at finding obvious bugs and validating code quality, but senior developers add essential domain context—use specialized agents for easy catches, then route to humans.
Atlassian processes 500+ customer feedback items daily with an agent that groups them by theme, enabling immediate reactions to problems instead of waiting for weekly sprints.
Recording 8–10 minute async product demos in Loom instead of Zoom calls increased engagement by 50% while removing calendar overhead and time-zone friction.
Context is the competitive advantage, not model intelligence; 69% of developers say their data isn't optimized for AI access; build a knowledge graph connecting Jira, code, design docs, and interviews.
Build agents for high-wait-time bottlenecks like test planning, release notes, legal review, and Kubernetes setup to reduce dependency delays without replacing specialists.
Survey developers every two months on speed, wait time, and independence to identify real pain points, then measure impact with concrete signals before and after AI intervention.
PR cycle time dropped 45% by combining AI reviewers with smaller pull requests and human review; measure both the metric and the factors influencing it.
Establish company-wide AI goals focused on developer joy and useful outcomes, not just adoption; assign leadership accountability to hero use cases across all departments.
Work shifts from heavy execution to heavy planning and validation; you become a manager of agents and their outputs, requiring fundamental changes to how teams organize their efforts.
Atlassian's Sven Peters argues that treating AI as a speed tool rather than reimagining workflows caps gains at 10-15%. High-performing teams instead map human bottlenecks—PR review cycles, specialist wait time—build rich context graphs linking code to requirements and domain knowledge, and deploy agents precisely at those friction points.
Human review emerges as the critical bottleneck for AI-assisted code at scale; 75% of Google's code is AI-generated, but every change requires human review.
Statically-typed languages like Java offer advantages for AI-generated code because LLMs can compile and verify types without human intervention, unlike dynamically-typed languages.
Open-source models and weights required for defense, regulated, and airgapped environments due to regulatory constraints and need for full auditability; closed models unsuitable.
AI code quality depends heavily on training data: models excel on green-field modern code (Python, JavaScript) but struggle with legacy, domain-specific systems not in training sets.
Open-source maintainers face overwhelming pull-request burdens from AI-generated contributions; detecting AI-generated code automatically remains difficult despite attempts at guard rails.
Human-in-the-loop approval mandatory for critical decisions; AI agents can propose changes but humans must authorize actual implementation, especially in defense and resource allocation.
Training infrastructure, tooling, and ecosystem matter more than marginal model performance differences for long-term coding success.
API design for AI usability now critical consideration alongside human readability; developers should test whether LLMs understand their APIs during design phase.
LLM usage should match problem complexity; many problems solvable with simple UI or traditional methods are being unnecessarily solved with large language models.
Large language models represent only a small part of AI; green AI alternatives and cost-effective techniques exist but remain less fashionable and commercially promoted.
JNation panel with practitioners from Google, IBM, and defense contracting surfaces three friction points: human review becomes the scaling bottleneck at organizations with high AI code volume, regulated industries require open-weight models for auditability, and open-source maintainers are flooded with low-quality AI PRs from contributors who don't understand their own submissions.
