AI productivity claims vs evidence
Independent research finds AI coding tools deliver 4% productivity gain, not 55% GOTO ConferencesTL;DW
- Study claiming 55.8% AI productivity gains lacks credibility; follow-up research found only 4% boost and zero significant labor market impact on earnings or hours.
- Reasoning models perform worse on high-complexity tasks, taking orders of magnitude longer; agents excel only in low-medium complexity tasks in well-tested, debt-free codebases.
- 57% of code written with AI copilot tools is involved in bugs; code churn, duplication, and refactoring activity all increased significantly since AI adoption.
- AI-generated work slop masquerading as quality reduces trust: 53% report annoyance receiving it, and 50% view colleagues who send it as less creative, capable, and trustworthy.
- Writing a 100-word email with AI consumes 140 watt-hours of energy (seven phone charges); training GPT-4 used 50 gigawatt-hours—equivalent to 6,000 US homes' annual consumption.
- Stop automating broken processes with AI; eliminate them instead. Adding AI to dysfunctional workflows creates insatiable demand for more reports, not solutions.
- No customer is asking for AI chatbots, AI emails, or AI interaction; talk directly to users about what's actually hard, slow, and painful before building anything.
- Organizations need pioneers (ideators), settlers (productizers), and town planners (commoditizers), but asking one person to fill all three roles guarantees failure.
- Context-switching across multiple projects kills shipping; the best way to fail at inventing something is making it a part-time job alongside existing responsibilities.
- Build small, ship fast to production, measure actual user behavior, and roll back quickly—then market working solutions as AI-powered to capitalize on hype without chasing false productivity claims.
TL;DW
- Study claiming 55.8% AI productivity gains lacks credibility; follow-up research found only 4% boost and zero significant labor market impact on earnings or hours.
- Reasoning models perform worse on high-complexity tasks, taking orders of magnitude longer; agents excel only in low-medium complexity tasks in well-tested, debt-free codebases.
- 57% of code written with AI copilot tools is involved in bugs; code churn, duplication, and refactoring activity all increased significantly since AI adoption.
- AI-generated work slop masquerading as quality reduces trust: 53% report annoyance receiving it, and 50% view colleagues who send it as less creative, capable, and trustworthy.
- Writing a 100-word email with AI consumes 140 watt-hours of energy (seven phone charges); training GPT-4 used 50 gigawatt-hours—equivalent to 6,000 US homes' annual consumption.
- Stop automating broken processes with AI; eliminate them instead. Adding AI to dysfunctional workflows creates insatiable demand for more reports, not solutions.
- No customer is asking for AI chatbots, AI emails, or AI interaction; talk directly to users about what's actually hard, slow, and painful before building anything.
- Organizations need pioneers (ideators), settlers (productizers), and town planners (commoditizers), but asking one person to fill all three roles guarantees failure.
- Context-switching across multiple projects kills shipping; the best way to fail at inventing something is making it a part-time job alongside existing responsibilities.
- Build small, ship fast to production, measure actual user behavior, and roll back quickly—then market working solutions as AI-powered to capitalize on hype without chasing false productivity claims.
Rasmus Lystrøm contrasts vendor-cited efficiency claims against recent independent studies showing only 4% improvement, 57% of AI-assisted code involving bugs, and reasoning models performing worse on complex tasks. Also covers trust erosion from code quality degradation and GPT-4 training consuming energy equivalent to 6,000 US homes.