Prompt injection power is bounded by agent permissions—control planner output controls plans; control tool-use agent controls tool execution.
Observability is critical: collect telemetry at LLM-to-code seams (where system prompt meets dynamic content) to detect attacks early.
Mirror system prompt patterns (markdown, spacing, tool argument names) when crafting prompt injections for higher success rates.
Data exfiltration often requires chaining LLM compromise with infrastructure hacks (CSP bypasses, expired domain purchases, credential misuse).
Stored prompt injections via RAG documents can persistently infect user long-term memory, enabling lateral platform attacks across multiple users.
Use lightweight prompt guards (Purple Llama's 300M-parameter model on CPU) for fast detection on dynamic content only, not full prompts.
Enforce tool-call policies: orchestrators must validate that agents call tools in standard ways with correct arguments and permissions.
LLM-as-judge with few-shot examples of platform-specific prompt injections generates medium-to-weak detection signal when deployed in parallel.
Scope agent capabilities per task: grant minimal permissions for each session, revoke after completion to limit blast radius.
Attacks are non-deterministic—prompt injections failing initially doesn't mean success is impossible; attackers retry dozens to hundreds of times.
Maps the attack surface across orchestration frameworks with five CVEs—VS Code Copilot, Outlook Copilot, Salesforce agents—showing kill chains from RAG poisoning to CSP-bypass exfiltration. Defenses focus on context firewalls, scoped per-session capabilities, and telemetry at LLM-to-code boundaries.
AI coding delivers 2-3x task speedup, but initial velocity gains disappear after 2 months due to AI-induced code complexity if code health isn't maintained.
Healthy code (code health score 10) reduces AI defect rates dramatically; unhealthy code (below 9) causes AI break rates to escalate beyond acceptable levels and increase defects by 60%.
Average enterprise codebase has code health of 5.15—far below the 9.5 minimum needed for AI safety; legacy code will bottleneck agentic adoption without uplift.
AI frequently generates code with low modularity, deep nesting, missing error handling, and poor structure—unhealthy code it cannot reliably maintain or extend itself.
Use MCP servers integrated with AI assistants to enforce code health checks automatically; with feedback loops, AI fixed 90-100% of code health issues versus only 50-55% without guidance.
Require 100% code coverage on new/modified code and existing codebase to prevent AI from deleting failing tests and ensure verification; coverage became one of speaker's most important KPIs.
Focus manual code review on tests, not implementation; define specifications as executable test code first, then trust automated safeguards (MCP, linting) for implementation verification.
Healthy code reduces token consumption by 29-50% compared to unhealthy code for identical tasks; as token pricing increases, code health becomes a financial imperative.
Architectural design principles (CLEAR framework) must complement code health to limit blast radius during evolution and enable safe agentic architecture at scale—still largely unsolved.
The majority of software costs (up to 95%) occur after first release during evolution and maintenance, where code quality and architecture determine success with agentic tools.
Adam Tornhill presents research showing 2-3x task speed gains evaporate in weeks as AI-induced complexity accumulates. Covers three mitigations: MCP server health enforcement, mandatory 100% test coverage, and CLEAR architectural principles — plus evidence that healthy code cuts token consumption 29-50%.
Scaling model size shows decreasing returns; GPT-4.5, Llama 4, and Mixtral releases failed to justify their computational costs despite larger sizes.
Small models now frequently outperform large ones on benchmarks; most neural network weights are redundant and can be removed after training with minimal performance loss.
Post-training, test-time scaling, and adaptive compute now offer better returns than pre-training compute; frontier labs unlikely to 4x model size again this year.
Adaptation and continuous learning emerge as the frontier; efficiency matters most because speed of learning from new information determines competitive advantage.
Optimization in data space is now cheaper than ever; targeted data curation and generation can steer model behavior toward rare parts of distributions without massive pre-training costs.
Auto Scientist automates end-to-end fine-tuning and outperforms human researchers at hyperparameter configuration by searching wider model families than domain experts typically optimize.
Small labs with strong data and training strategies can now compete; test-time compute doesn't require collocated infrastructure like pre-training, enabling distributed innovation.
Transformers are saturated architectures; hardware is overfit to matrix multiplication, making alternative architectures (capsule networks, sparse models) empirically difficult to succeed despite theoretical merit.
Pre-training, post-training, and test-time scaling serve different functions; keep data fresh across stages by injecting new information rather than repeating, and reserve parametric capacity for skills while using retrieval for facts.
Adaptive interfaces matter as much as models; code and design enable rich feedback loops absent in chat-interface thumbs-up systems—future interfaces should enable human-AI collaboration, not just mimic human behavior.
Hooker presents evidence that smaller models now outperform larger ones, model weights carry severe redundancy, and recent releases like GPT-4.5 and Llama 4 showed returns too poor to justify serving costs. The talk covers three vectors: post-training optimization, test-time compute on high-uncertainty examples, and continuous learning — illustrated by Auto Scientist, which outperformed human researchers on fine-tuning configuration search.
Economic value in AI is concentrating in chips ($300B+ revenue), not applications—Nvidia's data center business earns ~75% gross margins while app-layer profitability ranges 0-30%.
AI triangle resembles 2004 cloud economics; it took AWS eight years (2004-2012) to flip from infrastructure to applications dominance—this AI inversion may take longer due to substrate complexity.
Chip inference workloads represent ~40% of Nvidia GPU fleet utilization, training ~60%; inference share expected to grow as applications scale, unlocking profitability.
ChatGPT reached ~1 billion users monetized at $10/user/year; comparison: Alphabet monetizes 4 billion users at $100/user/year—growth requires expanding beyond knowledge work into mandatory daily utility status.
Hyperscaler ASICs (Google TPU, Meta MTIA, Amazon, OpenAI efforts) represent the likeliest repricing catalyst for the semis layer; breakthrough success would reshape dominance.
Vertical integration winners: Google (internet), Apple (mobile), Meta (social), but cloud remains heterogeneous—full vertical integration may be necessary for AI super-cycle dominance.
Revenue concentration risk: last two years added $350B to AI ecosystem; 75% accrued to semis, 90% of applications revenue split between two companies (OpenAI, Anthropic/Google).
AI application monetization will likely shift from subscriptions ($10/user) to intent-based advertising with better attribution and pricing than mobile ads—no model yet proven on phone-scale consumer AI.
Apps layer infrastructure quality poses bottleneck: hardest part of AI ecosystem is getting the substrate right—until solved, profitability remains trapped in chips.
Feature versus platform distinction critical for infrastructure startups: companies solving narrow problems risk becoming AWS features rather than standalone businesses.
Maps the generative AI value chain across semiconductors, infrastructure, and applications, showing $350B in new revenue concentrated at Nvidia despite 10x application growth over two years. Covers why near-zero marginal cost breaks down when serving users burns GPU compute, and what conditions—custom ASICs, inference dominance, hyperscaler integration—could reprice the stack.
Language models generalize better from in-context information than from fine-tuned parameters—reversals and syllogisms hit 99% accuracy in context but near chance after fine-tuning.
The reversal curse persists even when training models from scratch on synthetic data, showing it's a fundamental limitation of parametric learning, not just fine-tuning.
In-context learning succeeds because structural patterns (reversals, logical implications) appear frequently in natural training data, making them learnable as flexible procedures.
Parametric learning ties knowledge to explicit surface forms in training data, while in-context learning preserves richer detail that enables flexible reuse.
Offline data augmentation—using the model's in-context reasoning to generate latent inferences and adding them to training data—matches or exceeds pure in-context performance.
Test-time episodic retrieval (bringing learned documents back into context via oracle memory) restores generalization on reversal and other latent-structure tasks.
Reinforcement learning can train models to regenerate needed information via chain-of-thought, generalizing reasoning patterns to new domains, though it struggles with symmetry-breaking tasks like reversals.
These three methods trade off compute cost: offline augmentation is expensive to train but cheap at test time; test-time retrieval is cheap to train but expensive at inference.
Hippocampus-like episodic memory and cortex-like parametric learning are complementary in humans—fast episodic storage plus slow parametric integration prevents interference while enabling rapid single-trial learning.
Parametric systems need statistical structure to efficiently constrain inference; pure symbolic reasoning lacks feasibility for real-world generalization, requiring the models learn content-sensitive reasoning patterns.
Controlled experiments on facts, syllogisms, and encodings show fine-tuned models fail to reverse relations or compose logical chains, while the same models nearly ace both tasks given the data in context. Three mitigations tested: offline data augmentation, episodic retrieval at inference time, and RL-driven regeneration, each trading training cost for inference cost.
