Full-duplex speech models (like Moshi) enable overlapping speech and back-channeling, making conversations robust to human interaction patterns; half-duplex models break when users naturally interrupt or provide feedback.
Cascaded systems (speech-to-text → LLM → text-to-speech) require entire stack to complete in ~200ms for natural conversation, but TTS alone costs 200ms+, making human-like latency impossible without tool-call optimization.
Full-duplex models won't replace cascaded systems until they offer equal reliability, intelligence, observability, and personalization—current speech-to-speech models lack agent capabilities, tool calling, and abuse detection needed for production.
Tool latency (500ms–4s) is now the main bottleneck, not TTS; filler-based solutions where LLMs keep conversation flowing during tool calls help mask unpredictable delays.
On-device TTS models (<100M parameters) on smartphone CPUs eliminate cloud API costs and privacy risks, enabling profitable consumer voice apps without paying per-inference fees at scale.
Voice API costs are unsustainable for consumer apps; hyperscalers run voice models at a loss as marketing, but TTS dominates production costs—consumer builders burning entire fundraising rounds on API bills.
Zeghidour draws a line between naturalness and production readiness: full-duplex models like Moshi handle interruptions and paralinguistic cues but lack tool use, observability, and scalable economics. Cascaded STT-LLM-TTS pipelines stay dominant; Gradium's on-device TTS (Phonon) targets the cost problem by eliminating API round-trips.
Proprietary data, not frontier models, creates competitive advantage—less than 1% of enterprise data is publicly available on internet.
Bring models to your data, not data to models; contextual information about business-specific scenarios is unavailable to any general AI model.
Most AI deployments today add zero revenue impact; they improve efficiency 20-30% but lack proprietary context for critical business decisions.
Build agentic systems left-to-right: first define data needs, then governance, then pick a reasoning engine—not right-to-left by starting with model choice.
Three critical questions for agentic future: Who wins the model race? What could go wrong with agent sprawl? What's my unique competitive moat?
AI without governance is risk; centralize enterprise data from all systems (Salesforce, Workday, trading tools) into single lakehouse with role-based access control.
RBC Capital Markets built equity research agent reducing report time at 99% accuracy; MasterCard monetizes proprietary real-time transaction data via intelligent services.
Enron case study: unified data access reveals fraud red flags—mismatches between reported valuations and internal models, suspicious methodology changes, CFO-ordered calculation changes.
Start with business challenge, not technology; too often teams pick models first then search for problems—reverse this pattern and define use cases before model selection.
Jamin Nakai argues frontier models are commoditizing fast, so financial institutions' edge shifts to data strategy—grounding agents in proprietary context via a governed lakehouse. Covers agentic sprawl risks, RBC Capital Markets and Mastercard as real examples, and a hypothetical Enron fraud-detection case to show what context-aware reasoning unlocks.
Self Flow is a self-supervised training method that eliminates external encoders by combining representation learning and generation in a single flow using student-teacher noise levels.
Self Flow trains one model jointly across multiple modalities—images, video, audio, and actions—without separate specialized encoders for each, enabling true multimodal generative AI.
Models trained with Self Flow outperform baselines in text rendering, anatomy, and video coherence while converging faster and still reducing loss after baseline plateau.
Flux Klein generates and edits images in under 500ms (editing) and 300ms (generation)—near real-time—while matching or exceeding quality of slower open-source competitors like Kwen at 15+ seconds.
Self Flow enables joint video-and-audio generation from a single model trained on images, video, and audio without mode-specific alignments or encoder compromises.
Black Forest Labs is expanding beyond image generation toward physical AI, training models to predict robot actions and movements for automation and self-driving applications.
Self Flow removes the scaling ceiling imposed by fixed external encoders, allowing student and teacher models to scale up together without encoder limitations.
Prior encoder-based training showed unpredictable alignment failures—DinoV3 outperformed DinoV2 technically but worsened generative model performance with no clear explanation.
World models trained via Self Flow simulate geometry, relationships, and world interactions to enable training agents in generative environments for scaled robotics and manufacturing automation.
Real-time multimodal generation enables interactive visual engines for gaming and film where creators render content at the speed of prompting, not waiting seconds or minutes.
Self Flow uses dual noise streams—one heavily noised, one lightly noised—to jointly learn generation and representation in a single model, eliminating external vision encoders. Converges faster, fixes anatomy and text artifacts, and generalizes across images, video, audio, and robot action prediction.
Universal Commerce Protocol (UCP) enables agents to make purchases through structured API calls and JSON responses instead of scraping HTML forms and clicking buttons.
Shared Payment Token mechanism allows agents to send payment credentials to sellers with enforced spending limits—Stripe will decline charges exceeding the pre-agreed amount.
Machine Payments Protocol (MPP) uses HTTP 402 status code to declare resources requiring payment, enabling agents to pay micropayments for API calls and data on-demand.
Agents require new payment capabilities: machines need to buy API access, compute resources, and data—not t-shirts—creating distinct commerce patterns from human purchasing.
Crypto and blockchain settlement offer faster instant payment for agent-to-agent commerce, likely to dominate as instant digital transactions between machines eclipse traditional human commerce volume.
Open protocols and standards prevent siloing and lock-in that plagued centralized platforms like social media—critical safeguard when agentic commerce becomes 100x more powerful.
Agents can handle multiple competing protocols and payment methods simultaneously, unlike humans who struggle with complexity—protocols don't need to converge as quickly as they did historically.
Verification and acceptance testing loops close the gap between what agents are instructed to do and what they actually execute—essential new pattern for trustworthy agentic commerce.
Sellers retain full control of their backend and payment stack with UCP; agents simply consume standardized APIs instead of automating human checkout flows.
Machine Payments Protocol works with fiat, crypto, and shared payment tokens, supporting both subscription and usage-based billing models emerging in agent commerce.
Universal Commerce Protocol replaces HTML scraping with API-driven checkout and cryptographically enforced spending-limit tokens; Machine Payments Protocol revives HTTP 402 for per-request settlement on digital goods and API calls. Both protocols support fiat and crypto, with panelists from Block and Alchemy pressing for open standards over proprietary silos.
AI red teaming is fundamentally different from traditional security testing: attacks manipulate behavior, not exploit code bugs, with zero repeatability even at temperature zero.
LLMs hallucinate convincingly and reliably produce false outputs; black-box red teaming is useless without accessing real data to validate whether findings are genuine.
Prompt injection is an inherent flaw in LLM architecture due to lack of hierarchy between trusted system data and untrusted user/tool data in the context window.
Indirect and cross-prompt injection vectors like resume uploads, RAG databases, emails, logs, and poisoned web content are harder to detect than direct prompts.
Common attack techniques include jailbreaks (defeating model guardrails), prompt injection (attacking deployment framework), crescendo attacks (gradual escalation), and adversarial suffixes (mathematically optimized tokens).
Agents are far more dangerous than chatbots because they take real-world actions based on hallucinations and can exhibit emergent behaviors when multiple agents interact.
Existing AI red team tools are immature: they test for easy toxicity and jailbreaks but don't test real security concerns like data exfiltration or agent permission escalation.
Shift-left security into AI-native CICD pipelines with baseline benchmark testing and attack prompt libraries so developers can evaluate model safety during development.
Semantic analysis using AI is required for defense and attack because language is nearly infinite; naive string filtering (ignoring keywords) fails immediately and cannot prevent prompt injection.
Accept that you cannot eliminate prompt injection risk; instead, build mitigating controls around agents and clearly communicate residual risk to stakeholders.
Transformer architecture makes prompt injection structurally unavoidable, so NDC's session shifts focus to creative adversarial testing: jailbreaks, context poisoning, crescendo attacks, and adversarial suffixes. Covers Crop Duster and Tapper for AI-powered red teaming, and argues current vendor tooling misses real business risks like agent misbehavior and data exfiltration.
