surveillance infrastructure capability creep

CCC: State surveillance infrastructure expands by structural logic, not deliberate overreach media.ccc.de (Chaos Computer Club)
TL;DW
  • Privilege escalation in surveillance infrastructure happens through gradual capability expansion, not malicious intent—systems built for one purpose become reusable tools across contexts.
  • Infrastructure reduces friction and shapes what appears possible; once surveillance capabilities exist, they become self-reinforcing and difficult to roll back.
  • Combination mechanisms (like Palantir Gotham) enable authorities to correlate existing data across new contexts, creating probabilistic suspicion rather than evidence-based prosecution.
  • TK technical standards have evolved continuously since 1995 to cover new communications channels (email, messenger, SMS, RCS), demonstrating how infrastructure self-perpetuates.
  • Data collection for one purpose (e.g., law enforcement) automatically gets repurposed when fed into analytics systems—no new data harvesting required, major shift in use.
  • Age verification infrastructure built for social media becomes reusable for pornography sites, border enforcement, and other controls—single capability, multiple applications.
  • Encryption-breaking pressure drives capability expansion: source TK (device-level interception) is more restricted than network TK, creating incentive for Vorratsdatenspeicherung.
  • Courts can strike down surveillance laws as unconstitutional, but underlying infrastructure, workflows, and institutional commitments persist, enabling rapid re-legalization.
  • Governments must respond to public feelings of insecurity regardless of crime statistics; unaddressed anxiety creates political pressure for visible capabilities over effective structural reform.
  • Chat control, biometric mass surveillance, and hackback proposals advance despite research showing ineffectiveness—because demonstrable action matters more electorally than measurable results.

Martin Dukek distinguishes legal authorization (Befugnis) from operational capability, arguing that once agencies build interception infrastructure—like the 1995 telecom tap interfaces—institutional inertia, feature creep, and dataset combination (e.g., Palantir Gotham) drive expansion to email, messengers, and encrypted channels without fresh legal justification, even after courts strike down the regulatory basis.

when software rewrites are justified

Glauber Costa: rewrite in Rust only when fundamental conditions change, not language preference Confreaks
TL;DW
  • NVMe devices forced CPU-bound workloads instead of IO-bound, making Cassandra's Java architecture suboptimal—this architectural shift justified Scylla's C++ rewrite, not language preference.
  • Rewrite when fundamental conditions change (new hardware like NVMe, cost constraints, AI workload patterns), not because you prefer Rust or dislike the current language.
  • Software accumulates battle scars and undocumented quirks over time (e.g., SQLite's mysterious 1GB page insertion); rewrites lose this implicit knowledge unless you document it obsessively.
  • Inherit the ecosystem by keeping external interfaces stable—Scylla stayed Cassandra-compatible to reuse drivers/connectors; Terso stays SQLite-compatible rather than inventing new databases.
  • Database-per-tenant architecture with AI agents justifies Terso's SQLite rewrite: file-based nature enables billions of isolated databases, features like encryption and concurrent writes require async-first redesign impossible in C.
  • Avoid rewrites driven solely by language preference or desire for a blank canvas; resist the temptation to replace SQL or create entirely new systems when existing interfaces serve you well.
  • When truly rewriting, you're often building something architecturally different disguised as a rewrite—treat it as new software and use the language best suited (Rust now) rather than translating line-by-line.
  • Small compatibility bugs emerge when replacing established software; plan to hunt and fix them iteratively rather than achieving perfect compatibility upfront with new implementations.
  • The Cassandra community's skepticism of Scylla faded as NVME performance gains became undeniable; justified rewrites prove themselves through performance improvements that old architecture physically cannot match.
  • Cost reduction (40x performance, reduced memory spending at Datadog) can justify architectural rewrite when legacy system's fundamental design prevents optimization within the original framework.

Drawing on three rewrites—Cassandra in C++, a Datadog in-memory DB, and SQLite in Rust—Costa argues rewrites succeed when external conditions shift (NVMe, AI isolation needs, cost) and when the rewrite preserves the original's external interfaces to inherit ecosystem adoption rather than starting from scratch.