vertical-first stream processing
Stoflow beats Kafka Streams 13x on P99 latency by ditching horizontal scaling by default Plain Schwarz (Berlin Buzzwords, Haystack)TL;DW
- Stoflow challenges distributed stream processing default by designing for vertical scaling on single instances instead of horizontal scaling across multiple nodes.
- Horizontal scaling requires repartitioning data through network and Kafka, serialization overhead up to 40-50% of compute, state migration complexity, and rebalancing storms—collectively called the 'distribution tax'.
- Stoflow architecture uses single consumer, dispatcher for event-time ordering and key hashing, parallel lanes with virtual threads, shared RocksDB state, and barrier-based exactly-once transactions—eliminating repartition topics.
- P99 end-to-end latency 13x lower, CPU usage 3.5-4x less, memory 9x less than Kafka Streams in benchmarks across five representative topologies.
- Single 8-core machine processes 124k events/sec (1KB payloads) or 2M events/sec (word count), enabling 10 billion records/day throughput—ceiling higher than most real-world mission-critical workloads.
- Visa processes ~8,200 transactions/sec globally; SWIFT ~600 msgs/sec; most stream applications operate far below horizontal scaling thresholds but default to complex distributed setups.
- Eliminates Kafka Streams operational pain: no rebalancing, no state migration, simple stop-upgrade-restart cycles with persistent volumes preserve local state.
- Virtual threads enable blocking I/O (database enrichment, REST APIs, LLM calls) natively without blocking cores—previously considered anti-pattern in Kafka Streams.
- All state globally accessible in memory without sharding; concurrent processing with atomic primitives and locking; enables foreign-key joins without data movement.
- Stoflow decouples parallelism from source topic partition count using virtual threads; achieves hundreds/thousands of concurrent lanes with minimal resource cost versus Flink's practical limits.
TL;DW
- Stoflow challenges distributed stream processing default by designing for vertical scaling on single instances instead of horizontal scaling across multiple nodes.
- Horizontal scaling requires repartitioning data through network and Kafka, serialization overhead up to 40-50% of compute, state migration complexity, and rebalancing storms—collectively called the 'distribution tax'.
- Stoflow architecture uses single consumer, dispatcher for event-time ordering and key hashing, parallel lanes with virtual threads, shared RocksDB state, and barrier-based exactly-once transactions—eliminating repartition topics.
- P99 end-to-end latency 13x lower, CPU usage 3.5-4x less, memory 9x less than Kafka Streams in benchmarks across five representative topologies.
- Single 8-core machine processes 124k events/sec (1KB payloads) or 2M events/sec (word count), enabling 10 billion records/day throughput—ceiling higher than most real-world mission-critical workloads.
- Visa processes ~8,200 transactions/sec globally; SWIFT ~600 msgs/sec; most stream applications operate far below horizontal scaling thresholds but default to complex distributed setups.
- Eliminates Kafka Streams operational pain: no rebalancing, no state migration, simple stop-upgrade-restart cycles with persistent volumes preserve local state.
- Virtual threads enable blocking I/O (database enrichment, REST APIs, LLM calls) natively without blocking cores—previously considered anti-pattern in Kafka Streams.
- All state globally accessible in memory without sharding; concurrent processing with atomic primitives and locking; enables foreign-key joins without data movement.
- Stoflow decouples parallelism from source topic partition count using virtual threads; achieves hundreds/thousands of concurrent lanes with minimal resource cost versus Flink's practical limits.
Hartmut Armbruster's Stoflow uses a single consumer with virtual threads, in-memory lane-based processing, and transactional epochs for exactly-once semantics—eliminating repartition topics and rebalancing storms. Benchmarks on an 8-core VM show 3-4x less CPU, 9x less memory, and up to 2M events/sec, handling ~10B records/day on modest hardware.