Amazon Web Services · Amazon MSKCurrentAug 2025 – present
Kafka at fleet scale
KRaftupgradingZooKeeperTelemetry pingOne dot per broker, in clusters of six. An illustration, not to scale.
My team owns data plane workflows and the observability agents in Amazon MSK (Amazon Managed Streaming for Apache Kafka): telemetry collectors that gather logs and metrics on hundreds of thousands of Kafka brokers.
- Owned the in-place ZooKeeper-to-KRaft migration program, with about 12 engineers across three teams. It launched in August 2026: customers move an MSK cluster on Kafka 3.9 to KRaft in one online upgrade. Kafka 4.x supports only KRaft.
- Set the program's test strategy, acceptance criteria and regression bars. Reviewed designs and code, and fixed bugs.
- Reduced silent failures in the observability agents by making monitoring part of the team's Definition of Done.
- Cut the time from merged code to the first production region, using failure data to decide which dependency failures should block a release.










