Situation: Our platform team was split down the middle on whether our new notification engine should be event-driven using Apache Kafka or rely on a lightweight Redis Streams architecture, causing heated debates and stalling project kick-off.
Task: As engineering lead, I had to bring the team to consensus, align everyone on our business constraints, and maintain mutual trust and high morale.
Action: I stepped in to structure the decision. I asked both sides to pause ideological debates and collaborate on an Architecture Decision Record (ADR). We agreed upon our core constraints: expected throughput (10,000 events/sec), team operational expertise, infrastructure cost, and delivery timeline (six weeks). We timeboxed a three-day spike where each pair built a prototype against the same load profile. The spike revealed that while Kafka offered superior long-term partitioning, Redis Streams satisfied our five-year volume projections with zero added operational overhead, since our on-call team was already proficient in Redis.
Result: The team unanimously agreed to adopt Redis Streams for the immediate service with clean interface boundaries to adopt Kafka if event volumes scaled tenfold. We shipped on schedule, and both sides felt heard and respected because the outcome was determined by shared data.