What Event Driven Architecture Really Means
Event driven architecture (EDA) is a style where components talk through events—immutable facts that something happened. Instead of direct calls, services publish an event such as "OrderPlaced" or "PaymentAuthorized". Any other service that cares subscribes and reacts. This loose coupling lets teams evolve features fast, recover from failure gracefully, and scale horizontally without rewriting the world.
Core Concepts in One Breath
An event is a plain data record: a timestamp, a type, a payload, and maybe a correlation id. A publisher fires and forgets. A broker durably stores the event until every subscriber acknowledges it. Subscribers are idempotent: running the same event twice produces the same system state. Combine those ideas and you get a reactive mesh that is eventually consistent but highly available.
Events vs Commands vs Queries
Beginners often mix the three. Commands say "do this" and expect a response; they create coupling. Queries ask "tell me this" and need an answer immediately. Events simply announce "this happened" and carry no expectation about who listens. Keep commands inside a bounded context, expose queries through thin APIs, and let events flow across contexts.
Blueprint: How a Shopping Cart Uses Events
Imagine three microservices: Cart, Payment, and Shipping, each owning its database. The flow looks like this:
- User presses checkout; Cart service writes a local "CartLocked" event.
- Cart publishes an "OrderPlaced" event to the broker.
- Payment service consumes the event, charges the card, and publishes "PaymentAuthorized" or "PaymentFailed".
- Shipping service listens for both outcomes; only on success it creates a consignment.
No service ever calls another over HTTP, so Payment can be down for an hour without affecting Cart or Shipping. When it comes back it catches up by replaying queued events.
Choosing an Event Broker
Apache Kafka shines for high throughput and ordering guarantees. RabbitMQ is easier to run and supports complex routing. Managed options like AWS EventBridge, Google Pub/Sub, or Azure Event Grid remove ops burden. For local development test with the embedded broker in your stack—Kafka using Testcontainers or Rabbit with Docker Compose—to keep tests fast and deterministic.
Exactly Once Is a Myth: Idempotency Saves You
Brokers guarantee "at-least-once" delivery. Design every consumer so that processing the same event twice is safe. Store the event id in a unique column or use upserts. Wrap database writes and offset commits in a single transaction if your broker supports it; otherwise track offsets manually in your application tables.
Schema Evolution Without Breaking the World
Use a schema registry such as Confluent Schema Registry or AWS Glue. Start with Apache Avro, Protobuf, or JSON Schema. Enforce backward compatibility so new publishers do not crash old subscribers. Add only optional fields, never rename required ones, and bump the version in the event header. Automate compatibility checks in CI so bad builds never reach staging.
Mini Patterns You Will Actually Use
Event Sourcing
Instead of storing current state, append every domain event to an ordered log. To rebuild state, replay events. This gives an audit trail for free but adds complexity; pair it with snapshots so restarts stay fast.
Command Query Responsibility Segregation (CQRS)
Keep write models optimized for commands and read models (projections) for queries. Changes propagate asynchronously via events. Users see slightly stale data, yet writes stay simple and reads stay fast.
Outbox Pattern
Write events to a local "outbox" table inside the same transaction that updates business data. A relay process publishes the rows to the broker and deletes them after ack. You get transactional guarantees without two-phase commit support from the broker.
Eventual Consistency in Real Life
Accept that users will sometimes see a order in status "payment pending". Surface that fact in the UI with a spinner and provide a manual refresh button. Support teams can query the same read projections as customers, avoiding hidden state that only engineers understand.
Testing Strategy: From Unit to Production
Unit Tests
Pass an event object to a pure function and assert on returned state or new events. No broker needed; tests run in milliseconds.
Contract Tests
Use Pact or a home-baked fixture to verify that a publisher and subscriber agree on the event shape. Run these in CI on both sides to catch drift early.
Integration Tests
Spin up the real broker in Docker, publish a scenario, and wait until all consumers log acks. Kill containers midway to ensure retry logic works.
Chaos Tests
Inject broker latency or random broker restarts in staging. Measure end-to-end latency and error budget burn; fix retries or circuit breakers that collapse.
Observability: Tracing Events Across Services
Include a correlation id created at the edge—say, by the API gateway—and pass it in every event header. Use OpenTelemetry to export traces to Jaeger or Zipkin. A waterfall chart should show Cart → Broker → Payment → Shipping, making root cause analysis trivial even when logs scatter across pods.
Security Checklist
- Encrypt events in transit (TLS) and at rest (broker-level encryption).
- Sign events with JWS or use mTLS between publishers and broker for tamper evidence.
- Filter sensitive data; never place credit card numbers in payloads. Instead, publish a token and let the consumer call a secure API if it truly needs the detail.
- Apply topic ACLs so only the Payment service can publish to "payment.*" events.
Common Pitfalls and How to Dodge Them
Chatty Events
Spraying thousands of tiny field-level updates clogs the broker and replays slowly. Batch related changes or redesign coarse events.
Distributed Monolith
If every service must be up for a checkout to work you built a distributed monolith. Redraw boundaries so core flows survive single service loss.
Over-reliance on Broker Ordering
Kafka provides partition order, but a rebalance can move a consumer to another instance and reorder across partitions. Design algorithms that tolerate order variance or use partition keys wisely.
Technology Radar: When to Pick What
- Greenfield microservice system with uncertain load? Start with managed cloud broker and JSON events; optimize later.
- High-volume telemetry (IoT, logs)? Kafka with Snappy compression keeps disk and network sane.
- Financial transactions? Combine Kafka with exactly-once transactional producers, idempotent consumers, and outbox pattern.
Gradual Migration from Request-Response
Legacy APIs need not disappear overnight. Insert an anti-corruption layer: expose the old REST endpoint, translate the call into a command, publish resulting events, and return 202 Accepted with a status URL. Over time, upstream callers can switch to listening for events instead of polling.
Capacity Planning Baseline
A single Kafka partition easily handles 10 MB/s of events. Measure your peak payload size, multiply by 1.5 for growth, and create at least as many partitions as you expect peak consumer instances. Monitor broker disk to ensure a seven-day retention fits comfortably below 70 %.
Team Topology That Works
Align small cross-functional squads with each bounded context. Give every team full ownership of schema, code, and consumer lag dashboards. A central platform team maintains the broker cluster and tooling but does not approve every topic—self-service provisioning via GitOps keeps speed high.
Key Takeaways
Event driven architecture turns tight coupling into asynchronous flows, boosting autonomy and resilience. Start simple: publish immutable events through a broker, guarantee idempotency, and evolve schemas safely. Pair with patterns like CQRS or outbox when you need stronger guarantees, and always invest in observability before complexity snowballs. Master these pieces and you can build systems that bend without breaking—and ship features at the speed of ideas.
Disclaimer: This article is an educational overview generated by an AI language model. It does not constitute professional engineering advice. Verify broker documentation and run proofs of concept before production use.