Catalog / Event-Driven Architecture Cheatsheet
Event-Driven Architecture Cheatsheet
A quick reference guide to Event-Driven Architecture (EDA) principles, patterns, and technologies. Covers key concepts, benefits, and practical implementation details.
Core Concepts
Fundamental Principles
|
Events: Significant state changes or occurrences within a system. |
|
Producers: Services that emit events. They don’t need to know who consumes them. |
|
Consumers: Services that subscribe to and process events. They are decoupled from producers. |
|
Event Router/Broker: An intermediary that receives events from producers and routes them to appropriate consumers (e.g., Kafka, RabbitMQ). |
|
Asynchronous Communication: Producers and consumers operate independently and don’t wait for direct responses. |
Key Benefits
|
Decoupling |
Services operate independently, reducing dependencies and improving resilience. |
|
Scalability |
Individual services can be scaled independently based on their event processing needs. |
|
Flexibility |
New services can be added to consume existing events without modifying producers. |
|
Real-time Processing |
Enables immediate reaction to events, supporting real-time analytics and decision-making. |
Event Types
|
Event Notification: Simple notification about a state change. Consumers typically fetch related data. |
|
Event-Carried State Transfer: Event contains the data needed by consumers. |
|
Event-Carried Change Notification: Event contains the changed data. |
Common Patterns
Event Sourcing
|
Capturing all changes to an application’s state as a sequence of events. The current state can be reconstructed by replaying the events. |
|
Benefits: Auditability, temporal queries, easier debugging. |
|
Considerations: Event storage, replay mechanisms, eventual consistency. |
CQRS (Command Query Responsibility Segregation)
|
Separating read and write operations. Write operations (Commands) result in events that update read models (Queries). |
|
Benefits: Optimized read and write performance, simplified data models. |
|
Considerations: Eventual consistency, complexity in managing separate models. |
Saga Pattern
|
Managing distributed transactions by breaking them into a sequence of local transactions. Each local transaction publishes an event to trigger the next transaction in the saga. |
|
Compensation Transactions: If one transaction fails, a series of compensating transactions are executed to undo the previous transactions. |
|
Types: Choreography-based (implicit coordination) and Orchestration-based (explicit coordination). |
Technology Stack
Message Brokers
|
Apache Kafka |
High-throughput, fault-tolerant, distributed streaming platform. Suitable for large-scale event processing and data pipelines. |
|
RabbitMQ |
Versatile message broker that supports multiple messaging protocols. Good for complex routing and guaranteed delivery. |
|
Amazon SNS/SQS |
Cloud-based messaging services. SNS for pub/sub and SQS for message queues. Highly scalable and managed. |
Event Processing Frameworks
|
Apache Flink |
Distributed stream processing engine for stateful computations over unbounded data streams. Suitable for real-time analytics and complex event processing. |
|
Apache Spark Streaming |
Extension of Spark for processing real-time data streams. Supports micro-batching approach. |
|
Spring Cloud Stream |
Framework for building message-driven microservices. Provides abstractions for connecting to different message brokers. |
Data Storage
|
Event Store: Database optimized for storing event streams. Examples: EventStoreDB, AxonDB. |
|
NoSQL Databases: MongoDB, Cassandra, etc. Suitable for storing denormalized read models in CQRS. |
|
Relational Databases: PostgreSQL, MySQL, etc. Can be used for read models, but may require careful optimization. |
Implementation Considerations
Consistency
|
Eventual Consistency: Data may not be immediately consistent across all services. Requires careful handling of race conditions and conflicts. |
|
Idempotency: Consumers should be able to process the same event multiple times without side effects. |
|
Exactly-Once Semantics: Ensuring that each event is processed exactly once. Difficult to achieve in distributed systems. Often approximated with at-least-once delivery and idempotency. |
Error Handling
|
Dead Letter Queues (DLQ): Events that cannot be processed are sent to a DLQ for further investigation. |
|
Retry Mechanisms: Implement retry policies for transient errors. Use exponential backoff to avoid overwhelming the system. |
|
Circuit Breakers: Prevent cascading failures by temporarily stopping event processing when a service is unavailable. |
Monitoring and Observability
|
Event Tracking: Monitor event flow and processing latency. |
|
Correlation IDs: Include a correlation ID in each event to track it across different services. |
|
Metrics and Logging: Collect metrics about event processing and log errors and warnings. |