The conference Integrasjonsdagene 2012 took place in Halden 30th and 31st of August. I delivered the section of the presentation which is titled “Our Experiences and Challenges with BizTalk”.
On 25th of May I presented NServiceBus at NNUG Oslo. The presentation is now available at Slideshare (in Norwegian):
On the next Norwegian .Net User Group I will give a talk on NServiceBus. Topics which will be covered include an overview of the architecture and capabilities of NServiceBus and configuration options provided by the framework.
Ole-Marius Moe-Helgesen will give an introduction to Event Driven Architecture and messaging and he will also share experiences from a project at an insurance company which made use of NServiceBus.
Jan Ove Skogheim Olsen will share project experiences with NServiceBus from Call Norwegian.
More information available here: http://www.nnug.no/Avdelinger/Oslo/Moter/Brukergruppemote-25-mai-2010/
Command and Query Responsibility Segregation (CQRS) is a pattern where reading of data and commands for updating the domain model are separated into separate services. Architectures for distributed systems built on the CQRS pattern offers high scalability and reliability and has gained in popularity during the last couple of years.
Greg Young visited this month’s javaBin meeting in Oslo for a talk on CQRS based architectures, and in this blog post I will share some of the new insight I got into CQRS.
Event sourcing and eventual consistency are two essential concepts which fit well together with a CQRS based architecture, and previously I considered these two concepts to be mandatory in order to make the architecture scalable and reliable. However, the complexity introduced to a system by using these two concepts may scare many brave developers away from building real production systems which makes the most out of an architecture built on the CQRS pattern.
Most developers feel more comfortable with using well-known architectures built on a relational model stored in a RDBMS supporting ACID capable transactions. The mind shift required when changing to event sourcing and eventual consistency may seem too big and risky.
Figure 1, typical architecture utilizing the CQRS pattern
In the javaBin talk Greg Young actually advised against using eventual consistency when starting to implement a new system and rather gradually introduce the concept in parts of the system as it evolves and scalability issues appear. This will simplify the initial implementation and make it easier to get started with CQRS.
The simplest alternative: No event sourcing and no eventual consistency
This is the simplest option for handling consistency and concurrency because the domain model and denormalized read model can be updated in a single transaction.
The domain model and the denormalized read model can either be stored in the same database server or in different servers. A distributed transaction is required if updates are made on different threads or to different database servers, which will have an impact on the performance. An ORM is typically used for persistence of the domain model.
The read model can even be implemented as views on top of an existing schema modeled for OLTP.
Figure 2, Domain model and read model updated in a single transaction
Solving scaling issues as they arise: Event sourcing and no/partial eventual consistency
Greg Young prefers using event sourcing rather than a relational schema when persistence of the domain model. To quote his thoughts about ORMs: “Using an ORM is like kissing your sister!”
The events can for example be stored in a RDBMS, an object database, a document database or be serialize to flat files. The event store must support transactions.
As the system evolves and scalability issues surface, an event queue (and hence eventual consistency) for updates to the read model can be party introduced.
Figure 3, event store and read model is updated in a transaction.
An event queue and eventual consistency is introduced in areas where scaling issues arise.
The most complex and powerful alternative: Event sourcing and eventual consistency
In this alternative the domain events are stored in an event storage, and a queue is used to update the read model.
Two different queues can be used when updating the read model. The most traditional architecture is to publish the event to a separate queue in the same transaction as updates the event store. Tools like NServiceBus are typically used when publishing to the queue.
The second alternative was described in a recent blog post by Greg Young and uses the event storage as a queue. This means that there are no requirements for distributed transactions, as the only write happening when processing a command is to the event store. The read model is updated from the events in the event store and not from a separate queue. This has the advantage that there is only one version of the truth; it’s not possible to publish events which have different content from the ones stored in the event storage.
Figure 4, event store and event queue
Figure 5, using the event store as a queue
There is a wide range of options available for how to design persistence in a CQRS based architecture. The most important thing to consider is that the persistence requirements for the domain model on the command service usually will not conform well to the data retrieval requirements for the read service (think OLTP vs. OLAP).
Other factors which must be taken into consideration when designing the persistence models are the cost requirements, is it a greenfield or a brownfield project, the skills and competency of the developers, SLAs and enterprise architecture guidelines for the organization.