Imagine we’re building yet another e-commerce platform. One of its crucial business processes is, of course, processing an order. After a successful payment, the Orders module (domain) has to call Warehouse asynchronously to prepare purchased goods. Yet, they may not be there. Usually, it’s not a big deal, since we can get them from our suppliers. But what if any of the items are not available anymore? The order has been already placed!
The world has changed quite a lot since 1999 when Eric Brewer formulated his famous conjecture, known as the CAP Theorem today. Back then, the systems were running mostly on a self-hosted and rather expensive infrastructure, often related to the software components in use (due to licensing, etc.). A proper understanding of availability & consistency trade-offs in the presence of network partitions was simply yet another responsibility on the list.
There seems to be a lot of interest in improving application startup time these days, driven by the discussions on languages and technologies (like Go, Node, or GraalVM), that provide almost instant startup for serverless functions (a.k.a. lambdas) being run in the cloud. However, such a speedup brings benefits to other types of processing as well.
In this post, I’d like to present a different perspective on the topic of application startup time.
Apache Kafka has been designed with scalability and high-performance in mind. Thanks to its architecture and unique ordering guarantees (only within the topic’s partition), it is able to easily scale to millions of messages. However, there are some specific situations when using a topic with just one partition (despite being against mentioned features) might be a valid and simple solution to some complex problems of the distributed world.
In this post, I’d like to describe the so-called single-partition topic pattern and to list some valid use cases for it.