Stream processing means high throughput and different functionality compared to a message queue. This allows you to adopt a reactive programming approach with a publish-subscribe pattern. With Kafka, you can still scale your system to enable parallel processing, but you can also add different kinds of consumers that execute various types of logic when a single event occurs. This makes Kafka a message broker or a streaming platform. Kafka lets you replay messages to allow for reactive programming, but more crucially, Kafka lets multiple consumers process different logic based on a single message. These messages will not be removed when a consumer retrieves them, making them persistent messages. Unlike a message queue, Kafka enables you to publish messages (or events) to Kafka topics.
In other words, the messages in a message queue are more like commands, suited for imperative programming, while Kafka manages events that are suited for reactive programming. You can scale a message queue by having multiple consumers, but this doesn't necessarily enable you to trigger independent actions from the same event. In this sense, message queues are very similar to imperative programming in that once something happens, the queue decides that a certain action should occur downstream in the system. Queues typically allow for some transaction, to ensure the message's desired action was successfully executed, and then the message is removed from the queue entirely. A subscriber can pull a single message or a batch of messages at once. In its simplest form, a message queue allows subscribers to pull a message from the end of the queue for processing. To better understand what Kafka can do for you, it's important to first understand the benefits and limitations of a basic message queue.
0 kommentar(er)
