Kafka Event Publisher
Introduction
Kafka is an open source distributed event streaming platform and Kafka works well as a replacement for more traditional message broker.
In the Drupal developer ecosystem developers use hooks or Symfony events and write modules using the APIs provided. In such a case entire business logic or the data processing which would be triggered either on a hook or Symfony events is part of the module. Few of the problems with this pattern is the often that these modules can't be scaled indenpendently due to tight coupling.
The motivation of Kafka Event Publisher module is to decouple the processing of these Symfony events to Kafka (message broker).
How it works?
Kafka Event Publisher modules allows developers to map Drupal events to Kafka topics hence decoupling the processing of the business logic upon the trigger of that Drupal event to an independent microservice.
Requirements
-
= PHP 7.4, libkafka v0.11.0, php extension rdkafka
Installation
- Install as you would normally install a contributed Drupal module. Visit https://www.drupal.org/node/895232/ for further information.
Configuration
- Goto /admin/config/kafka-event-publisher
- Check Enable Kafka Event Publishing checkbox
- Enter the Kafka brokers
- Map Kafka topics to Drupal Events. The supported format of the mapping is
kafka_topic_name|entity_bundle_name:events, e.g.- Publishing newly created article to article_notify topic:
article_notify|article:create - Updating Article:
article_update|article:update - Deleting Article:
article_delete|article:delete - Viewing Article:
article_view|article:view - And similarly for other entity i.e. for user
user_notify|user:create
- Publishing newly created article to article_notify topic: