1,829 views
Information systems generate a large number of events during their operation, which can be related both to the system itself and to the implemented subject area. But the generated events need to be delivered to recipients - other components of the information system, and in this case, message queue brokers and streaming systems come to the rescue. In this video, we will talk about the most popular event streaming system - Apache Kafka and will cover the following topics: Brief description of data streams Kafka terminology Running Apache Kafka locally Using CLI utilities to send and receive messages Sending and receiving messages in a Java project 00:00:00 Introduction 00:01:35 About data streams 00:06:36 Kafka terminology and theory 00:21:00 Installing and starting Kafka 00:31:39 Sending and receiving messages in the CLI 00:47:49 Creating a sender in Java 00:55:21 Using transactions 01:02:55 Non-idempotent senders and write confirmation 01:06:12 Creating a receiver in Java 01:08:28 Manually assigning partitions to a receiver 01:11:14 Receiving messages 01:16:57 Subscribing to a topic in a group recipients 01:25:00 Finding an offset 01:39:55 Committing read offsets to the recipient 01:55:30 About delivery semantics 02:01:35 An example of Exactly Once semantics 02:06:40 Conclusions Articles related to the video: About message queues: https://alexkosarev.name/2024/10/25/a... Getting started with Kafka: https://alexkosarev.name/2024/10/31/k... Launching Apache Kafka in a cluster: https://alexkosarev.name/2024/11/04/k... Apache Kafka: partitions and replicas: https://alexkosarev.name/2024/11/08/k... Sending messages to Kafka: https://alexkosarev.name/2024/12/03/k... Receiving messages from Kafka: https://alexkosarev.name/2024/12/04/k... Telegram channel: https://t.me/+TZCuO38vG3oqu_Jq Support the project: https://www.tinkoff.ru/cf/4PEOiVCZQuS