Kafka Terminology
Apache Kafka™ is a distributed streaming platform. Kafka runs as a cluster in one or more servers. The Kafka cluster stores streams of records in categories called topics. Each record has a key, value, and a timestamp.
For more information of Apache Kafka, refer to the Kafka documentation at https://kafka.apache.org/. Some of the key terms of Apache Kafka are defined as follows:
- Kafka Topic
- In Kafka, a topic is a logical channel to which records are published. Each topic can have one or many consumers which subscribe to the data written to the topic.
- Kafka Partition
- A Kafka Topic is grouped into several partitions for scalability. Each partition is an sequence of records that are continually added to a structured commit log. A sequential ID number called the offset is assigned to each record in the partition.
- Kafka Producer
- A Kafka producer is a program that sends messages to one or more Kafka topics. The Kafka producer can optionally specify a key, which determines the partition that the message is routed to.
- Kafka Consumers and Consumer Group
- A Kafka consumer is a program that receives messages from a Kafka topic. Each Kafka topic partition is assigned to one processing thread in a Kafka Consumer. In Kafka, consumers logically group themselves in consumer groups. A Kafka topic partition is assigned to a particular consumer in a group. Within a consumer group, all consumers work in a load-balanced mode, that is, a consumer in a group only receives one message from a partition. If the consumer goes away, the partition is allocated to another consumer in the group. If there are more consumers in a group than partitions, some consumers go idle. If there are less consumers in a group than partitions, some consumers might receive messages from more than one partition.
- Message Ordering and Client Acknowledgments
- Messages from a certain partition are always delivered in order in which they are received by the partition. A consumer client might choose to specify a message offset and all messages from that offset are delivered to the consumer.
Copyright © TIBCO Software Inc. All rights reserved.