Apache Kafka Monitoring

The tibkmongateway process polls a configured set of Apache Kafka-related services like Brokers, ZooKeepers, Connectors, Streams, Producers & Consumers. It pushes the polled information to an InfluxDB database for retrieval, display, and analysis using customizable Grafana-based dashboards.

For ease of configuration, a .yaml file is used to configure the monitoring behavior, allowing easy configuration for each Apache Kafka component, including jmx-host and jmx-port settings, security settings, and other component-specific settings.

Scope

The TIBCO Apache Kafka Monitoring library gathers monitoring data from the following Apache Kafka components (the default sampling rate is every 60 seconds, but this value can be modified):

  • Apache Kafka Brokers
  • Apache ZooKeeper
  • Apache Kafka producers and consumers
  • Apache Kafka stream modules
  • Apache Kafka Connectors