Configuration and Script Examples

To verify that your TIBCO Messaging Monitor for Apache Kafka installation is ready for configuration and deployment, perform the following general tasks and confirmations. This topic contains simple default configurations and usage examples; your requirements may vary.

Before you begin
  • Ensure that you have Python 3 and Java installed before you begin.
  • Run the following script to install any Python dependencies required by Apache Kafka Monitoring:
    pip3 install -r /opt/tibco/msgmon/akd/requirements.txt

Edit the config/kafka_config.yaml file

Begin by editing the kafka_config.yaml file, which is located in /opt/tibco/msgmon/akd/config/

This file contains configuration information pertaining to the Apache Kafka cluster to be monitored. Edit this file to enter specifics about your Apache Kafka Cluster and Apache ZooKeeper nodes. You may need to modify the following fields:

settings:
    grafana:
        url, username, password
        datasource:
            type, access, basicAuth, isDefault

    influxDB:
        url, username, password

The default configuration in MSGMON Core at /opt/tibco/msgmon/config/ should be sufficient to start the InfluxDB and Grafana services.

Start Kafka Monitoring

Note: Note: The configured maximum number of file handlers must be high enough for InfluxDB to monitor multiple Apache Kafka services in parallel. You can see the file handlers limit by executing "ulimit -n". You can set the file handlers limit by executing "ulimit -n <int>".

Start monitoring using the following procedure:

  1. Change to the monitoring directory:
    cd /opt/tibco/msgmon/akd/
  2. Run the start script.
    python3 bin/msgmon-kafka-start.py -kc config/kafka_config.yaml

    You see an output similar to the following:

    *************************************
    TIBCO Messaging Monitoring - Apache Kafka Plugin: 1.1.x
    *************************************
     INFO    [argument_parser.py:126] MSGMON-Kafka logging level set to INFO
     INFO    [measurements.py:67] Successfully loaded all templates and drivers.
     INFO    [validate.py:40] Validation successful. Kafka-config @/opt/tibco/msgmon/akd/config/kafka_config.yaml 
     INFO    [generator.py:321] my_zookeeper_1..zookeepers..my_cluster_1 added to list of jmxTrans_Servers
     INFO    [generator.py:321] my_zookeeper_2..zookeepers..my_cluster_1 added to list of jmxTrans_Servers
     INFO    [generator.py:321] my_zookeeper_3..zookeepers..my_cluster_1 added to list of jmxTrans_Servers
     INFO    [generator.py:225] sample_broker_1..brokers..my_cluster_1 added to list of jmxTrans_Servers
     INFO    [generator.py:225] sample_broker_2..brokers..my_cluster_1 added to list of jmxTrans_Servers
    *
    *
    *
    INFO [influx.py:265] InfluxDB server at http://localhost:8086 reports version 1.8.2 INFO [influx.py:363] Retention policy _persistent created with INF duration INFO [influx.py:363] Retention policy _tibco_kafka_rp created with INF duration
Note: Note: To see all available options, use the -h option.

Check Monitoring Status

To check that the monitoring for all of the cluster elements has started, enter:

python3 bin/msgmon-kafka-status.py  

You see an output similar to the following:

*************************************

TIBCO Messaging Monitoring - Apache Kafka Plugin: 1.1.x

*************************************
 INFO    [msgmon-kafka-status.py:144] MSGMON-Kafka logging level set to INFO
 INFO    [msgmon-kafka-status.py:94] Status File loaded from: /var/tmp/phonnaka/tibco-msgmon/status/msgmon-kafka.yaml

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  jmxTrans Status : Running

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To see detailed information about the status of jmxTrans and Apache Kafka clusters being monitored, use the command using the --detailed option as follows:

python3 bin/msgmon-kafka-status.py --detailed. 

Stop Monitoring

To stop monitoring of your Apache Kafka cluster, run the stop script:

python3 bin/msgmon-kafka-stop.py

You see an output similar to the following:

*************************************

TIBCO Messaging Monitoring - Apache Kafka Plugin: 1.1.x

*************************************
 INFO    [msgmon-kafka-stop.py:173] MSGMON-Kafka logging level set to INFO
 INFO    [msgmon-kafka-stop.py:112] Status File loaded from: /var/tmp/phonnaka/tibco-msgmon/status/msgmon-kafka.yaml
 INFO    [msgmon-kafka-stop.py:120] Stopped process jmxTrans running with pid: 54890
 INFO    [msgmon-kafka-stop.py:142] Status File written to : /var/tmp/phonnaka/tibco-msgmon/status/msgmon-kafka.yaml