Fault Tolerance

Kafka plug-in supports fault tolerance.

This is an example to understand the fault tolerance feature.

  • Two brokers (A & B)
  • Partitions (P0 & P1)
  • P0 - Leader is A, Slave is B
  • P1 - Leader is B, Slave is A

Two brokers (A & B) - A topic having 2 partitions (P0 & P1) and 2 replication factors where each broker is acting as a leader for each of the partitions.

When you send messages on a particular partition (P0), leader (A) of that partition (P0) is terminated abruptly. As a result, for partition (P0), new leader (B) gets elected internally and messages are continuously sent and received on the Kafka plug-in without any error.

As A is terminated for P0, Slave B gets elected as a leader, and messages are seamlessly sent and received on Kafka plug-in.

The plug-in supports multiple brokers, partitions, and replication factors.

Note: In case of any errors, try increasing the ZooKeeper timeout using any one of the following:
  • In the server.properties file, increase the connection time of property zookeeper.connection.timeout.ms.
  • Add property zookeeper.connection.timeout.ms on the Advanced tab and increase the value (connection time) to support Kafka fault tolerance.