Using Ignite CDC

Apache Ignite is an open source, in-memory computing platform. It provides the foundation for GridGain. You can also replicate cache data by using Apache Ignite's Change Data Capture (CDC) extension. The CDC extension provides Active-Passive or Active-Active types of cross-cluster replication.

Note: Ignite CDC should be implemented in a multihost environment as it is not supported in a single-host environment.

The CDC extension provides the following cross-cluster replication options:

Active-Passive Replication

Only the sender implements the CDC by input to the Receiver's address. There is no change required at the receiver's end.
Active-Active Replication
In this, both the sender and the receiver must implement Ignite CDC and input the Receiver's address.

To set up a cross-cluster replication, the CDC extension module provides the following methods:

  • IgniteToIgniteClientCdcStreamer
  • IgniteToIgniteCdcStreamer
  • IgniteToKafkaCdcStreamer combined with KafkaToIgniteCdcStreamer

TIBCO BusinessEvents implements IgniteToIgniteClientCdcStreamer for cross-cluster replication. See Apache Ignite documentation for more information,.

Before you begin
  • TIBCO FTL or Apache Ignite must be configured as the cluster provider. For more information, see Configuring TIBCO FTL as the Cluster Provider or Configuring Apache Ignite as the Cluster Provider
  • If you have configured Apache Ignite as the cluster provider, then you do not need to define the common connection settings. The application uses values for these settings from the cluster provider configuration.
  • However, if you have selected TIBCO FTL as the cluster provider, then you have to configure the Apache Ignite cache properties and the cluster properties at once.
  • To use the Apache Ignite's CDC replication feature, set up Ignite CDC extension JAR in the library of BusinessEvents installation. To set up Ignite CDC extension JAR, see Setting up CDC Extension JAR.
  • Set up the configurations to Shared Nothing as persistence options. See Persistence Options for more information.
    Procedure
  1. In TIBCO BusinessEvents Studio, open the project CDD file and on the Cluster tab, select Object Management.

  2. In the Configuration window, select the following:

    • From the Object Management Mode list, select Cache.
    • From the Cache Provider list, select Apache Ignite.

  3. Define the cache settings as follows.

    Apache Ignite Cache Properties
    Field Description

    Cache Agent Quorum

    Specifies a minimum number (quorum) of storage-enabled nodes that must be active in the cluster when the system starts up before the other agents in the cluster become fully active.

    Default value: 1

    Note: The inference agent does not start processing events if the number of cache members in the cluster does not satisfy the quorum count.
    Number of Backup Copies

    The number of back-up copies specifies the number of members of the distributed cache service that hold the back-up data for each unit of storage in the cache.

    There is no guarantee of no data loss if the Number of Backup Copies is set to 0.

    Default value: 1

    For a back-up count of 1 to be effective, you need at least two cache agents (or storage enable nodes).

  4. (Optional) To enable encryption for the stored data when you use Shared Nothing persistence, select the Enable Storage encryption checkbox.
  5. (Optional) If you select the Enable Storage encryption checkbox, provide the value to the Identity File Path field. The identity file provides a keystore with type and password.

  6. (Optional) To replicate cache data between clusters by using the Ignite CDC extension, select Ignite CDC from the Choose cache replication method dropdown menu.

  7. Configure the following Ignite CDC properties.
    Ignite CDC Configuration Properties
    FieldDescription
    Data Center IDSets a numeric value as an identifier for this data center node.
    Receiver Address for Ignite CDC

    Sets the IP address and port number of a receiver node.

    For example: 127.0.0.1:50001

    To add multiple receiver nodes, provide the IP addresses of each receiver node with their port numbers in a comma-separated list format.

    Note: To set active-active replication mode, set the Receiver's Address at the Sender node and the Sender's address at the Receiver node.
  8. Configure the following CDD properties in the Properties tab of the .cdd file.
    CDD Properties Configuration
    FieldDescription
    ignite.cdc.failed.connect.attempt.retry.interval

    The time interval after which the Ignite CDC will retry to connect to the destination cluster for a failed attempt.

    Default value: 5000 milliseconds
    ignite.cdc.connect.retry.attempt.threshold

    The number of times that the Ignite CDC will retry to reconnect to the destination cluster.

    Default value: 5
  9. Save the project CDD file.
What to do next