Persistence Cluster Configuration

Creating a Persistence Cluster

To create a cluster, enter Edit Mode, navigate to the Persistence Cluster Table, and click the New Cluster button. Inside this dialog, all you need to do to create a new persistence cluster is provide a name. You can edit the disk persistence mode, add and remove persistence servers, and set up Disaster Recovery from this dialog as well:

Property Description
Name

Name of the persistence cluster.

Cluster names must be unique.

All names are limited to 256 characters.

Disk Persistence Mode

Only persistent (replicated) messages are persisted on disk.

None: Disable disk persistence.

Sync: The send-message call returns once the message has been written persistently to a majority of disks.

Async: The send-message call returns once the OS has buffered the write of the message. A tail end of data can be lost if a majority of server hosts crash simultaneously.

Warning: Changing this setting requires a restart of all persistence servers in the cluster
DR Replication Enables replication to the Disaster Recovery Set.

To edit advanced cluster and persistence server properties, toggle the advanced view. See Advanced Persistence Cluster Configuration for more information.

For background information, see Persistence Services and Clusters in TIBCO FTL® - Enterprise Edition Administration.

Editing a Persistence Cluster

To edit a configured persistence cluster, click the three vertical dots in the cluster row of the Persistence Cluster Table and select Edit. To modify a Persistence Cluster properties other than those outlined in Creating a Persistence Cluster, toggle the Advanced edit view. For more information, see Advanced Persistence Cluster Configuration.

For background information, see Persistence Services and Clusters in TIBCO FTL® - Enterprise Edition Administration.

For consequences of modifying this configuration, see Persistence Modifications: Store and Durable and Quorum Behaviors in TIBCO FTL® - Enterprise Edition Administration.

Deleting a Persistence Cluster

To delete a configured Persistence Cluster, open the actions menu for that cluster row, and select Delete.

For background information, see Persistence Services and Clusters in TIBCO FTL® - Enterprise Edition Administration.

For consequences of modifying this configuration, see Persistence Modifications: Store and Durable and Quorum Behaviors in TIBCO FTL® - Enterprise Edition Administration.

Advanced Persistence Cluster Configuration

Enable Advanced View in the create or edit Cluster Dialog to modify the following Advanced Persistence Cluster properties.

For background information, see Persistence Services and Clusters in TIBCO FTL® - Enterprise Edition Administration.

For consequences of modifying this configuration, see Persistence Modifications: Store and Durable and Quorum Behaviors in TIBCO FTL® - Enterprise Edition Administration.

Property Description
Indexes

When enabled, message data, indexes, and metadata for persistent messages are stored on disk. No memory is consumed to store persistent messages. Also, when the persistence service restarts, it can recover from disk immediately (without having to read message metadata from disk).

Warning: Changing this setting requires a restart of all persistence services in the cluster.
DR Protocol

If you enable disaster recovery replication, the services in the cluster use this transport protocol to establish a communication bus for that purpose.

You can set other parameters of the transport for each service in the cluster.

Force Quorum Delay

Automatically force a quorum after this delay, in seconds. Zero is a special value indicating that automatic force formation is disabled.

This setting can be used to ensure that the quorum always forms without administrator intervention. A majority of non-empty servers is usually enough to form a quorum, however in some situations (such as DR activation or recovery from a backup file), the quorum cannot form unless all members are present. If some servers are still missing, this option forces the quorum to form automatically after the specified delay.

Administrators can still manually force a quorum if this setting is configured. See Quorum Behaviors and Before Forcing a Quorum in TIBCO FTL® - Enterprise Edition Administration for more information.

Byte Limit

This parameter limits the volume of message data in the cluster. If publishing another message would exceed this limit, the cluster rejects that message.

The cluster can grow until it exhausts available memory or disk of any one of its persistence services.

Zero is a special value, indicating no limit. The cluster can grow until it exhausts available memory of any one of its services.

When messages are stored on disk, message metadata is still stored in memory. Use this setting to constrain memory for message metadata. See Size Units Reference in TIBCO FTL® - Enterprise Edition Administration. for more information.

Message Limit

Limits the number of messages that can be held by the cluster. When this limit is reached, the client receives an exception on tibPublisher_Send or tibPublisher_SendMessages (when cluster_confirm_send mode is set on the cluster). However, the client library retries the sends. To have the exception presented to the user, the client application must specify a finite persistence retry duration.

Note: Even when messages are stored on disk, message metadata is still stored in memory. Hence, use this setting to constrain memory use.

Zero is the default value indicating no limit. The cluster can grow until it exhausts available memory or disk of any one of its persistence services.

Message Swapping

When enabled, message data is swapped to disk. This allows the persistence service to store more message data than available memory. This setting affects both persistent (replicated) and nonpersistent (non-replicated) messages.

Note: If indexes on disk feature is enabled, persistent messages are always swapped to disk, regardless of this setting.
Warning: Changing this setting requires a restart of all persistence services in the cluster.
Auto Compaction

Set to enable or disable automatic disk compaction. This setting applies to both non-persistent messages (if swapping is enabled) and persistent messages.

Note: If the indexes on disk is enabled, automatic compaction is always enabled for persistent messages, regardless of this setting.
Note: Changing the setting does not require a restart.
Compact when in-use ratio below

Set a value between 0 and 1, inclusive. The persistence service attempts to keep the ratio of disk inuse size to disk allocated size at or above the specified value. If the ratio of in-use to allocated size falls below this value, the persistence service can start a compaction. Automatic compaction does not occur if the disk inuse size is extremely small.

Note: Automatic compaction must be enabled for this value to take effect.

If the indexes on disk is enabled, this threshold is ignored for persistent messages. Automatic compaction is always enabled for persistent messages.

Changing the setting does not require a restart.

Server Client Timeout

When the heartbeat of the leader is silent for this interval, in seconds, its clients seek to connect to a new leader from among the other services in the cluster. The default is 5 seconds.

When a client's heartbeat is silent for this interval, the persistence service clears all state associated with the client.

Server Server Timeout When the heartbeat of any service in the cluster is silent for this interval, in seconds, the remaining services attempt to form a new quorum. The default is 3 seconds.
Cluster Cluster Timeout When the heartbeat from a leader persistence service in a second cluster is silent for this interval, in seconds, the leader of the first cluster attempts to reconnect. The default is 5 seconds.
Server Client Heartbeat The leader persistence service sends heartbeats to its clients at this interval, in seconds. The default is 2 seconds.
Server Server Heartbeat Persistence services in the cluster exchange heartbeats at this interval, in seconds. The default is 0.5 seconds.
Cluster Cluster Heartbeat The leader persistence services of different persistence clusters in a forwarding zone exchange heartbeats at this interval, in seconds. The default is 2 seconds.

Advanced Set Properties

Property Description
Cluster Protocol

Persistence services with a cluster’s service set communicate with one another using this transport.

As needed, set the host, subnet, and port parameters in the persistence server-level configuration, which can be accessed by clicking the pencil icon next to a persistence server’s name in the dialog.

Client Protocol

Client applications communicate with persistence services in the cluster using the client transport.

As needed, set the host, subnet, and port parameter in the persistence server-level configuration, which can be accessed by clicking the pencil icon next to a persistence server’s name in the dialog.

Notice that all the services in a cluster use the same client protocol, though individual services can supply different transport parameters.

For connection-oriented protocols, such as static TCP, the persistence service always is the listening end, and clients the connecting end.

Zone Protocol Optional. When two persistence services need to communicate across the WAN to forward wide-area stores, they establish a transport bus using this protocol.
Externally Reachable Addresses

When using persistence with zones of clusters that use an auto inter-cluster transport to address the cluster (or to address a load balancer's interface), use this field to identify one or more ports for the core servers of this cluster (or the port for the load balancer).

Host: Either an IP address or resolvable hostname.

Port: The port number to use to access this host

Advanced Persistence Server Properties

Name Description
Weight

Optional. This value must be an integer in the range [1,10]. Larger values indicate greater preference that the service becomes the leader.

When absent, the default weight is 10.

Cluster Transport

Persistence Servers inherit the cluster transport protocol from the service set and communicate with one another using this transport.

  • For connection-oriented transports, set the host name and port for establishing a connection (both are required).

  • For dynamic TCP transports, you can set a subnet mask and port (both are optional).

Client Transport

Persistence Servers inherit the client transport protocol from the service set. Client applications use this transport to communicate with persistence services in the cluster.

  • For connection-oriented transports, set the host name and port for establishing a connection (both are required).

  • For dynamic TCP transports, you may set a subnet mask and port (both are optional).

  • Backlog Buffer Size: limits the number of messages that can be delivered at once (Messages may also be limited by the prefetch count parameter. See [Durable Prefetch Count]).

See also Client Transport and Non-Blocking Send in TIBCO FTL® - Enterprise Edition Administration.

DR Transport

Optional. When two persistence services need to communicate across the WAN for disaster recovery replication, they establish a transport using this definition.

Persistence services inherit the disaster recovery transport protocol from the cluster. Set the host name and port for establishing a connection (both are required). For background information, see Disaster Recovery in TIBCO FTL® - Enterprise Edition Administration.

Alternate Client Transport

Optional. When defined, clients on the associated hosts communicate with this persistence service using the alternate transport. All other clients use the regular client transport defined for the cluster.

To define an alternate client transport for clients on specific host computers, supply a transport protocol and other transport parameters. Add hosts to the list of Associated Client Host Names.

For background information, see Alternate Client Transport. See also Client Transport and Non-Blocking Send in TIBCO FTL® - Enterprise Edition Administration.

Zone Transport Optional. When two persistence services need to communicate across the WAN to forward wide-area stores, they establish a transport bus using parameter values in this definition.