Apache Ignite troubleshooting and tuning

Spotfire Server uses Apache Ignite for server cluster management. Property settings can help you manage such concerns such as cluster robustness and how quickly problems are reported.

Use the following Spotfire Server configuration options to help you manage the cluster with Apache Ignite.

Table 1. Apache Ignite configuration properties in Spotfire Server
Property name	Description
`clustering.apacheignite.caches.defaults.backups`	Sets a specific number of Apache Ignite configuration replication or backups. The default is `1`.
`clustering.apacheignite.caches.defaults.cache-mode`	Replicates the Apache Ignite configuration across the cluster. Options are `REPLICATED` or `PARTITIONED`. The default is `PARTITIONED`.
`clustering.apacheignite.timeouts.cache-operations-milliseconds`	Determines the timeout for retrieving or modifying cached values. The default is `-1` (Disabled).
`clustering.apacheignite.timeouts.client-failure-detection-timeout`	Sets the failure detection timeout for the client connection. The default is `30000` milliseconds.

For information about configuring the Apache Ignite process, see the command-line command reference config-external-ignite-process.

Increasing robustness for Spotfire Server across the cluster

When Spotfire Server instances keep failing, or when they fail to communicate with other instances in the cluster, you can make the system more robust by replicating Ignite data to all instances. To do this, override the defaults for the following key properties for all Ignite caches.

Note: Making this change can lead to lower write performance and increase the use of memory resources.

For replication across the entire cluster, use the following command:

config.sh set-config-prop --name=clustering.apacheignite.caches.defaults.cache-mode --value=REPLICATED

For replication or backing up to more instances if you do not need or want a full replication, use the following command:

config.sh set-config-prop --name=clustering.apacheignite.caches.defaults.backups --value=3

Configuring a timeout for Ignite cache operations

In the situation where Ignite networking is unstable or stalls, you might gain some benefit from configuring the timeout for retrieving or modifying cached values. For example, it improves error detection from a client perspective because the error rate increases. Also, when this type of timeout occurs, the load balancer health URL indicates that the Spotfire Server is UNHEALTHY. Knowing these conditions as early as possible can help you fix problems earlier.

To configure this timeout, set the property clustering.apacheignite.timeouts.cache-operations-milliseconds to the wanted timeout value (in milliseconds). The default for this property is -1 (disabled). For example, to set the value to 20 seconds, use the following command:

config.sh set-config-prop --name=clustering.apacheignite.timeouts.cache-operations-milliseconds --value=20000

For more information about this setting, see the Apache Ignite Cache Configuration documentation.

Detect client connection failures earlier

When you need to detect client connection failures earlier, you can lower the time it takes for the client to reach timeout. For your server cluster, you can change this value in the setting clustering.apacheignite.timeouts.client-failure-detection-timeout.

The default for this setting is 30000 milliseconds (30 seconds). Change the value to the number of milliseconds before timeout. For example, to set to 10 seconds, change the value using the following command.

config.sh set-config-prop --name=clustering.apacheignite.timeouts.client-failure-detection-timeout --value=10000

For more information about this setting, see the Apache Ignite Network Configuration documentation for the Ignite properties IgniteConfiguration.clientFailureDetectionTimeout and IgniteConfiguration.failureDetectionTimeout (the property that is configured with the Spotfire Server configuration command clustering.apacheignite.timeouts.failure-detection-timeout).