Service resource management scenarios

You can use a combination of the Spotfire Service for Python custom properties, including the pruning properties engine.prune and dynamic.prune.threshold, to ensure the best usage of the Spotfire Service for Python engines that the Spotfire Service for Python allocates.

The custom properties engine.session.max and engine.queue.size determine the number of engines that are available, and the number of engines allowed in the queue, respectively. These values are determined by the number of logical processors available on the node where the Spotfire Service for Python is running. Additionally, you might want to set properties that control how long a Spotfire Service for Python engine in a session can remain idle, how long to run an execution before timing out, or the percentage of engines that can run in a session before pruning is triggered.

The following two configuration examples, and their associated scenarios, demonstrate the resource management for different combinations of custom properties. These non-exhaustive usage scenarios are provided only to give two of many configurations for engine pruning and engine idle timeout. Your needs can vary, depending on your job sizes, the number of users, and the number of available logical processors.

Configuration A

Assume the following configuration values, where three engines are created and waiting in the queue for jobs. 

#Configuration A
engine.execution.timeout: 60
engine.session.maxtime: 120
# by default, these are set to number of logical processors on the system
engine.session.max: 3
engine.queue.size: 3
# the idle timeout
engine.prune: 10
# The service capacity at which idle pruning is engaged, as a percentage value.
#   0 = always idle prune.
# 100 = never idle prune.
dynamic.prune.threshold: 100

The following three scenarios show how this configuration affects the jobs that users submit.

Scenario Result
1A: A single user submits a job that runs for more than 60 seconds Because this job runs for longer than the value set for engine.execution.timeout, the execution is halted and the engine is destroyed. Results for this long job are not returned. When the user submits another job, a new engine is provided from the queue.
2A: A single user submits a job that runs for 5 seconds

The job completes and returns results, and the engine persists.

The capacity of the service is equal to the number of engines in use divided by the maximum sessions. In this case, the capacity is 1/3, or 33%, which is below the dynamic.prune.threshold value of 100. The user can access this same engine for up to engine.session.maxtime (in this case 120 seconds).

3A: Four users submit jobs that execute for 5 seconds each Because dynamic.prune.threshold is set to 100 (specifying never idle prune), the first three user jobs claim the engines (engine.session.max) for the entire duration of engine.session.maxtime, or 120 seconds. The fourth user must wait until that time expires to claim a new engine, which are then made available on a first-come-first-served basis.

Configuration B

Change the configuration as follows, where only the dynamic.prune.threshold has been changed from 100 to 0. 

#Configuration B
engine.execution.timeout: 60
engine.session.maxtime: 120
# by default, these are set to number of logical processors on the system
engine.session.max: 3
engine.queue.size: 3
# the idle timeout
engine.prune: 10
# The service capacity at which idle pruning is engaged, as a percentage value.
#   0 = always idle prune.
# 100 = never idle prune.
dynamic.prune.threshold: 0

The same user scenarios show how this changed configuration affects the jobs that users submit.

Scenario Result
1B: A single user submits a job that runs for more than 60 seconds No change in behavior from Configuration A.
2B: A single user submits a job that runs for 5 seconds

The job completes and return results.

The capacity is at 33, which is now higher than the dynamic.prune.threshold of 0.
  • If the user does not submit a new job within 10 seconds defined by the engine.prune idle timeout, then the used engine is destroyed.
  • If the user submits a new job before the timeout occurs, then the second job uses the same engine.
  • If the user submits a new job after the timeout occurs, then another engine is pulled from the queue.
3B: Four users submit jobs that execute for 5 seconds each

The first three user jobs get engines, and the fourth user job must wait for an engine to become available, because the engine.session.max has been reached.

However, dynamic.prune.threshold is set to always idle prune (0), so after submitting a job and getting results, if a user sits idle for longer than engine.prune of 10 seconds, the engine is destroyed. More engines are created and made available to the fourth user.

Conclusion

In scenario 3A (where four users submit jobs, and the dynamic.prune.threshold is set to 100), the fourth user might have to wait for up to 2 minutes for an available engine (the engine.session.maxtime), whereas in scenario 3B (where four users submit jobs, and the dynamic.prune.threshold is set to 0), the fourth user could wait for just 15 seconds (job run of 5 seconds and engine.prune idle timeout of 10 seconds).

By default, the dynamic.prune.threshold is set to 60, because this setting balances both access for a high volume of users and faster response times for a lower volume of users. The default values for engine.execution.timeout and engine.session.maxtime are set to balance security and availability. For your on-premises usage, you might find it useful to increase execution timeouts or disable idle timeouts altogether.

See Engine pruning for more information about these custom properties. See Configuring Spotfire Service for Python for information about setting all custom properties.