Service resource management scenarios

You can use a combination of the TERR service custom properties, including the pruning properties engine.prune and dynamic.prune.threshold, to ensure the best usage of the TERR engines that the TERR service allocates.

The custom properties engine.session.max and engine.queue.size determine the number of engines that are available, and the number of engines allowed in the queue, respectively. These values are determined by the number of logical processors available on the node where the TERR service is running. Additionally, you might want to set properties that control how long a TERR engine in a session can remain idle, how long to run an execution before timing out, or the percentage of engines that can run in a session before pruning is triggered.

The following two configuration examples, and their associated scenarios, demonstrate the resource management for different combinations of custom properties. These non-exhaustive usage scenarios are provided only to give two of many configurations for engine pruning and engine idle timeout. Your needs can vary, depending on your job sizes, the number of users, and the number of available logical processors.

Configuration A

Assume the following configuration values, where three engines are created and waiting in the queue for jobs.

#Configuration A
engine.execution.timeout: 60
engine.session.maxtime: 120
# by default, these are set to number of logical processors, minus 1, on the system
engine.session.max: 3
engine.queue.size: 3
# the idle timeout
engine.prune: 10
# The service capacity at which idle pruning is engaged, as a percentage value.
#   0 = always idle prune.
# 100 = never idle prune.
dynamic.prune.threshold: 100

The following three scenarios show how this configuration affects the jobs that users submit.


Scenario	Result
1A: A single user submits a job that runs for more than 60 seconds	Because this job runs for longer than the value set for `engine.execution.timeout`, the execution is halted and the engine is destroyed. Results for this long job are not returned. When the user submits another job, a new engine is provided from the queue.
2A: A single user submits a job that runs for 5 seconds	The job completes and returns results, and the engine persists. The capacity of the service is equal to the number of engines in use divided by the maximum sessions. In this case, the capacity is 1/3, or 33%, which is below the `dynamic.prune.threshold` value of `100`. The user can access this same engine for up to `engine.session.maxtime` (in this case `120` seconds).
3A: Four users submit jobs that execute for 5 seconds each	The first three user jobs get engines, and the fourth user job must wait for an engine to become available, because the `engine.session.max` has been reached. Because `dynamic.prune.threshold` is set to `100`, which specifies never idle prune, the first three users claim their engines for the duration of the value of the `engine.session.maxtime` of `120` seconds. At this point, the engines are destroyed and new engines are made available on a first-come-first-served basis.

Configuration B

Change the configuration as follows, where only the dynamic.prune.threshold has been changed from 100 to 0.

#Configuration B
engine.execution.timeout: 60
engine.session.maxtime: 120
# by default, these are set to number of logical processors on the system
engine.session.max: 3
engine.queue.size: 3
# the idle timeout
engine.prune: 10
# The service capacity at which idle pruning is engaged, as a percentage value.
#   0 = always idle prune.
# 100 = never idle prune.
dynamic.prune.threshold: 0

The same user scenarios show how this changed configuration affects the jobs that users submit.


Scenario	Result
1B: A single user submits a job that runs for more than 60 seconds	No change in behavior from Configuration A.
2B: A single user submits a job that runs for 5 seconds	The job completes and return results. The capacity is at 33 which is now higher than the `dynamic.prune.threshold` of `0`, so the engine is destroyed after the `engine.prune` (idle timeout) of `10` seconds. If the user submits a new job, a new engine is created from the queue.
3B: Four users submit jobs that execute for 5 seconds each	The first three user jobs get engines, and the fourth user job must wait for an engine to become available, because the `engine.session.max` has been reached. However, `dynamic.prune.threshold` is set to always idle prune (`0`), so after submitting a job and getting results, if a user sits idle for longer than `engine.prune` of `10` seconds, the engine is destroyed. More engines are created and made available to the fourth user.

Conclusion

In scenario 3A (where four users submit jobs, and the dynamic.prune.threshold is set to 100), the fourth user might have to wait for up to 2 minutes for an available engine (the engine.session.maxtime), whereas in scenario 3B (where four users submit jobs, and the dynamic.prune.threshold is set to 0), the fourth user could wait for just 15 seconds (job run of 5 seconds and engine.prune idle timeout of 10 seconds).

By default, the dynamic.prune.threshold is set to 60, because this setting balances both access for a high volume of users and faster response times for a lower volume of users. The default values for engine.execution.timeout and engine.session.maxtime are set to balance security and availability. For your on-premises usage, you might find it useful to increase execution timeouts or disable idle timeouts altogether.

See Engine pruning for more information about these custom properties. See Configuring the Service for information about setting all custom properties.