Direct Data Transfer

By default, GridServer uses Direct Data Transfer, or peer-to-peer communication, to optimize data throughput between Drivers and Engines. Without Direct Data Transfer, all task inputs and outputs must be sent through the Manager. Sending the inputs and outputs through the Manager results in higher memory and disk use on the Manager, and might lower throughput overall.

With Direct Data Transfer, only lightweight messages are sent through the Manager, and the “heavy lifting” is done by the Driver and Engine nodes themselves.