File Collection Parallelism - Overview
File Parallelism allows you to configure the degree of parallelism in the file collector.
Up to 100 concurrent file collections at the rule level can be configured.
By default, LogLogic LMI processes a single file pull rule at a time by delivering files associated with a single rule. Where the user feels appropriate and where the file transfer is relatively slow, LogLogic LMI allows for simultaneous file delivery for several rules simultaneously. This creates a higher throughput of file log processing because LogLogic LMI is capable of processing files at a relatively high speed compared to the time it takes to pull the files into LogLogic LMI. Users can experiment by increasing the parallelism until there is a backlog in the /loglogic/data/filecollector/archiver directory.
File Parallelism is configured through command-line options that pass to the File Collector when it is launched. These options are specified in /loglogic/conf/node_config.xml.
The degree of parallelism is specified by the command-line option -p x (where x is the degree of parallelism). The maximum degree of parallelism is 100, and the default is 1 (when the command-line option is absent).