Dataload Parallelization for Performance

Previously, the data loading mechanism of orders, logs, and alarms was too slow regarding KPSA performance. Also, querying the orders and logs was too slow.

The data loading mechanism has been improved with the following features:

1. Improve the way FP orders are dumped in database by FPOMS engine.
2. Add indexes to FPOMS tables for fast searches.
3. Provide statistics on database injection to measure real injection rates.

To improve the database injection mechanism, the FP orders, logs and alarms are not using yml format but use a CSV format. Generated files are lighter and can be directly loaded by database itself using a specific command shipped with database (PostgreSQL, MySQL, and ORACLE) for huge data injection.

All dates are translated from string to unsigned long. Also, indexes are added to tables for faster queries.

Use the following command to see the statistics:

 provadmin display fpoms stats=true

For example, the command may show the following result:

 FP OMS Web Admin URL: http://localhost:18081 
 
        Node:1 
         
        Order:5003 
        Order Line:5002 
        Technical Order:5003 
        Work Order:5003 
        Parameter:5003 
        Log:446624 
        Action:22962 
        Alarm:0 
 
    Statistics (DB injection rates): 
        17.90 Order/s 
        1406.73 rows/s 
        1 Order corresponds to 78.60 rows (avg) 
        1.47 Logs/s