Model Processing

This topic explains model processing details such as model storage, data extraction, server cache population, cache validity, repository cache rebuild, and parallel processing of models.

Model Storage and Publishing

  1. The product, offer ID mapping, action, discount, and price models are stored in the DATA_MODEL table in the database so that they can be loaded from the database on the next engine restart.

    The plan fragment model is stored in the PLAN_FRAGMENT table so that a particular plan fragment model can be accessed by Orchestrator anytime during the plan execution.

  2. The plan fragment model payload is converted into the process component model and published on a designated topic mentioned in the table below. The JeOMS component subscribes to this topic. On receiving the message, JeOMS creates in-memory objects for using the models readily.

    The DATA_MODEL table is the primary storage location for all loaded models.

Model Published on JMS Topic
Product tibco.aff.ocv.events.products.publish
Action tibco.aff.ocv.events.actions.publish
Offer ID Mapping tibco.aff.ocv.events.products.publish
Discount tibco.aff.ope.events.discount.publish
Price tibco.aff.ope.events.price.publish
Plan Fragment (Process Component) tibco.aff.jm.model.processComponent.pub

Data Extraction

The process of extracting model information is the same regardless of the online or offline model processing mode. The XML data models information is extracted from the files and the in-memory cache objects are populated with that information.

Server Cache Population

As part of performance improvements in the model loading process, the member instance designated to process models writes in-memory cache repository information generated by reading the models to the database SERVER_CACHE table.
  1. Model cache is broken down into multiple model types. These types are PRODUCT, PRICE, ACTION, DISCOUNT, and OFFERID. Based on the published models, cache information for each category is populated in the SERVER_CACHE table.
  2. Based on the maximum size of the in memory model information for each model cache, this information is broken down into multiple rows.
  3. When AOPD, OPE, or both (deployed either as a standalone or colocated instance) starts up, embedded model load engine components first checks to see if valid cache information exists in the SERVER_CACHE table. If cache information does not exist in the table or is not valid, the XML models are read from the DATA_MODEL table and the in-memory server cache information is regenerated.
  4. If valid cache information exists in the SERVER_CACHE table, this information is read instead of processing the XML models. This step saves considerable amount of processing time as SERVER_CACHE information stores the pre-processed XML information.
  5. When a designated member instance processes all the models and writes information to the SERVER_CACHE table, this instance sends out a notification on the TIBCO EMS topic (tibco.aff.oms.events.refresh.cache) for all AOPD and OPE instances to read the updated cache information from the SERVER_CACHE table. Until this notification is processed, AOPD and OPE continues to hold the existing in-memory information.
  6. While processing models for creating in-memory repository information, all the models are processed. The existing configuration parameter "Max No of Model cached" is still honored by AOPD and OPE.

Cache Validity

  • The SERVER_CACHE table maintains cache validity that signifies if the cache information stored in the table is valid or not. One (1) in the value signifies a VALID cache and zero (0) in the value signifies an INVALID cache.
  • The cache information for a particular model type is switched between valid (1) and invalid (0) before any full or partial publish is processed. When a new partial or full publish is received from TIBCO Fulfillment Catalog, either in the offline folders or on TIBCO EMS, cache validity for that model category is first switched to an invalid state (value 0 in the table).
  • Now all the models for that category are processed. After processing all the models, existing invalid entries from the table are deleted, and the updated cache information is stored in the SERVER_CACHE table. After all the rows for a particular model are stored in the table, the validity for the cache is switched to valid (1).
  • Whenever cache information is requested by a model load engine, it is first checked to see if the cache information in the table is valid. Cache information is read from the table from one valid state to another.

Repository Cache Rebuild

When the SERVER_CACHE table is empty or has an invalid cache validity, the restarting application server processes the corresponding XML models from the DATA_MODEL table and repopulates the cache information in the SERVER_CACHE table. It also sets the validity of all the rows for the model type to a VALID (value of 1) status.

Parallel Processing of Models

As part of speeding up model processing time, designated member instance processes all models (PRODUCT, PRICE, ACTION, DISCOUNT, and OFFERID Mapping) in parallel.