Data Models

LogLogic LMI parses log data into structured formats to enhance search and analysis. Based on the log source type, you can define how to parse your data and which columns to extract.

From the Management > Advanced Features > Data Models page, you can view all data models available on your appliance.

Using data models, LogLogic LMI parses log data in a structured format to enhance search and analysis. Based on the log source type, you can define parsing rules within the data models to decide how to parse your data and which columns to extract.

The data models in LogLogic LMI can be broadly classified into the following categories:

• Advanced Data Models, which use parser types such as syslog, Regex, JSON, and so on.
• GP Parser-Based Data Models, which are data models that use a grouped-pattern parser (GP parser). GP parsers are especially efficient in handling complex Regex parsing rules that work on heterogeneous free-text logs.

Functions of Data Models

Using data models you can:

• Define parsing rules that extract columns from your data.
• Define a schema for an event.
• Name and specify the data types for extracted columns.

Parsing Rules

A data model can be associated with multiple parsing rules. Sometimes within the same source, some logs are completely different to others, and it is not practical, or even possible, to match them all with a single rule. You need a different way of parsing for each kind of log, and you can do that by defining several rules, each targeting one type of log.

If a data model has more than one parsing rule defined, then the extracted column set is the union of the column sets of all parsing rules and the additional system-defined columns. For example, create a data model and define a parsing rule, Rule1, to extract four defined columns and Rule2, to extract eight different defined columns. Now, when you run a search query on this data model, the 12 columns are displayed.

Parsing rules are applied top to bottom in the order they are defined in a data model. For example, if Rule1 matches some of your data then it is used to extract column values. If Rule1 fails to match with your data, then only Rule2 is applied, and so on. You can change the order of parsing rules.

For an overview of the parsers in advanced data models, see Types of Parsers in Advanced Data Models. In GP parser-based data models, there is only one type of parser. See GP Parser-Based Data Models