Event Group

An event group describes the criteria events should meet to be part of a rule.

Event groups can be of the following 3 types :

  • Required: the rule cannot be triggered if no event(s) matches this event group. This is the default type.
  • Excluded: the rule will NOT be triggered if event(s) matches this event group.
  • Optional: If events are matching this event group, they will be part of the triggering rule if other criteria are met.

An event group may have the following parts:

  • conditions on the number of events
  • a filtering clause
  • a grouping clause
  • a set of having clause
  • upper limits on the number of groups and events that may be created while this rule is run. This is a safeguard against a memory overflow.

The event group can be defined as follows: 

Event Group <identifier> [ Is ( Required | Optional | Excluded ) ] [ With Delayed Evaluation ]
[ At Least <integer> Events ]
[ At Most <integer> Events ]
[ <identifier environment> ]
[ Where <expression> ]
[ With The Same <expression> [ As <identifier> ]
( , <expression> [ As <identifier> ] )* ]
( Having <having clause> )*
[ Limits <integer> Groups And <integer> Events ]

When At Least parameter is defined, it requires at least an integer more than 0. If it is omitted, this implies at least 1.

If Where clause is defined, it should match the expressions. It is evaluated as a Boolean. For details, see Expressions.

Default Limits are 10000 groups and 100000 events.

Expressions

Expressions can be used to express how to compute a value in many situations.

The different situations can be:

  • in a condition
  • in a grouping definition
  • in field assignment 
    [ ( + | - ) ] <double>
[ ( + | - ) ] <long>
"<String>"
{ ( d | t | ts ) yyyy-MM-dd hh:mm:ss }
True
False
Null
<IPv4 address>
<IPv6 address>
<key identifier>
$<identifier>(<expression>)
( <expression> )
<expression> * <expression>
<expression> / <expression>
<expression> % <expression>
<expression> + <expression>
<expression> - <expression>
<expression> Is [ Not ] Null
Exists <expression>
<expression> [ Not ] Like <expression>
<expression> [ Not ] [ Any | All ] Contains <expression>
<expression> [ Not ] [ Any | All ] Regexp <expression>
<expression> [ Any | All ] = <expression>
<expression> [ Any | All ] != <expression>
<expression> [ Any | All ] > <expression>
<expression> [ Any | All ] >= <expression>
<expression> [ Any | All ] <= <expression>
<expression> [ Any | All ] < <expression>
<expression> [ Any | All ] <> <expression>
<expression> [ Any | All ] In ( <expression>, expression, … )
<expression> In <expression>/<expression>
<expression> [ Any | All ] Between <expression> And <expression>
Case <expression>
( When <expression> Then <expression ) +
[ Else <expression> ]
<function name> ( [ <expression> ] , [ <expression> ] , … )
<aggregation function>

The following operators are supported:

  • Equals (=)
  • Not equals (!=), (<>)
  • Lower than (<)
  • Lower or equal (<=)
  • Greater than (>)
  • Greater or equal (>=)
  • In:
    • <list of expressions>: Checks if value matches any one of the values in a set. Supports all data types.
    • <network>/<net length>: Checks whether an IP address matches a network, defined as a network IP address and a network bitmask length.
  • Between <expression> And <expression>: Supports Timestamps, Long, Integers and Float
  • AND, OR

    Examples: 

    ( sys_eventType = “1234”) and ( sys_body like “%login failed%”)
( sys_bodySize > 30) and (sys_bodySize < 20)
( ll_eventID is not null) and ( ll_eventID > -1 )
  • Functions
  • Aggregation Functions
  • Identifier Environment

Functions

Functions are used to compute a value as output from parameters as input.

Some functions are predefined in the language. It is also possible to call a static Java function provided by the user.

The Pre-defined Functions that are available in ECL are listed in the following table.

Function Name Arguments Returns
String functions
len

char_length

character_length

length

(String) Length of string 1.
lower (String) Lower case of string 1.
upper (String) Upper case of string 1.
trim (String) Trimmed string 1 (without leading and trailing spaces).
substitute (String 1, String 2, String 3) Substitute string 2 by string 3 in string 1.
left (String, Int) <int> left characters of string 1.
right (String, Int) <int> right characters of string 1 .
mid

substr

substring

(String, Int 1, Int 2) Characters from string1 starting at offset <int1> for a length of <int2>.
find

position

(String 1, String 2) Index of the first occurrence of string2 within string1, -1 if no occurrence is found.
concatenate (String 1, String 2, …) Concatenation of all strings passed as arguments.
TransformString (stringToTransform, regularExpression, template) or (stringToTransform, regularExpression, template, defaultValue) It tries to match the stringToTransform with the regular expression, and then returns the template with references to groups in the regular expression substituted with the actual values. To refer to groups, use $1, $2, etc to refer to numbered groups, and $<name> to refer to named groups. If the string doesn't match, or is there any other error, the default value will be returned (or NULL if not specified).
List functions
size List Size of the list.
Conditional functions
IIF Condition, then, else Returns the 'Then' value if condition is true, otherwise it should return the 'Else' value.

For example: IIF(true, “a”, “b”) returns “a” IIF(false, ”a”,”b”) returns “b”

Smart List functions
lookup (String 1, String 2) The value associated with String2 in the smart list named String1.
isInList (String 1, String 2) True if the value String2 is defined in smart list named String1.
Conversion functions
ToTimestamp (expression, formatString) or (expression, formatString, timezone) or (expression, formatString, timezone, defaultValue) The expression, which should evaluate to a string, is interpreted as a time according to the supplied formatString. If the conversion fails, null is returned, unless a default string is provided, which is interpreted as a time and returned.
Note: If timezone is omitted or is empty, the system default timezone is used.
ToIP (expression_ or (expression, defaultValue) Convert the expression to an IP address (Java InetAddress). If the conversion fails, null is returned, unless a default string is provided, which is interpreted as an IP address and returned.
ToTimestampString (expression, formatString) or (expression, formatString, timezone) or (expression, formatString, timezone, defaultValue) Same as ToTimestamp, except the conversion is in the opposite direction to get a printable timestamp.
Note: If timezone is omitted or is empty, the system default timezone is used.
ToInt (expression) or (expression, defaultValue) The obvious conversion to integer with default value taken if not convertible.
ToLong (expression) or (expression, defaultValue) The obvious conversion to Long with default value taken if not convertible.
ToString (expression) or (expression, defaultValue) The obvious conversion to String with default value taken if not convertible.
ToFloat (expression) or (expression, defaultValue) The obvious conversion to Float with default value taken if not convertible.
Note: LogLogic uses double precision (that is 64 bits) when storing floating point numbers.
ToBool (expression) or (expression, defaultValue) The obvious conversion to Boolean with default value taken if not convertible.
ToDouble (expression) or (expression, defaultValue) The obvious conversion to Double with default value taken if not convertible.
ExtractJson (expression, extraction path) or (expression, extraction path, default value) The expression, which is a JSON string is parsed. A field is extracted from the expression using the extraction path. If either the extpression or the path are invalid, an optional default value is returned.
ExtractKvp (expression, extraction path) or (expression, extraction path, nested KVP delimiters /default "{}"/) or (expression, extraction path, nested KVP, delimiter / default ","/) or (expression, extraction path, nested KVP, delimiter, separator /default "="/) or (expression, extraction path, nested KVP, delimiter, separator, escape character / default "\\"/) or (expression, extraction path, nested KVP, delimiter, separator, escape character, default value) The expression, which is a nested KVP string is parsed. A field is extracted from the expression using the extraction path. If either the expression or the path are invalid, an optional default value is returned.

Aggregation Functions

Expressions used in the Having clause must contain at least one aggregation function. 

Count ( * )
Count ( [ Distinct | All ] <expression> Limit <integer> )
Sum ( [ Distinct | All ] <expression> Limit <integer> )
Avg ( [ Distinct | All ] <expression> Limit <integer> )
Max ( [ Distinct | All ] <expression> Limit <integer> )
Min ( [ Distinct | All ] <expression> Limit <integer> )
Var ( [ Distinct | All ] <expression> Limit <integer> )
Stdev ( [ Distinct | All ] <expression> Limit <integer> )
Function Option Definition
Count() ( * ) This applies the function for any event with no additional constraints.
All This applies the function on all values that are not null.
Distinct This only applies the function once per distinct values.
Sum This is the total value.
Avg This is the average value.
Max This is the maximum value.
Min This is the minimum value.
Var This is the variance.
Stdev This is the standard deviation function.

Having Clause

The Having clause adds additional constraints on the events that have passed the filter and are grouped by the rule. 

At (Least | Most) <integer> 
Distinct <expression> 
As <identifier> 
Limit <integer> 
Count Of <expression> Being <expression> (Greater | Less) Than <integer> 
Percentage Of <expression> Being <expression> (Greater | Less) Than <integer> %<condition>
Note: The Having clause expression must contain at least 1 aggregation function.

The supported parameters are:

  • Count Of: count the number of time two expressions are equals and check that this value is greater or less than a boundary.
  • Percentage Of: count the number of time two expressions are equals and make a ratio of this count versus the number of events in the group, then check whether the value is less or more than a value expressed as percent.

The Having clause can also be an expression using aggregation functions and resolving to a boolean.