Adding an Aggregation Rule

When a new aggregation rule is created, it starts computing the metrics for the given GROUP BY expressions only for new events arriving into the system. It does not aggregate data for the events generated before the rule creation time. When an aggregation query matches an aggregation rule, the query results are fetched from the collected aggregation data. When the aggregation rule is created, a filter Blok is created automatically for that rule and is used in future searches.
Attention:
  • When creating aggregations for small periods, for example using GROUP BY minutes or seconds, use the data type as long for the columns in your custom data model.
  • When using your own data model to create aggregation rules, if you edit any column types of the data model after creating an aggregation rule, then for the changes to take effect you must either create a new aggregation rule again, or edit and save the existing aggregation rule.

Procedure

  1. Navigate to Management > Advanced Features > Rule Management and click the Aggregation tab.
  2. From the Aggregation page, click to add a new aggregation rule.
    The Add Aggregation Rule dialog box is displayed.
  3. In the Created by field, the user name who created the aggregation rule is displayed automatically.
  4. Enter the rule name in the Name field.
    This field name cannot be blank, cannot contain special characters, and does not allow duplicate names.
  5. (Optional) Enter the description of the rule in the Description field.
  6. Enter the search query in the Query field.
    Make sure you enter a valid syntax of a search query. An EQL or SQL query that contains GROUP BY statement and aggregated projections are supported. For details, see GROUP BY Statement. Note the following exceptions:
    • Time functions in the GROUP BY statement can only have sys_eventTime as an argument. Any other timestamp column is not supported.
    • Restrictions on GROUP BY statement:
      • GROUP BY statement can include both column names or expressions. However, only time functions are supported, for example, days(sys_eventTime), days(sys_eventTime). Generic expressions such as length(Process)/10 are not supported.
      • Special characters in column names should be escaped, for example, [Process Name] or [% Memory]
    • Restrictions on the projection clause (COLUMNS):
      • The projection items must contain aggregate functions such as sum(), avg().
      • Projection items can be a complex expression, for example, concatenate(max(cpu), '_vs_', min(cpu)), but it should contain aggregation functions. An aggregation function can contain only simple functions as arguments. For example, max(Memory) is supported but max(sqrt(Memory)) is not.

    Click Validate to verify the search statement.

  7. Select or clear the appropriate Aggregation time check boxes to add or remove time functions in GROUP BY clause of the query.
    When you select this option, the time aggregation you selected is inserted in the GROUP BY clause of the rule query. The default time aggregation functions are:
    • weeks(sys_eventTime)
    • days(sys_eventTime)
    • hours(sys_eventTime)
    • minutes(sys_eventTime)
  8. To activate the aggregation rule, click the Enabled slider to ON.
    The optimization starts after the rule is enabled. The disabled rule stops computing real-time events.
  9. Enter the retention time in the Retention period field for which the computed aggregation values remain stored in the aggregated data.
    You can search based on the same aggregation functions until the specified time has passed. By default, it is set to -1w (1 week).
    For example, if the retention period is -2w (2 weeks), then pre-computed results remain in the system for 2 weeks since the time of rule creation.
  10. Click Save to add the new aggregation rule.
    The newly added rule is displayed on the Aggregation page.

Saving an Advanced Search Query as an Aggregation Rule

If the query you type on the Advanced Search contains aggregation fields such as GROUP BY and COLUMNS state, you can save the query as an Aggregation Rule.

If the query is not already included in any saved Aggregation Rule, then the Optimize icon is displayed.

Procedure

  1. Click the optimize icon to open the Add Aggregation Rule dialog box.
    The query that you typed on the Advanced Search page is automatically filled in the Query field.
  2. Enter the other fields in the Add Aggregation Rule dialog box as per Step 4 onwards and then click Save.
    The Aggregation Rule is saved in the system.