Creating an Aggregation Rule

Procedure
1. Go to Management > Advanced Features > Rules and click the Aggregation tab.
2. Click Create New Rule.
3. In the Add Aggregation Rule dialog box, provide the following information:
   1. Parent Group: Select a parent group where you want to save the rule. 

      You can create a new group or select the User group, or select any user-created group

      Default parent group: When creating a nested group within any 'All' group (for example, All Rules, All Bloks, and so on), the User group is the default group. Otherwise, the current parent group is selected as the default group.

   2. Name: The name can include letters, numbers, hyphen, or underscore (_). The rule name cannot start with a number.
   3. (Optional) Description of the rule.
   4. Query: Ensure that you enter a valid syntax of a search query. An EQL or SQL query that contains GROUP BY statement and aggregated projections are supported. For details, see GROUP BY Statement. Note the following exceptions:
      • Time functions in the GROUP BY statement can only have sys_eventTime as an argument. Any other timestamp column is not supported.
      • Restrictions on GROUP BY statement:
        • GROUP BY statement can include both column names or expressions. However, only time functions are supported, for example, days(sys_eventTime), weeks(sys_eventTime). Generic expressions such as length(Process)/10 are not supported.
        • Special characters in column names should be escaped, for example, [Process Name] or [% Memory]
      • Restrictions on the projection clause (COLUMNS):
        • The projection items must contain aggregate functions such as sum(), avg().
        • Projection items can be a complex expression, for example, concatenate(max(cpu), '_vs_', min(cpu)), but it should contain aggregation functions. An aggregation function can contain only simple functions as arguments. For example, max(Memory) is supported but max(sqrt(Memory)) is not.

      Click Validate to verify the query statement.

   5. Select or clear the appropriate Aggregation time check boxes to add or remove time functions in GROUP BY clause of the query.
      When you select this option, the time aggregation you selected is inserted in the GROUP BY clause of the rule query. The default time aggregation functions are:

      • weeks(sys_eventTime)
      • days(sys_eventTime)
      • hours(sys_eventTime)
      • minutes(sys_eventTime)
   6. In the Retention period field, enter the retention time for which the computed aggregation values remain stored in the aggregated data.
      You can search based on the same aggregation functions until the specified time has passed. By default, it is set to -1w (1 week).
      For example, if the retention period is -2w (2 weeks), then pre-computed results remain in the system for 2 weeks since the time of rule creation.
   7. In the Compute aggregation for field, specify the time when the aggregation computing must start. A one-time calculation of aggregation happens on the data that was already collected prior to the creation of the aggregation rule. To start computing as soon as the rule is created, leave the field empty. To compute aggregation for the data that exists prior to rule creation, specify the time relative to rule creation time. For example, -1d, -1w, and so on.
      Note:

      • The Compute aggregation for period cannot be more than the retention period.
      • To modify the Compute aggregation for period, first modify the retention period and then the Compute aggregation for period.

      The data model Loglogic_Aggregation_Rules_Metrics provides a disk usage statistics of all aggregation rules in the system. You can run the infrastructure query Aggregation Rule Metrics and view the results.

   8. In the Maximum Aggregated Data Storage Size in MB field, enter the maximum storage size of aggregated data. The minimum value is 1 MB, the maximum value is 2147483647, and the default is 1024 MB if the field is empty or 0. You can specify the limit based on the storage availability in your setup.
      Data is purged daily, based on the retention period and the maximum storage values. Purging is triggered 24 hours after rule creation time. When one of these values is exceeded, the oldest 24 hours data is purged to match the values that you specified in the aggregation rule. For example, with the retention period as -1w and maximum storage size as 1024 MB, suppose that the aggregation data on the first day is 1000 MB and on the second day, is 2024 MB. On the second day, the oldest 24 hours data (in this example, 1000 MB data of the first day) is purged to match the storage size limit of 1024 MB.
   9. To activate the aggregation rule, set Enabled to ON.
      The optimization starts only after the rule is enabled. The disabled rule does not compute real-time events.
4. Click Save.

Saving an Advanced Search Query as an Aggregation Rule

Procedure
1. On the Advanced Search page, click the optimize icon to open the Add Aggregation Rule dialog box.
   The query that you typed on the Advanced Search page is automatically filled in the Query field.
2. Enter the other fields in the Add Aggregation Rule dialog box and then click Save.
   The aggregation rule is saved in the system.