Distributed Advanced Search

Running a query on one appliance returns results only from that appliance. Distributed Advanced Search queries can be run on a Management Station and distributed to multiple Remote Appliances in the setup. The results from all appliances are consolidated by the Management Station.

To run a distributed Advanced Search query on multiple appliances, specify a list of appliances by using the sys_concentratorId column or the DeviceInGroup query function within the query that you run on the Management Station.
Important: The query is sent to only those appliances to which you have access permissions. Furthermore, on the Remote Appliances, the query is run only on those log sources to which you have access permissions. If you run a query on the Management Station and omit the appliance list, the query is run only on the Management Station and not on the Remote Appliances.

Requirements to run distributed Advanced Search queries

You can use distributed queries in Advanced Search if all of the following requirements are met:

  • The Management Station setup includes the required Remote Appliances. For information about how to set up a Management Station, see Manage Appliances with Management Station in the TIBCO LogLogic® Log Management Intelligence Administration.
  • You have permission to access the Remote Appliances and devices on which the search query is to be run.
  • Advanced Features are enabled on the Management Station and Remote Appliances.
  • The data node port (9621) on each Remote Appliance is accessible by the Management Station.

Using the sys_concentratorId column

You can specify the value of sys_concentratorId using =, ==, or the IN construct. Valid values for sys_concentratorId can be any of the following:

  • Explicit appliance IP addresses, where the IP addresses must exist in the Management Station list on the Management > Management Station page.
  • Appliance grouping shortcuts, similar to those used in the Log Source Picker in Advanced Search.
  • Enrichment lists
Examples with IP addresses and shortcuts used in the sys_concentratorId column
Type of value Value of sys_concentratorId The query is sent to...
Appliance IP address sys_concentratorId = "127.0.0.1" Only to the Management Station, which is always specified as 127.0.0.1. This is the default value if sys_concentratorId is not specified.
sys_concentratorId IN ("127.0.0.1, <IP1>") The Management Station itself and the specified Remote Appliance having IP address IP1.
sys_concentratorId="<IP2>" Only to the specified Remote Appliance having IP address IP2.
Appliance grouping shortcuts sys_concentratorId IN "ALL" All members of the Management Station setup.
sys_concentratorId IN "ALL_LX"

sys_concentratorId IN "ALL_ST"

sys_concentratorId IN "ALL_MX"

sys_concentratorId IN "ALL_LX_MX"

sys_concentratorId IN "ALL LX MX"

sys_concentratorId IN "ALL LX/MX"

The types of appliances specified in the shortcut.

Using Enrichment Lists in the sys_concentratorId column

Enrichment lists can also be used in distributed Advanced Search queries to specify sets of appliances to which the query must be sent.

For example, you can create an enrichment list called MyApplianceSets with mappings such as:

{ "set1" : "127.0.0.1, <IP1>, <IP2>", 
"set2" : "<IP3>" }
If you run a query like this:
USE system | sys_concentratorId = $MyApplianceSets('set1') | sys_body CONTAINS 'joe'
the query is sent to the IP addresses in set1.
The following query is an example of using an enrichment list when using a device group in a distributed Advanced Search:
USE system | DeviceInGroup('MyLogLogicAppliances', $MyApplianceSets('set1') ) | sys_body CONTAINS 'joe'
Note: The IP addresses in the enrichment list must match those in the Management Station setup.
Examples with enrichment lists used in the sys_concentratorId column
Type of value Value of sys_concentratorId The query is sent to...
Enrichment list sys_concentratorId=$MyLogLogicAppliances("set1") All appliances in set1 from the Enrichment List MyLogLogicAppliances
Enrichment list and IN sys_concentratorId IN ($MyLogLogicAppliances("set2") ) All appliances in set2 from the Enrichment List MyLogLogicAppliances
Combination sys_concentratorId IN ( "<IP_1>", "ALL_LX", $MyLogLogicAppliances("set1") ) The following appliances:
  • The appliance with IP address IP_1
  • All LogLogic LX Appliances
  • All appliances in set1 from the Enrichment List MyLogLogicAppliances.

Using the DeviceInGroup function

You can specify a list of local or global device groups by using the DeviceInGroup function:
DeviceInGroup("<Group Name>", "<IP list or appliance grouping shortcuts>")
where:
  • <Group Name> is the name of the device group
  • <IP list or appliance grouping shortcuts> specifies the list of appliance concentrator IP addresses on which the device group is to be evaluated
Examples of the DeviceInGroup function
Value of DeviceInGroup The query is sent to...
DeviceInGroup("MyLogLogicAppliances", "ALL") All log sources in the device group MyLogLogicAppliances on all appliances on which you have access permission.
DeviceInGroup("GlobalGroup", "ALL_LX") All log sources in the device group GlobalGroup on all LogLogic LX Appliances which have group members and on which you have access permission.
DeviceInGroup("LocalGroup", "<IP1>") All log sources in the device group LocalGroup on the appliance at IP address IP1.
DeviceInGroup("LocalGroup", "<IP2>, <IP3>") All log sources in the device group LocalGroup on the appliances at IP address IP2 and IP3.
DeviceInGroup("GlobalGroup", $MyLogLogicAppliances("set1") ) All log sources in the global group GlobalGroup on the appliances specified in the set1 entry in the Enrichment list MyLogLogicAppliances. The query is sent to only those appliances with members in GlobalGroup and on which you have permission.
Local device groups
You can also run the query on local device groups that use the same name on multiple appliances. For example, if the local device group MyLocalGroup exists on two appliances, Appliance1 (having IP address IP1) and Appliance2 (having IP address IP2), and you have permissions on MyLocalGroup on both appliances, then using the following function in a distributed Advanced Search query on the Management Station:
DeviceInGroup("MyLocalGroup", "<IP1>, <IP2>")
causes the query to be sent to both appliances Appliance1 and Appliance2, for their corresponding local group MyLocalGroup. Search results from all devices belonging to the MyLocalGroup device group on both appliances are sent back to the Management Station.
Global device groups
If you create a global device group GlobalGroup on the Management Station that includes log sources from Appliance1 and Appliance2, you can use the global group name in the DeviceInGroup function:
DeviceInGroup("GlobalGroup", "ALL")
This query is sent to Appliance1 and Appliance2.
Global Dynamic Device Groups
In regular expression search, it is not possible to run distributed queries on dynamic device groups because dynamic device groups cannot have a global scope.

However, for distributed Advanced Search queries, if you create local dynamic device groups that have the same group name on the appliances involved, and run a distributed Advanced Search query for those local dynamic device groups, you can achieve the effect of a global search result on dynamic device groups.

For more information about device groups, seeDevice Group Management in TIBCO LogLogic® Log Management Intelligence Administration.

Limitations and errors

Limitations

You cannot use the Tail functionality in distributed Advanced Search.

Errors
An error is displayed if:
  • The specified appliance shortcut is invalid.
  • An appliance specified in the distributed query is not a member of the Management Station setup.

Examples of distributed Advanced Search queries

These are a few examples of complete queries that you can use in distributed Advanced Search:
Using sys_concentratorId:
  • USE system | sys_concentratorId="ALL" | sys_eventTime in -10m
  • USE system | sys_concentratorId IN ("127.0.0.1") | sys_eventTime in -10m
  • USE system | sys_concentratorId IN ($MyApplianceGroups("remotes") ) | sys_eventTime in -10m
Using DeviceInGroup function:
  • USE system | DeviceInGroup("<group name>", "ALL") | sys_eventTime in -1y
  • USE system | DeviceInGroup("<group name>", "RemoteAppliance1") | sys_eventTime in -1y
Using DeviceInGroup in the projection:
  • USE system | sys_deviceType = "General Syslog" | COLUMNS sys_collectIP, DeviceInGroup("LocalGroup1") as dig | sys_concentratorId = "ALL" | sys_eventTime in -1y
  • USE system | COLUMNS sys_deviceType, sys_device, sys_eventTime, DeviceInGroup("All Other UNIX") as isOtherUnix | sys_eventTime in -1h