Fetching the Ingested Data

Query

USE LogLogic_System_Ingest_And_Index_Stats

Description

The data ingest count is taken when the files are collected by the appliance, whereas the index count is taken when those files (or, in the case of large pulled files, parts of those files) are actually indexed.

If a large amount of data is ingested in 1 hour, most of it might be indexed in the subsequent hour, resulting in a higher index count than the ingest count for that hour.

In a relatively quiet system, the data ingest count is updated only periodically, whereas the index count is updated every time a file is indexed. Therefore, the index count might be updated before the data ingest count. As a result, the index count might be more than the data ingest count for the most recent hour.

Therefore, for up-to-date values, check the count shortly after the end of any particular hour.

Query result

The following fields are returned in the search results:

Field Description
lls_time Time period to be queried
lls_ingestBytes Number of data bytes ingested during the specified time period
lls_indexBytes Number of bytes indexed during the specified time period
lls_indexSizeDelta Change in the size of the physical disk space consumed by the index
lls_indexMsgCount Number of messages indexed
lls_ingestMsgCount Number of messages ingested
lls_unIndexBytes Number of bytes that are not indexed yet

Examples

  1. USE LogLogic_System_Ingest_And_Index_Stats

    Returns: the results in one-hour buckets

  2. USE LogLogic_System_Ingest_And_Index_Stats | COLUMNS lls_time, lls_ingestBytes / 1024.0 / 1024.0 / 1024.0 AS IngestGB, lls_indexBytes / 1024.0 / 1024.0 /1024.0 AS IndexGB

    Returns: the results in GB

  3. USE LogLogic_System_Ingest_And_Index_Stats | COLUMNS DAYS(lls_time) as myTime, SUM(lls_ingestBytes ) / 1024.0 / 1024.0 / 1024.0 AS IngestGBPerDay, SUM(lls_indexBytes ) / 1024.0 / 1024.0 / 1024.0 AS IndexGBPerDay| GROUP BY DAYS(lls_time)

    Returns: the result in one-day buckets in GB