Destroying LogLogic LMI Event Data

To comply with the GDPR regulations, LogLogic LMI provides a CLI utility to destroy selected data.

You can use the llshred utility on the result of an Advanced Search query to destroy a list of events from the result. After running the utility, the log data is not deleted from the appliance. Instead, the utility replaces every byte in the message with an X. The number of Xs is equal to the number of bytes in the original message.

For example, the following log entry includes 55 characters: 

llfeed,03/23/2018 15:28:06.584,tcp,10252,19208,attitude

The utility replaces the log entry with the string LLSHREDDER followed by 45 Xs: 

LLSHREDDERXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

If an event is removed or its length changed, the references to the subsequent events become invalid. Therefore, the utility shreds the log events rather than deleting them. To shred events ingested into the appliance after running the utility, you must run the utility again.

Procedure

  1. Back up the data on your appliance.
    Caution: The only way to recover shredded events is by restoring a backup. Therefore, you must take a backup before running the utility.
  2. Run an Advanced Search query for the events you want to shred. The query must include at least the sys_eventKey column in the projection. The sys_eventKey column can be in any position in the projection. For example: 
    SQL Query: 
select sys_eventKey, sys_body from system
where sys_collectIP='192.168.56.1' and sys_body contains 'attitude'

EQL query:
use system |'attitude'| sys_collectIP='192.168.56.1'| columns sys_eventKey, sys_body
  3. Save the search result as a .csv file.
  4. Copy the .csv file to the appliance using a file transfer tool such as SCP.
  5. Run the CLI command llshred on the data in the .csv file to destroy the data.
    For more information about the CLI command, see llshred Command.

Impact of llshred Utility on Search and Reports

Advanced Search
If you specify a search term using Regex or contains, no result is returned. When you use any of the following fields (exclusively) in a where clause or in the source filter of a data model, the search result returns the shredded log events:
  • sys_eventTime
  • sys_collectIP
  • sys_deviceType
  • sys_device
  • sys_collectorDomain
  • sys_eventKey
  • sys_filename
Index Search, Real-time Reports
Index search and real-time reports return the shredded logs (containing the Xs) in the results even after running the llshred utility.
Regex Search
Regex search does not return the shredded logs (containing the Xs) in the results after running the llshred utility.