Exporting Access Logs

The Log Service writes logs to a mounted persistent volume and are available at /mnt/data.

Other than the default file destination, the Log Service can forward logs to Kafka, Elasticsearch, HTTP endpoint, or a TCP socket. To forward to these destinations, use one of the recipes below and follow these steps:
  1. Edit <path-to-installer-directory>/tmgc-deploy/aws/k8s/properties/tml_log_properties.json
  2. Ensure that the hosts for any of the destinations are reachable via the network and that the required ports are not blocked for access.
  3. Follow installation steps.

Properties for access log sync to MOM (tethered mode)

Mashery Local can forward access logs to MOM. The following properties must be set. (See properties table for valid values.) 
{
    "apiKey": "<MOM API Key>",
    "apiSecret": "<MOM API Secret>",
    "auto-binding": "ON",
    "td_agent_output_channelType": "DEFAULT",
    "mom-host": "<MOM HOST in https endpoint format eg: https://api-mom.mashery.com>"
}

Properties for different output channels

The main property that needs to be set is the output channel marked by tml_log_service_output_channelType. The accepted values are DEFAULT, ELASTICSEARCH, HTTP, KAFKA and TCP. The values are case-sensitive.
Note:
  • These properties are defined in a JSON format and all JSON validation rules apply.
  • All required values have to be provided, For example, even if destinations are using default ports, those have to be provided per recipe.
  • All values must be set before starting the deployment.

Recipe for File Destination

This destination does not have any additional properties.

tml_log_properties.json: 
{
  "tmg-internal-statusCheck":"OFF",
  "tml_log_service_logLevel" : "info",
  "logserviceURL" : "localhost",
  "td_agent_output_channelType" : "DEFAULT"
}

Recipe for Elasticsearch

The Logging service requires certain properties to write to Elasticsearch destinations. See the Log Service Configuration Properties table for valid values.

tml_log_properties.json 
{
  "tmg-internal-statusCheck":"OFF",
  "tml_log_service_logLevel" : "info",
  "logserviceURL" : "localhost",
  "td_agent_output_channelType" : "ELASTICSEARCH",
  "td_agent_out_elasticsearch_host" : "<network reachable elastic search host>",
  "td_agent_out_elasticsearch_port" : "<valid ES port>",
  "td_agent_out_elasticsearch_index" : "<elastic search index to update for new entries>"
 }

Recipe for HTTP Endpoints

See the Log Service Configuration Properties table for valid values.

tml_log_properties.json 
{
  "tmg-internal-statusCheck":"OFF",
  "tml_log_service_logLevel" : "info",
  "logserviceURL" : "localhost",
  "tml_log_service_output_channelType" : "HTTP",
  "tml_log_service_out_channel_httpURI" : "<network reachable HTTP endpoint>"
 }

Recipe for Kafka

See the Log Service Configuration Properties table for valid values.

tml_log_properties.json 
{
  "tmg-internal-statusCheck":"OFF",
  "tml_log_service_logLevel" : "info",
  "logserviceURL" : "localhost",
  "td_agent_output_channelType" : "KAFKA",
  "td_agent_out_kafka_brokers" : "<network reachable comma separated list of kafka broker:port>",
  "td_agent_out_kafka_topic" : "<Kafka topic to publish to>"
 }

Recipe for TCP Socket

See the Log Service Configuration Properties table for valid values.

tml_log_properties.json 
{
  "tmg-internal-statusCheck":"OFF",
  "tml_log_service_logLevel" : "info",
  "logserviceURL" : "localhost",
  "td_agent_output_channelType" : "TCP",
  "td_agent_out_tcp_host" : "<network reachable hostname or IP address>",
  "td_agent_out_tcp_port" : "<valid TCP port number>"
}