FAQs

For k8s cluster:

1. Run:

   kubectl get svc

   This lists all the services and external IPs of the Load Balancer. Select the Load Balancer IP of the reporting app named "reporting-app-0".
2. Access the Grafana dashboard: http://<External_IP_Of_Loadbalancer_Of_ReportingApp>:3000

For OpenShift cluster:

1. Run:

   oc get svc

   This lists all the services and external IPs of the Load Balancer. Select the Load Balancer IP of the reporting app named "reporting-app-0".
2. Access the Grafana dashboard: http://<External_IP_Of_Loadbalancer_Of_ReportingApp>:3000

For Swarm cluster:

1. Access the Grafana dashboard using the Swarm master's public IP: http://<SWARM_MASTER_IP>:3000

For k8s cluster:

kubectl label nodes <nodename>
node-name=reporting

For OpenShift cluster:

oc label nodes <nodename>
node-name=reporting

To see the labels on the nodes for k8s cluster:

kubectl get nodes --show-labels

To see labels on the nodes for OpenShift cluster:

oc get nodes --show-labels

Verify that one of the nodes is labelled as "node-name=reporting".

To see on which node the reporting pod in running on a k8s cluster:

kubectl get pods -o wide

To see on which node the reporting pod in running on an OpenShift k8s cluster:

oc get pods -o wide

To see on which node the reporting pod in running on a Swarm cluster, run the following command on the node which has a placement constraint for the reporting container:

docker ps

There four different services that are running on reporting pod/container:

• Grafana's logs can be checked at /var/log/grafana/
• Prometheus's logs can be checked at /var/log/prometheus/
• Loki's logs can be checked at /var/log/loki/
• Fluentd logs can be checked at /var/log/fluentd/

• Retention for metrics related to traffic and access logs is 1 year.
• Retention for all other metrics is 4 days.
• Retention for verbose metadata is 1 day.
• Retention for Container's log data is 4 days.

Create a file cleanup_prometheus_data.ini and add the following details:

[DELETION_PERIOD]
# Deletion period of prometheus data, its in days(number of days for which data has to be kept)
VERBOSE_METRICS_DATA_DELETION={number_of_days}
PROCESS_METRICS_DATA_DELETION={number_of_days}

Save this file and follow instructions in the TML-Reporting Configuration topic to apply the changes during deployment.

Create a file loki-docker-config.yaml and paste the following content. Change the value of variable "{hours e.g. 96h}" to keep the application logs:

auth_enabled: false
 
server:
  http_listen_port: 3100
 
ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0
  chunk_target_size: 1536000
 
schema_config:
  configs:
    - from: 2020-07-15
      store: boltdb
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: {hours e.g. 96h}
 
storage_config:
  boltdb:
    directory: /mnt/data/loki/index
  filesystem:
    directory: /mnt/data/loki/chunks
 
limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: {hours e.g. 96h}
  ingestion_rate_mb: 16
  ingestion_burst_size_mb: 16
 
chunk_store_config:
  max_look_back_period: 0s
 
table_manager:
  retention_deletes_enabled: true
  retention_period: {hours e.g. 96h}

Save this file and follow instructions in the Loki Configuration topic to apply the changes during deployment.

For K8s cluster:

kubectl get pods | grep -i reporting-set-0-0

For OpenShift cluster:

oc get pods | grep -i reporting-set-0-0

For Swarm cluster: Navigate to the node which hosts the reporting pod, then run the command:

docker ps -a | grep -i reporting