Platform log files

DCImanager 6 saves information about the operation of services in log files. The data from log files can be used to diagnose the platform operation. Log files are stored in the /var/log directories of docker containers.

Log files list

Main log files on platform server

Container	File path	Contents
dci_auth_back_1	/var/log/licupdate.log	logs of the license validation
dci_back	/var/log/dci_1_writer.log /var/log/dci_1_reader.log	logs of the main DCImanager 6 service
	/var/log/dci_location_check_update.log	logs of the update service for containers at the location. The service starts when the container is restarted and updates the containers at the location, if necessary
	/var/log/dci_locationctl.log /var/log/dci_ansiblectl_location_<location id>.log /var/log/ansible.log	logs of creating and configuring a location
	/var/log/dci_operationctl.log	log of operations performed at the location
dci_consumer_1	/var/log/worker_daemon.log	logs of periodic equipment polls and obtaining statistical data
dci_dns_proxy_1	/var/log/dns_proxy_integration.log /var/log/dns_proxy_service_1_reader.log /var/log/dns_proxy_service_1_writer.log	logs of the modules: Integration with PowerDNS Integration with DNSmanager 6
dci_ipmgr_1	/var/log/ip_1_reader.log /var/log/ip_1_writer.log	logs of address space management service
dci_ipmi_proxy_v2_client_1	/var/log/ipmi_proxy_goproxy.log	goproxy service logs. The service transmits data to the VNC client and proxies Websocket requests to the VNC server
dci_ipmi_proxy_v2_client_1	/var/log/ipmi_proxy_update_checker.log	logs of the update service for BMC proxy module at the location. The service starts each time the container is restarted and updates the module if necessary
vm_ldap_1	/var/log/isp_ldap.log	logs of synchronization with the LDAP directory
migrator	/var/log/migrate.log /var/log/bill_migrate.log	migration of objects from DCImanager 5
dci_input_1	/var/log/nginxctl.log	SSL certificate issue log

The platform installation log is saved in the file /opt/ispsystem/dci/install.log.

The log files of the diagnostics operation are listed in the Diagnostics check article.

The authorization service logs are written to the standard output of the dci_auth_back4_1 container.

Main log files on location server

Container	File path	Contents
eservice_handler	/var/log/dci_switch_<id>_handler.log	logs of switch handlers
	/var/log/dci_pdu_<id>_handler.log	logs of PDU handlers
	/var/log/dci_ipmi_<id>_handler.log	logs of BMC handlers
	/var/log/dci_intel_amt_handler.log	logs of Intel AMT handlers

Comments

Working with logs

Copy log file from a container

To copy a log file from the container to the current directory, run the command:

docker cp <container_name>:<path_to_log> ./

Comments to the command

Example command

docker cp dci_auth_back_1:/var/log/licupdate.log ./

View log files in a container

Enter the required container:
```
docker exec -it <container_name> sh
```
Comments to the command
<container_name> — container name
Example command
```
docker exec -it dci_back sh
```
View the log file using standard Linux utilities. For example, cat, tail, less, grep, etc.
Example command: Find out when the switch was polled
```
grep 'Switch status start' /var/log/dci_switch_1_handler.log
```
Exit the container:
```
exit
```

Viewing log files via stdout

Separate log files are not created for some services. Logs of these services are written to stdout. To view such logs, run the command:

docker logs <container_name> --tail <lines> --since <period>

Comments to the command

Example command: Display the last 100 log lines of the authorization service created in the last 60 minutes

docker logs dci_consul_1 --tail 100 --since 60m

Collecting platform logs

To collect all the log files of the platform in one directory:

Connect to the server with the platform via SSH.

Create the bash script getlog.sh with the following content:

#!/bin/bash

BEGIN_DATE=$(date +'%Y-%m-%d %H:%M:%S' --date '-1 day')
SINCE=${1:-${BEGIN_DATE}}

/bin/echo "Remove old files"
/bin/rm -rf /home/logs && /bin/mkdir -p /home/logs/container && /bin/mkdir -p /home/logs/service && /bin/cd /home/logs

/bin/echo "---===Step 1/3===---"
DOCKER_CONTAINER_NAMES=`journalctl --field CONTAINER_NAME`
CONTAINERS=($DOCKER_CONTAINER_NAMES)

/bin/echo "Get logs of the containers"
for container in ${CONTAINERS[@]}
do
    /bin/echo "Get log of the container: ${container}"
    /bin/journalctl --no-pager CONTAINER_NAME=${container} --since "${SINCE}" > /home/logs/container/${container}.log
done

/bin/echo "---===Step 2/3===---"
DOCKER_SERVICES_NAMES=`journalctl --field SYSLOG_IDENTIFIER`
SERVICES=($DOCKER_SERVICES_NAMES)

/bin/echo "Get logs of the services"
for service in ${SERVICES[@]}
do
    /bin/echo "Get log of the service: ${service}"
    /bin/journalctl --no-pager SYSLOG_IDENTIFIER=${service} --since "${SINCE}" > /home/logs/service/${service}.log
done

/bin/echo "---===Step 3/3===---"
ARCHIVE_NAME="logs_$(date +'%Y_%m_%d_%H_%M_%S').tar.gz"
tar -cvzf /home/$ARCHIVE_NAME -C /home/logs .

/bin/echo "Done"

Run the script using one of the commands:
Before collecting logs, the script deletes all files from the /home/logs/ directory.
Collect logs for the last day
```
bash getlog.sh
```
Collect logs since a certain date
```
bash getlog.sh "YYYY-MM-DD"
```
Collect logs since a certain date and time
```
bash getlog.sh "YYYY-MM-DD HH:MM:SS"
```
Comments to the commands

YYYYY-MM-DD — date from which logs should be collected
HH:MM:SS — time from which logs should be collected

The script:
1. Saves container logs in the /home/logs/container/ directory.
2. Saves service logs in the /home/logs/service/ directory.
3. Creates an archive logs_<current_date>_.tar.gz in the /home/ directory.

Logging in journald

Logging settings

The platform uses the journald system service to collect logs. journald collects logs for all containers except dci_consul_1. You can change the journald logging level, read more in the Logging level management.

The journald service on the platform server and the location server is configured automatically. The service configuration is saved in the /usr/lib/systemd/journald.conf.d/dci.conf file. If the service has already been configured on the server, the configuration set by the platform will have a higher priority.

Configuring the journald service may fail if the syslog and rsyslog service settings on the server have been changed.

Configuration parameters set by the platform:

Storage=persistent — store files on disk;
Compress=yes — compress data in logs;
SystemMaxFileSize=500M — the maximum size of the log file is 500 Mb. When this value is reached, logs are rotated;
SystemMaxUse=20G — the maximum disk size for storing logs is 20 GB. When this value is reached, older files will be deleted.

Operations with logs

journald saves logs in the /var/log/journal/ directory. You can view the logs using the journalctl utility. Examples of how to use the utility:

Get id of containers for which logging is performed

journalctl --field CONTAINER_NAME

View container logs

journalctl -f CONTAINER_NAME=<container_id>

Get id of services for which logging is performed

journalctl --field SYSLOG_IDENTIFIER

View service logs

journalctl -f SYSLOG_IDENTIFIER=<service_id>

Get logs of the authorization service from the certain date

journalctl --no-pager CONTAINER_NAME=dci_auth_back4_1 --since "YYYY-MM-DD"

Comments to the command

Useful tips