Docker, a popular OS-level virtualization platform that allows us to deliver applications in packages known as containers which in simple terms are isolated environments with their own software, libraries, and configuration files.
Like any other modern software, logging events and messages like warnings and errors is an inherent part of the Docker platform, which allows you to debug your applications and production issues.
We’ll be covering some simple ways in which you can manage and monitor logs for your containers. So let’s get started.
Docker Logs Command
The basic syntax to fetch logs of a container is:
$ docker logs [OPTIONS] <CONTAINER-NAME OR ID>OR
$ docker container logs [OPTIONS] <CONTAINER-NAME OR ID>Both of the syntaxes are essentially the same, so we’ll focus on the rest of the commands in this article as docker logs.
Though do note here that the above command is only functional for containers that are started with the json-file or journald logging driver.
Here OPTIONS refers to the available supported flags with docker logs command, which are listed below:
| Name, Shorthand | Default | Description |
|---|---|---|
--details |
Show extra details provided to logs. | |
--follow, -f |
Follow log output | |
--since |
Show logs since timestamp (e.g. 2021-08-28T15:23:37Z) or relative (e.g. 56m for 56 minutes) | |
--tail , -n |
all |
Number of lines to show from the end of the logs |
--timestamps, -t |
Show timestamps | |
--until |
API 1.35+ Show logs before a timestamp (e.g. 2021-08-28T15:23:37Z) or relative (e.g. 56m for 56 minutes) |
Example:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
28913415ed22 nginx "/docker-entrypoint.…" 2 seconds ago Up 1 second 80/tcp gifted_edison
$ docker logs 28913415ed22
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/08/28 09:02:59 [notice] 1#1: using the "epoll" event method
2021/08/28 09:02:59 [notice] 1#1: nginx/1.21.1
2021/08/28 09:02:59 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/08/28 09:02:59 [notice] 1#1: OS: Linux 5.8.0-1039-azure
2021/08/28 09:02:59 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/08/28 09:02:59 [notice] 1#1: start worker processes
2021/08/28 09:02:59 [notice] 1#1: start worker process 31
2021/08/28 09:02:59 [notice] 1#1: start worker process 32
$Docker Logs Location
Docker, by default, captures the standard output (and standard error) of all your containers and writes them in files using the JSON format. This is achieved using JSON File logging driver or json-file. These logs are by default stored at container-specific locations under /var/lib/docker filesystem.
/var/lib/docker/containers/<container_id>/<container_id>-json.logAs an example, for my redis container listed below, I can check its json logfile as shown in the snippet below:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
551c9273bbea redis "docker-entrypoint.s…" 19 minutes ago Up 19 minutes 6379/tcp redis
6cc871763df1 nginx "/docker-entrypoint.…" 7 hours ago Up 7 hours 0.0.0.0:8080->80/tcp, :::8080->80/tcp nostalgic_wescoff
$ sudo ls -l /var/lib/docker/containers/551c9273bbea6eaf66523ed735866b9ebe6924c3b504dfeb44bef90e69d59c73/551c9273bbea6eaf66523ed735866b9ebe6924c3b504dfeb44bef90e69d59c73-json.log
-rw-r----- 1 root root 1437 Aug 28 16:53 /var/lib/docker/containers/551c9273bbea6eaf66523ed735866b9ebe6924c3b504dfeb44bef90e69d59c73/551c9273bbea6eaf66523ed735866b9ebe6924c3b504dfeb44bef90e69d59c73-json.log
$ sudo tail -10 /var/lib/docker/containers/551c9273bbea6eaf66523ed735866b9ebe6924c3b504dfeb44bef90e69d59c73/551c9273bbea6eaf66523ed735866b9ebe6924c3b504dfeb44bef90e69d59c73-json.log
{"log":"1:C 28 Aug 2021 16:53:42.160 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo\n","stream":"stdout","time":"2021-08-28T16:53:42.16031257Z"}
{"log":"1:C 28 Aug 2021 16:53:42.160 # Redis version=6.2.5, bits=64, commit=00000000, modified=0, pid=1, just started\n","stream":"stdout","time":"2021-08-28T16:53:42.160337871Z"}
{"log":"1:C 28 Aug 2021 16:53:42.160 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf\n","stream":"stdout","time":"2021-08-28T16:53:42.160342171Z"}
{"log":"1:M 28 Aug 2021 16:53:42.160 * monotonic clock: POSIX clock_gettime\n","stream":"stdout","time":"2021-08-28T16:53:42.160792578Z"}
{"log":"1:M 28 Aug 2021 16:53:42.161 * Running mode=standalone, port=6379.\n","stream":"stdout","time":"2021-08-28T16:53:42.161148683Z"}
{"log":"1:M 28 Aug 2021 16:53:42.161 # Server initialized\n","stream":"stdout","time":"2021-08-28T16:53:42.161170984Z"}
{"log":"1:M 28 Aug 2021 16:53:42.161 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.\n","stream":"stdout","time":"2021-08-28T16:53:42.161186984Z"}
{"log":"1:M 28 Aug 2021 16:53:42.161 * Ready to accept connections\n","stream":"stdout","time":"2021-08-28T16:53:42.161484389Z"}
$Show Extra Details
To show extra details provided to logs, use --details flag.
Example:
$ docker logs 6cc871763df1 --details
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/08/28 10:29:05 [notice] 1#1: using the "epoll" event method
2021/08/28 10:29:05 [notice] 1#1: nginx/1.21.1
2021/08/28 10:29:05 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/08/28 10:29:05 [notice] 1#1: OS: Linux 5.8.0-1039-azure
2021/08/28 10:29:05 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/08/28 10:29:05 [notice] 1#1: start worker processes
2021/08/28 10:29:05 [notice] 1#1: start worker process 33
2021/08/28 10:29:05 [notice] 1#1: start worker process 34
172.17.0.1 - - [28/Aug/2021:10:29:26 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
$Follow Log Output
You can use --follow or -f flag to follow the log output. This allows you to monitor new updates in the log stream from continuously STDOUT and STDERR.
Example:
$ docker logs 6cc871763df1 -f
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/08/28 10:29:05 [notice] 1#1: using the "epoll" event method
2021/08/28 10:29:05 [notice] 1#1: nginx/1.21.1
2021/08/28 10:29:05 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/08/28 10:29:05 [notice] 1#1: OS: Linux 5.8.0-1039-azure
2021/08/28 10:29:05 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/08/28 10:29:05 [notice] 1#1: start worker processes
2021/08/28 10:29:05 [notice] 1#1: start worker process 33
2021/08/28 10:29:05 [notice] 1#1: start worker process 34
172.17.0.1 - - [28/Aug/2021:10:29:26 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"Tail Logs
Container logs can be tailed to limit the number of output shown on the screen with --tail or -n flag. By default, this flag assumes all as an argument that shows the complete log stream. To show a fixed number of lines from the end of the logs, specify a positive integer number following --tail or -n flag.
Example:
$ docker logs 6cc871763df1 -n 10
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/08/28 10:29:05 [notice] 1#1: using the "epoll" event method
2021/08/28 10:29:05 [notice] 1#1: nginx/1.21.1
2021/08/28 10:29:05 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/08/28 10:29:05 [notice] 1#1: OS: Linux 5.8.0-1039-azure
2021/08/28 10:29:05 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/08/28 10:29:05 [notice] 1#1: start worker processes
2021/08/28 10:29:05 [notice] 1#1: start worker process 33
2021/08/28 10:29:05 [notice] 1#1: start worker process 34
172.17.0.1 - - [28/Aug/2021:10:29:26 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
$Show Logs Since
We can limit log output by using --since flag and giving a timestamp like an absolute value with syntax 2021-08-28T15:23:37Z or a relative one like 56m for 56 minutes.
The --since option shows only the container logs generated after a given date. You can specify the date as an RFC 3339 date, a UNIX timestamp, or a Go duration string (e.g. 1m30s, 3h). The local time zone on the client will be used if you do not provide either a Z or a +-00:00 time zone offset at the end of the timestamp. You can combine the --since option with either or both of the --follow or --tail options.
Example:
$ docker logs --since=1m nostalgic_wescoff
172.17.0.1 - - [28/Aug/2021:15:19:24 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
172.17.0.1 - - [28/Aug/2021:15:19:25 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
$In the above example, logs since only 2 minutes are shown where nostalgic_wescoff is the auto-generated name assigned for the nginx container.
Show Logs Until
Like --since flag, docker logs also support --until flag, which shows logs before the given timestamp. Similarly, the timestamp follows a similar convention as earlier and can be specified as an absolute value with syntax 2021-08-28T15:23:37Z or a relative one like 56m for 56 minutes.
Example:
$ docker logs --until=1h30m nostalgic_wescoff
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/08/28 10:29:05 [notice] 1#1: using the "epoll" event method
2021/08/28 10:29:05 [notice] 1#1: nginx/1.21.1
2021/08/28 10:29:05 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/08/28 10:29:05 [notice] 1#1: OS: Linux 5.8.0-1039-azure
2021/08/28 10:29:05 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/08/28 10:29:05 [notice] 1#1: start worker processes
2021/08/28 10:29:05 [notice] 1#1: start worker process 33
2021/08/28 10:29:05 [notice] 1#1: start worker process 34
172.17.0.1 - - [28/Aug/2021:10:29:26 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
$In the above example, all logs before 1 hour 30 minutes are shown.
Show Timestamps
Many container applications offer timestamps built in their log output, so Docker also shows them with docker logs command. If you need Docker to explicitly prefix its timestamps in the output, use --timestamps or -t flag.
Example:
$ docker logs -t redis
2021-08-28T16:53:42.160312570Z 1:C 28 Aug 2021 16:53:42.160 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2021-08-28T16:53:42.160337871Z 1:C 28 Aug 2021 16:53:42.160 # Redis version=6.2.5, bits=64, commit=00000000, modified=0, pid=1, just started
2021-08-28T16:53:42.160342171Z 1:C 28 Aug 2021 16:53:42.160 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
2021-08-28T16:53:42.160792578Z 1:M 28 Aug 2021 16:53:42.160 * monotonic clock: POSIX clock_gettime
2021-08-28T16:53:42.161148683Z 1:M 28 Aug 2021 16:53:42.161 * Running mode=standalone, port=6379.
2021-08-28T16:53:42.161170984Z 1:M 28 Aug 2021 16:53:42.161 # Server initialized
2021-08-28T16:53:42.161186984Z 1:M 28 Aug 2021 16:53:42.161 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2021-08-28T16:53:42.161484389Z 1:M 28 Aug 2021 16:53:42.161 * Ready to accept connections
$Merge Flags
Docker offers to combine certain flags to get more filtered output rather than print all of the log contents on the screen. As a simple example, we can combine --tail flag with --since to get more restricted output.
Example:
$ docker logs --since=2h -f nostalgic_wescoff
172.17.0.1 - - [28/Aug/2021:15:19:24 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
172.17.0.1 - - [28/Aug/2021:15:19:25 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"This can work with other flags as well.
Filter With Shell Utilities
Linux shell utilities can also be used for more dexterity in the log output. Utilities like grep, head, tail etc. can be piped to docker logs output for more advanced operations.
Example:
$ docker logs --since=7h nostalgic_wescoff 2>&1 | grep GET
172.17.0.1 - - [28/Aug/2021:10:29:26 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
172.17.0.1 - - [28/Aug/2021:15:19:24 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
172.17.0.1 - - [28/Aug/2021:15:19:25 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.68.0" "-"
$Do note here that we need to redirect log streams to provide single piped input for grep using 2>&1.
Summary 👩💻
Docker is a versatile platform that offers numerous features to administer its environment. Managing logs for a system is one of the essential skills which every system administrator should know. Managing logs in Docker is easy once you know the available command and possible flags as per your requirements.
For further read on Docker and its functionalities, refer to Docker’s documentation.