Collect Docker container logs

Overview

Docker’s default json-file logging driver writes each container’s stdout and stderr to /var/lib/docker/containers/<id>/<id>-json.log. The OpenTelemetry Collector’s filelog receiver tails those files, parses each JSON line, and ships the logs to Osuite over OTLP.

Caution

docker_stats is a metrics receiver — it collects container CPU, memory, and network metrics, not logs. Do not use it for log collection. For container metrics, see Docker infrastructure monitoring.

Prerequisites

• Containers using the json-file logging driver (Docker’s default). Other drivers do not write to /var/lib/docker/containers.
• The Collector must have read access to the host’s container log directory. Run the Collector with the directory mounted read-only:

-v /var/lib/docker/containers:/var/lib/docker/containers:ro

The json-file path does not need the Docker socket — that is only required by the docker_stats metrics receiver.

Collector config

This otel-collector.yaml tails every container’s log file, promotes the message to the record body, keeps the stream name, and tags each record with its container ID from the file path.

receivers:
  filelog:
    include:
      - /var/lib/docker/containers/*/*-json.log
    start_at: end
    include_file_path: true
    operators:
      - type: json_parser
        timestamp:
          parse_from: attributes.time
          layout_type: gotime
          layout: "2006-01-02T15:04:05.999999999Z07:00"
      - type: move
        from: attributes.log
        to: body
      - type: move
        from: attributes.stream
        to: attributes["log.iostream"]
      - type: regex_parser
        parse_from: attributes["log.file.path"]
        regex: '^/var/lib/docker/containers/(?P<container_id>[^/]+)/'

processors:
  batch: {}

exporters:
  otlp/osuite:
    endpoint: ingest.<region>.osuite.io:443
    tls:
      insecure: false
    headers:
      x-osuite-ingest-token: <your-ingest-token>

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch]
      exporters: [otlp/osuite]

Parsing and enrichment

• Each json-file line has the shape {"log":"...","stream":"stdout","time":"..."}. The json_parser reads it, the time field becomes the record timestamp, and the two move operators put the message in the body and the stream in log.iostream.
• The regex_parser extracts the 64-character container ID from the file path into a container_id attribute, so you can filter by container in the explorer.
• This config collects logs from every container on the host, including the Collector’s own. Add an exclude list to the receiver to drop containers you do not want.

Verify in Osuite

What you should see

Within a minute, container stdout and stderr appear in the Logs Explorer, each record tagged with container_id and log.iostream. Filtering by a container’s ID isolates its output.

Troubleshooting

If no files are found, confirm the containers use the json-file driver (docker inspect <container> --format '{{.HostConfig.LogConfig.Type}}') and that /var/lib/docker/containers is mounted into the Collector. A permission-denied error means the Collector user cannot read the directory — mount it read-only and run the Collector with sufficient privileges.

No data in Osuite after a few minutes? Work through these checks.

• Endpoint — confirm the exporter targets exactly ingest.<region>.osuite.io:443 for your region, over TLS.
• Token — confirm the ingest header carries a valid <your-ingest-token> and has not been rotated.
• Pipeline — confirm the signal you expect (traces, logs, or metrics) is wired into an active pipeline with the otlp/osuite exporter attached.
• Export errors — check the application or Collector logs for OTLP export failures (auth, DNS, TLS, connection refused).
• Network — confirm the host has outbound access to the ingest endpoint on port 443.
• Timing — allow up to a minute for the first data to appear before assuming a failure.

Still stuck? Ask the Investigation Agent or contact support.