Collect nginx logs with OpenTelemetry

Overview

nginx writes an access log and an error log to disk. The OpenTelemetry Collector’s filelog receiver tails both files, parses each line into structured attributes, and ships the records to Osuite over OTLP. This page collects logs; for connection and request-rate metrics see Monitor NGINX with OpenTelemetry.

Prerequisites

• nginx writing logs to disk. On Debian and Ubuntu the defaults are /var/log/nginx/access.log and /var/log/nginx/error.log; RHEL, CentOS, and Fedora use the same paths.
• The OpenTelemetry Collector installed on the host, with read access to /var/log/nginx.
• An Osuite ingest token and endpoint — see Getting Started.

Note

The official nginx container image symlinks the access log to stdout and the error log to stderr. In containers, collect those streams with Docker container logs or Kubernetes pod logs instead of tailing files.

Collector config

This otel-collector.yaml uses two filelog receivers — one for the access log and one for the error log — because the two files have different formats. The regexes below assume the stock combined access format and the default nginx error format.

receivers:
  filelog/nginx_access:
    include:
      - /var/log/nginx/access.log
    start_at: end
    operators:
      - type: regex_parser
        regex: '^(?P<remote_addr>\S+) - (?P<remote_user>\S+) \[(?P<time_local>[^\]]+)\] "(?P<request>[^"]*)" (?P<status>\d{3}) (?P<body_bytes_sent>\d+) "(?P<http_referer>[^"]*)" "(?P<http_user_agent>[^"]*)"'
        timestamp:
          parse_from: attributes.time_local
          layout_type: strptime
          layout: '%d/%b/%Y:%H:%M:%S %z'

  filelog/nginx_error:
    include:
      - /var/log/nginx/error.log
    start_at: end
    operators:
      - type: regex_parser
        regex: '^(?P<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?P<level>\w+)\] (?P<pid>\d+)#(?P<tid>\d+): (?P<message>.*)'
        timestamp:
          parse_from: attributes.time
          layout_type: strptime
          layout: '%Y/%m/%d %H:%M:%S'
        severity:
          parse_from: attributes.level
          mapping:
            debug: debug
            info:
              - info
              - notice
            warn: warn
            error:
              - error
              - crit
            fatal:
              - alert
              - emerg

processors:
  batch: {}
  resourcedetection:
    detectors: [system]
  resource:
    attributes:
      - key: service.name
        value: nginx
        action: upsert

exporters:
  otlp/osuite:
    endpoint: ingest.<region>.osuite.io:443
    tls:
      insecure: false
    headers:
      x-osuite-ingest-token: <your-ingest-token>

service:
  pipelines:
    logs:
      receivers: [filelog/nginx_access, filelog/nginx_error]
      processors: [batch, resourcedetection, resource]
      exporters: [otlp/osuite]

Parsing and enrichment

• Access log format assumption — the access regex parses nginx’s stock combined format ($remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"). If you have customised log_format, adjust the capture groups to match your directive.
• Status code — the access parser extracts status as an attribute so you can filter for 5xx and 4xx responses in the explorer. Access records carry no log level, so no severity is set for them.
• Error severity — the error parser reads nginx’s level word (error, warn, crit, and so on) and maps it to the record’s SeverityNumber. See severity.
• Host and service attributes — the resourcedetection processor tags each record with the host name, and the resource processor sets a stable service.name so nginx logs group together in the explorer.
• Timestamps — nginx writes time_local in the server’s local time zone. The %z offset in the layout keeps the parsed time correct across zones.

Verify in Osuite

What you should see

Within a minute, access and error records appear in the Logs Explorer tagged with service.name=nginx and the host name. Filtering error records by severity returns the expected levels, and access records expose the status attribute.

Troubleshooting

If the parsed timestamp looks wrong, the layout does not match the log’s time format — check for a custom log_format or a non-default error log style. If a record keeps its whole line in the body with no attributes, the regex did not match; test it against a real log line and confirm the format assumption above.

No data in Osuite after a few minutes? Work through these checks.

• Endpoint — confirm the exporter targets exactly ingest.<region>.osuite.io:443 for your region, over TLS.
• Token — confirm the ingest header carries a valid <your-ingest-token> and has not been rotated.
• Pipeline — confirm the signal you expect (traces, logs, or metrics) is wired into an active pipeline with the otlp/osuite exporter attached.
• Export errors — check the application or Collector logs for OTLP export failures (auth, DNS, TLS, connection refused).
• Network — confirm the host has outbound access to the ingest endpoint on port 443.
• Timing — allow up to a minute for the first data to appear before assuming a failure.

Still stuck? Ask the Investigation Agent or contact support.