Collect PostgreSQL logs with OpenTelemetry

Overview

PostgreSQL writes server logs — startup, connections, statements, and errors — to a log file. The OpenTelemetry Collector’s filelog receiver tails that file, parses each entry, and ships the records to Osuite over OTLP. This page collects logs; for connection, transaction, and replication metrics see Monitor PostgreSQL with OpenTelemetry.

Prerequisites

• PostgreSQL configured to log to a file (logging_collector = on, or a distribution default). On Debian and Ubuntu the file is /var/log/postgresql/postgresql-<version>-main.log; on RHEL and CentOS it is under /var/lib/pgsql/<version>/data/log/.
• The OpenTelemetry Collector installed on the host, with read access to the log directory.
• An Osuite ingest token and endpoint — see Getting Started.

Collector config

This otel-collector.yaml tails the PostgreSQL log and parses the default stderr format. The multiline block keeps multi-line statements and error contexts together as one record. Adjust the include path for your version and distribution.

receivers:
  filelog/postgresql:
    include:
      - /var/log/postgresql/postgresql-*-main.log
    start_at: end
    multiline:
      line_start_pattern: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'
    operators:
      - type: regex_parser
        regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d+ \w+) \[(?P<pid>\d+)\](?: (?P<user>\S+)@(?P<database>\S+))? (?P<level>\w+):\s+(?P<message>(?s).*)'
        timestamp:
          parse_from: attributes.timestamp
          layout_type: strptime
          layout: '%Y-%m-%d %H:%M:%S.%f %Z'
        severity:
          parse_from: attributes.level
          mapping:
            debug: DEBUG
            info:
              - INFO
              - NOTICE
              - LOG
            warn: WARNING
            error: ERROR
            fatal:
              - FATAL
              - PANIC

processors:
  batch: {}
  resourcedetection:
    detectors: [system]
  resource:
    attributes:
      - key: service.name
        value: postgresql
        action: upsert

exporters:
  otlp/osuite:
    endpoint: ingest.<region>.osuite.io:443
    tls:
      insecure: false
    headers:
      x-osuite-ingest-token: <your-ingest-token>

service:
  pipelines:
    logs:
      receivers: [filelog/postgresql]
      processors: [batch, resourcedetection, resource]
      exporters: [otlp/osuite]

Parsing and enrichment

• Format assumption — the regex parses the default stderr format with a log_line_prefix of %m [%p] %q%u@%d (timestamp with milliseconds and zone, PID, and an optional user@database). If your log_line_prefix differs, adjust the leading capture groups to match.
• Severity — PostgreSQL uses its own level words (LOG, NOTICE, WARNING, ERROR, FATAL, PANIC). The mapping translates them to the record’s SeverityNumber — note that LOG maps to info, not a distinct level. See severity.
• Multiline — statements, DETAIL, HINT, and STATEMENT lines wrap under a single entry. The line_start_pattern treats a line beginning with a timestamp as a new record and folds the rest into the previous one.
• csvlog and jsonlog — if you set log_destination = csvlog, replace the regex_parser with a csv_parser; on PostgreSQL 15+ with log_destination = jsonlog, use a json_parser instead. Both are more robust than parsing stderr text.
• Host and service attributes — the resourcedetection and resource processors tag each record with the host name and a stable service.name.

Verify in Osuite

What you should see

Within a minute, PostgreSQL records appear in the Logs Explorer tagged with service.name=postgresql and the host name. Filtering by severity isolates ERROR and FATAL entries, and multi-line statements arrive as single records.

Troubleshooting

If timestamps fail to parse, your log_line_prefix likely differs from the assumed %m [%p] — check SHOW log_line_prefix; and adjust the layout and leading capture groups. If multi-line statements are split into separate records, the line_start_pattern does not match your timestamp format; align it with the actual line prefix.

No data in Osuite after a few minutes? Work through these checks.

• Endpoint — confirm the exporter targets exactly ingest.<region>.osuite.io:443 for your region, over TLS.
• Token — confirm the ingest header carries a valid <your-ingest-token> and has not been rotated.
• Pipeline — confirm the signal you expect (traces, logs, or metrics) is wired into an active pipeline with the otlp/osuite exporter attached.
• Export errors — check the application or Collector logs for OTLP export failures (auth, DNS, TLS, connection refused).
• Network — confirm the host has outbound access to the ingest endpoint on port 443.
• Timing — allow up to a minute for the first data to appear before assuming a failure.

Still stuck? Ask the Investigation Agent or contact support.