Collect log files

Overview

The filelog receiver tails one or more files on disk, just like tail -f, and ships each line to Osuite over OTLP. It handles multiline records, log rotation, and position checkpointing, so it resumes where it left off after a restart. For logs written by your own application code, prefer the SDK bridge in Collect application logs — it adds trace correlation that a file tail cannot.

Prerequisites

• The OpenTelemetry Collector installed on the host, with read access to the log files.
• An Osuite ingest token and endpoint — see Getting Started.

Collector config

This otel-collector.yaml tails a JSON log file, parses the timestamp and severity, and exports to Osuite. Adjust the include paths for your files.

receivers:
  filelog:
    include:
      - /var/log/myapp/*.log
    start_at: end
    include_file_path: true
    include_file_name: true
    operators:
      - type: json_parser
        timestamp:
          parse_from: attributes.timestamp
          layout: "%Y-%m-%dT%H:%M:%S.%fZ"
      - type: severity_parser
        parse_from: attributes.level

processors:
  batch: {}
  resourcedetection:
    detectors: [system]

exporters:
  otlp/osuite:
    endpoint: ingest.<region>.osuite.io:443
    tls:
      insecure: false
    headers:
      x-osuite-ingest-token: <your-ingest-token>

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch, resourcedetection]
      exporters: [otlp/osuite]

Parsing and enrichment

• JSON logs — the json_parser operator turns each line into structured attributes. Point its timestamp block at the field holding the event time and match the layout to your format.
• Severity — the severity_parser operator reads a text level (INFO, error, WARN) from an attribute and sets the record’s SeverityNumber, so filtering by level works consistently. See severity.
• Plain-text logs — for logs that are not JSON, drop the json_parser and use a regex_parser operator to extract fields, or send the raw line as the body with no parser at all.
• Host attributes — the resourcedetection processor with the system detector tags every record with the host name, so you can filter by machine in the explorer.
• Multiline — for stack traces or other multiline records, add a multiline block to the receiver with a line_start_pattern so each entry is captured as one log.

Verify in Osuite

What you should see

Within a minute, records from the tailed files appear in the Logs Explorer, tagged with the host name and file path. Filtering by severity returns the expected levels once the severity parser is wired.

Troubleshooting

If the parsed timestamp looks wrong, the layout does not match the field’s format — the record falls back to the time it was read. If severity filtering is empty, confirm the severity_parser points at the field that actually holds the level.

No data in Osuite after a few minutes? Work through these checks.

• Endpoint — confirm the exporter targets exactly ingest.<region>.osuite.io:443 for your region, over TLS.
• Token — confirm the ingest header carries a valid <your-ingest-token> and has not been rotated.
• Pipeline — confirm the signal you expect (traces, logs, or metrics) is wired into an active pipeline with the otlp/osuite exporter attached.
• Export errors — check the application or Collector logs for OTLP export failures (auth, DNS, TLS, connection refused).
• Network — confirm the host has outbound access to the ingest endpoint on port 443.
• Timing — allow up to a minute for the first data to appear before assuming a failure.

Still stuck? Ask the Investigation Agent or contact support.