Skip to content
Osuite Osuite Docs

Trace sampling and the ingestion pipeline

By default, Osuite ingests every trace your services send. As trace volume grows, sampling lets you keep the traces that carry signal — errors and slow requests — while dropping routine healthy ones, so ingestion stays useful without staying expensive.

Tail-based sampling

Section titled “Tail-based sampling”

Tail-based sampling makes the keep-or-drop decision after a trace is complete, rather than at the moment the first span starts. Because the whole trace is available when the decision is made, the outcome of the request can drive it.

That is what makes it the right default for observability data. You can:

  • keep every trace that contains an error
  • keep every trace slower than a latency threshold
  • keep only a fraction of the remaining routine, successful traces

You never lose the traces you would actually open during an incident, and ingestion volume becomes predictable.

Collector-side sampling

Section titled “Collector-side sampling”

If you run the OpenTelemetry Collector in front of Osuite, you can apply tail-based sampling before data leaves your network using the Collector’s tail_sampling processor. The recipe below keeps all error traces and all traces over one second, and samples 10% of everything else.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318


processors:
  # tail_sampling requires every span of a trace to reach this collector instance
  tail_sampling:
    decision_wait: 10s
    num_traces: 50000
    policies:
      - name: keep-errors
        type: status_code
        status_code:
          status_codes: [ERROR]
      - name: keep-slow
        type: latency
        latency:
          threshold_ms: 1000
      - name: sample-the-rest
        type: probabilistic
        probabilistic:
          sampling_percentage: 10


exporters:
  otlp/osuite:
    endpoint: ingest.<region>.osuite.io:443
    tls:
      insecure: false
    headers:
      x-osuite-ingest-token: <your-ingest-token>


service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [tail_sampling]
      exporters: [otlp/osuite]

Because the keep-or-drop decision needs the complete trace, every span of a given trace must reach the same collector instance. If you run several collectors behind a load balancer, route spans by trace ID (for example with the loadbalancing exporter in a two-tier collector setup) so each trace lands whole on one sampling collector.

Section titled “Related”