Monitor Elasticsearch with OpenTelemetry

Osuite monitors Elasticsearch through the OpenTelemetry Collector’s elasticsearch receiver, which reads the node stats and cluster health APIs and exports them over OTLP. Cluster health, disk headroom, and JVM pressure arrive with the same resource attributes as your services, so a slow search in APM lines up with the cluster serving it.

Note

The elasticsearch receiver ships in the Collector contrib distribution. Use otelcol-contrib (or any build that includes it), not the core otelcol binary.

Setup

Prerequisites

• A reachable Elasticsearch cluster (7.9 or later).
• When security is enabled, a user with the monitor cluster privilege.

Create a role and user with the monitor privilege:

curl -u elastic:changeme -X POST "http://localhost:9200/_security/role/otel_monitor" \
  -H 'Content-Type: application/json' -d '{"cluster":["monitor"]}'

curl -u elastic:changeme -X POST "http://localhost:9200/_security/user/otel" \
  -H 'Content-Type: application/json' \
  -d '{"password":"a-strong-password","roles":["otel_monitor"]}'

Export the password to the Collector’s environment:

export ELASTICSEARCH_PASSWORD='a-strong-password'

Configure the Collector

1. Add the elasticsearch receiver, the otlp/osuite exporter, and a metrics pipeline. Pick the tab that matches where your Collector runs — only the receiver endpoint changes.

   Bare host

   receivers:
     elasticsearch:
       endpoint: http://localhost:9200
       username: otel
       password: ${env:ELASTICSEARCH_PASSWORD}
       collection_interval: 30s
   
   processors:
     batch:
     resourcedetection:
   
   exporters:
     otlp/osuite:
       endpoint: ingest.<region>.osuite.io:443
       headers:
         x-osuite-ingest-token: <your-ingest-token>
   
   service:
     pipelines:
       metrics:
         receivers: [elasticsearch]
         processors: [batch, resourcedetection]
         exporters: [otlp/osuite]

   Docker

   receivers:
     elasticsearch:
       endpoint: http://elasticsearch:9200
       username: otel
       password: ${env:ELASTICSEARCH_PASSWORD}
       collection_interval: 30s
   
   processors:
     batch:
     resourcedetection:
   
   exporters:
     otlp/osuite:
       endpoint: ingest.<region>.osuite.io:443
       headers:
         x-osuite-ingest-token: <your-ingest-token>
   
   service:
     pipelines:
       metrics:
         receivers: [elasticsearch]
         processors: [batch, resourcedetection]
         exporters: [otlp/osuite]

   Kubernetes

   receivers:
     elasticsearch:
       endpoint: http://elasticsearch.default.svc.cluster.local:9200
       username: otel
       password: ${env:ELASTICSEARCH_PASSWORD}
       collection_interval: 30s
   
   processors:
     batch:
     resourcedetection:
   
   exporters:
     otlp/osuite:
       endpoint: ingest.<region>.osuite.io:443
       headers:
         x-osuite-ingest-token: <your-ingest-token>
   
   service:
     pipelines:
       metrics:
         receivers: [elasticsearch]
         processors: [batch, resourcedetection]
         exporters: [otlp/osuite]

2. Start the Collector. On Docker and Kubernetes, the Collector and Elasticsearch must share a network so the endpoint resolves.

Tip

By default the receiver collects stats for every node in the cluster through the contacted endpoint. When you run one Collector per node (a daemonset), set nodes: ["_local"] on each so node metrics are not duplicated.

Validate

What you should see

Metrics named elasticsearch.* and jvm.* appear in Dashboards → Explore metrics within a minute, tagged with the cluster and node names.

Data collected

Metric	Description	Unit
elasticsearch.cluster.health	Cluster health status (green/yellow/red)	{status}
elasticsearch.cluster.nodes	Nodes in the cluster	{nodes}
elasticsearch.cluster.shards	Shards by state	{shards}
elasticsearch.node.documents	Documents on a node	{documents}
elasticsearch.node.fs.disk.available	Free disk space available to the node	By
elasticsearch.node.operations.completed	Operations completed by a node	{operations}
elasticsearch.node.operations.time	Time spent on node operations	ms
elasticsearch.node.thread_pool.tasks.queued	Queued thread-pool tasks	{tasks}
jvm.memory.heap.used	JVM heap memory in use	By
jvm.gc.collections.count	Garbage collections performed	1

Per-index metrics are available by listing indices in the receiver’s indices field.

Dashboards and alerts

Once metrics arrive, build an Elasticsearch dashboard in the dashboard builder from the series above. Alerts in Osuite are PromQL-based — see Alerts. Useful starting conditions: cluster health leaving green (elasticsearch.cluster.health), low elasticsearch.node.fs.disk.available, and sustained high JVM heap usage.

Troubleshooting

• 401 Unauthorized — confirm the otel user has the monitor cluster privilege and that the credentials match.
• TLS errors — when the cluster serves HTTPS, use an https:// endpoint and configure the receiver’s tls block with the cluster CA.

No data in Osuite after a few minutes? Work through these checks.

• Endpoint — confirm the exporter targets exactly ingest.<region>.osuite.io:443 for your region, over TLS.
• Token — confirm the ingest header carries a valid <your-ingest-token> and has not been rotated.
• Pipeline — confirm the signal you expect (traces, logs, or metrics) is wired into an active pipeline with the otlp/osuite exporter attached.
• Export errors — check the application or Collector logs for OTLP export failures (auth, DNS, TLS, connection refused).
• Network — confirm the host has outbound access to the ingest endpoint on port 443.
• Timing — allow up to a minute for the first data to appear before assuming a failure.

Still stuck? Ask the Investigation Agent or contact support.