Guides

Telemetry & Observability

Monitor tileserver-rs with OpenTelemetry traces and metrics

tileserver-rs exports traces and metrics via OpenTelemetry OTLP gRPC. This guide covers setup with common observability backends.

Quick Start

Enable telemetry in your config.toml:

[telemetry]
enabled = true
endpoint = "http://localhost:4317"

This enables both traces and metrics, exported to an OTLP-compatible collector on port 4317 (gRPC).

What's Exported

Traces

Every HTTP request creates a span with method, path, status, and duration. Traces use the standard tracing crate integration — all structured log events in the codebase become span events.

Metrics

MetricTypeUnitDescription
http.server.request.countCounterrequestsTotal HTTP requests
http.server.request.durationHistogramsecondsRequest duration distribution
http.server.response.body.sizeHistogrambytesResponse body size distribution

All metrics include these attributes:

AttributeExample
http.request.methodGET
http.response.status_code200
url.path/data/openmaptiles/12/2176/1493.pbf

Configuration Reference

[telemetry]
enabled = true                      # Enable OpenTelemetry
endpoint = "http://localhost:4317"   # OTLP gRPC endpoint
service_name = "tileserver-rs"       # Service name in traces/metrics
sample_rate = 1.0                    # Trace sampling (0.0-1.0)
metrics_enabled = true               # Enable metrics (default: true)
metrics_export_interval_secs = 60    # Metrics push interval (default: 60)
Metrics are only active when both enabled and metrics_enabled are true. When disabled, all metric instruments are no-ops with zero performance overhead.

Backend Setup Examples

Grafana Alloy + Tempo + Prometheus

This is the recommended stack for production. Grafana Alloy (formerly Grafana Agent) receives OTLP and forwards traces to Tempo and metrics to Prometheus.

# compose.yml
services:
  tileserver:
    image: ghcr.io/vinayakkulkarni/tileserver-rs:latest
    ports:
      - "8080:8080"
    volumes:
      - ./data:/data:ro
      - ./config.toml:/app/config.toml:ro

  alloy:
    image: grafana/alloy:latest
    ports:
      - "4317:4317"   # OTLP gRPC
      - "12345:12345" # Alloy UI
    volumes:
      - ./alloy-config.alloy:/etc/alloy/config.alloy
    command: ["run", "/etc/alloy/config.alloy"]

  tempo:
    image: grafana/tempo:latest
    ports:
      - "3200:3200"
    volumes:
      - ./tempo.yaml:/etc/tempo.yaml
    command: ["-config.file=/etc/tempo.yaml"]

  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
# config.toml
[telemetry]
enabled = true
endpoint = "http://alloy:4317"

Jaeger (Development)

For quick local development, Jaeger's all-in-one image accepts OTLP directly:

# compose.yml
services:
  tileserver:
    image: ghcr.io/vinayakkulkarni/tileserver-rs:latest
    ports:
      - "8080:8080"
    volumes:
      - ./data:/data:ro
      - ./config.toml:/app/config.toml:ro

  jaeger:
    image: jaegertracing/jaeger:latest
    ports:
      - "4317:4317"   # OTLP gRPC
      - "16686:16686" # Jaeger UI
    environment:
      - COLLECTOR_OTLP_ENABLED=true
# config.toml
[telemetry]
enabled = true
endpoint = "http://jaeger:4317"
metrics_enabled = false  # Jaeger is traces-only

Open http://localhost:16686 to view traces.

OpenTelemetry Collector

For maximum flexibility, use the official OTel Collector to fan-out to multiple backends:

# otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  batch:
    timeout: 10s

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true
  prometheusremotewrite:
    endpoint: http://prometheus:9090/api/v1/write

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/tempo]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheusremotewrite]
# config.toml
[telemetry]
enabled = true
endpoint = "http://otel-collector:4317"

Performance Considerations

  • Traces use batch export — spans are buffered and sent periodically, not per-request
  • Metrics use a PeriodicReader (default: every 60 seconds) — metric data points are aggregated in-memory and pushed at the configured interval
  • Instruments (counters, histograms) are lock-free — recording a metric is an atomic operation with negligible overhead
  • When disabled, all instruments are no-ops. The only per-request cost is a single atomic load from OnceLock

Sampling

For high-traffic deployments, reduce trace sampling to control export volume:

[telemetry]
enabled = true
sample_rate = 0.1  # Only export 10% of traces

Metrics are always exported at full fidelity regardless of sample_rate — sampling only affects traces.

Troubleshooting

No data appearing in backend

  1. Verify the endpoint is reachable: curl -v http://localhost:4317
  2. Check tileserver logs for OpenTelemetry initialized with metrics=true
  3. Ensure your collector accepts OTLP gRPC (not HTTP) on the configured port
  4. Wait at least metrics_export_interval_secs for the first metrics push

High memory usage

If metrics cardinality is too high (many unique URL paths), consider using a collector with attribute filtering to drop or group url.path values.