Design an Observability / APM Data Pipeline
Problem
Design New Relic's core: an APM/observability pipeline ingesting traces, metrics, and logs from customer apps.
Requirements
Functional:
- Ingest spans/metrics/logs from agents
- Store and query for dashboards
- Distributed tracing (trace assembly)
- Alerting on SLOs
Non-functional:
- Huge write volume, multi-tenant
- Low-latency queries
- Cost-efficient retention
Discussion points
- Agent ingestion and sampling
- Trace assembly from spans
- Time-series + trace storage
- Query and dashboards
- Multi-tenant isolation and back-pressure
added …