8.1. The Three Pillars: Logs, Metrics, Traces
The three pillars are not interchangeable; each answers a different question at a different cost. Metrics tell you something is wrong cheaply and in aggregate; logs tell you what happened in detail but expensively at high cardinality; traces tell you where in a request’s path across services the time or the error went. This chapter covers structured logging and correlation IDs, the metric types and the RED/USE methods for using them, distributed tracing with context propagation, and how to correlate all three from a single failure event.
Topics Covered
Section titled “Topics Covered”- 8.1.1. Structured Logging: JSON Logs and Correlation IDs: Covers structured JSON logging and correlation IDs that stitch a single request’s logs across services.
- 8.1.2. Log Aggregation: ELK Stack (Elasticsearch, Logstash, Kibana): Explains centralizing logs with the ELK stack: Elasticsearch, Logstash, and Kibana.
- 8.1.3. Metrics: Counter, Gauge, Histogram, Summary: Covers the four metric types — counter, gauge, histogram, and summary — and what each measures.
- 8.1.4. The RED Method (Rate, Errors, Duration) vs. the USE Method: Compares the RED method for services with the USE method for resources as complementary monitoring lenses.
- 8.1.5. Distributed Tracing: Span, Trace, and Baggage Propagation: Explains distributed tracing with spans, traces, and the context propagation that follows a request across services.
- 8.1.6. Correlation: Joining Three Signals from a Single Failure Event: Covers correlating logs, metrics, and traces from a single failure event using exemplars and shared IDs.