7.4. Chaos Engineering
Chaos engineering is the practice of injecting failure into production on purpose, on the theory that the only way to know your system survives a failure is to cause one under controlled conditions — before it happens on its own at 3am. It is not random breakage: it is a disciplined experiment with a hypothesis, a bounded blast radius, and an abort condition. This chapter covers the principles pioneered by Netflix’s Chaos Monkey, the fault-injection techniques, GameDay drills, and the tooling that automates it all.
Topics Covered
Section titled “Topics Covered”- 7.4.1. Chaos Monkey: Netflix’s Pioneering Approach: Covers Netflix’s Chaos Monkey, the tool that pioneered randomly killing production instances to force resilience.
- 7.4.2. Principles of Chaos Engineering: Hypothesis, Blast Radius, Observation: Explains the principles of chaos engineering: a hypothesis, a bounded blast radius, and steady-state observation.
- 7.4.3. Fault Injection: Simulating Latency, Exceptions, and Partitions: Covers injecting latency, exceptions, and network partitions to test how a system degrades under stress.
- 7.4.4. GameDay: Disaster Scenario Drills: Explains GameDay drills that rehearse disaster scenarios with the team before they happen for real.
- 7.4.5. Automating Chaos: Litmus, Chaos Toolkit: Covers automating chaos experiments with Litmus and Chaos Toolkit for continuous resilience validation.