Circuit breaker pattern
Need(s)
Services collaborate when handling one client request. When one service invokes another one, there is always the possibility that the second one is unavailable or is exhibiting such high latency so it is essentially unusable. The failure of one service can potentially cascade to other services throughout the application. How to prevent a network or service failure from cascading to other services ?
Circuit breaker pattern definition
In his excellent book "Release It"1, Michael Nygard popularized the circuit breaker pattern to prevent catastrophic failure cascade. The basic idea behind the circuit breaker is very simple. You wrap a protected function call in a circuit breaker object, which monitors for failures. Once the failures reach a certain threshold, the circuit breaker trips, and all further calls to the circuit breaker return with an error, without the protected call being made at all. Usually you'll also want some kind of monitor alert if the circuit breaker trips.
In contrast to the Retry pattern2 which enables services to retry an operation, the circuit breaker prevents the service from performing an operation that is likely to fail. For example, a client service can use a circuit breaker to prevent further remote calls over the network when a downstream service is not functioning properly. It can also prevent the network from becoming congested by a sudden spike in failed retries by one service to another, and it can also prevent cascading failures. Self-healing circuit breakers check the downstream service at regular intervals and reset the circuit breaker when the downstream service starts functioning properly.
1. “Release It!: Design and Deploy Production-Ready Software”, Michael Nygard, https://www.amazon.com/gp/product/0978739213?ie=UTF8&tag=martinfowlerc-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=0978739213 ↩
2. “Retry pattern”, This content ↩