Must-Known Resiliency Patterns for Distributed Systems

The page groups resiliency patterns for distributed systems into downstream patterns used by callers and upstream patterns used by service owners. It presents them as proactive measures to reduce cascading failures, manage overload, and keep services stable under higher complexity.

Reading notes

#

• Downstream patterns are applied by the service caller so a failure in one service does not trigger a domino effect.
• Timeouts stop a service from waiting indefinitely for a downstream response and help avoid resource exhaustion.
• Circuit breakers monitor success and failure rates and stop calls temporarily when failures cross a threshold.
• Retries are useful for transient failures, and exponential backoff with jitter reduces pressure on the downstream service.
• Upstream patterns are used by service owners to protect service health during high traffic or failures.
• Load shedding rejects part of the incoming traffic when a service is overloaded and can prioritize critical requests.
• Rate limiting caps how many requests a client can make in a time window and helps prevent abuse and unfair resource usage.
• Bulkheading isolates parts of the system so failures in one component do not cascade to others.
• Health checks let load balancers avoid unhealthy instances and support smoother rolling updates.