How to Retry Failed API Calls in Python With Exponential Backoff and Jitter

Every API call you make over a network can fail for reasons that have nothing to do with your code: a server restarts, a load balancer briefly runs out of capacity, a database connection pool is momentarily exhausted. These are transient failures, and the standard fix is to retry the call. But a retry loop written the wrong way can make an already-struggling service worse instead of better, and it can do it in a way that only shows up under real production load, never in your own testing.

This tutorial builds a retry helper in Python from scratch. You will reproduce the exact failure mode that makes naive retries dangerous (a synchronized pile-on known as the thundering herd problem), fix it with exponential backoff, discover that backoff alone does not actually fix it, fix that with jitter (randomized delay), and then layer on the two things most retry tutorials skip: honoring a server’s own Retry-After header, and knowing which errors should never be retried at all. Every command and every number below was run for real while writing this post; nothing is invented.

What You Will Build

A small, dependency-light Python toolkit with no third-party retry library involved, so you can see exactly what those libraries are doing for you under the hood:

• A flaky FastAPI service you control, so you can reproduce a transient outage on demand instead of just reading about one
• A naive retry loop, to see how aggressively an unprotected retry hammers a struggling dependency
• A concurrency experiment with 12 simulated clients retrying at once, to make the thundering herd problem visible as data, not just as a warning in a blog post
• Four backoff strategies (fixed delay, plain exponential backoff, full jitter, and decorrelated jitter) measured side by side against that same 12-client experiment
• retry_toolkit.py: a single reusable call_with_retry() function that combines full jitter backoff, Retry-After support, a status-code retry policy, and a hard cap on both attempt count and total elapsed time
• A pytest suite that verifies the retry logic deterministically, with no live server and no real waiting

Prerequisites

• Python 3.10 or newer (this tutorial used Python 3.13.14 on Windows; nothing here is platform-specific)
• Comfort with basic Python: functions, exceptions, and reading a small HTTP client library
• Basic familiarity with HTTP status codes and the request/response cycle
• No prior knowledge of backoff algorithms or jitter is assumed; both are explained from first principles below

Step 1: Set Up Your Environment

Create a project folder and an isolated virtual environment, then install the three libraries this tutorial uses: a web framework to build the flaky test service, a server to run it, and an HTTP client to call it.

mkdir retry-backoff-demo
cd retry-backoff-demo
python -m venv venv
venv\Scripts\activate   # on Linux/macOS: source venv/bin/activate
pip install fastapi uvicorn requests pytest

This tutorial was written against fastapi 0.141.1, uvicorn 0.52.4, requests 2.34.2, and pytest 9.1.1. None of the APIs used here are new or unstable, so newer versions should behave the same way.

Step 2: Build a Dependency That Fails on Purpose

To study retry behavior you need something that actually fails, predictably, so your results are reproducible. Create flaky_service.py:

from collections import defaultdict
from threading import Lock

from fastapi import FastAPI, Response
from fastapi.responses import JSONResponse

app = FastAPI()

_lock = Lock()
_attempts = defaultdict(int)

FAIL_UNTIL = 4  # each tag fails this many times, then succeeds


@app.get("/flaky")
def flaky(tag: str):
    """Fails with 503 the first FAIL_UNTIL times a given tag is requested, then succeeds."""
    with _lock:
        _attempts[tag] += 1
        attempt = _attempts[tag]
    if attempt <= FAIL_UNTIL:
        return JSONResponse(
            status_code=503,
            content={"error": "service_unavailable", "tag": tag, "attempt": attempt},
        )
    return {"status": "ok", "tag": tag, "attempt": attempt}


@app.get("/always-down")
def always_down(tag: str):
    """Never recovers. Used to prove a retry loop gives up instead of hanging forever."""
    with _lock:
        _attempts[tag] += 1
        attempt = _attempts[tag]
    return JSONResponse(
        status_code=503,
        content={"error": "service_unavailable", "tag": tag, "attempt": attempt},
    )


@app.get("/rate-limited")
def rate_limited(response: Response):
    """Always 429s and tells the caller exactly how long to wait."""
    response.headers["Retry-After"] = "3"
    return JSONResponse(
        status_code=429,
        content={"error": "rate_limited", "retry_after_seconds": 3},
        headers={"Retry-After": "3"},
    )


@app.get("/orders/{order_id}")
def get_order(order_id: int):
    """A normal client error. No amount of retrying will ever fix a 404."""
    if order_id == 999:
        return JSONResponse(status_code=404, content={"error": "not_found", "order_id": order_id})
    return {"order_id": order_id, "status": "ok"}


@app.post("/reset")
def reset():
    with _lock:
        _attempts.clear()
    return {"status": "reset"}

The /flaky endpoint fails with a 503 Service Unavailable for the first four requests carrying a given tag, then starts succeeding. Every demo in this tutorial uses its own tag, so runs never interfere with each other. /always-down never recovers, for testing what happens when a retry loop has to give up. /rate-limited always returns 429 Too Many Requests with a real Retry-After header. /orders/999 returns a plain 404, standing in for a normal client error that no amount of retrying will ever fix.

Start it in its own terminal and leave it running for the rest of this tutorial:

uvicorn flaky_service:app --host 127.0.0.1 --port 8811

Step 3: See Why Naive Retries Are Dangerous

The simplest possible retry loop calls the endpoint, and if it fails, immediately calls it again. Create step2_naive_retry.py:

import time

import requests

BASE = "http://127.0.0.1:8811"


def naive_retry(tag: str, max_attempts: int = 10):
    attempts = []
    for i in range(max_attempts):
        t0 = time.monotonic()
        r = requests.get(f"{BASE}/flaky", params={"tag": tag})
        elapsed = time.monotonic() - t0
        attempts.append((i + 1, r.status_code, elapsed))
        if r.status_code == 200:
            return attempts
    raise RuntimeError("exhausted attempts")


if __name__ == "__main__":
    start = time.monotonic()
    attempts = naive_retry("naive-demo")
    total = time.monotonic() - start
    for num, status, elapsed in attempts:
        print(f"attempt {num}: status={status} took {elapsed * 1000:.1f}ms")
    print(f"succeeded after {len(attempts)} attempts, total wall time {total * 1000:.1f}ms")

python step2_naive_retry.py

attempt 1: status=503 took 3.5ms
attempt 2: status=503 took 2.5ms
attempt 3: status=503 took 2.6ms
attempt 4: status=503 took 2.7ms
attempt 5: status=200 took 2.7ms
succeeded after 5 attempts, total wall time 14.0ms

From one client, this looks completely harmless: five requests, fourteen milliseconds, done. The problem is invisible at this scale and only appears once you multiply it by the number of clients a real service actually has. If a downstream dependency briefly returns errors for every one of your service’s 500 concurrent requests, all 500 naive retry loops fire again almost immediately, at essentially the same moment. That second wave of traffic lands on a dependency that was already struggling, which is exactly the load pattern most likely to keep it down. The fix is not to stop retrying: it is to control when retries happen.

Step 4: Reproduce the Thundering Herd Problem

The obvious next idea is to wait between retries instead of firing immediately. To see whether that actually solves the problem, you need more than one client. Create retry_strategies.py with four candidate wait-time functions:

import random


def fixed_delay(attempt: int, base: float = 2.0, cap: float = 30.0) -> float:
    return base


def exponential_no_jitter(attempt: int, base: float = 0.5, cap: float = 30.0) -> float:
    return min(cap, base * (2 ** attempt))


def full_jitter(attempt: int, base: float = 0.5, cap: float = 30.0) -> float:
    # AWS Architecture Blog, "Exponential Backoff And Jitter" (Marc Brooker):
    # sleep = random_between(0, min(cap, base * 2 ** attempt))
    return random.uniform(0, min(cap, base * (2 ** attempt)))


def decorrelated_jitter(prev_sleep: float, base: float = 0.5, cap: float = 30.0) -> float:
    # sleep = min(cap, random_between(base, sleep * 3))
    # Note: uses the PREVIOUS sleep value, not the attempt number.
    return min(cap, random.uniform(base, prev_sleep * 3))

Now create herd_experiment.py, which launches 12 simulated clients at the same instant, all retrying against the same flaky endpoint using whichever strategy you pick, and records the wall-clock timestamp of every single HTTP request any of them makes:

import sys
import time
from concurrent.futures import ThreadPoolExecutor, as_completed

import requests

from retry_strategies import decorrelated_jitter, exponential_no_jitter, fixed_delay, full_jitter

BASE = "http://127.0.0.1:8811"
N_CLIENTS = 12
MAX_ATTEMPTS = 8
BUCKET_SECONDS = 0.25


def reset():
    requests.post(f"{BASE}/reset")


def worker(client_id: int, strategy: str, experiment_start: float):
    tag = f"herd-{strategy}-{client_id}"
    timestamps = []
    attempt = 0
    sleep_time = 0.5  # used as the running state for decorrelated jitter
    while attempt < MAX_ATTEMPTS:
        t = time.monotonic()
        r = requests.get(f"{BASE}/flaky", params={"tag": tag})
        timestamps.append(t - experiment_start)
        attempt += 1
        if r.status_code == 200:
            return timestamps
        if strategy == "fixed":
            sleep_time = fixed_delay(attempt)
        elif strategy == "exp_no_jitter":
            sleep_time = exponential_no_jitter(attempt)
        elif strategy == "full_jitter":
            sleep_time = full_jitter(attempt)
        elif strategy == "decorrelated_jitter":
            sleep_time = decorrelated_jitter(sleep_time)
        else:
            raise ValueError(strategy)
        time.sleep(sleep_time)
    raise RuntimeError(f"client {client_id} exhausted attempts")


def run_strategy(strategy: str):
    reset()
    experiment_start = time.monotonic()
    all_timestamps = []
    with ThreadPoolExecutor(max_workers=N_CLIENTS) as pool:
        futures = [pool.submit(worker, i, strategy, experiment_start) for i in range(N_CLIENTS)]
        for f in as_completed(futures):
            all_timestamps.extend(f.result())
    return sorted(all_timestamps)


def bucket_and_print(strategy: str, timestamps: list):
    if not timestamps:
        print(f"{strategy}: no data")
        return
    max_t = max(timestamps)
    n_buckets = int(max_t / BUCKET_SECONDS) + 1
    counts = [0] * n_buckets
    for t in timestamps:
        counts[int(t / BUCKET_SECONDS)] += 1
    peak = max(counts)
    print(f"\n=== {strategy} ===")
    print(f"total requests: {len(timestamps)}  |  span: {max_t:.2f}s  |  peak requests in any {BUCKET_SECONDS}s window: {peak}")
    for i, c in enumerate(counts):
        bar = "#" * c
        print(f"  t={i * BUCKET_SECONDS:5.2f}s [{c:2d}] {bar}")


if __name__ == "__main__":
    strategy = sys.argv[1] if len(sys.argv) > 1 else "fixed"
    ts = run_strategy(strategy)
    bucket_and_print(strategy, ts)

Fixed Delay: Everyone Waits the Same Amount, So Everyone Retries Together

Run the experiment with a plain, constant two-second delay between attempts:

python herd_experiment.py fixed

=== fixed ===
total requests: 60  |  span: 8.05s  |  peak requests in any 0.25s window: 12
  t= 0.00s [12] ############
  t= 2.00s [12] ############
  t= 4.00s [12] ############
  t= 6.00s [12] ############
  t= 8.00s [12] ############
  (all other 0.25s windows: 0 requests)

All 12 clients started together, so they fail together, so they all sleep for exactly the same two seconds, so they all retry together. The peak load on the dependency is 12 simultaneous requests, which is exactly what it would have been with no delay at all. A fixed delay does nothing to spread the load out; it just delays the pile-on by a fixed amount and repeats it every two seconds.

Exponential Backoff Alone Does Not Fix It Either

The standard next step is exponential backoff: double the wait time after every failed attempt, so a struggling dependency gets progressively more breathing room. Run the same experiment with exponential_no_jitter, which computes delay = min(cap, base * 2 ** attempt) with no randomness:

python herd_experiment.py exp_no_jitter

=== exp_no_jitter ===
total requests: 60  |  span: 15.05s  |  peak requests in any 0.25s window: 12
  t= 0.00s [12] ############
  t= 1.00s [12] ############
  t= 3.00s [12] ############
  t= 7.00s [12] ############
  t=15.00s [12] ############
  (all other 0.25s windows: 0 requests)

This is the part most retry tutorials skip. The peak load is still 12 simultaneous requests, identical to the fixed-delay run; only the spacing between the spikes changed, growing to 1, 3, 7, and 15 seconds. Every client computes the exact same deterministic sequence of delays, because every client fails at the same moment and starts counting from the same attempt number. Marc Brooker’s AWS Architecture Blog post that popularized this analysis put it plainly: instead of reducing the number of clients competing in every round, plain exponential backoff just introduced times when no client is competing. Backoff alone changes the rhythm of the pile-on. It does not break the synchronization that causes it.

Step 5: Add Jitter to Actually Break the Synchronization

Jitter means adding randomness to the delay so that clients which failed at the same instant do not all wake up at the same instant. The term comes from electrical engineering, where it describes unwanted timing variation in a signal; retry logic borrows the word for timing variation you introduce on purpose. Two well-known formulas, both from the same AWS Architecture Blog post by Marc Brooker, cover most real use cases.

Full jitter throws away the deterministic delay entirely and picks a random point between zero and the exponential ceiling:

sleep = random_between(0, min(cap, base * 2 ** attempt))

Decorrelated jitter does not use the attempt number at all. It feeds the previous sleep value into the next one, so each wait grows from wherever the last one landed rather than from a fixed schedule:

sleep = min(cap, random_between(base, previous_sleep * 3))

Full Jitter in Practice

python herd_experiment.py full_jitter

=== full_jitter ===
total requests: 60  |  span: 10.05s  |  peak requests in any 0.25s window: 14
  t= 0.00s [14] ##############
  t= 0.25s [ 3] ###
  t= 0.50s [ 7] #######
  t= 0.75s [ 3] ###
  t= 1.00s [ 1] #
  t= 1.50s [ 3] ###
  t= 1.75s [ 1] #
  t= 2.00s [ 5] #####
  t= 2.25s [ 1] #
  t= 2.50s [ 1] #
  t= 3.00s [ 1] #
  t= 3.25s [ 1] #
  t= 3.75s [ 1] #
  t= 4.00s [ 3] ###
  t= 4.75s [ 1] #
  t= 5.25s [ 1] #
  t= 5.50s [ 1] #
  t= 6.00s [ 1] #
  t= 6.50s [ 1] #
  t= 6.75s [ 1] #
  t= 8.00s [ 2] ##
  t= 8.50s [ 1] #
  t= 8.75s [ 2] ##
  t= 9.00s [ 1] #
  t= 9.25s [ 1] #
  t= 9.50s [ 1] #
  t=10.00s [ 1] #
  (all remaining 0.25s windows in range: 0 requests)

Everything after the first burst spreads across the full ten-second span instead of landing in five sharp spikes. This is the real effect jitter buys you: a dependency that just came back up sees a trickle of retrying clients instead of a second collapse.

Look closely at t=0.00s first, though: fourteen requests, not spread out at all, and actually higher than the twelve clients in the experiment (a couple of clients got unlucky with a very short jitter value and fired their second attempt within the same quarter-second window as everyone else’s first). This is a genuinely easy point to miss: jitter only desynchronizes retries. It does nothing for the very first attempt, because every client makes that first call before any backoff or jitter logic has run at all. If your traffic spikes happen because of a shared trigger, a deploy, a cron job, a cache expiring at the top of the hour, jitter will not soften that initial spike. It only stops the second, third, and fourth waves from re-synchronizing on top of it.

Decorrelated Jitter: An Alternative With a Different Tradeoff

python herd_experiment.py decorrelated_jitter

=== decorrelated_jitter ===
total requests: 60  |  span: 16.09s  |  peak requests in any 0.25s window: 12
  t= 0.00s [12] ############
  t= 0.50s [ 3] ###
  t= 0.75s [ 2] ##
  t= 1.00s [ 2] ##
  t= 1.25s [ 6] ######
  t= 1.50s [ 1] #
  t= 1.75s [ 3] ###
  ... (spread continues, low counts per window, through t=16.09s)

In this run, decorrelated jitter spread the retries over a longer window than full jitter did (16.1 seconds versus 10.1 seconds), while never exceeding the original 12-client peak. Because both formulas involve random_between, the exact numbers will differ a little every time you run this yourself. The AWS post’s own conclusion is a genuine tradeoff rather than a clear winner, noting that full jitter tends to use less total work but slightly more time to finish than decorrelated jitter across many contending clients. This tutorial’s own experiment cannot confirm or contradict that specific finding: every strategy here makes exactly 60 requests in total, since FAIL_UNTIL is a fixed attempt count rather than a dependency that actually gets more overwhelmed under heavier load. What this experiment can show honestly is narrower and still useful: both jitter formulas break the synchronized-spike pattern that fixed delay and plain exponential backoff could not. For most applications, full jitter is the simpler mental model to reach for first (one call, one formula, no state to carry between attempts), which is what this tutorial builds into the final retry helper.

Step 6: Respect the Server’s Own Retry-After Header

Guessing at a backoff schedule is a fallback for when the server does not tell you what it wants. Often it does. MDN’s Retry-After reference describes the header plainly: it indicates how long the user agent should wait before making a follow-up request, as either an integer number of seconds or an HTTP date, and it commonly appears on 503 Service Unavailable and 429 Too Many Requests responses. Your flaky service’s /rate-limited endpoint always returns 429 with Retry-After: 3. A retry helper that ignores this and computes its own backoff instead is working against the server’s own instructions, and on a real API it can get you throttled harder or temporarily banned.

Step 7: Know What Not to Retry

Not every failure is transient, and retrying the wrong kind of failure wastes time at best and hides a real bug at worst. A useful status-code policy splits errors into two groups:

• Retryable: 429 (rate limited), 502/503/504 (upstream problems), and connection-level failures like a dropped connection or a timeout. These describe the server, not your request.
• Not retryable: 400, 401, 403, 404, and most other 4xx codes. These describe your request. If it was malformed, unauthorized, or pointed at something that does not exist, sending the identical request again will fail the identical way every time.

Hitting /orders/999, which always returns 404, should fail on the first attempt with no delay at all. You will confirm that in Step 9.

There is a second, sharper edge here worth calling out explicitly: retrying is only safe if the operation is idempotent, meaning running it twice has the same effect as running it once. A GET request is naturally idempotent. A POST that charges a card or places an order usually is not; blindly retrying one after a timeout can execute it twice even though the first attempt actually succeeded server-side and only the response got lost. This tutorial’s demo server only exposes GET endpoints so it can focus on backoff and jitter without that extra complexity. If you are retrying a non-idempotent write, see this site’s idempotency keys tutorial for how to make retries of a POST safe.

Step 8: Cap Both Attempt Count and Total Elapsed Time

A max-attempts limit alone is not enough. Because backoff delays grow, a handful of unlucky attempts can burn far more wall-clock time than you planned for, which matters a great deal if this retry loop is running inside a request handler with its own timeout. Run this against the permanently-broken /always-down endpoint with a generous attempt budget but a tight time budget:

Create step8_elapsed_cap.py:

import logging
import time

import requests

from retry_toolkit import RetryExhausted, call_with_retry

logging.basicConfig(level=logging.INFO, format="  %(message)s")

BASE = "http://127.0.0.1:8811"

if __name__ == "__main__":
    requests.post(f"{BASE}/reset")
    try:
        call_with_retry(
            lambda: requests.get(f"{BASE}/always-down", params={"tag": "elapsed-cap-demo"}),
            max_attempts=10,
            base=5.0,
            cap=30.0,
            max_elapsed=3.0,
        )
    except RetryExhausted as exc:
        print(f"gave up after only {exc.attempts} of 10 allowed attempts, {exc.elapsed:.2f}s elapsed")

python step8_elapsed_cap.py

  attempt 1 failed, backing off 9.05s (full jitter)
gave up after only 2 of 10 allowed attempts, 9.06s elapsed

A single unlucky jittered sleep of 9.05 seconds was enough by itself to blow past a 3-second time budget, so the loop correctly gave up on attempt 2 instead of grinding through eight more attempts the caller no longer had time for. Run it again and you will likely see a different sleep value (it is drawn from random.uniform(0, 10) here) that still blows the same budget almost every time, since 3 seconds is a small slice of that 10-second ceiling. An attempt-count cap and a wall-clock cap protect against two different failure modes, and a production retry helper needs both.

Step 9: Put It All Together, a Reusable Retry Helper

Everything above becomes one function. Create retry_toolkit.py:

import logging
import random
import time

import requests

logger = logging.getLogger("retry_toolkit")

RETRYABLE_STATUSES = {429, 502, 503, 504}


class RetryExhausted(Exception):
    def __init__(self, message, last_response=None, attempts=0, elapsed=0.0):
        super().__init__(message)
        self.last_response = last_response
        self.attempts = attempts
        self.elapsed = elapsed


def _full_jitter_sleep(attempt: int, base: float, cap: float) -> float:
    return random.uniform(0, min(cap, base * (2 ** attempt)))


def call_with_retry(
    request_fn,
    *,
    max_attempts: int = 6,
    base: float = 0.5,
    cap: float = 30.0,
    max_elapsed: float = 60.0,
    retryable_statuses=RETRYABLE_STATUSES,
):
    """Call request_fn() (a zero-arg callable returning a requests.Response) with
    full-jitter exponential backoff. Honors a server-supplied Retry-After header
    when present. Immediately raises on any status code outside retryable_statuses,
    since retrying a 404 or a 401 only wastes time.
    """
    start = time.monotonic()
    last_response = None
    for attempt in range(1, max_attempts + 1):
        try:
            response = request_fn()
        except (requests.exceptions.ConnectionError, requests.exceptions.Timeout) as exc:
            logger.warning("attempt %d: network error: %s", attempt, exc)
            response = None
        else:
            last_response = response
            if response.status_code < 400:
                logger.info("attempt %d: succeeded with status %d", attempt, response.status_code)
                return response
            if response.status_code not in retryable_statuses:
                logger.error(
                    "attempt %d: non-retryable status %d, giving up immediately",
                    attempt,
                    response.status_code,
                )
                response.raise_for_status()

        elapsed = time.monotonic() - start
        if attempt == max_attempts or elapsed >= max_elapsed:
            raise RetryExhausted(
                f"gave up after {attempt} attempts ({elapsed:.1f}s elapsed)",
                last_response=last_response,
                attempts=attempt,
                elapsed=elapsed,
            )

        retry_after = last_response.headers.get("Retry-After") if last_response is not None else None
        if retry_after is not None:
            sleep_time = float(retry_after)
            logger.info("attempt %d failed, server set Retry-After: %.1fs", attempt, sleep_time)
        else:
            sleep_time = _full_jitter_sleep(attempt, base, cap)
            logger.info("attempt %d failed, backing off %.2fs (full jitter)", attempt, sleep_time)
        time.sleep(sleep_time)

    raise RetryExhausted(
        "gave up",
        last_response=last_response,
        attempts=max_attempts,
        elapsed=time.monotonic() - start,
    )

call_with_retry() takes a zero-argument callable so it can wrap any HTTP call you already have, checks the response status against the retryable set, honors Retry-After when the server provides one, falls back to full jitter backoff when it does not, and raises a clear RetryExhausted exception carrying the last response and the real attempt count and elapsed time, rather than silently giving up or hanging forever.

Wire it up against all four scenarios you have built so far. Create final_demo.py:

import logging
import time

import requests

from retry_toolkit import RetryExhausted, call_with_retry

logging.basicConfig(level=logging.INFO, format="  %(message)s")

BASE = "http://127.0.0.1:8811"


def scenario_recovers():
    print("\n--- scenario 1: transient failures, then success ---")
    requests.post(f"{BASE}/reset")
    t0 = time.monotonic()
    response = call_with_retry(lambda: requests.get(f"{BASE}/flaky", params={"tag": "final-demo-1"}))
    print(f"RESULT: {response.status_code} {response.json()}  (took {time.monotonic() - t0:.2f}s)")


def scenario_non_retryable():
    print("\n--- scenario 2: 404 fails fast, no retries ---")
    t0 = time.monotonic()
    try:
        call_with_retry(lambda: requests.get(f"{BASE}/orders/999"))
    except requests.exceptions.HTTPError as exc:
        print(f"RESULT: raised immediately as expected: {exc}  (took {time.monotonic() - t0:.2f}s)")


def scenario_retry_after():
    print("\n--- scenario 3: 429 honors Retry-After instead of its own backoff ---")
    t0 = time.monotonic()
    try:
        call_with_retry(lambda: requests.get(f"{BASE}/rate-limited"), max_attempts=3)
    except RetryExhausted as exc:
        print(f"RESULT: exhausted after {exc.attempts} attempts, {exc.elapsed:.1f}s elapsed (~3s per wait, as instructed)")


def scenario_gives_up():
    print("\n--- scenario 4: permanently-down dependency, gives up cleanly ---")
    requests.post(f"{BASE}/reset")
    t0 = time.monotonic()
    try:
        call_with_retry(
            lambda: requests.get(f"{BASE}/always-down", params={"tag": "final-demo-4"}),
            max_attempts=4,
            base=0.3,
        )
    except RetryExhausted as exc:
        print(f"RESULT: raised RetryExhausted after {exc.attempts} attempts, {exc.elapsed:.2f}s elapsed, last status={exc.last_response.status_code}")


if __name__ == "__main__":
    scenario_recovers()
    scenario_non_retryable()
    scenario_retry_after()
    scenario_gives_up()

python final_demo.py

--- scenario 1: transient failures, then success ---
  attempt 1 failed, backing off 0.95s (full jitter)
  attempt 2 failed, backing off 1.66s (full jitter)
  attempt 3 failed, backing off 1.00s (full jitter)
  attempt 4 failed, backing off 5.32s (full jitter)
  attempt 5: succeeded with status 200
RESULT: 200 {'status': 'ok', 'tag': 'final-demo-1', 'attempt': 5}  (took 8.96s)

--- scenario 2: 404 fails fast, no retries ---
  attempt 1: non-retryable status 404, giving up immediately
RESULT: raised immediately as expected: 404 Client Error: Not Found for url: http://127.0.0.1:8811/orders/999  (took 0.00s)

--- scenario 3: 429 honors Retry-After instead of its own backoff ---
  attempt 1 failed, server set Retry-After: 3.0s
  attempt 2 failed, server set Retry-After: 3.0s
RESULT: exhausted after 3 attempts, 6.0s elapsed (~3s per wait, as instructed)

--- scenario 4: permanently-down dependency, gives up cleanly ---
  attempt 1 failed, backing off 0.01s (full jitter)
  attempt 2 failed, backing off 0.60s (full jitter)
  attempt 3 failed, backing off 0.81s (full jitter)
RESULT: raised RetryExhausted after 4 attempts, 1.43s elapsed, last status=503

All four behaviors you built individually now work through one shared function: recovery after transient failures, an immediate failure on a non-retryable status, exact Retry-After compliance instead of guessed backoff, and a clean, informative exception instead of a hang when the dependency never comes back.

Step 10: Verify It With an Automated Test Suite

Hitting a live server manually proves the happy path works once. A test suite proves the logic is correct on every path, every time, without waiting through real sleeps. Create test_retry_toolkit.py, which fakes HTTP responses and patches time.sleep so the whole suite runs in milliseconds:

import requests
import pytest

from retry_toolkit import RetryExhausted, call_with_retry


class FakeResponse:
    def __init__(self, status_code, headers=None, body=None):
        self.status_code = status_code
        self.headers = headers or {}
        self._body = body or {}

    def json(self):
        return self._body

    def raise_for_status(self):
        if self.status_code >= 400:
            raise requests.exceptions.HTTPError(f"{self.status_code} error")


def test_succeeds_after_transient_failures(monkeypatch):
    monkeypatch.setattr("retry_toolkit.time.sleep", lambda s: None)
    calls = []

    def flaky():
        calls.append(1)
        if len(calls) < 3:
            return FakeResponse(503)
        return FakeResponse(200, body={"ok": True})

    response = call_with_retry(flaky, max_attempts=5, base=0.1)
    assert response.status_code == 200
    assert len(calls) == 3


def test_non_retryable_status_raises_immediately(monkeypatch):
    monkeypatch.setattr("retry_toolkit.time.sleep", lambda s: (_ for _ in ()).throw(AssertionError("should not sleep")))
    calls = []

    def not_found():
        calls.append(1)
        return FakeResponse(404)

    with pytest.raises(requests.exceptions.HTTPError):
        call_with_retry(not_found, max_attempts=5)
    assert len(calls) == 1


def test_honors_retry_after_header_over_backoff(monkeypatch):
    sleeps = []
    monkeypatch.setattr("retry_toolkit.time.sleep", lambda s: sleeps.append(s))

    def rate_limited():
        return FakeResponse(429, headers={"Retry-After": "7"})

    with pytest.raises(RetryExhausted) as exc_info:
        call_with_retry(rate_limited, max_attempts=3, base=0.1, cap=1.0)

    assert sleeps == [7.0, 7.0]
    assert exc_info.value.attempts == 3


def test_gives_up_after_max_attempts(monkeypatch):
    monkeypatch.setattr("retry_toolkit.time.sleep", lambda s: None)
    calls = []

    def always_down():
        calls.append(1)
        return FakeResponse(503)

    with pytest.raises(RetryExhausted) as exc_info:
        call_with_retry(always_down, max_attempts=4, base=0.1)
    assert len(calls) == 4
    assert exc_info.value.attempts == 4
    assert exc_info.value.last_response.status_code == 503


def test_connection_error_is_retried(monkeypatch):
    monkeypatch.setattr("retry_toolkit.time.sleep", lambda s: None)
    calls = []

    def flaky_network():
        calls.append(1)
        if len(calls) == 1:
            raise requests.exceptions.ConnectionError("boom")
        return FakeResponse(200)

    response = call_with_retry(flaky_network, max_attempts=3, base=0.1)
    assert response.status_code == 200
    assert len(calls) == 2

pytest test_retry_toolkit.py -v

test_retry_toolkit.py::test_succeeds_after_transient_failures PASSED     [ 20%]
test_retry_toolkit.py::test_non_retryable_status_raises_immediately PASSED [ 40%]
test_retry_toolkit.py::test_honors_retry_after_header_over_backoff PASSED [ 60%]
test_retry_toolkit.py::test_gives_up_after_max_attempts PASSED           [ 80%]
test_retry_toolkit.py::test_connection_error_is_retried PASSED           [100%]

5 passed in 0.07s

The test_honors_retry_after_header_over_backoff test is the one most worth reading twice: it asserts the recorded sleep times are exactly [7.0, 7.0], proving the helper used the server’s stated value rather than computing its own jittered guess. That is the kind of regression a manual test against a live server would probably never catch, since it only shows up if someone later reorders the status-code check and the header check inside the function.

Common Mistakes and Gotchas

• Retrying without jitter and calling it fixed. As Step 4 showed with real data, exponential backoff alone still produces full-sized synchronized spikes; it just spaces them further apart. If your dashboards show periodic load spikes on a downstream service that line up with an earlier outage, this is a common cause.
• Retrying non-idempotent writes. A POST that is not protected by an idempotency key can execute twice if you retry it after a timeout, even though the first attempt actually succeeded and only the response was lost in transit.
• Retrying every status code, including 4xx. Retrying a 401 or a 404 six times with backoff just delays the correct error by however long your backoff schedule takes, and it can mask a real bug in the request itself.
• Capping attempts but not elapsed time. Step 8 showed a single jittered sleep can consume more wall-clock time than a tight budget allows. If this retry loop runs inside a request handler that itself has a timeout, an attempt cap alone will not stop it from blowing past that timeout.
• Forgetting jitter cannot help the very first burst. Every client’s first attempt fires before any backoff logic runs, so a shared trigger (a deploy, a cron job, a cache expiring) will always produce a synchronized first wave no matter how good your jitter is. Jitter only prevents the second wave from re-synchronizing on top of it.
• Losing decorrelated jitter’s state across a refactor. Decorrelated jitter depends on the previous sleep value, not the attempt number. If you refactor the retry loop and accidentally reset that value to base on every iteration instead of carrying it forward, you silently turn it into a plain bounded-random jitter and lose the intended escalation.

How to Confirm It All Works End to End

Before trusting this in anything real, confirm each of these against your own run:

• python herd_experiment.py fixed and exp_no_jitter both show a peak of 12 simultaneous requests with no delay between them; full_jitter and decorrelated_jitter spread that same 12-client load across many more, smaller windows.
• python final_demo.py shows all four scenarios: recovery after failures, an immediate 404 failure with 0.00s elapsed, exact Retry-After compliance, and a clean RetryExhausted instead of a hang.
• pytest test_retry_toolkit.py -v passes all five tests in well under a second, with no live server running.

If any of those do not match, the most common culprit is a stale flaky_service.py process left over from an earlier run holding old attempt counts. Restart it and re-run /reset (both herd_experiment.py and final_demo.py already call it for you) before comparing numbers.

Next Steps

Hand-rolling this once is worth doing, so you know exactly what happens when a library does it for you. In production, reach for a tested implementation rather than maintaining your own:

• urllib3.util.Retry, which powers the HTTPAdapter used by requests, already implements most of this tutorial. Its backoff_factor and backoff_jitter parameters control backoff and randomness, status_forcelist controls which codes get retried, and respect_retry_after_header (on by default) honors the header from Step 6 automatically, for the status codes in its own RETRY_AFTER_STATUS_CODES set (413, 429, and 503 by default, notably not 502 or 504, so check that set matches your own retry policy before relying on it).
• tenacity, a popular standalone retry library, if you want a decorator-based API with pluggable wait strategies and stop conditions instead of wrapping every call by hand.
• This site’s circuit breaker tutorial is the natural companion to this one: backoff and jitter govern how a single call recovers, while a circuit breaker stops calling a dependency altogether once it is clearly down, so you are not still computing (and waiting out) backoff delays against something you already know has failed.
• This site’s rate limiter tutorial covers the same problem from the other side: instead of a client deciding how to back off, a server decides how to shed load before it gets overwhelmed in the first place.