How to Investigate a Legacy Codebase With AI Before You Change It

When you inherit a codebase nobody on your team wrote, the fastest way to break production is to ask an AI coding assistant to clean it up before anyone understands what it actually does. A function full of database calls, magic numbers, and conditionals looks like a mess, but some of that mess encodes real business rules: a compliance threshold, a currency quirk, a downstream system that depends on one exact string value. Refactor first and you can quietly delete the reason the code exists.

This tutorial teaches a different first move: use AI to investigate a codebase before you change anything in it. You will build a small, deliberately messy Python order-processing service, then run it through a repeatable sequence of prompts that map the repository, trace one capability end to end, separate business rules from plumbing, hunt for hidden side effects and duplicated logic, and mark what still isn’t understood. At every step you will check the AI’s answers against the real source code, the test suite, and the git history, because you are about to see, with real transcripts, that a confident, well-formatted AI answer is not automatically a correct one. Twice in this tutorial, checking the AI’s work catches it being wrong.

The workflow is adapted from freeCodeCamp’s How to Understand a Legacy Codebase Using AI Before Changing it, which teaches the same investigate-before-refactor discipline against a TypeScript example. This version rebuilds the exercise from scratch in Python against a local Ollama model, with an original example codebase designed to hide a real bug, a real case of duplicated logic that has quietly drifted apart, and a real piece of buried business context. Every prompt and AI response shown below is a genuine, captured transcript, not a paraphrase, including the parts where the model gets something wrong.

What You Will Learn

• Why understanding has to come before refactoring, and what concretely goes wrong when you skip it
• A sequence of investigation prompts you can reuse on any unfamiliar codebase
• How to verify an AI’s claims against source code, tests, and git blame, including two real cases where that verification catches the AI being wrong
• How to turn what you learn into a small, evidence-backed fix, with a regression test proving it

Prerequisites

• Comfortable reading Python and basic SQL
• Git installed, and a basic understanding of what git log and git blame show you
• Python 3.11 or newer, with pytest installed (pip install pytest)
• Ollama installed and running locally, with a model pulled. This tutorial uses qwen3.5:4b (ollama pull qwen3.5:4b), but the workflow does not depend on this specific model. Any local or hosted AI assistant that can read pasted code works the same way, which is the entire point of the exercise: the discipline matters more than the model
• No Docker, cloud account, or paid API key is required. Everything in this tutorial runs on your own machine

Step 1: Build an Intentionally Messy Example Codebase

To make this exercise honest, you need code with real problems in it, not a toy example that is already clean. You are going to build a small order-processing service for an online store. It calculates order totals, applies a currency-specific rounding rule, applies a loyalty discount, flags some orders for manual review, writes to a database, and notifies the customer. A separate fulfillment process reads from the same database later.

Create the project and initialize git:

mkdir -p legacy-archaeology-demo/src/orders
mkdir -p legacy-archaeology-demo/tests
mkdir -p legacy-archaeology-demo/data
cd legacy-archaeology-demo
git init
git config user.email "[email protected]"
git config user.name "Your Name"
printf "data/*.db\n__pycache__/\n*.pyc\n.pytest_cache/\n" > .gitignore
touch src/__init__.py src/orders/__init__.py tests/__init__.py
git add .gitignore
git commit -m "Add gitignore"

The Parts That Do Not Change

Create src/orders/pricing.py:

def calculate_total(items):
    return sum(item["unit_price"] * item["qty"] for item in items)

Create src/orders/notifications.py:

def notify_customer(customer_id, message):
    # In production this would call an email/SMS provider. Logged here instead.
    print(f"[notify] to customer {customer_id}: {message}")

Create tests/test_orders.py with the first five tests. You will add a sixth test later, once the investigation earns it:

import os
import sys

sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))

import pytest
from orders.service import process_order, init_db


@pytest.fixture
def db_path(tmp_path):
    path = str(tmp_path / "orders.db")
    init_db(path)
    return path


def test_jpy_orders_round_to_nearest_100(db_path):
    order = {
        "id": "o1",
        "customer_id": "c1",
        "country": "JP",
        "currency": "JPY",
        "items": [{"sku": "X", "qty": 1, "unit_price": 1234}],
    }
    result = process_order(order, db_path=db_path)
    assert result["total"] == 1200


def test_loyalty_discount_requires_minimum_total(db_path):
    order = {
        "id": "o2",
        "customer_id": "c2",
        "country": "US",
        "currency": "USD",
        "customer_loyalty_years": 5,
        "items": [{"sku": "X", "qty": 1, "unit_price": 50}],
    }
    result = process_order(order, db_path=db_path)
    # total is 50, below the 100 floor, so no discount applies
    assert result["total"] == 50


def test_loyalty_discount_applies_above_floor(db_path):
    order = {
        "id": "o3",
        "customer_id": "c3",
        "country": "US",
        "currency": "USD",
        "customer_loyalty_years": 5,
        "items": [{"sku": "X", "qty": 1, "unit_price": 200}],
    }
    result = process_order(order, db_path=db_path)
    assert result["total"] == 190.0


def test_orders_over_5000_outside_us_go_to_review(db_path):
    order = {
        "id": "o4",
        "customer_id": "c4",
        "country": "DE",
        "currency": "USD",
        "items": [{"sku": "X", "qty": 1, "unit_price": 6000}],
    }
    result = process_order(order, db_path=db_path)
    assert result["status"] == "PENDING_REVIEW"


def test_orders_over_5000_inside_us_are_approved(db_path):
    order = {
        "id": "o5",
        "customer_id": "c5",
        "country": "US",
        "currency": "USD",
        "items": [{"sku": "X", "qty": 1, "unit_price": 6000}],
    }
    result = process_order(order, db_path=db_path)
    assert result["status"] == "APPROVED"

Build service.py the Way Real Legacy Code Accumulates

This is the file you will spend most of this tutorial investigating. Instead of writing the finished version in one shot, build it up across several real commits with real commit messages, the way it would actually happen at a company over a couple of years. That history is not set dressing: it is what makes Step 10 of this tutorial possible.

Start with the base version: calculate a total, save it, notify the customer.

import sqlite3
import logging
from datetime import datetime, timezone

from .pricing import calculate_total
from .notifications import notify_customer

logger = logging.getLogger(__name__)

DB_PATH = "data/orders.db"


def init_db(db_path=DB_PATH):
    conn = sqlite3.connect(db_path)
    conn.execute(
        """
        CREATE TABLE IF NOT EXISTS orders (
            id TEXT PRIMARY KEY,
            customer_id TEXT,
            total REAL,
            currency TEXT,
            status TEXT,
            created_at TEXT
        )
        """
    )
    conn.commit()
    conn.close()


def process_order(order, db_path=DB_PATH):
    total = calculate_total(order["items"])
    status = "APPROVED"

    conn = sqlite3.connect(db_path)
    conn.execute(
        "INSERT OR REPLACE INTO orders (id, customer_id, total, currency, status, created_at) "
        "VALUES (?, ?, ?, ?, ?, ?)",
        (
            order["id"],
            order["customer_id"],
            total,
            order["currency"],
            status,
            datetime.now(timezone.utc).isoformat(),
        ),
    )
    conn.commit()
    conn.close()

    notify_customer(order["customer_id"], f"Order {order['id']} approved for {total} {order['currency']}")

    logger.info("ORDER_PROCESSED order_id=%s status=%s total=%s", order["id"], status, total)

    return {"order_id": order["id"], "status": status, "total": total}

git add src/orders/pricing.py src/orders/notifications.py src/orders/service.py \
    src/__init__.py src/orders/__init__.py tests/test_orders.py tests/__init__.py
git commit -m "Initial order processing: totals, persistence, notification"

A few weeks later, someone adds a currency-specific rounding rule. Edit process_order so it starts like this:

def process_order(order, db_path=DB_PATH):
    total = calculate_total(order["items"])

    if order["currency"] == "JPY":
        total = round(total / 100) * 100

    status = "APPROVED"

git commit -am "Round JPY totals to the nearest 100: yen has no minor currency unit"

Then a loyalty discount ships:

    if order["currency"] == "JPY":
        total = round(total / 100) * 100

    if order.get("customer_loyalty_years", 0) >= 3 and total >= 100:
        total = total * 0.95

    status = "APPROVED"

git commit -am "Add 5 percent loyalty discount for 3+ year customers (Growth team request, ticket GRW-118). Floor of \$100 keeps the discount from applying to trivial test orders."

Then a manual-review rule is added for certain high-value orders:

    if order.get("customer_loyalty_years", 0) >= 3 and total >= 100:
        total = total * 0.95

    if total > 5000 and order["country"] != "US":
        status = "PENDING_REVIEW"
    else:
        status = "APPROVED"

git commit -am "Flag high-value international orders for manual review per Compliance (SOX finding JIRA-4821). US orders are exempt because they already pass through the separate KYC pipeline before reaching this service."

Finally, someone adds a small in-memory cache for an admin debug tool:

logger = logging.getLogger(__name__)

DB_PATH = "data/orders.db"

_RECENT_TOTALS_CACHE = {}

    if order.get("customer_loyalty_years", 0) >= 3 and total >= 100:
        total = total * 0.95

    _RECENT_TOTALS_CACHE[order["id"]] = total

    if total > 5000 and order["country"] != "US":
        status = "PENDING_REVIEW"
    else:
        status = "APPROVED"

git commit -am "Cache last computed total per order id for the admin recalculate-total tool"

At this point src/orders/service.py should look like this in full:

import sqlite3
import logging
from datetime import datetime, timezone

from .pricing import calculate_total
from .notifications import notify_customer

logger = logging.getLogger(__name__)

DB_PATH = "data/orders.db"

_RECENT_TOTALS_CACHE = {}


def init_db(db_path=DB_PATH):
    conn = sqlite3.connect(db_path)
    conn.execute(
        """
        CREATE TABLE IF NOT EXISTS orders (
            id TEXT PRIMARY KEY,
            customer_id TEXT,
            total REAL,
            currency TEXT,
            status TEXT,
            created_at TEXT
        )
        """
    )
    conn.commit()
    conn.close()


def process_order(order, db_path=DB_PATH):
    total = calculate_total(order["items"])

    if order["currency"] == "JPY":
        total = round(total / 100) * 100

    if order.get("customer_loyalty_years", 0) >= 3 and total >= 100:
        total = total * 0.95

    _RECENT_TOTALS_CACHE[order["id"]] = total

    if total > 5000 and order["country"] != "US":
        status = "PENDING_REVIEW"
    else:
        status = "APPROVED"

    conn = sqlite3.connect(db_path)
    conn.execute(
        "INSERT OR REPLACE INTO orders (id, customer_id, total, currency, status, created_at) "
        "VALUES (?, ?, ?, ?, ?, ?)",
        (
            order["id"],
            order["customer_id"],
            total,
            order["currency"],
            status,
            datetime.now(timezone.utc).isoformat(),
        ),
    )
    conn.commit()
    conn.close()

    notify_customer(order["customer_id"], f"Order {order['id']} approved for {total} {order['currency']}")

    logger.info("ORDER_PROCESSED order_id=%s status=%s total=%s", order["id"], status, total)

    return {"order_id": order["id"], "status": status, "total": total}

Now add the two other modules that round out the system. First, code nobody has touched in a long time, kept around for a nightly batch job:

Create src/legacy_utils.py:

def apply_legacy_discount(customer_loyalty_years, total):
    """Used by the nightly batch reconciliation job (batch_reconcile.py, retired 2024).

    Kept here because a support script still imports it.
    """
    if customer_loyalty_years > 3:
        return total * 0.95
    return total

git add src/legacy_utils.py
git commit -m "Add apply_legacy_discount for the nightly batch reconciliation job"

Second, a separate process (imagine it runs on its own schedule, in its own container) that reads orders back out of the same database:

Create src/fulfillment.py:

import sqlite3


def get_orders_ready_for_fulfillment(db_path="data/orders.db"):
    conn = sqlite3.connect(db_path)
    cur = conn.execute("SELECT id, customer_id, total FROM orders WHERE status = 'APPROVED'")
    rows = cur.fetchall()
    conn.close()
    return rows

git add src/fulfillment.py
git commit -m "Add fulfillment center polling job"

Finally, add a command-line entry point and commit everything you have built:

Create src/cli.py:

import argparse
import json

from orders.service import process_order, init_db


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("order_json")
    args = parser.parse_args()

    init_db()
    order = json.loads(args.order_json)
    result = process_order(order)
    print(json.dumps(result))


if __name__ == "__main__":
    main()

git add src/cli.py
git commit -m "Add CLI entry point for manual order processing"
git add tests/test_orders.py tests/__init__.py
git commit -m "Add regression tests for order processing"

Run the suite once to confirm you are starting from a healthy state:

$ python -m pytest -v
tests/test_orders.py::test_jpy_orders_round_to_nearest_100 PASSED        [ 20%]
tests/test_orders.py::test_loyalty_discount_requires_minimum_total PASSED [ 40%]
tests/test_orders.py::test_loyalty_discount_applies_above_floor PASSED   [ 60%]
tests/test_orders.py::test_orders_over_5000_outside_us_go_to_review PASSED [ 80%]
tests/test_orders.py::test_orders_over_5000_inside_us_are_approved PASSED [100%]

5 passed in 0.13s

Confirm the CLI works too:

$ python src/cli.py '{"id": "o100", "customer_id": "c100", "country": "DE", "currency": "USD", "customer_loyalty_years": 4, "items": [{"sku": "WIDGET", "qty": 3, "unit_price": 45}]}'
[notify] to customer c100: Order o100 approved for 128.25 USD
{"order_id": "o100", "status": "APPROVED", "total": 128.25}

You now have a small, working system with ten real commits behind it and five passing tests. From here on, pretend you just joined the team and this is the first time you have opened this repository. Do not re-read the commits you just wrote; the rest of this tutorial is about recovering that context using AI, and then checking whether the AI actually recovered it correctly.

Step 2: Map the Repository Before You Read Any Code

The first instinct when you open an unfamiliar repository is to start opening files. Resist it for one more step. Give the AI just the file listing, with no source code at all, and ask it to guess the architecture. This does two things: it gives you a fast orientation, and it tells you what the AI does not yet know, which is useful for spotting later when it starts guessing instead of reading.

Prompt (paraphrased for space; the exact prompt text is in the repo): Given only the repository’s file listing (no source code), what appear to be the main architectural layers, which files likely hold the core business logic, which look like entry points, and what questions should be answered before touching any of this code?

Model response (qwen3.5:4b, real captured output):

Based on the file listing provided:

1.  **Architectural Layers/Responsibilities**
    *   **Entry Point:** `src/cli.py` (handles command-line execution).
    *   **Domain/Business Logic:** Likely split between `src/orders/service.py` and `src/fulfillment.py`. The presence of a dedicated service layer suggests an MVC or similar pattern.
    *   **Infrastructure/Miscellaneous:** `src/legacy_utils.py` (likely contains deprecated code, helper functions, or glue logic).
    *   **Notifications/Pricing:** Specific business capabilities within the orders domain (`src/orders/pricing.py`, `src/orders/notifications.py`).

2.  **Core Business Logic**
    The most likely candidates are:
    *   `src/orders/service.py` (Central orchestration for order processing).
    *   `src/fulfillment.py` (Logic related to shipping, inventory, or delivery status).
    *   `src/orders/pricing.py` and `src/orders/notifications.py` (Specific rules within the domain).

3.  **Entry Points**
    *   `src/cli.py`: This is almost certainly where execution starts when running commands via terminal (`python -m src.cli ...`).

4.  **Questions Before Touching Code**
    *   What specific business entities exist? (e.g., Orders, Customers, Products). The code references "orders" and "fulfillment," but the data model is unknown.
    *   How does `legacy_utils.py` interact with modern logic? Is it a migration bridge or just unused historical code?
    *   What are the valid states for an order in this system? (e.g., Pending, Shipped, Cancelled).
    *   Are there external dependencies defined elsewhere that aren't visible here (e.g., database drivers, payment gateways)?

The model correctly guessed that cli.py is the entry point and that service.py is central, and it asked a genuinely useful clarifying question about legacy_utils.py: is it a migration bridge, or just unused historical code? Hold onto that question. You will get a real answer to it in Step 8, and it will not be the answer you expect.

Step 3: Verify the Entry Point Yourself

Before trusting the AI’s guess about where execution starts, check it with a plain search. This is fast, and it establishes a habit you should keep for the rest of this tutorial: when a claim is cheap to verify directly, verify it directly instead of asking the AI a follow-up question.

$ grep -rn "__main__\|def main\|argparse" src/
src/cli.py:1:import argparse
src/cli.py:7:def main():
src/cli.py:8:    parser = argparse.ArgumentParser()
src/cli.py:18:if __name__ == "__main__":

Confirmed: there is exactly one entry point, src/cli.py. The AI’s structural guess was right. Its guess about the exact invocation syntax in Step 2 was slightly off, which is a small, harmless example of the same lesson: even a correct-sounding detail is worth a second look when it costs you ten seconds to check.

Step 4: Trace One Capability End to End

Now give the AI the real source: cli.py, service.py, pricing.py, and notifications.py. Ask it to trace a single capability, processing one order, from the entry point through every function it touches, listing the file, function, input, output, state change, external call, and error behavior at each step. This is the single most useful prompt in this whole workflow, because it forces the AI (and you) to slow down to one step at a time instead of summarizing.

Prompt (paraphrased for space; the exact prompt text is in the repo): Trace the “process an order” capability from its entry point (src/cli.py) through every function it calls, until all observable side effects are complete. Number each step and show file, function, input, output, state change, external call, and error behavior for each one. Do not summarize multiple steps into one, and do not propose any refactor.

Model response (qwen3.5:4b, real captured output):

Here is the trace of the "process order" capability from entry point through completion:

### Step 1
*   **File**: `src/cli.py`
*   **Function**: `main()`
*   **Input**: Command line arguments containing a JSON string for an order (e.g., `"order_json": '{"id":"ORD001", ...}'`).
*   **Output**: None.
*   **State Change**: No internal state change; prepares execution environment.
*   **External Call**: `argparse` module to parse arguments.
    *   Reads standard input/command line for the JSON string.
*   **Error Behavior**: If argument parsing fails or no order_json is provided, an error message from argparse is printed and program exits with non-zero status before reaching this step's logic.

### Step 2
*   **File**: `src/cli.py`
*   **Function**: `main()`
*   **Input**: None (internal call).
*   **Output**: Returns control to the caller of `init_db`.
*   **State Change**: Initializes database schema if it does not exist. Creates file `data/orders.db` with an `orders` table containing columns: id, customer_id, total, currency, status, created_at.
*   **External Call**: SQLite3 (`sqlite3.connect`, `execute`, `commit`). Connects to local filesystem path `data/orders.db`.
*   **Error Behavior**: If the directory `data/` does not exist or disk write fails, a `FileNotFoundError` or `OSError` is raised by sqlite3.

### Step 3
*   **File**: `src/cli.py`
*   **Function**: `main()`
*   **Input**: String argument from command line (e.g., `'{"id":"ORD001", "customer_id":"CUST123", ...}'`).
*   **Output**: Python dictionary representing the order object.
*   **State Change**: None.
*   **External Call**: `json` module (`json.loads`). Parses JSON string into native Python data structures (dict, list).
*   **Error Behavior**: If the input is not valid JSON or missing required keys referenced later in `process_order`, a `ValueError` or `KeyError` will be raised when passed to Step 4.

### Step 4
*   **File**: `src/cli.py`
*   **Function**: `main()`
*   **Input**: Dictionary object representing the order (from Step 3).
*   **Output**: None (internal call).
*   **State Change**: No state change; delegates processing logic.
*   **External Call**: Calls into module `orders.service`. Specifically invokes function `process_order` passing the dictionary and default DB path.

### Step 5
*   **File**: `src/orders/service.py`
*   **Function**: `process_order()`
*   **Input**: Order dictionary (e.g., `{'id': 'ORD001', ...}`).
*   **Output**: None (internal call).
*   **State Change**: Updates internal module-level variable `_RECENT_TOTALS_CACHE`. Key: order ID, Value: calculated total.
*   **External Call**: Calls into submodule `orders.pricing`, specifically function `calculate_total` passing the list of items from the order dictionary.

### Step 6
*   **File**: `src/orders/pricing.py`
*   **Function**: `calculate_total()`
*   **Input**: List of item dictionaries (e.g., `[{'unit_price': 10, 'qty': 2}, ...]`).
*   **Output**: Float representing the sum of unit price multiplied by quantity.
    *   Calculation: $\sum (\text{item}["\text{unit\_price}") \times \text{item}["\text{qty"}])$.
*   **State Change**: None (pure function).
*   **External Call**: None. Reads data from the input list provided by `service.py`.

### Step 7
*   **File**: `src/orders/service.py`
*   **Function**: `process_order()`
*   **Input**: Float total calculated in Step 6; Order dictionary (from Step 5).
*   **Output**: None.
*   **State Change**: Modifies the variable `total`. Applies rounding logic if currency is JPY, applies discount if loyalty $\ge$ 3 years and total $\ge$ 100. Updates `_RECENT_TOTALS_CACHE` with new value (Step 5 was for initial calc).
    *   Logic: If condition met, $total = \text{round}(total / 100) \times 100$. Else if loyalty/amount conditions met, $total = total \times 0.95$.
*   **External Call**: None (pure logic based on input).

### Step 8
*   **File**: `src/orders/service.py`
*   **Function**: `process_order()`
*   **Input**: Order dictionary; calculated final `total`.
*   **Output**: String status ("PENDING_REVIEW" or "APPROVED").
*   **State Change**: None. Determines logic for database insertion based on total amount and country.
    *   If $total > 5000$ AND country $\neq$ "US": `status` = "PENDING_REVIEW".
    *   Else: `status` = "APPROVED".
*   **External Call**: None (pure logic).

### Step 9
*   **File**: `src/orders/service.py`
*   **Function**: `process_order()`
*   **Input**: Order dictionary, final status string, final total float.
*   **Output**: None.
*   **State Change**: Writes data to SQLite database file `data/orders.db`. Inserts or replaces a row with: id, customer_id, calculated_total, currency, determined_status, current UTC timestamp (ISO format). Commits transaction and closes connection immediately after write.
    *   Database State: Row added/updated in table `orders`.
*   **External Call**: SQLite3 (`sqlite3.connect`, `execute` with parameterized query, `commit`). Writes to filesystem via database engine.

### Step 10
*   **File**: `src/orders/service.py`
*   **Function**: `process_order()`
*   **Input**: Order dictionary (specifically customer_id). Message string constructed from order ID and total/currency.
    *   Example message: `"Order ORD001 approved for 500 JPY"`.
*   **Output**: None.
*   **State Change**: Prints to standard output stream prefixed with `[notify]`. This is the only observable side effect outside of database writing and logging in this step.
    *   Console Output: `to customer CUST123: Order ORD001 approved for 500 JPY` (formatted as per code).
*   **External Call**: Standard output stream (`print`). Simulates external notification but logs to console instead of calling an email/SMS provider.

### Step 11
*   **File**: `src/orders/service.py`
*   **Function**: `process_order()`
*   **Input**: Order dictionary, status string, total float (from previous steps).
*   **Output**: None.
*   **State Change**: Logs an informational message to the logging system configured for this module.
    *   Log Output: `[INFO] ORDER_PROCESSED order_id=ORD001 status=APPROVED total=500` (formatted as per code).
*   **External Call**: Python `logging` module (`logger.info`). Writes log entry to standard error or configured file handler.

### Step 12
*   **File**: `src/orders/service.py`
*   **Function**: `process_order()`
*   **Input**: Order dictionary (from initial call).
*   **Output**: Dictionary result object: `{"order_id": "ORD001", "status": "APPROVED", "total": 500}`.
*   **State Change**: None. Returns control to caller (`cli.py`).
*   **External Call**: None.

### Step 13
*   **File**: `src/cli.py`
*   **Function**: `main()`
*   **Input**: Dictionary result object from `process_order`.
*   **Output**: JSON string representation of the dictionary (e.g., `'{"order_id": "ORD001", ...}'`).
*   **State Change**: None. Formats output for user consumption.
*   **External Call**: `json` module (`json.dumps`). Converts Python dict to compact JSON string.

### Step 14
*   **File**: `src/cli.py`
*   **Function**: `main()`
*   **Input**: JSON string from previous step.
*   **Output**: None (internal call).
*   **State Change**: Writes formatted result to standard output stream, visible on terminal/console.
    *   Console Output: The raw JSON object returned by the service.
*   **External Call**: Standard output stream (`print`). Displays final result to user.

### Step 15
*   **File**: `src/cli.py`
*   **Function**: `main()` (Execution End)
*   **Input**: None.
*   **Output**: Program terminates successfully with exit code 0.
*   **State Change**: All observable side effects complete: Database updated, Console notification printed, Log entry written, Final JSON output displayed to user.
*   **External Call**: OS process termination signal sent back to shell/parent process.

This took just over two minutes on a local 4B model and produced 15 numbered steps. It is detailed and, on a first read, convincing. Keep it open. In Step 6 you will ask a narrower, more pointed question about one of the side effects it describes here, and the answer will not quite match what this trace implies.

Step 5: Separate Business Rules From Infrastructure

Now zoom into service.py alone and ask the AI to label every block as either a business rule (a decision that reflects how the company operates) or infrastructure (persistence, logging, caching mechanics that would look the same at any company). This split matters because business rules are the part you cannot safely infer or invent during a refactor. Infrastructure code you can usually rewrite freely.

Prompt (paraphrased for space; the exact prompt text is in the repo): Analyze process_order() without proposing a refactor. Label every block of code as either BUSINESS RULE or INFRASTRUCTURE, with a one-line quote of the code each label applies to.

Model response (qwen3.5:4b, real captured output):

### **INFRASTRUCTURE**
> `import sqlite3`, `logger = logging.getLogger(__name__)`, `_RECENT_TOTALS_CACHE` (caching mechanism), and the explicit database connection logic (`conn.execute(...)`) represent infrastructure components that handle persistence, logging, and caching mechanics independent of business logic.

### **BUSINESS RULE**
> The conditional checks for currency conversion rounding (`if order["currency"] == "JPY"`), loyalty discount eligibility (`>= 3` years AND `total >= 100`), status determination based on amount/country thresholds (`> 5000` and country != US), and the final notification message reflect decisions defining how the company operates its pricing, customer rewards, risk management, and communication policies.

This response is shorter and coarser than the instructions asked for: it groups things by category instead of labeling each block individually. That is a realistic outcome, not a scripted one. Local models do not always follow formatting instructions precisely. When that happens, either re-prompt with a stricter instruction, or, as here, treat the coarser answer as directionally correct and confirm the details yourself against the source, which is exactly what the rest of this tutorial keeps doing.

Step 6: Find Hidden Side Effects

A side effect is anything a function does besides compute and return a value. The dangerous ones are the ones a return-value-focused reader would never notice. Ask directly for this list, with instructions to mark anything uncertain as unknown rather than guessing.

Prompt (paraphrased for space; the exact prompt text is in the repo): List every observable side effect produced directly or indirectly by process_order(). For each one, give the side effect, where it happens, whether it is synchronous, whether an earlier failure would prevent it, and whether it is easy to miss on a first read. Mark uncertain answers as unknown.

Model response (qwen3.5:4b, real captured output):

Here is the list of observable side effects produced by `process_order()`, analyzed based on your criteria.

### 1. Side Effect: Database Table Creation (Schema Initialization)
*   **Description:** Creates or ensures the existence of an `orders` table in SQLite with specific columns (`id`, `customer_id`, etc.). If the database file does not exist, this creates it; if it exists but lacks the schema, it updates it.
*   **Location:** Inside `init_db()` (lines 20–31). Note: This function is called *before* `process_order` in typical usage patterns, so strictly speaking, `process_order` does not call this. However, if `DB_PATH` points to a new file and the caller hasn't run `init_db`, the database creation happens implicitly when `sqlite3.connect()` attempts to open/write to it (depending on SQLite flags) or fails immediately upon write attempt.
    *   *Correction for strict code analysis:* Looking strictly at the provided snippet, **`process_order` does not call `init_db`.** Therefore, this side effect is **not produced directly by `process_order()`**. It relies on an external prerequisite step (`init_db`) or assumes the table exists. If we assume standard execution where tables might be missing:
    *   **Revised Location:** Implicitly triggered when executing the `INSERT OR REPLACE` statement (lines 46–53) if the database file is new and no schema was pre-created, causing SQLite to fail with an error rather than creating a table automatically. Thus, there is **no successful side effect** of "creating the table" inside this function alone unless `init_db` was called externally first.
    *   **Decision:** I will exclude this from the list as it requires external invocation or results in failure (not a success state).

### 2. Side Effect: Cache Population (`_RECENT_TOTALS_CACHE`)
*   **Description:** Stores the calculated total price for the specific order ID into a module-level dictionary cache, potentially causing memory growth over time and preventing future calculations from being fresh if not cleared.
*   **Location:** Line 35 (`_RECENT_TOTALS_CACHE[order["id"]] = total`).
*   **Synchronous?** Yes. It blocks until the assignment completes (which is instantaneous).
*   **Failure Prevention?** No. Even if `conn.commit()` fails later, this line executes successfully first. However, failure in *earlier steps* (like network latency for external services) does not affect it since no I/O occurs here yet. Failure of a hypothetical "cache clearing" step would prevent the cache from being useful, but that is outside this function's scope.
    *   **Failure Prevention:** No earlier step within `process_order` prevents this; in fact, this happens before any potential failure points later in the flow (except if an exception jumps out).
*   **Easy to Miss?** Yes. It modifies a global state object (`_RECENT_TOTALS_CACHE`) rather than returning data or modifying local variables only. A reader focusing on input/output might miss that memory is being consumed and polluted with stale data for other orders processed later in the same process lifetime.

### 3. Side Effect: Database Write (Order Insertion/Update)
*   **Description:** Writes a new row to the `orders` table or updates an existing one, persisting the order details including status and timestamp. This involves disk I/O and locking mechanisms within SQLite.
*   **Location:** Lines 46–53 (`conn.execute(...)`, followed by commit). Specifically triggered inside the block starting at line 42 (after `status` determination) but executed after connecting to DB on line 44.
    *   *Note:* The code connects, executes, commits, and closes before notifying. This is a distinct side effect from the cache population.
*   **Synchronous?** Yes. It waits for SQLite to commit the transaction (write to disk/transaction log) before proceeding to `notify_customer`. If this step fails or times out, execution stops here.
*   **Failure Prevention?** Yes. Failure in earlier steps would prevent it:
    *   If `calculate_total()` raises an exception, line 35 runs but subsequent lines (including DB write) are skipped due to the jump-out of control flow.
    *   If `order["id"]` or other required fields are missing/invalidated by a prior check not shown here, it might fail during execution. But specifically, if an earlier step in this function failed (e.g., calculation error), this side effect is prevented from happening at all.
*   **Easy to Miss?** Yes. The return value only contains the order ID and status; the actual persistence of data into `orders.db` is not reflected in the output arguments or local variables visible immediately after execution returns.

### 4. Side Effect: Customer Notification (`notify_customer`)
*   **Description:** Sends a notification to the customer (potentially via email, SMS, push, etc.). This involves network I/O and external service interaction.
*   **Location:** Line 56 (`notify_customer(...)`). Note that this call happens *after* `conn.close()` on line 54.
    *   *Critical Observation:* The code structure is: Calculate -> Cache -> Determine Status -> Write DB -> Close Conn -> Notify Customer. This means the notification relies on the order being successfully written to the database (or at least connected), but it does not wait for the external service's confirmation of delivery.
*   **Synchronous?** Partially/Yes regarding execution flow, No regarding completion guarantee. The function returns immediately after calling `notify_customer`. It waits for the *call* to return from Python's perspective, which usually implies waiting for the network call to complete (unless the library is asynchronous and non-blocking). Assuming standard blocking behavior: Yes, it blocks until the notification service responds or times out.
    *   **Refinement:** If `notify_customer` uses a fire-and-forget pattern internally, this side effect happens but doesn't block. However, based on "observable," we assume it attempts to send. Let's mark as Synchronous regarding function control flow (it waits for the call result), though not necessarily waiting for external confirmation if the library is async.
*   **Failure Prevention?** Yes. Failure in earlier steps prevents this:
    *   If `calculate_total()` fails, execution stops before reaching line 56.
    *   If `conn.commit()` or `conn.close()` raises an exception (e.g., disk full), the function exits via try/except logic not shown but implied by standard Python behavior; if no explicit error handling is present in this snippet and an unhandled exception occurs, execution stops here. Even without exceptions, logical flow prevents it: earlier steps must complete for code to reach line 56.
*   **Easy to Miss?** Yes. The function returns a dictionary with order info (`{"order_id": ..., "status": ...}`). It does not return the notification status (success/failure/delay). A reader might assume the transaction is atomic and complete, missing that an external network call has been initiated which could fail silently or take time to resolve outside this function's scope.

### 5. Side Effect: Logging (`logger.info`)
*   **Description:** Writes a log entry containing order ID, status, and total to the configured logging backend (e.g., file, console). This involves writing to disk/network depending on the logger configuration.
*   **Location:** Line 59 (`logger.info(...)`).
    *   *Timing:* Like `notify_customer`, this happens after DB operations but before return. It is a side effect of execution flow.
*   **Synchronous?** Yes (typically). Logging frameworks usually flush to disk or buffer; if configured for immediate flushing, it blocks until the log entry is written. Even with buffering, it executes synchronously in the Python call stack.
*   **Failure Prevention?** Yes. As noted above, any exception raised before line 59 prevents this side effect from occurring (e.g., calculation error).
*   **Easy to Miss?** Moderate/Yes. Log messages are often considered "debug" or informational noise rather than core data flow by casual readers who focus on return values and main logic branches. The log level is `INFO`, which might be filtered out in production environments where only errors (`ERROR`) are visible, making the side effect effectively invisible to an observer unless they check logs specifically.

### Summary Table

| Side Effect | Location (Line/Block) | Synchronous? | Prevented by earlier failure? | Easy to Miss? |
| :--- | :--- | :--- | :--- | :--- |
| **Cache Update** (`_RECENT_TOTALS_CACHE`) | Line 35 | Yes | No (within this function's logic flow) | **Yes** (Global state mutation, no return value change) |
| **DB Write** (`INSERT OR REPLACE`) | Lines 46–53 | Yes | **Yes** (Calculation or prior steps failing stops execution here) | **Yes** (Return dict omits persistence confirmation) |
| **Notification Send** (`notify_customer`) | Line 56 | Partially/Yes* | **Yes** (Execution flow dependency on earlier lines) | **Yes** (No return value indicating success/failure of send) |
| **Logging Entry** (`logger.info`) | Line 59 | Yes | **Yes** (Execution flow dependency) | **Moderate** (Often filtered out in production, low priority signal) |

*\*Note on Notification Synchronicity:* If `notify_customer` is an async library call that returns immediately without waiting for the external service to finish sending, then it is not synchronous. However, based strictly on the Python code provided (`func()`), execution pauses until `notify_customer(...)` returns from its internal implementation. Without knowing the specific implementation of `.notifications.notify_customer`, we assume standard blocking behavior or at least that the function call completes before returning control to `process_order`.

This is a strong, careful answer covering the cache write, the database insert, the customer notification, and the log line, each with a real assessment of how easy it is to miss. Hold onto its description of the notification step specifically. It says the notification “relies on the order being successfully written to the database” and does not mention any other condition on when it fires. Keep that description in mind for Step 7.

Step 7: Discover Implicit Contracts Between Modules

An implicit contract is an assumption one part of the system makes about another, with nothing (no shared constant, type, or schema check) enforcing it. These are the most dangerous findings in this entire workflow, because breaking one produces no error message. It just produces wrong behavior somewhere else, later, that nothing points back to the change that caused it.

service.py writes rows to a shared SQLite table. fulfillment.py, a separate process, reads from that same table later. Give the AI both files and ask what one assumes about the other that is not enforced anywhere.

Prompt (paraphrased for space; the exact prompt text is in the repo): service.py writes order rows to a shared SQLite table. fulfillment.py reads from that table later, in a separate process. Identify every implicit contract between them: anything fulfillment.py assumes about the data service.py produces that is not enforced by a shared constant, type, or schema check. For each one, explain what breaks if service.py changes without updating fulfillment.py.

Model response (qwen3.5:4b, real captured output):

Based on the code provided in `service.py` and `fulfillment.py`, here are every implicit contract between them where assumptions exist but are not enforced by explicit constants, type hints, or schema validation.

### 1. The "JPY" Currency Rounding Assumption
*   **The Contract:** `fulfillment.py` assumes that the `total` column in the database will always be a clean integer (or at least representable without floating-point noise) because it performs arithmetic on these values later. Specifically, if you calculate $10 \times 2 = 20$, Python's float representation might store this as `20.000000000000004`.
*   **Why it exists:** In `service.py`, there is logic: `if order["currency"] == "JPY": total = round(total / 100) * 100`. This forces JPY totals to be multiples of 100. However, for other currencies (e.g., USD), the code does **not** enforce rounding or integer conversion before insertion.
*   **Consequence if `service.py` changes:** If a future version removes the specific currency check but keeps floating-point math elsewhere, or if it introduces new logic that results in non-integer floats for other currencies:
    *   When `fulfillment.py` calculates discounts (e.g., applying 10% off), it might get unexpected precision errors.
    *   If `fulfillment.py` tries to sort orders by total, floating-point inconsistencies could cause incorrect ordering of items that should be equal.

### 2. The "Status" String Exact Match Assumption
*   **The Contract:** `fulfillment.py` assumes the string `'APPROVED'` is exactly what will appear in the database for approved orders. It uses this exact literal to filter rows: `WHERE status = 'APPROVED'`.
*   **Why it exists:** In `service.py`, the logic determines status based on conditions (`total > 5000 and country != "US"`). If these conditions are met, it sets `status = "PENDING_REVIEW"`. Otherwise, it defaults to `"APPROVED"`. There is no validation ensuring that *only* approved orders exist or that the string case matches exactly.
*   **Consequence if `service.py` changes:**
    *   If a bug causes `service.py` to set status to `'approved'` (lowercase) instead of `'APPROVED'`, `fulfillment.py` will return an empty list, effectively breaking the fulfillment pipeline for those orders.
    *   If `service.py` adds new statuses like `"READY_TO_SHIP"` and decides that only these should be fulfilled, `fulfillment.py` (which still looks for `'APPROVED'`) would miss them entirely unless updated to look for multiple values or a superset string.

### 3. The "Customer Loyalty" Logic Dependency
*   **The Contract:** `service.py` implements complex business logic regarding customer loyalty (`customer_loyalty_years >= 3`). It assumes that if this condition is met and the total is high enough, it applies a discount (multiplies by 0.95) *before* determining the final status or writing to DB? Actually, looking closely:
    The code calculates `total`, then checks loyalty conditions on the calculated total. If loyal, it reduces the total again (`total = total * 0.95`). Then it decides status based on whether this **new** (discounted) total is > 5000.
*   **The Assumption:** `fulfillment.py` assumes that the final value in the database reflects all applicable discounts and loyalty calculations performed by `service.py`. It does not know *how* or *why* a number was reduced; it just sees the result.
*   **Consequence if `service.py` changes:** If `service.py` is refactored to apply the discount after checking status, or if the threshold for applying the discount changes (e.g., requires 5 years instead of 3), the value written to the DB will change. Since `fulfillment.py` has no knowledge of this logic, it cannot compensate; it simply reads whatever number is there and proceeds with its own calculations based on that potentially altered figure.

### 4. The "Country" Field Assumption
*   **The Contract:** `service.py` uses the field `order["country"]` to determine if an order needs review (`!= "US"`). This implies a contract that every valid order object passed to `process_order()` contains a `"country"` key with a string value.
*   **Why it exists:** The code accesses `order["country"]` directly without `.get()`. If the country is missing, this will raise a KeyError in `service.py`, crashing the process before data reaches the DB. However, if we assume valid inputs are passed:
    *   Implicitly, `fulfillment.py` assumes that orders with status `'APPROVED'` have had their countries validated by `service.py`. It does not check for invalid country codes in its own logic (it doesn't use the field at all).
*   **Consequence if `service.py` changes:** If a future update allows "US" to be treated as an exception where no review is needed, but sets status differently based on other factors, or if it accidentally skips setting the country key for some orders:
    *   While this primarily breaks `service.py`, if `fulfillment.py` were extended later to check countries (e.g., "Do not fulfill from unknown regions"), its lack of schema enforcement means it might crash immediately upon encountering a missing key, whereas currently it ignores the field entirely.

### 5. The Cache Side-Effect Assumption
*   **The Contract:** `service.py` populates `_RECENT_TOTALS_CACHE[order["id"]] = total`. This is an internal state variable in Python's memory space (not shared between processes).
*   **Why it exists:** It assumes that the next time this specific module runs, or perhaps a testing framework running the same process again, will find these cached values.
*   **Consequence if `service.py` changes:** If `_RECENT_TOTALS_CACHE` is cleared (e.g., via an import reload in tests) or modified to store different data structures:
    *   Any code relying on this cache within the module itself might behave unexpectedly between runs, though it doesn't directly affect `fulfillment.py`. However, if a developer assumes "the total calculated here is cached for quick lookup" and tries to read from there in another part of their own refactor, they will get stale or missing data.

### 6. The Notification Message Format Assumption
*   **The Contract:** `service.py` constructs the notification string: `f"Order {order['id']} approved for {total} {order['currency']}"`. It assumes that if an order is processed, it will be marked as "approved". Note the logic flow: The status variable (`status`) might end up being `"PENDING_REVIEW"` even though the log message says `...approved for ...`.
*   **Why it exists:** There is a disconnect between the logging statement and the actual database write. If the condition `total > 5000` triggers, `status` becomes "PENDING_REVIEW", but the logger still prints "... approved".
*   **Consequence if `service.py` changes:** This isn't strictly an issue for `fulfillment.py`, as it doesn't read logs. However, it represents a broken contract between the *log output* and the *database state*. If someone assumes "If I see 'approved' in the log, the order is ready," they will be misled because high-value non-US orders are logged as approved but stored as pending review.

### Summary of Critical Risks for `fulfillment.py`
The most dangerous implicit contracts that could break `fulfillment.py` if `service.py` changes behavior without updating it are:

1.  **Floating Point Precision:** If `service.py` stops rounding non-JPY totals, math in `fulfillment.py` will drift slightly over time (e.g., summing many orders).
2.  **String Literal Mismatch:** If `service.py` changes the casing of "APPROVED" to something else due to a typo or logic change, `get_orders_ready_for_fulfillment()` returns nothing immediately.

Five of these six findings are solid: the exact-string match on 'APPROVED' is real and is the single most dangerous line in the system (change the casing anywhere and fulfillment silently stops finding orders, with no exception, no log line, nothing). But look closely at finding #6. It claims there is “a disconnect between the logging statement and the actual database write,” and that when an order goes to PENDING_REVIEW, “the logger still prints ‘… approved’.”

That is not quite what the code says. Look at the two lines it is describing:

    notify_customer(order["customer_id"], f"Order {order['id']} approved for {total} {order['currency']}")

    logger.info("ORDER_PROCESSED order_id=%s status=%s total=%s", order["id"], status, total)

The logger.info call the AI blamed does not contain the word “approved” anywhere. It logs the real status value dynamically, so a pending-review order logs status=PENDING_REVIEW, exactly as you would want. The AI attributed the disconnect to the wrong line.

But look at the notify_customer line just above it. It hardcodes the word “approved” into a message sent to the customer, and it is called unconditionally, with no check on status at all. So the AI’s underlying instinct, that something here tells the customer “approved” regardless of the real outcome, was correct. It just named the wrong function. This is a genuinely common failure mode: an AI’s high-level conclusion is right, but a specific supporting detail is wrong, and if you only skim the conclusion you will misdiagnose which line to fix.

Step 8: Validate the Finding Against the Real System

Do not act on Step 7’s finding until you have reproduced it yourself. Run the actual code with an order that should land in PENDING_REVIEW, and look at what the customer would actually see:

$ python -c "
import sys; sys.path.insert(0, 'src')
from orders.service import process_order, init_db
init_db('data/repro.db')
order = {
    'id': 'repro1', 'customer_id': 'cust-repro', 'country': 'DE', 'currency': 'USD',
    'items': [{'sku': 'X', 'qty': 1, 'unit_price': 6000}],
}
result = process_order(order, db_path='data/repro.db')
print('RESULT:', result)
"
[notify] to customer cust-repro: Order repro1 approved for 6000 USD
RESULT: {'order_id': 'repro1', 'status': 'PENDING_REVIEW', 'total': 6000}

Confirmed, with real output: the stored status is PENDING_REVIEW, but the customer-facing message says “approved.” This is a genuine bug, not a hypothetical one, and you found it by combining an AI-generated hypothesis with your own reproduction, exactly the order those two things should happen in. Skipping the reproduction step and shipping a fix based only on the AI’s explanation would have had you editing the wrong function, since the AI’s own words pointed at the logger.

Step 9: Use AI to Find Duplicated Business Rules

Recall the question Step 2’s AI response asked, unprompted: is legacy_utils.py a migration bridge, or just unused historical code? Here is how you answer that kind of question: check whether its logic still matches the logic it was supposedly replaced by. Both service.py and legacy_utils.py apply a loyalty discount. Ask the AI to compare them line by line.

Prompt (paraphrased for space; the exact prompt text is in the repo): Both files implement a loyalty discount. Compare the discount logic in each line by line. Are the conditions identical? If they differ, list every difference precisely, and give a concrete input that the two implementations would treat differently.

Model response (qwen3.5:4b, real captured output):

### Comparison of Discount Logic

The two files implement **different** discount logic. They are not identical in conditions, thresholds, or operator usage. Below is the line-by-line comparison and analysis.

#### Line-by-Line Analysis

| Feature | `src/orders/service.py` (Current) | `src/legacy_utils.py` (Legacy) | Difference? |
| :--- | :--- | :--- | :--- |
| **Loyalty Condition** | `>= 3` (Greater than or equal to 3 years) | `> 3` (Strictly greater than 3 years) | **Yes**: Uses different operators (`>=` vs `>`). |
| **Total Threshold** | None required for the discount logic itself. The condition checks loyalty first, then applies a check on total *inside* the same block implicitly? Wait, let's re-read carefully.<br><br>Code: `if order.get("customer_loyalty_years", 0) >= 3 and total >= 100:`<br>The discount is applied **only if** BOTH conditions are met. | None required for the discount logic itself.<br>Code: `if customer_loyalty_years > 3`<br>The discount is applied based solely on loyalty years in this snippet (no check against a minimum total). | **Yes**: The current file requires `total >= 100`. The legacy file does not. |
| **Action** | Multiplies `total` by `0.95`. | Returns `total * 0.95`. | Functionally identical action, but triggered differently. |

---

### Precise Differences List

1.  **Loyalty Threshold Operator**:
    *   Current: Uses `>=` (greater than or equal to). Customers with exactly 3 years get the discount.
    *   Legacy: Uses `>` (strictly greater than). Customers with exactly 3 years do **not** get the discount; they must have at least 4 years.

2.  **Minimum Total Threshold**:
    *   Current: Requires an explicit minimum spend (`total >= 100`). If a customer has $\ge$ 3 years but spends less than $100, no discount is applied.
    *   Legacy: No minimum total check exists in this function definition. Any amount qualifies if the loyalty condition is met (though practically limited by business rules elsewhere).

---

### Concrete Input Example Showing Different Behavior

Consider a customer with **exactly 3 years** of loyalty and an order total of **$50**.

| Feature | `src/orders/service.py` Result | `src/legacy_utils.py` Result |
| :--- | :--- | :--- |
| **Input**:<br>`customer_loyalty_years = 3`<br>`total = 50` | **No Discount Applied** (`$50.00`) <br>*(Reason: While loyalty is $\ge$ 3, the total $50 < 100$, failing the `and` condition)* | **Discount Applied** (`$47.50`) <br>*(Reason: Loyalty $3 > 3$? No... Wait.)<br>**Correction on Logic Trace**: Let's re-evaluate the legacy logic for this specific input.* |

*Let me correct the trace above based strictly on the code provided:*

**Scenario A: Customer with exactly 3 years, Total = $50**
1.  **Current (`service.py`)**: `if 3 >= 3 and 50 >= 100:` $\rightarrow$ False (because $50 < 100$). Result: **No discount**.
2.  **Legacy (`legacy_utils.py`)**: `if 3 > 3:` $\rightarrow$ False. Result: **No discount**.

*This input does not show a difference.* Let's try an input that triggers the operator difference specifically (ignoring total for legacy since it has no check).

**Scenario B: Customer with exactly 4 years, Total = $50**
1.  **Current (`service.py`)**: `if 4 >= 3 and 50 >= 100:` $\rightarrow$ False (because $50 < 100$). Result: **No discount**.
2.  **Legacy (`legacy_utils.py`)**: `if 4 > 3:` $\rightarrow$ True. Returns `$50 * 0.95 = \$47.50`.

**Scenario C (The Definitive Difference): Customer with exactly 3 years, Total = $150**
This input highlights the **Total Threshold** difference perfectly.

*   **Input**: `customer_loyalty_years = 3`, `total = 150`
    *   **Current (`service.py`)**:
        *   Condition: `(3 >= 3) and (150 >= 100)` $\rightarrow$ `True and True` $\rightarrow$ **TRUE**.
        *   Result: Discount applied ($142.50).
    *   **Legacy (`legacy_utils.py`)**:
        *   Condition: `(3 > 3)` $\rightarrow$ **FALSE**.
        *   Result: No discount applied (Returns $150.00).

**Scenario D (The Operator Difference): Customer with exactly 4 years, Total = ANY amount ($>0)$**
This input highlights the **Loyalty Operator** difference perfectly.

*   **Input**: `customer_loyalty_years = 3`, `total = 5` (Low total to avoid triggering current file's $100 limit)
    *   **Current (`service.py`)**: `(3 >= 3)` is True, but `(5 >= 100)` is False. Result: No discount ($5).
    *   **Legacy (`legacy_utils.py`)**: `if (3 > 3):` is False. Result: No discount ($5).

*Let's try the exact boundary for Loyalty in Legacy, assuming we ignore Total or assume Total doesn't matter there.*

**Scenario E (Strict Operator Difference - Ignoring Total constraint of current file):**
If we run `legacy_utils.py` with 3 years and $100:
*   **Legacy**: `if 3 > 3:` is False. No discount.
*   **Current**: `if 3 >= 3 and 100 >= 100:` is True. Discount applied ($95).

### Final Conclusion on Differences & Concrete Input

**Are the conditions identical?**
No. The current implementation requires both high loyalty AND a minimum spend, whereas the legacy implementation only checks for strictly higher loyalty (and ignores total in this specific function snippet).

**Concrete Input Treated Differently:**
*   **Input**: `customer_loyalty_years = 3`, `total = 150`
    *   **Current Implementation (`service.py`)**: Applies a **9.5% discount**. (Because $3 \geq 3$ AND $150 \geq 100$). Final price: **$137.25** ($150 \times 0.95$).
    *   **Legacy Implementation (`legacy_utils.py`)**: Applies **NO discount**. (Because $3 > 3$ is False). Final price: **$150.00**.

*Alternative Input for Operator Difference only:*
If we assume the legacy function is called in a context where total doesn't matter or is high enough, but focus purely on years:
*   **Input**: `customer_loyalty_years = 3`, `total = 200` (Assuming legacy ignores total)
    *   **Current (`service.py`)**: Applies discount ($190).
    *   **Legacy (`legacy_utils.py`)**: No discount ($200), because $3$ is not strictly greater than $3$.

The reasoning is real and, once you follow it through, correct: the current file requires total >= 100 and uses >= on years; the legacy file has no total floor at all and uses strict > on years. A customer with exactly 3 loyalty years and a $150 order gets a discount from one implementation and not the other. But check its own arithmetic in the final summary: it writes “Applies a 9.5% discount… Final price: $137.25 ($150 × 0.95)”. Two separate numbers in that one sentence are wrong. Multiplying by 0.95 is a 5% discount, not 9.5%. And 150 × 0.95 = 142.50, not 137.25, which the model’s own earlier “Scenario C” in the same response had already computed correctly. It contradicted its own correct answer two paragraphs later. The logic was right. Two numbers in its own summary of that logic were wrong. Verify the arithmetic separately from the reasoning; getting one right says nothing about the other.

The duplication itself is real and worth flagging for a fix even though this tutorial does not act on it (the JPY rounding and review-threshold rules on the “current” path make a full reconciliation a bigger job than this tutorial’s scope; see the migration plan in Step 11). What matters here is the habit: the AI can get the reasoning right and still hand you a wrong number in the same breath.

Step 10: Build a Dependency Map, Then Check It Against a Static Tool

A dependency map, even a rough one, is useful for planning a refactor: it tells you what you can change in isolation and what you cannot. Ask the AI to build one from the import statements alone, without the file contents this time, to see how far imports alone can take you.

Prompt (paraphrased for space; the exact prompt text is in the repo): Given only each file’s top-level imports (not full source), build a lightweight dependency map: what each file depends on, and what depends on it. Then answer specifically: based on imports alone, does fulfillment.py depend on orders/service.py, or the reverse, or neither?

Model response (qwen3.5:4b, real captured output):

### Dependency Map

Based on the provided file listings and import statements, here is the lightweight dependency map for each module:

| File | Imports (Depends On) | Depends On By (Is Imported By) |
| :--- | :--- | :--- |
| **src/cli.py** | `argparse`, `json`<br>`orders.service` | *(None listed)* |
| **src/orders/service.py** | `sqlite3`, `logging`, `datetime`<br>`orders.pricing`<br>`orders.notifications` | `src/cli.py` |
| **src/orders/pricing.py**| *None* (Local imports only) | `src/orders/service.py` |
| **src/orders/notifications.py** | *None* (Local imports only) | `src/orders/service.py` |
| **src/legacy_utils.py** | *None* (No local imports listed)<br>*(Note: Likely has external dependencies not shown)* | *(None listed)* |
| **src/fulfillment.py**| `sqlite3`<br>`orders.service` *(Inferred from context of typical CLI usage, but strictly speaking only `sqlite3` is explicitly stated in your text. However, looking at the pattern of other files importing `.service`, let's re-read carefully: "src/cli.py... imports orders.service". It does **not** say fulfillment imports it yet. Let's stick strictly to what was written.)* | *(None listed)* |

*Correction on `fulfillment.py` based strictly on your text:*
Your text states: `"src/fulfillment.py: imports sqlite3"`. It does not explicitly list an import of `.service`. However, in many such architectures, fulfillment might interact with orders. But to be rigorous and answer the second part correctly without making assumptions beyond the text provided:

**Strict Dependency Map (Based strictly on your input):**
1.  **src/cli.py**: Depends on `argparse`, `json`, `orders.service`. Imported by nothing listed here.
2.  **src/orders/service.py**: Depends on `sqlite3`, `logging`, `datetime`, `pricing`, `notifications`. Imported by `cli.py`.
3.  **src/orders/pricing.py**: No dependencies (locally). Imported by `service.py`.
4.  **src/orders/notifications.py**: No dependencies (locally). Imported by `service.py`.
5.  **src/legacy_utils.py**: No local imports listed. Dependencies unknown from this snippet.
6.  **src/fulfillment.py**: Depends on `sqlite3`. **(Crucially: Does not list an import of orders.service in your text).**

---

### Analysis Question Answer

**Question:** Based *only* on the imports provided, does `fulfillment.py` depend on `orders/service.py`, or does `orders/service.py` depend on `fulfillment.py`, or neither?

**Answer: Neither.**

**Reasoning:**
1.  **Check if `fulfillment.py` depends on `orders/service.py`:**
    The text explicitly states that `src/fulfillment.py` "imports sqlite3". It does not list an import for `.service`. Therefore, based strictly on the provided grep results, there is no evidence of this dependency.

2.  **Check if `orders/service.py` depends on `fulfillment.py`:**
    The text states that `src/orders/service.py` imports: `sqlite3`, `logging`, `datetime`, `.pricing`, and `.notifications`. It does not list an import for `.service/fulfillment`. Therefore, there is no evidence of this dependency.

*Note on Context:* While it is highly probable in a real-world application that these two modules interact (e.g., fulfilling orders), the prompt specifically asks to reason **based on imports alone** from the provided text. Since neither file lists an import statement for the other module, the relationship does not exist according to this specific data set.

The AI’s answer is correct, and that is the point of this step: based on imports alone, the answer really is “neither.” fulfillment.py never imports service.py. Now confirm that mechanically instead of trusting the AI’s reasoning about its own reasoning. A small script using Python’s built-in ast module can compute the same map without an LLM at all:

import ast
import os
import sys


def find_py_files(root):
    for dirpath, _, filenames in os.walk(root):
        if "__pycache__" in dirpath:
            continue
        for fn in filenames:
            if fn.endswith(".py"):
                yield os.path.join(dirpath, fn)


def module_name(path, root):
    rel = os.path.relpath(path, root)
    rel = rel[:-3] if rel.endswith(".py") else rel
    return rel.replace(os.sep, ".").removesuffix(".__init__")


def analyze(root="src"):
    edges = {}
    for path in find_py_files(root):
        mod = module_name(path, ".")
        with open(path, encoding="utf-8") as f:
            tree = ast.parse(f.read(), filename=path)
        calls = set()
        for node in ast.walk(tree):
            if isinstance(node, ast.ImportFrom) and node.module:
                target = node.module
                if node.level:
                    # relative import, resolve against the importing module's package
                    parts = mod.split(".")[:-1]
                    target = ".".join(parts[: len(parts) - node.level + 1] + [node.module])
                calls.add(target)
            elif isinstance(node, ast.Import):
                for alias in node.names:
                    calls.add(alias.name)
        edges[mod] = sorted(calls)
    return edges


if __name__ == "__main__":
    edges = analyze("src")
    for mod, deps in sorted(edges.items()):
        print(f"{mod}:")
        if not deps:
            print("  (no local/stdlib module-level imports of interest)")
        for d in deps:
            print(f"  -> {d}")

$ python dep_map.py
src:
  (no local/stdlib module-level imports of interest)
src.cli:
  -> argparse
  -> json
  -> orders.service
src.fulfillment:
  -> sqlite3
src.legacy_utils:
  (no local/stdlib module-level imports of interest)
src.orders:
  (no local/stdlib module-level imports of interest)
src.orders.notifications:
  (no local/stdlib module-level imports of interest)
src.orders.pricing:
  (no local/stdlib module-level imports of interest)
src.orders.service:
  -> datetime
  -> logging
  -> sqlite3
  -> src.orders.notifications
  -> src.orders.pricing

The static analysis agrees with the AI: no import edge connects fulfillment.py to orders.service. And that agreement is exactly what makes this step worth including. A dependency map built purely from imports, whether an AI infers it or a script computes it deterministically, is structurally blind to the one coupling in this codebase that actually matters: the 'APPROVED' string contract you found in Step 7. That coupling runs through a shared database table at runtime, not through an import statement, so no import-based tool, AI or otherwise, will ever surface it. This is why Step 7’s contract-tracing has to happen as its own step. A dependency map tells you what is wired together. It does not tell you what is silently coupled through shared state.

Step 11: Mark What You Do Not Know, Then Check the Version History

The review-threshold condition, total > 5000 and order["country"] != "US", is exactly the kind of magic-looking logic that invites a confident but made-up explanation. Ask the AI about it in isolation, with explicit instructions to mark a guess as a guess rather than presenting it as fact.

Prompt (paraphrased for space; the exact prompt text is in the repo): Here is one condition from process_order(), shown with no other context: if total > 5000 and order[“country”] != “US”: status = “PENDING_REVIEW”. Why might a threshold of exactly 5000 and a US carve-out exist? Give your best guess, but mark it explicitly as a guess if you cannot confirm it from the code alone. Do not invent a confident-sounding explanation if you are not sure.

Model response (qwen3.5:4b, real captured output):

Based strictly on the provided code snippet without access to external business rules or documentation, here is an analysis of why these specific conditions might exist:

**The Threshold (`> 5000`)**
*   **High-Risk Transaction Limit:** The most common reason for a monetary threshold in order processing systems is fraud prevention. A limit set at exactly $5,000 suggests that transactions exceeding this amount are considered "high value" and therefore require manual verification to ensure they are legitimate before being automatically approved or shipped.
*   **Customs/Import Fees:** If the system handles international shipping, there may be specific customs regulations where orders over a certain dollar value trigger mandatory review for duty assessment or import clearance documentation.

**The Carve-out (`order["country"] != "US"`)**
*   **Domestic vs. International Policy:** The exclusion of the US likely indicates that domestic (US-based) transactions are handled under different, perhaps lower-risk protocols than international ones. This could be because:
    *   Domestic orders generally have fewer fraud risks or faster payment verification methods available within the local ecosystem.
    *   There is a specific regulatory exemption for US customers regarding high-value transaction reviews that does not apply to foreign buyers (e.g., different banking compliance standards).

**Conclusion / Best Guess**
It is highly probable that this logic implements an **automated fraud mitigation strategy**: Orders over $5,000 originating from outside the United States are flagged as "PENDING_REVIEW" due to higher perceived risk associated with international high-value transactions. Conversely, orders under $5,000 or those coming directly from US customers bypass this specific check and proceed automatically ("APPROVED").

*Note: This is a hypothesis based on standard software engineering patterns for e-commerce; the actual business justification (e.g., specific fraud rates in certain regions vs. others) cannot be confirmed without reading the surrounding code, database logs, or product requirements.*

This is a well-behaved answer: a plausible hypothesis (fraud prevention on high-value international orders), clearly labeled as a hypothesis, with an explicit note that it “cannot be confirmed without reading… product requirements.” That is the AI doing exactly what you asked. Now go find out if the guess is actually right.

You do not need product requirements. You have something better: the commit that introduced this exact line. Run git blame on it.

$ git blame -L 44,47 --date=short src/orders/service.py
870f1015 (Your Name 2026-08-22 44)     if total > 5000 and order["country"] != "US":
870f1015 (Your Name 2026-08-22 45)         status = "PENDING_REVIEW"
870f1015 (Your Name 2026-08-22 46)     else:
870f1015 (Your Name 2026-08-22 47)         status = "APPROVED"

$ git log -1 --format="%s%n%n%b" 870f1015
Flag high-value international orders for manual review per Compliance (SOX finding JIRA-4821). US orders
are exempt because they already pass through the separate KYC pipeline before reaching this service.

The real answer is not the AI’s guess. It is not fraud prevention, and the US carve-out is not about lower domestic risk. It is a compliance requirement tied to a specific SOX audit finding, and US orders are exempt because they are already screened by a completely different system (a KYC pipeline) before they ever reach this code. The AI’s hedged guess was plausible, professionally worded, and wrong in the way that matters most for a refactor: if you “simplified” this threshold based on the fraud-prevention theory, for example by making it apply to US orders too, you would silently violate the actual compliance requirement while believing you had improved the code.

This is the entire argument for the version-history step in one example. A well-hedged AI guess that admits its own uncertainty is still a guess. Grep and git blame are not smarter than the model, but on a question like this one they are not guessing at all, they are quoting the person who wrote the line and said why.

Step 12: Turn the Investigation Into a Validated Fix

You now have a short, evidence-backed list:

• Confirmed bug (Steps 7 and 8): notify_customer unconditionally tells the customer “approved,” even when the stored status is PENDING_REVIEW. Reproduced with real output. Safe and small to fix.
• Confirmed drift, not yet fixed (Step 9): legacy_utils.apply_legacy_discount and the inline loyalty discount in service.py have silently diverged (> vs >=, and a missing total floor). Reconciling them safely means first finding out whether the nightly batch job that calls legacy_utils.py still runs at all, which this tutorial’s scope does not cover. Flag it, do not touch it yet.
• Confirmed dangerous contract (Step 7): the 'APPROVED' string is load-bearing for a separate process and enforced nowhere. Worth a shared constant in a real refactor. Out of scope for this tutorial’s one fix.
• Confirmed, now-explained business rule (Step 11): the $5,000 non-US review threshold is a compliance requirement, not a fraud heuristic. Do not touch it without Compliance sign-off, regardless of what any future AI investigation guesses about it.

Only the first item is small, safe, and fully understood enough to fix right now. Apply it:

    if status == "APPROVED":
        notify_customer(order["customer_id"], f"Order {order['id']} approved for {total} {order['currency']}")

Add a regression test that encodes exactly what you found, so this cannot silently regress again:

def test_pending_review_orders_do_not_send_an_approved_notification(db_path, capsys):
    order = {
        "id": "o6",
        "customer_id": "c6",
        "country": "DE",
        "currency": "USD",
        "items": [{"sku": "X", "qty": 1, "unit_price": 6000}],
    }
    result = process_order(order, db_path=db_path)
    captured = capsys.readouterr()
    assert result["status"] == "PENDING_REVIEW"
    assert "approved" not in captured.out.lower()

Run it against the code before your fix, to prove the test actually catches the bug and is not a test that would pass regardless:

$ python -m pytest tests/test_orders.py::test_pending_review_orders_do_not_send_an_approved_notification -v
FAILED tests/test_orders.py::test_pending_review_orders_do_not_send_an_approved_notification
AssertionError: assert 'approved' not in '[notify] to...r 6000 usd\n'
  'approved' is contained here:
    [notify] to customer c6: order o6 approved for 6000 usd

Now apply the fix shown above and run the full suite:

$ python -m pytest -v
tests/test_orders.py::test_jpy_orders_round_to_nearest_100 PASSED        [ 16%]
tests/test_orders.py::test_loyalty_discount_requires_minimum_total PASSED [ 33%]
tests/test_orders.py::test_loyalty_discount_applies_above_floor PASSED   [ 50%]
tests/test_orders.py::test_orders_over_5000_outside_us_go_to_review PASSED [ 66%]
tests/test_orders.py::test_pending_review_orders_do_not_send_an_approved_notification PASSED [ 83%]
tests/test_orders.py::test_orders_over_5000_inside_us_are_approved PASSED [100%]

6 passed in 0.14s

Commit it with a message that records not just what changed, but how you know it was safe to change:

git commit -am "Only notify the customer of approval when status is APPROVED.
AI-assisted codebase archaeology surfaced that pending-review orders were
incorrectly told they were approved; confirmed by reproduction and added
a regression test."

That commit message is doing real work. The next person who runs git blame on this exact line, maybe using this same workflow, will see not just what the fix was, but that it was investigated and verified rather than guessed at. You are paying the same debt forward that the “SOX finding JIRA-4821” commit paid for you in Step 11.

Common Mistakes and Gotchas

Asking for a refactor before you ask for an explanation

“Clean this up” and “explain what this does, and do not change it” are different prompts that produce different modes of response. The first one puts the model into solution mode, where it starts making design decisions before it has surfaced the constraints those decisions need to respect. Ask investigation questions first, explicitly telling the model not to propose changes, and only ask for a refactor once you have a validated list of what has to be preserved.

Trusting a conclusion without checking its supporting details

Step 7’s AI response reached a correct high-level conclusion (a status/message mismatch exists) while misattributing the specific cause. If you had fixed the file the AI actually named, you would have “fixed” a working logging call and left the real bug in place. Read the specific lines a finding points to, not just its summary sentence.

Trusting AI arithmetic because the reasoning around it was correct

Step 9 is a clean example: the comparison logic was right, and the model even computed the correct dollar figure once, then produced two different wrong numbers in its own summary of that same, correct reasoning a few lines later. Logic and arithmetic are different failure modes. Verifying one tells you nothing about the other.

Accepting a well-hedged guess as if hedging made it true

The Step 11 response about the $5,000 threshold was honest about its own uncertainty, and still wrong. A model that says “this might be X, but I cannot confirm it” has done its job correctly. Your job is to go find the confirmation, not to round a well-worded hypothesis up to a fact because it came with an appropriate disclaimer.

Treating an import-based dependency map as a complete picture

Step 10’s static analysis and the AI’s own answer agreed with each other, and both missed the one coupling in the codebase that actually mattered. Agreement between two tools that work the same way is not the same as completeness. If two parts of a system share a database, a message queue, a file on disk, or any other channel that is not an import statement, no import-based map will ever show you that they are connected.

Skipping the reproduction step

Every finding in this tutorial that changed anything (Step 8’s bug, in particular) was reproduced with real, captured output before it was acted on. An AI’s description of what code does is a hypothesis about the code, not a report of having run it, unless you have explicitly given the model tool access to actually execute it and shown its real output. Run the code yourself.

How to Confirm the Whole Workflow Holds Together

Before moving on, walk back through what you can now prove, not just what you were told:

• The full test suite passes, including the new regression test: python -m pytest -v reports 6 passed
• The reproduction from Step 8, re-run after your fix, no longer prints a false “approved” notification for a PENDING_REVIEW order
• git log --oneline -- src/orders/service.py shows your fix commit sitting on top of a real, readable history, including the compliance commit that explains the threshold
• You can state, from evidence rather than inference, why the $5,000 review threshold exists, what the AI’s dependency map cannot see, and exactly which line was actually responsible for the false “approved” message

If all four of those check out, you have done more than fix one bug. You have built a repeatable habit: map first, trace one real capability end to end, separate rules from plumbing, hunt side effects and contracts and duplication, mark what you do not know instead of filling the gap with a guess, and validate everything against source, tests, and history before you act on it. That sequence works the same way on an eleven-file demo project and on a codebase with fifteen years of history behind it. The demo project just lets you see, with concrete transcripts, exactly where an AI’s help stops being reliable enough to trust without checking.

Next Steps

• Run this same sequence of prompts against a real repository you are unfamiliar with, ideally one with a genuine multi-year commit history to mine in the “mark unknown” step
• Once you have a validated understanding of a legacy function, pair this workflow with characterization tests before refactoring it; sxz.io’s How to Safely Refactor Legacy Python Code Using Characterization Tests picks up exactly where this tutorial leaves off
• If your investigation prompts keep needing the same background explained every time, save them as reusable instructions; sxz.io’s How to Write and Organize CLAUDE.md Files for Claude Code covers how
• If an AI investigation keeps producing vague or unconfirmed answers on your own codebase, sxz.io’s How to Fix Vague AI Agent Answers With Prompt and Context Engineering covers why, and what to feed the model instead