How to Audit Python Dependencies With CycloneDX SBOMs and pip-audit, and Find a Malicious Release PyPI Has Already Deleted

On October 21, 2026, a poisoned build of LiteLLM, version 1.107.0, was live on PyPI for 1 hour 43 minutes. By LiteLLM’s own timestamps, as our earlier report put it, its wheels (PyPI’s installable files) were up from 11:39:51 to 13:23 UTC. The malware record for that release in the OSV database says each wheel shipped three extra files at the site-packages root, so the payload was triggered at every Python interpreter start, “without importing litellm.” Once PyPI took the release down, every team running Python had the same question: did any of our services install it?

Table Of Content

What you will build, and the words you need first
Prerequisites
Step 1: Create the tools environment
Step 2: Create five pretend services
Step 3: Generate an SBOM from an installed environment
How to read it
Three mistakes that look like success
Pointing the tool at the wrong Python
Getting different bytes for the same environment
Starting an environment you do not trust
Step 4: See what an SBOM built from requirements.txt misses
Step 5: Audit with pip-audit
Run it and read the table
What the exit status means
Three ways to run it, and what each one sees
Step 6: The deleted-release trap
When did the scanner learn about it?
What about pip-audit’s default mode?
Step 7: Archive an SBOM for every build
Step 8: Hunt for a package across every SBOM
Step 9: Re-audit the archive on a schedule
Step 10: Test it, then break it on purpose
Common mistakes and limits
Reading exit status 0 as “clean”
Auditing the file instead of the environment
Archiving requirements SBOMs and calling it an inventory
Counting rows
Treating an SBOM as proof of safety
Trusting the dependency graph too much
Check that everything works end to end
Where to go next

There are two ways to answer. You can ask a vulnerability scanner, or you can ask your own inventory. This tutorial teaches both and shows where each one fails. You will generate a CycloneDX software bill of materials (SBOM) for several Python services, audit them with pip-audit, archive the SBOMs, and write two small scripts: one that searches every archived SBOM for a package and a version range, and one that re-audits the whole archive on a schedule. Along the way I measured three things that surprised me. First, pip-audit, run against PyPI’s own data, answered “No known vulnerabilities found” with exit status 0 for the deleted LiteLLM release. Second, the first commit that put a record for 1.107.0 into the malware repository that feeds OSV is dated 21:46:03 UTC, more than ten hours after the first poisoned wheel appeared. Third, building an SBOM from a virtual environment starts that environment’s Python, which runs its .pth startup files, the same mechanism the LiteLLM payload used.

One scope note before you start. This tutorial audits software; it does not run anything malicious. The gateway service in the lab names the bad release in its requirements file but never installs it, and PyPI no longer serves it anyway. I did not download, unpack or run any copy of that release. What is said here about its contents comes from the OSV record and from our earlier report. Everything below covers Python packages from PyPI on one machine. Container images, operating system packages and other languages need other tools, and our container SBOM tutorial covers one of them.

What you will build, and the words you need first

By the end you will have these files in one folder:

make_fleet.py creates five pretend services, each with its own requirements.txt and virtual environment;
show_sbom.py, sha.py, pth_probe.py, advisories.py, compare_modes.py and probe_resolve.py are small helpers that read SBOMs, test a safety question and count what pip-audit finds;
gen_sboms.py writes one SBOM per service into an archive folder;
sbom_hunt.py searches the archive for a package and a version range and says which component pulled it in;
sbom_audit.py re-audits every archived SBOM against OSV with pip-audit;
test_sbom_tools.py holds 18 offline tests, and a table near the end shows eight deliberate bugs they catch.

A few terms, defined before they are used. A dependency is a package your code needs. A direct dependency is one you list yourself, and a transitive dependency is one that another package pulls in. A pin is a requirement written as an exact version, such as requests==2.19.1. A virtual environment (venv) is a private folder of installed packages for one project, made here with Python’s venv module.

A software bill of materials (SBOM) is a machine-readable list of what is inside a piece of software, like the ingredient list on a food package. CycloneDX is one SBOM format. It is developed under the OWASP Foundation and is also an Ecma standard (ECMA-424), and its specification overview says “Components describe the complete inventory of first-party and third-party components.” It also says “The dependency graph is capable of representing both direct and transitive relationships.” The current specification is version 1.7, and the tool you will use writes version 1.6 unless you ask for another, which is what the output below shows. A component is one entry in that inventory. Here each one is identified by a package URL, or purl, such as pkg:pypi/[email protected]. The purl specification describes its syntax as “a standardized URL-based syntax that uniquely identifies software packages, independent of their ecosystem or distribution channel.” A bom-ref is a label inside one file that lets the dependency graph point at a component.

An advisory is a published report of a flaw, or of malicious code, in specific versions of a package. Advisory IDs tell you which database wrote them. According to the OSV schema, GHSA is the GitHub Security Advisory Database, PYSEC is the PyPI Vulnerability Database and MAL is the Malicious Packages Repository, and CVE is the CVE program’s own ID. One flaw often has several IDs, and each record lists the others as aliases. OSV (Open Source Vulnerabilities) gathers these databases behind one API at api.osv.dev. pip-audit is a tool published under the PyPA (Python Packaging Authority) organization on GitHub that looks your packages up in a vulnerability service, and its README lists “multiple vulnerability services”, namely PyPI and OSV.

Prerequisites

Python 3.13. Every command below ran on Python 3.13.14 on Windows 11. The pip-audit README says it needs Python 3.10 or newer.
PowerShell on Windows. Paths use forward slashes, which PowerShell and these tools accept. Three files contain Windows folder names: Scripts (where a virtual environment keeps its programs) in make_fleet.py, and Lib/site-packages in compare_modes.py and pth_probe.py. On Linux and macOS those are bin and lib/python3.13/site-packages. I did not run anything on Linux or macOS.
Network access to PyPI and to api.osv.dev.
About 65 MB of disk for the five environments and about 45 minutes.
Background: you can run a Python script and read JSON. No security background is assumed.

The numbers here were captured on October 25, 2026. New advisories are only ever added, so your advisory counts may be higher than mine. Unpinned transitive packages, such as certifi, will be newer versions on your machine.

Step 1: Create the tools environment

Everything in this tutorial runs from one folder inside one virtual environment that holds the tools. Pinning the versions means you see what I saw:

mkdir sbom-lab
cd sbom-lab
python -m venv labvenv
labvenv\Scripts\Activate.ps1
python -m pip install cyclonedx-bom==7.5.0 pip-audit==2.10.1 packaging==26.3 pytest==9.1.1
python --version
pip-audit --version
cyclonedx-py --version

The activation line is for PowerShell. If PowerShell says that running scripts is disabled on your system, use cmd.exe instead, where the line is labvenv\Scripts\activate.bat. On Linux or macOS it is source labvenv/bin/activate. Keep this terminal open and in this folder; every file below is saved here, and every command assumes the environment is active.

Check: the last three commands should print these versions. The third line is the version of cyclonedx-py itself.

Python 3.13.14
pip-audit 2.10.1
7.5.0

If pip-audit is not found, the environment is not active, so run the activation line again.

Step 2: Create five pretend services

To audit something you need something to audit. This script builds a small “fleet” of five services. Each one gets a requirements.txt that pins only the packages its developers asked for, and each (except one) gets its own virtual environment with those pins installed. The pins are old on purpose, so the audit has something to find.

"""make_fleet.py: create five pretend services, each with a pinned requirements.txt
and (except the gateway) its own virtual environment with those pins installed.

Run from the lab folder:  python make_fleet.py
"""
import subprocess
import sys
import venv
from pathlib import Path

FLEET = Path("fleet")

SERVICES = {
    "orders-api": ["requests==2.19.1"],
    "reports-worker": ["jinja2==2.10"],
    "billing-api": ["requests==2.34.2", "jinja2==3.1.6"],
    "notifier": ["urllib3==1.25.3"],
    "gateway": ["litellm==1.107.0"],
}

# PyPI deleted litellm 1.107.0 and it was malicious, so we never install it.
NEVER_INSTALL = {"gateway"}


def venv_python(service_dir: Path) -> Path:
    return service_dir / ".venv" / "Scripts" / "python.exe"


def main() -> int:
    for name, pins in SERVICES.items():
        service_dir = FLEET / name
        service_dir.mkdir(parents=True, exist_ok=True)
        req_file = service_dir / "requirements.txt"
        req_file.write_text("\n".join(pins) + "\n", encoding="utf-8")
        if name in NEVER_INSTALL:
            print(f"{name}: wrote requirements.txt, no environment (never installed)")
            continue
        venv.create(service_dir / ".venv", with_pip=True, clear=True)
        py = str(venv_python(service_dir))
        subprocess.run(
            [py, "-m", "pip", "install", "-q", "--disable-pip-version-check",
             "-r", str(req_file)],
            check=True,
        )
        listing = subprocess.run(
            [py, "-m", "pip", "list", "--format=freeze", "--disable-pip-version-check"],
            check=True, capture_output=True, text=True,
        ).stdout
        packages = [ln for ln in listing.splitlines() if ln.strip() and not ln.lower().startswith("pip==")]
        print(f"{name}: installed {len(packages)} packages: {', '.join(packages)}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Read it from the top. SERVICES maps each service to its pins. orders-api asks for requests==2.19.1, a 2018 release. notifier pins urllib3==1.25.3 directly, so later you can compare a package someone chose with the same package that arrived by accident. gateway asks for litellm==1.107.0 and is listed in NEVER_INSTALL, so it only gets a requirements file. For the others, venv.create makes the environment (with clear=True, so a second run starts clean), pip install -r installs the pins, and pip list --format=freeze reports what actually ended up installed. Run it:

python make_fleet.py

orders-api: installed 5 packages: certifi==2026.10.23, chardet==3.0.4, idna==2.7, requests==2.19.1, urllib3==1.23
reports-worker: installed 2 packages: Jinja2==2.10, MarkupSafe==3.0.4
billing-api: installed 7 packages: certifi==2026.10.23, charset-normalizer==3.5.2, idna==3.20, Jinja2==3.1.6, MarkupSafe==3.0.4, requests==2.34.2, urllib3==2.8.0
notifier: installed 1 packages: urllib3==1.25.3
gateway: wrote requirements.txt, no environment (never installed)

It takes about half a minute. Look at the orders-api line: the requirements file named one package and pip installed five. The other four are transitive dependencies. Version 2.19.1 of requests declares caps on its own dependencies (chardet below 3.1.0, idna below 2.8, urllib3 below 1.24), which is why this environment holds chardet 3.0.4, idna 2.7 and urllib3 1.23. Nobody typed those three, but they are the packages that run on your servers, so they matter just as much.

Check: you should see five lines and a fleet folder with five service folders. The gateway folder has no .venv.

Step 3: Generate an SBOM from an installed environment

Now the first real task. The cyclonedx-py program reads a virtual environment and writes a CycloneDX file. Its documentation explains why this is the preferred mode: “By analyzing the actually installed packages, this will produce the most accurate and complete CycloneDX BOM.” First, a small helper to print an SBOM as a table, because raw CycloneDX JSON is long:

"""show_sbom.py: print a CycloneDX SBOM as a short table.
Usage: python show_sbom.py FILE.cdx.json [--count]     (--count prints only the first line)"""
import json
import sys
from pathlib import Path

path = Path(sys.argv[1])
sbom = json.loads(path.read_text(encoding="utf-8"))
components = sbom.get("components", [])
print(f"{path.name}: CycloneDX {sbom['specVersion']}, {len(components)} components")
if "--count" in sys.argv:
    sys.exit(0)
for comp in components:
    print(f"  {comp['name']:<12}{comp.get('version', '?'):<12}{comp.get('purl', '-')}")
print("dependencies:")
for dep in sbom.get("dependencies", []):
    if dep.get("dependsOn"):
        print(f"  {dep['ref']} -> {', '.join(dep['dependsOn'])}")

Then generate the SBOM for orders-api and look at it:

mkdir demo
cyclonedx-py environment fleet/orders-api/.venv -o demo/orders-api.env.cdx.json
python show_sbom.py demo/orders-api.env.cdx.json

orders-api.env.cdx.json: CycloneDX 1.6, 6 components
  certifi     2026.10.23  pkg:pypi/[email protected]
  chardet     3.0.4       pkg:pypi/[email protected]
  idna        2.7         pkg:pypi/[email protected]
  pip         26.1.2      pkg:pypi/[email protected]
  requests    2.19.1      pkg:pypi/[email protected]
  urllib3     1.23        pkg:pypi/[email protected]
dependencies:
  requests==2.19.1 -> certifi==2026.10.23, chardet==3.0.4, idna==2.7, urllib3==1.23
  urllib3==1.23 -> certifi==2026.10.23, idna==2.7

How to read it

The first line says the file follows CycloneDX 1.6 and has 6 components, although the requirements file named one. Each component line shows a name, a version and a purl. Under dependencies: every line is an edge in the graph, such as requests==2.19.1 -> certifi==2026.10.23, chardet==3.0.4, idna==2.7, urllib3==1.23. The strings on the left and right of the arrow are bom-refs, the labels that connect the graph to the component list. Behind this table the JSON file also records when it was made (metadata.timestamp) and a random serialNumber.

Three details are worth noticing now, because they matter later. First, pip itself is a component, because it is installed in the environment. Second, there is no root component: the file never says which application it describes. In this tutorial “top level” therefore means that no other installed package depends on the component, which is true of requests and of pip. Third, treat the graph as a clue about who pulled something in, not as proof. The file draws edges from urllib3 to certifi and idna, yet in urllib3 1.23’s own metadata file certifi appears only on a line marked extra == 'secure', and idna only for Python 2.7 or older with that extra.

Check: the component count is 6 and every component has a pkg:pypi/ purl.

Three mistakes that look like success

Pointing the tool at the wrong Python

If you run the command without naming an environment, it inventories the Python that is running the tool, which is your tools environment, not the service:

cyclonedx-py environment -o demo/wrong.cdx.json
python show_sbom.py demo/wrong.cdx.json --count

wrong.cdx.json: CycloneDX 1.6, 53 components

That is 53 components, none of them the service’s. The command succeeded, the file is valid, and it describes the wrong thing. Always name the environment, and glance at the component count. One more Windows detail: when I pointed the tool at fleet/orders-api/.venv/Scripts/python.exe with forward slashes it failed with “CRITICAL | CDX > [WinError 2] The system cannot find the file specified”. Passing the venv folder, as above, worked.

Getting different bytes for the same environment

Each run stamps a new serial number and time into the file, so two SBOMs of an unchanged environment differ. That is useful for an archive, because the timestamp tells you when the inventory was taken. It is a nuisance when you want to diff two SBOMs to see what changed. The --output-reproducible flag removes the random and time-based values. A tiny helper prints a short hash of each file so you can compare:

"""sha.py: print the first 12 hex digits of each file's SHA-256. Usage: python sha.py FILE [FILE ...]"""
import hashlib
import sys

for name in sys.argv[1:]:
    with open(name, "rb") as handle:
        print(hashlib.sha256(handle.read()).hexdigest()[:12], name)

cyclonedx-py environment fleet/orders-api/.venv --output-reproducible -o demo/r1.cdx.json
cyclonedx-py environment fleet/orders-api/.venv --output-reproducible -o demo/r2.cdx.json
cyclonedx-py environment fleet/orders-api/.venv -o demo/n1.cdx.json
cyclonedx-py environment fleet/orders-api/.venv -o demo/n2.cdx.json
python sha.py demo/r1.cdx.json demo/r2.cdx.json demo/n1.cdx.json demo/n2.cdx.json

bb79f53ffc34 demo/r1.cdx.json
bb79f53ffc34 demo/r2.cdx.json
3864838f0c2c demo/n1.cdx.json
cd168514b859 demo/n2.cdx.json

The two reproducible files (r1 and r2) have the same hash, and the two default files (n1 and n2) do not. Your hash values will differ from mine; what matters is which lines match.

Starting an environment you do not trust

This one matters most for the story that opened this tutorial. To find the packages in an environment, cyclonedx-py environment starts that environment’s own Python. Python, at startup, processes every .pth file in site-packages, and the site module documentation says “lines starting with import (followed by space or tab) are executed with exec().” (Python 3.15 deprecates this but still runs such lines during the deprecation period.) The LiteLLM record says the payload’s trigger was a file called cook.pth that “contains import cook.” So inventorying a compromised environment can run the compromise again. The tool’s own help hints at this: its -S option “Prevents evaluation of *.pth files, but may lead to incomplete component detection.” I tested it with a harmless stand-in. The script below plants a .pth file that only writes a marker file, then tries three ways to inventory the environment:

"""pth_probe.py: does inventorying a virtual environment run that environment's .pth files?
A harmless marker file stands in for a payload. Usage: python pth_probe.py"""
import json
import subprocess
import sys
import venv
from pathlib import Path

env_dir = Path("pth_demo")
marker = Path("pth_marker.txt").resolve()
venv.create(env_dir / ".venv", with_pip=False, clear=True)
site = env_dir / ".venv" / "Lib" / "site-packages"
# Python runs any line of a .pth file that starts with "import" when the interpreter starts.
(site / "marker.pth").write_text(f"import pathlib; pathlib.Path({str(marker)!r}).write_text('ran')\n", encoding="utf-8")

CYCLONEDX = [sys.executable, "-m", "cyclonedx_py", "environment"]
PIP_LIST = [sys.executable, "-m", "pip", "list", "--path", str(site), "--format=freeze"]
for label, command in [
    ("environment, no flag", CYCLONEDX + [str(env_dir / ".venv"), "-o", "pth_demo/sbom.cdx.json"]),
    ("environment -S", CYCLONEDX + ["-S", str(env_dir / ".venv"), "-o", "pth_demo/sbom.cdx.json"]),
    ("pip list --path", PIP_LIST),
]:
    marker.unlink(missing_ok=True)
    subprocess.run(command, check=True, capture_output=True)
    print(f"{label:<22}marker file created: {marker.exists()}")

subprocess.run(CYCLONEDX + ["-S", "fleet/orders-api/.venv", "-o", "pth_demo/orders-S.cdx.json"], check=True, capture_output=True)
found = len(json.loads(Path("pth_demo/orders-S.cdx.json").read_text(encoding="utf-8")).get("components", []))
print(f"environment -S on orders-api found {found} components")

python pth_probe.py

environment, no flag  marker file created: True
environment -S        marker file created: False
pip list --path       marker file created: False
environment -S on orders-api found 0 components

Without a flag, the marker appeared, which means the code in the .pth file ran. With -S it did not run, but on a real environment -S found 0 components instead of 6, and gave no error. That is an empty SBOM that looks valid. The third method, pip list --path, reads the package folders without starting the target’s Python, so the marker never appeared. Its output feeds cyclonedx-py requirements:

python -m pip list --path fleet/orders-api/.venv/Lib/site-packages --format=freeze > demo/orders-api.freeze.txt
cyclonedx-py requirements demo/orders-api.freeze.txt -o demo/orders-api.frozen.cdx.json
python show_sbom.py demo/orders-api.frozen.cdx.json --count

orders-api.frozen.cdx.json: CycloneDX 1.6, 6 components

You get all 6 components, but no dependency graph, because a frozen list has no edges. For a suspect environment, that trade is worth it. For your own build servers, use the plain environment mode. Our tutorial on auditing .pth startup hooks shows how to find these files.

Step 4: See what an SBOM built from requirements.txt misses

You can also build an SBOM from a requirements file without installing anything. It is quick and it is the only option for the gateway, but compare the result:

cyclonedx-py requirements fleet/orders-api/requirements.txt -o demo/orders-api.req.cdx.json
python show_sbom.py demo/orders-api.req.cdx.json

orders-api.req.cdx.json: CycloneDX 1.6, 1 components
  requests    2.19.1      pkg:pypi/[email protected]
dependencies:

One component instead of six. The file lists exactly what a human typed, so it cannot contain chardet, idna or urllib3. The bom-ref is requirements-L1, a label built from the line number in the file, and the dependency list is empty. If you archive SBOMs like this one, a later search for urllib3 will not find it in the orders-api SBOM, which wrongly suggests the service does not use it. Keep this result in mind; Step 8 shows the damage.

Check: you have two SBOMs for the same service, with 6 and 1 components. From now on, SBOMs of installed environments are the ones to archive, and requirements SBOMs are the fallback.

Step 5: Audit with pip-audit

Run it and read the table

Now check one service’s packages against known vulnerabilities. Start with the smallest case, reports-worker, which pins jinja2==2.10. The two flags tell pip-audit to take the file as it is: --no-deps is documented as “don’t perform any dependency resolution; requires all requirements are pinned to an exact version,” and --disable-pip “can only be used with hashed requirements files or if the --no-deps flag has been provided”.

pip-audit -r fleet/reports-worker/requirements.txt --no-deps --disable-pip

Found 12 known vulnerabilities in 1 package
Name   Version ID              Fix Versions
------ ------- --------------- ------------
jinja2 2.10    PYSEC-2021-66   2.11.3
jinja2 2.10    PYSEC-2019-217  2.10.1
jinja2 2.10    PYSEC-2019-217  2.10.1
jinja2 2.10    PYSEC-2021-66   2.11.3
jinja2 2.10    PYSEC-2026-1473 3.1.3
jinja2 2.10    PYSEC-2026-1474 3.1.4
jinja2 2.10    PYSEC-2026-1475 3.1.5
jinja2 2.10    PYSEC-2026-1471 3.1.6
jinja2 2.10    PYSEC-2026-1473 3.1.3
jinja2 2.10    PYSEC-2026-1471 3.1.6
jinja2 2.10    PYSEC-2026-1474 3.1.4
jinja2 2.10    PYSEC-2026-1475 3.1.5
(exit status 1)

(pip-audit also prints two warning lines about hashing your pinned dependencies. I left them out. The last line, (exit status 1), is not printed by the program; I added it to the captured text to show the exit status, and I do the same in later blocks.) The columns are: the package, its version, the advisory ID that pip-audit chose to show, and the versions that fix it. The highest fix version in the table is 3.1.6.

Now count. The summary says “Found 12 known vulnerabilities in 1 package,” but look at the IDs: PYSEC-2019-217 appears twice, and so do all the others. There are 6 advisories, each printed twice. The cause is visible in pip-audit’s source. In _audit.py for version 2.10.1, the docstring promises that “any two results for the same dependency that share an alias are collapsed into a single result with a union of all aliases.” But the code adds every result whose primary ID starts with PYSEC in a first pass without that check, and pip-audit gives the PYSEC ID to a GitHub record that lists a PYSEC alias. So a GitHub record and a PYSEC record for the same advisory both survive. For requests 2.19.1, OSV returned 10 records: five GHSA and five PYSEC IDs, each pair naming the other as an alias, and pip-audit printed 10 rows. Whenever you report numbers, count advisories, not rows. This small module does it by merging records that share any ID or alias:

"""advisories.py: count advisories, not the rows pip-audit prints."""


def unique_advisories(vulns: list) -> list:
    """Collapse records that share any id or alias into one advisory (a set of ids)."""
    groups: list = []
    for vuln in vulns:
        ids = {vuln["id"], *vuln.get("aliases", [])}
        for group in [g for g in groups if g & ids]:
            groups.remove(group)
            ids |= group
        groups.append(ids)
    return groups

If two records share even one ID, they are the same advisory, so their ID sets are merged. The loop also handles chains, where record A shares an ID with B and B with C. When you want to silence a single advisory, --ignore-vuln PYSEC-2018-28 works on the printed ID; in my test it removed both rows for that advisory (10 rows became 8). Use it sparingly and write down why.

What the exit status means

The README’s exit code section lists two codes: 0, “No known vulnerabilities were detected,” and 1, “One or more known vulnerabilities were found.” A CI job reads that status to pass or fail. Remember it, because Step 6 shows a case where a 0 hides something.

Three ways to run it, and what each one sees

pip-audit can read your requirements file as it is, resolve the file with pip first, or look at an installed environment. They are different audits. This script runs all three against both vulnerability services and counts what comes back:

"""compare_modes.py: run pip-audit six ways on one service and count what each way sees.

Run from the lab folder:  python compare_modes.py orders-api
"""
import json
import subprocess
import sys
import time
from pathlib import Path

from advisories import unique_advisories

service = sys.argv[1] if len(sys.argv) > 1 else "orders-api"
service_dir = Path("fleet") / service
req = str(service_dir / "requirements.txt")
site = str(service_dir / ".venv" / "Lib" / "site-packages")

SOURCES = {
    "requirements, --no-deps": ["-r", req, "--no-deps", "--disable-pip"],
    "requirements, resolved": ["-r", req],
    "installed environment": ["--path", site],
}

print(f"{'source':<26}{'backend':<7}{'packages':>9}{'skipped':>8}{'rows':>6}{'unique':>7}{'exit':>6}{'secs':>6}")
for label, source_args in SOURCES.items():
    for backend in ("pypi", "osv"):
        started = time.time()
        proc = subprocess.run(
            [sys.executable, "-m", "pip_audit", *source_args, "-s", backend,
             "-f", "json", "--progress-spinner", "off"],
            capture_output=True, text=True,
        )
        deps = json.loads(proc.stdout)["dependencies"]
        rows = sum(len(d.get("vulns", [])) for d in deps)
        unique = sum(len(unique_advisories(d.get("vulns", []))) for d in deps)
        skipped = sum(1 for d in deps if "skip_reason" in d)
        print(f"{label:<26}{backend:<7}{len(deps):>9}{skipped:>8}{rows:>6}{unique:>7}"
              f"{proc.returncode:>6}{time.time() - started:>6.1f}")

python compare_modes.py orders-api

source                    backend packages skipped  rows unique  exit  secs
requirements, --no-deps   pypi           1       0    10      5     1   0.6
requirements, --no-deps   osv            1       0    10      5     1   1.1
requirements, resolved    pypi           5       0    37     19     1  13.8
requirements, resolved    osv            5       0    38     19     1  14.8
installed environment     pypi           6       0    39     20     1   0.8
installed environment     osv            6       0    40     20     1   2.3

Read the table row by row. The --no-deps audit saw 1 package and 5 unique advisories: it audited only what a human typed. The resolved audit had pip work out the full set in a scratch environment, found 5 packages and 19 advisories, and took about 15 seconds. The audit of the installed environment saw 6 packages and 20 advisories in one or two seconds, and the extra advisory belongs to pip itself (more on that below). The two services, PyPI and OSV, agree on the unique counts; the row counts differ by one because OSV returned a second record for one urllib3 advisory. So: audit what is actually installed, right after installing it in CI, because that is what ships. Use --no-deps only for fully pinned lock files that already list every transitive package.

About that pip advisory: the pip that venv installed (26.1.2) is itself flagged. OSV’s summary reads “pip would incorrectly handle doubly-encoded package URLs from indexes,” and the fix is pip 26.2. Upgrade pip in your build step (python -m pip install --upgrade pip) and the noise disappears from every service at once.

Step 6: The deleted-release trap

Now audit the gateway, whose only requirement is the deleted release. Run pip-audit three ways: with the default PyPI data, again with --strict, and with the OSV service:

$ pip-audit -r fleet/gateway/requirements.txt --no-deps --disable-pip
No known vulnerabilities found
Name    Skip Reason
------- ------------------------------------------------------------------------
litellm Dependency not found on PyPI and could not be audited: litellm (1.107.0)
(exit status 0)

$ pip-audit -r fleet/gateway/requirements.txt --no-deps --disable-pip --strict
ERROR:pip_audit._cli:litellm: Dependency not found on PyPI and could not be audited: litellm (1.107.0)
(exit status 1)

$ pip-audit -r fleet/gateway/requirements.txt --no-deps --disable-pip -s osv
Found 1 known vulnerability in 1 package
Name    Version ID           Fix Versions
------- ------- ------------ ------------
litellm 1.107.0 PYSEC-2026-2
(exit status 1)

(Again, I dropped pip-audit’s two warnings about hashes.) Read the first result slowly. It says “No known vulnerabilities found” and the exit status is 0, so a CI job would pass. Only the table under it, “Dependency not found on PyPI and could not be audited,” says the release was skipped. pip-audit’s PyPI service asks for one JSON page per release, here pypi.org/pypi/litellm/1.107.0/json, and that page is gone: when I checked with curl, it returned 404 while the page for 1.106.2 returned 200. The README’s security model says you can assume pip-audit will make a best effort to resolve your dependencies and “either fully audit each or explicitly state which ones it has skipped, as well as why it has skipped them.” It does state it, in a table, and still exits 0.

The second run shows the fix on the PyPI side. The --strict flag is documented as “fail the entire audit if dependency collection fails on any dependency,” and here it turns the skip into an error and exit status 1. The third run uses OSV, which answers by package name and version and does not need the release to exist on PyPI, so it reports the advisory. The ID column says PYSEC-2026-2; with JSON output (-f json) its aliases list MAL-2026-2144, the malicious-package record. There are no fix versions, because the whole version is bad.

When did the scanner learn about it?

The OSV answer only helps once a record exists. I read the history of that record in GitHub’s public API to see when. The record, MAL-2026-2144, was first published on March 24 for two earlier bad LiteLLM versions, 1.82.7 and 1.82.8, and its text says 1.107.0 was “uploaded to PyPI on 2026-10-21 between 11:39 and 11:40 UTC.” This is the timeline of what the repository holds, in UTC on October 21:

Time	What happened	Where I read it
11:39:51	The first poisoned wheel is live on PyPI	LiteLLM’s timestamps, as reported in our earlier post
13:23	PyPI no longer serves the wheels	same
21:46:03	A commit titled “Add litellm 1.107.0 (PyPI): credential stealer and backdoor (#1655)” is dated in the OpenSSF malicious-packages repository	GitHub commits API
21:48:09	A bot commit titled “Assign IDs” replaces the pending file with a change to MAL-2026-2144	GitHub commits API
22:01:01	OSV’s API reports MAL-2026-2144 as modified	api.osv.dev

By my arithmetic from those times, the first commit for 1.107.0 came 10 hours 6 minutes after the first wheel appeared, and 8 hours 23 minutes after PyPI stopped serving the wheels. I could see the repository’s history, not the moment OSV’s servers first returned the entry, so treat the 21:46 commit as the earliest time I can show for a scanner that reads OSV. The report’s own published field reads 12:23:26, about 44 minutes after the upload, but that is when its author wrote the record, not when the repository accepted it. None of this is a criticism of the people who filed and merged the report; it is how advisory feeds work. A scanner can only know what someone has already written down. In the hours before that, a scanner that reads OSV could not have told you whether you had installed it; your own inventory could.

What about pip-audit’s default mode?

The runs above used --no-deps --disable-pip. Without those flags, pip-audit asks pip to resolve the requirements in a scratch environment first, and for a version PyPI no longer has, that resolution cannot succeed. I ran this default mode on the deleted pin five times on this machine, with pip-audit 2.10.1 on Windows 11. It never printed a result and never returned: I stopped runs after 90 seconds, after 100 seconds, twice after 240 seconds and once after more than four minutes. I did not find out why it keeps running. The practical lesson is to give any CI step that runs pip-audit a time limit, as this probe does:

"""probe_resolve.py: run pip-audit in its default, pip-resolving mode on the deleted LiteLLM pin
and stop it if it is still running after 240 seconds."""
import subprocess
import sys

with open("probe.out", "w", encoding="utf-8") as out:
    proc = subprocess.Popen(
        [sys.executable, "-m", "pip_audit", "-r", "fleet/gateway/requirements.txt", "-s", "pypi"],
        stdout=out, stderr=out,
    )
    try:
        proc.wait(timeout=240)
        print("finished with exit status", proc.returncode)
    except subprocess.TimeoutExpired:
        subprocess.run(["taskkill", "/PID", str(proc.pid), "/T", "/F"], capture_output=True)
        print("still running after 240 seconds, so I stopped it")
with open("probe.out", encoding="utf-8", errors="replace") as handle:
    lines = handle.read().splitlines()
print(f"pip-audit had printed {len(lines)} lines, none of them a result")

python probe_resolve.py

still running after 240 seconds, so I stopped it
pip-audit had printed 0 lines, none of them a result

The probe always uses its full four minutes, so start it, let it finish while you read on, and do not wait for it.

Step 7: Archive an SBOM for every build

An SBOM answers questions only if you kept it. The script below walks the fleet and writes one SBOM per service into sboms/archive: an environment SBOM wherever there is an environment, and a requirements SBOM for the gateway, which we never installed. It also writes a requirements SBOM for every service into sboms/req, so we can compare the two kinds in Step 8.

"""gen_sboms.py: write one CycloneDX SBOM per service into sboms/archive (and sboms/req for comparison).

Run from the lab folder:  python gen_sboms.py
Services with a .venv get an environment SBOM in the archive; the gateway, which we never
install, gets a requirements SBOM there instead.
"""
import subprocess
import sys
from pathlib import Path

FLEET = Path("fleet")
ARCHIVE = Path("sboms") / "archive"
REQ = Path("sboms") / "req"


def cyclonedx(*args) -> None:
    subprocess.run([sys.executable, "-m", "cyclonedx_py", *map(str, args)], check=True)


def main() -> int:
    ARCHIVE.mkdir(parents=True, exist_ok=True)
    REQ.mkdir(parents=True, exist_ok=True)
    for service_dir in sorted(p for p in FLEET.iterdir() if p.is_dir()):
        name = service_dir.name
        req_file = service_dir / "requirements.txt"
        cyclonedx("requirements", req_file, "-o", REQ / f"{name}.cdx.json")
        venv_dir = service_dir / ".venv"
        if venv_dir.is_dir():
            cyclonedx("environment", venv_dir, "-o", ARCHIVE / f"{name}.cdx.json")
            source = "environment"
        else:
            cyclonedx("requirements", req_file, "-o", ARCHIVE / f"{name}.cdx.json")
            source = "requirements"
        print(f"{name}: archived an SBOM built from the {source}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

python gen_sboms.py

billing-api: archived an SBOM built from the environment
gateway: archived an SBOM built from the requirements
notifier: archived an SBOM built from the environment
orders-api: archived an SBOM built from the environment
reports-worker: archived an SBOM built from the environment

Notice that the script runs sys.executable -m cyclonedx_py instead of looking for a program on the path. That guarantees it uses the tool installed in the active environment. In a real pipeline you would run this in the build job right after pip install, name the file after the service and the commit (for example billing-api/3f9c2e1.cdx.json) and store it next to the build artifact. Keep each SBOM for as long as that build might still be running anywhere. Here the archive keeps one file per service, so each run overwrites the last.

Check: sboms/archive and sboms/req each hold five .cdx.json files.

Step 8: Hunt for a package across every SBOM

This is the script that answers “did any of our services install it?” in seconds. It reads every SBOM in a folder and reports each component whose name matches and whose version falls in a range you give. It also says which component pulled it in and when the SBOM was made. The code is longer than the others, so read the paragraphs after it:

"""sbom_hunt.py: find a package version range across a folder of CycloneDX SBOMs.

Usage:    python sbom_hunt.py SBOM_FOLDER PACKAGE SPECIFIER
Example:  python sbom_hunt.py sboms/archive urllib3 "<1.26.5"
Exit status: 0 nothing found, 1 at least one hit, 2 bad input or unreadable SBOM.
"""
from __future__ import annotations

import argparse
import json
import sys
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path

from packaging.specifiers import InvalidSpecifier, SpecifierSet
from packaging.utils import canonicalize_name
from packaging.version import InvalidVersion, Version

SUFFIX = ".cdx.json"


@dataclass(frozen=True)
class Hit:
    service: str
    name: str
    version: str
    status: str  # "affected" or "unreadable-version"
    pulled_in_by: tuple  # names of the components that depend on it; empty means top level
    built: str = "(unknown)"  # when the SBOM was generated, in UTC


def plural(count: int, one: str, many: str) -> str:
    return f"{count} {one if count == 1 else many}"


def iter_components(components):
    """Yield every component, including components nested inside other components."""
    for comp in components or []:
        yield comp
        yield from iter_components(comp.get("components"))


def built_at(sbom: dict) -> str:
    """The SBOM's own timestamp as 'YYYY-MM-DD HH:MM' in UTC, or '(unknown)'."""
    raw = (sbom.get("metadata") or {}).get("timestamp")
    if not raw:
        return "(unknown)"
    try:
        return datetime.fromisoformat(raw).astimezone(timezone.utc).strftime("%Y-%m-%d %H:%M")
    except ValueError:
        return str(raw)[:16]


def parents_of(sbom: dict) -> dict:
    """Map a component's bom-ref to the 'name==version' labels of the components that depend on it."""
    by_ref = {c["bom-ref"]: c for c in iter_components(sbom.get("components")) if "bom-ref" in c}
    parents: dict = {}
    for dep in sbom.get("dependencies", []):
        parent = by_ref.get(dep.get("ref"))
        if parent is None:
            continue
        label = f"{parent.get('name')}=={parent.get('version')}"
        for child_ref in dep.get("dependsOn", []):
            parents.setdefault(child_ref, []).append(label)
    return parents


def hunt(sbom: dict, service: str, name: str, spec: SpecifierSet) -> list:
    """Return a Hit for every PyPI component called `name` whose version falls inside `spec`."""
    wanted = canonicalize_name(name)
    parents = parents_of(sbom)
    built = built_at(sbom)
    hits = []
    for comp in iter_components(sbom.get("components")):
        purl = comp.get("purl", "")
        if purl and not purl.startswith("pkg:pypi/"):
            continue  # same name, different ecosystem
        if canonicalize_name(comp.get("name", "")) != wanted:
            continue
        raw = comp.get("version", "")
        try:
            version = Version(raw)
        except InvalidVersion:
            # Never skip silently: a component we cannot compare is a component a human must check.
            hits.append(Hit(service, comp["name"], raw or "(none)", "unreadable-version", (), built))
            continue
        if spec.contains(version, prereleases=True):
            hits.append(Hit(service, comp["name"], raw, "affected",
                            tuple(parents.get(comp.get("bom-ref"), ())), built))
    return hits


def hunt_folder(folder: Path, name: str, spec: SpecifierSet):
    """Hunt through every *.cdx.json file in `folder`. Returns (hits, scanned_count, problems)."""
    hits, problems, scanned = [], [], 0
    for path in sorted(folder.glob("*" + SUFFIX)):
        service = path.name[: -len(SUFFIX)]
        try:
            sbom = json.loads(path.read_text(encoding="utf-8"))
        except (OSError, ValueError) as exc:
            problems.append(f"{path.name}: {exc}")
            continue
        scanned += 1
        hits.extend(hunt(sbom, service, name, spec))
    return hits, scanned, problems


def main(argv=None) -> int:
    ap = argparse.ArgumentParser(description="Find a package version range in archived CycloneDX SBOMs.")
    ap.add_argument("folder", type=Path)
    ap.add_argument("package")
    ap.add_argument("specifier", help='PEP 440 specifier, for example "<1.26.5" or "==1.107.0"')
    args = ap.parse_args(argv)
    try:
        spec = SpecifierSet(args.specifier)
    except InvalidSpecifier as exc:
        print(f"bad specifier: {exc}", file=sys.stderr)
        return 2
    if not args.folder.is_dir():
        print(f"not a folder: {args.folder}", file=sys.stderr)
        return 2

    hits, scanned, problems = hunt_folder(args.folder, args.package, spec)
    for line in problems:
        print(f"UNREADABLE {line}", file=sys.stderr)
    if hits:
        print(f"{'SERVICE':<16}{'PACKAGE':<10}{'VERSION':<10}{'STATUS':<20}{'BUILT (UTC)':<18}PULLED IN BY")
        for h in hits:
            who = ", ".join(h.pulled_in_by) or "(top level)"
            print(f"{h.service:<16}{h.name:<10}{h.version:<10}{h.status:<20}{h.built:<18}{who}")
    print(f"scanned {plural(scanned, 'SBOM', 'SBOMs')}, {plural(len(hits), 'hit', 'hits')} for {args.package} {args.specifier}")
    if problems:
        return 2
    return 1 if hits else 0


if __name__ == "__main__":
    sys.exit(main())

Four details carry most of the weight. First, names are compared with canonicalize_name. The Python Packaging User Guide’s name normalization rule lowercases a name and collapses runs of dots, hyphens and underscores into one hyphen, so Jinja2, jinja2 and JINJA2 are one package and so are ruamel.yaml and ruamel_yaml. A plain string comparison would miss real hits. Second, versions are compared as versions with SpecifierSet, not as text, because as text “1.10.0” sorts before “1.9”. PEP 440 also says that for a specifier without a local version label, “the local version label of any candidate versions MUST be ignored when matching versions,” so ==1.107.0 matches a build labelled 1.107.0+internal, which is what you want when a company rebuilds a package. Third, a component whose version cannot be read is reported as unreadable-version, never skipped, and an SBOM file that cannot be parsed makes the whole run exit 2. A hunt that quietly skips what it cannot read gives false comfort. Fourth, components in other ecosystems are ignored, so an npm package that happens to be called requests does not count.

Now run five hunts. The exit status is 1 when something is found, 0 when nothing is, and 2 when the input was bad. First, the question that started this tutorial:

python sbom_hunt.py sboms/archive litellm "==1.107.0"

SERVICE         PACKAGE   VERSION   STATUS              BUILT (UTC)       PULLED IN BY
gateway         litellm   1.107.0   affected            2026-10-25 20:40  (top level)
scanned 5 SBOMs, 1 hit for litellm ==1.107.0
(exit status 1)

The gateway has it, and the BUILT column says the SBOM was made at that time, so a hit means that the build which produced this SBOM contained the package. Always check whether the service has been rebuilt since. Next, a package nobody chose. Suppose an advisory says urllib3 before 1.25.9 is affected; PYSEC-2020-148, for CVE-2020-26137, says “urllib3 before 1.25.9 allows CRLF injection if the attacker controls the HTTP request method”.

python sbom_hunt.py sboms/archive urllib3 "<1.25.9"

SERVICE         PACKAGE   VERSION   STATUS              BUILT (UTC)       PULLED IN BY
notifier        urllib3   1.25.3    affected            2026-10-25 20:40  (top level)
orders-api      urllib3   1.23      affected            2026-10-25 20:40  requests==2.19.1
scanned 5 SBOMs, 2 hits for urllib3 <1.25.9
(exit status 1)

Two services have it. notifier pins it itself, and orders-api has it only because requests==2.19.1 pulled it in, which the last column shows. Now run the same hunt against the SBOMs built from requirements files:

python sbom_hunt.py sboms/req urllib3 "<1.25.9"

SERVICE         PACKAGE   VERSION   STATUS              BUILT (UTC)       PULLED IN BY
notifier        urllib3   1.25.3    affected            2026-10-25 20:40  (top level)
scanned 5 SBOMs, 1 hit for urllib3 <1.25.9
(exit status 1)

Only notifier shows up. orders-api is still running a vulnerable urllib3, and this archive cannot see it. That is the false negative from Step 4, and it is the strongest reason to archive SBOMs of installed environments. The last two hunts show name matching and the empty case:

python sbom_hunt.py sboms/archive JINJA2 "<3"

SERVICE         PACKAGE   VERSION   STATUS              BUILT (UTC)       PULLED IN BY
reports-worker  Jinja2    2.10      affected            2026-10-25 20:40  (top level)
scanned 5 SBOMs, 1 hit for JINJA2 <3
(exit status 1)

python sbom_hunt.py sboms/archive requests ">=3"

scanned 5 SBOMs, 0 hits for requests >=3
(exit status 0)

Searching for JINJA2 found the component stored as Jinja2. The last hunt found nothing and exited 0. Because of the exit statuses you can also use the script as a build gate: keep a short list of known-bad releases such as litellm ==1.107.0, run one hunt per entry on each new SBOM, and fail the build when any returns 1.

Step 9: Re-audit the archive on a schedule

Hunting needs you to know what to look for. Auditing finds what a database already knows. Here is the missing piece: re-audit the SBOMs you stored, so that when a new advisory appears, yesterday’s builds are checked against it without anyone touching the services. The script turns each SBOM back into name==version pins, audits them with pip-audit against OSV, and prints unique advisories per package:

"""sbom_audit.py: re-audit archived CycloneDX SBOMs against OSV with pip-audit.

Usage:    python sbom_audit.py SBOM_FOLDER
Exit status: 0 no advisories, 1 advisories found, 2 pip-audit or an SBOM failed.
"""
from __future__ import annotations

import argparse
import json
import subprocess
import sys
import tempfile
from pathlib import Path

from advisories import unique_advisories
from sbom_hunt import SUFFIX, iter_components, plural


def pins_from_sbom(sbom: dict) -> list:
    """Return sorted, de-duplicated 'name==version' pins for every PyPI component that has a version."""
    pins = set()
    for comp in iter_components(sbom.get("components")):
        purl = comp.get("purl", "")
        if purl and not purl.startswith("pkg:pypi/"):
            continue
        if comp.get("name") and comp.get("version"):
            pins.add(f"{comp['name']}=={comp['version']}")
    return sorted(pins, key=str.lower)


def run_pip_audit(pins: list) -> list:
    """Audit exact pins against OSV. Returns pip-audit's 'dependencies' list."""
    with tempfile.TemporaryDirectory() as tmp:
        req = Path(tmp) / "requirements.txt"
        req.write_text("\n".join(pins) + "\n", encoding="utf-8")
        proc = subprocess.run(
            [sys.executable, "-m", "pip_audit", "-r", str(req), "--no-deps", "--disable-pip",
             "-s", "osv", "-f", "json", "--progress-spinner", "off"],
            capture_output=True, text=True, encoding="utf-8",
        )
    # pip-audit exits 1 when it finds advisories, so the exit code alone cannot tell
    # "found something" from "crashed". Valid JSON on stdout is the success signal.
    try:
        return json.loads(proc.stdout)["dependencies"]
    except (ValueError, KeyError):
        raise RuntimeError(f"pip-audit failed (exit {proc.returncode}): {proc.stderr.strip()[-300:]}")


def main(argv=None) -> int:
    ap = argparse.ArgumentParser(description="Re-audit archived CycloneDX SBOMs with pip-audit and OSV.")
    ap.add_argument("folder", type=Path)
    args = ap.parse_args(argv)

    found_any, failed = False, False
    for path in sorted(args.folder.glob("*" + SUFFIX)):
        service = path.name[: -len(SUFFIX)]
        try:
            pins = pins_from_sbom(json.loads(path.read_text(encoding="utf-8")))
            deps = run_pip_audit(pins)
        except (OSError, ValueError, RuntimeError) as exc:
            print(f"{service}: FAILED {exc}")
            failed = True
            continue
        flagged = [(d, unique_advisories(d.get("vulns", []))) for d in deps if d.get("vulns")]
        total = sum(len(groups) for _, groups in flagged)
        print(f"{service}: {plural(len(pins), 'package', 'packages')}, {len(flagged)} with advisories, "
              f"{plural(total, 'unique advisory', 'unique advisories')}")
        for dep, groups in flagged:
            print(f"    {dep['name']}=={dep['version']}: {plural(len(groups), 'unique advisory', 'unique advisories')}")
        found_any = found_any or bool(flagged)
    if failed:
        return 2
    return 1 if found_any else 0


if __name__ == "__main__":
    sys.exit(main())

Two decisions in run_pip_audit deserve a mention. It uses OSV, because in Step 6 the PyPI service could not see a deleted release. And it does not treat exit status 1 as a failure, since that is what pip-audit returns when it finds advisories; instead it requires valid JSON on standard output, and raises an error, so the whole script exits 2, if there is none. Run it:

python sbom_audit.py sboms/archive

billing-api: 8 packages, 1 with advisories, 1 unique advisory
    pip==26.1.2: 1 unique advisory
gateway: 1 package, 1 with advisories, 1 unique advisory
    litellm==1.107.0: 1 unique advisory
notifier: 2 packages, 2 with advisories, 12 unique advisories
    pip==26.1.2: 1 unique advisory
    urllib3==1.25.3: 11 unique advisories
orders-api: 6 packages, 4 with advisories, 20 unique advisories
    idna==2.7: 2 unique advisories
    pip==26.1.2: 1 unique advisory
    requests==2.19.1: 5 unique advisories
    urllib3==1.23: 12 unique advisories
reports-worker: 3 packages, 2 with advisories, 7 unique advisories
    jinja2==2.10: 6 unique advisories
    pip==26.1.2: 1 unique advisory
(exit status 1)

All five SBOMs took about 12 seconds. The gateway is flagged for litellm==1.107.0 straight from its stored SBOM, even though the service was never installed. billing-api, the one built from current packages, is flagged only for pip, the advisory from Step 5. orders-api has 20 unique advisories across four packages, including 12 for the urllib3 nobody chose. Note that exit status 1 here means something was found, so in a scheduled job you would send the printed report somewhere a person reads it, not just fail the run.

To schedule it, run python sbom_audit.py sboms/archive from Windows Task Scheduler, cron or your CI system’s scheduled pipeline, nightly is a sensible start, with a time limit as in Step 6. I did not set up a schedule in this lab.

Step 10: Test it, then break it on purpose

The scripts decide whether you hear about a bad package, so they deserve tests. These 18 tests run offline, using small SBOMs built in code:

"""test_sbom_tools.py: offline tests for sbom_hunt.py and sbom_audit.py. Run with: python -m pytest -q"""
import json
import subprocess

import pytest
from packaging.specifiers import SpecifierSet

import advisories
import sbom_audit
import sbom_hunt


def comp(name, version, ref=None, purl=None, **extra):
    c = {"type": "library", "name": name, "bom-ref": ref or f"{name}=={version}",
         "purl": purl or f"pkg:pypi/{name.lower()}@{version}"}
    if version is not None:
        c["version"] = version
    c.update(extra)
    return c


def sbom_of(*components, dependencies=()):
    return {"bomFormat": "CycloneDX", "specVersion": "1.6", "components": list(components),
            "dependencies": list(dependencies)}


def test_iter_components_finds_nested_components():
    inner = comp("urllib3", "1.23")
    outer = comp("vendored-bundle", "1.0", components=[inner])
    names = [c["name"] for c in sbom_hunt.iter_components([outer])]
    assert names == ["vendored-bundle", "urllib3"]


@pytest.mark.parametrize("component_name, searched", [
    ("Jinja2", "jinja2"), ("jinja2", "JINJA2"), ("ruamel.yaml", "ruamel_yaml"), ("Flask_Cors", "flask-cors"),
])
def test_names_are_compared_in_canonical_form(component_name, searched):
    hits = sbom_hunt.hunt(sbom_of(comp(component_name, "1.0")), "svc", searched, SpecifierSet("<2"))
    assert len(hits) == 1


def test_versions_compare_as_versions_not_as_text():
    # As text, "1.10.0" < "1.9" is True. As versions it is False.
    sbom = sbom_of(comp("demo", "1.10.0"))
    assert sbom_hunt.hunt(sbom, "svc", "demo", SpecifierSet("<1.9")) == []
    assert len(sbom_hunt.hunt(sbom, "svc", "demo", SpecifierSet(">=1.9"))) == 1


def test_local_and_pre_release_versions_are_matched():
    sbom = sbom_of(comp("litellm", "1.107.0+internal"), comp("other", "1.107.0rc1"))
    assert len(sbom_hunt.hunt(sbom, "svc", "litellm", SpecifierSet("==1.107.0"))) == 1
    assert len(sbom_hunt.hunt(sbom, "svc", "other", SpecifierSet("==1.107.0rc1"))) == 1


@pytest.mark.parametrize("version, shown", [("latest", "latest"), (None, "(none)")])
def test_components_with_unreadable_versions_are_reported_not_skipped(version, shown):
    hits = sbom_hunt.hunt(sbom_of(comp("demo", version)), "svc", "demo", SpecifierSet("<2"))
    assert [(h.status, h.version) for h in hits] == [("unreadable-version", shown)]


def test_other_ecosystems_with_the_same_name_are_ignored():
    sbom = sbom_of(comp("requests", "1.0.0", purl="pkg:npm/[email protected]"))
    assert sbom_hunt.hunt(sbom, "svc", "requests", SpecifierSet("<2")) == []


def test_hit_reports_which_component_pulled_it_in():
    sbom = sbom_of(
        comp("requests", "2.19.1"), comp("urllib3", "1.23"),
        dependencies=[{"ref": "requests==2.19.1", "dependsOn": ["urllib3==1.23"]}, {"ref": "urllib3==1.23"}],
    )
    (hit,) = sbom_hunt.hunt(sbom, "orders-api", "urllib3", SpecifierSet("<1.26.5"))
    assert hit.pulled_in_by == ("requests==2.19.1",)
    (top,) = sbom_hunt.hunt(sbom, "orders-api", "requests", SpecifierSet("<3"))
    assert top.pulled_in_by == ()


def test_hit_reports_when_the_sbom_was_built_in_utc():
    sbom = sbom_of(comp("demo", "1.0"))
    sbom["metadata"] = {"timestamp": "2026-10-25T22:08:29.189219+02:00"}
    (hit,) = sbom_hunt.hunt(sbom, "svc", "demo", SpecifierSet("<2"))
    assert hit.built == "2026-10-25 20:08"
    sbom["metadata"] = {}
    assert sbom_hunt.hunt(sbom, "svc", "demo", SpecifierSet("<2"))[0].built == "(unknown)"


def test_requirements_style_sbom_cannot_see_transitive_packages():
    # An SBOM built from requirements.txt lists only what was typed, so a hunt for urllib3 finds nothing.
    sbom = sbom_of(comp("requests", "2.19.1", ref="requirements-L1"))
    assert sbom_hunt.hunt(sbom, "orders-api", "urllib3", SpecifierSet("<1.26.5")) == []


def write_sbom(folder, name, sbom):
    (folder / f"{name}.cdx.json").write_text(json.dumps(sbom), encoding="utf-8")


def test_cli_exit_codes(tmp_path, capsys):
    write_sbom(tmp_path, "svc-a", sbom_of(comp("litellm", "1.107.0")))
    write_sbom(tmp_path, "svc-b", sbom_of(comp("litellm", "1.106.2")))
    assert sbom_hunt.main([str(tmp_path), "litellm", "==1.107.0"]) == 1
    out = capsys.readouterr().out
    assert "svc-a" in out and "svc-b" not in out and "scanned 2 SBOMs, 1 hit for litellm ==1.107.0" in out
    assert sbom_hunt.main([str(tmp_path), "litellm", "==9.9.9"]) == 0
    assert sbom_hunt.main([str(tmp_path), "litellm", "not a specifier"]) == 2
    assert sbom_hunt.main([str(tmp_path / "missing"), "litellm", "==1"]) == 2


def test_unreadable_sbom_file_fails_the_hunt_instead_of_being_skipped(tmp_path, capsys):
    write_sbom(tmp_path, "good", sbom_of(comp("demo", "1.0")))
    (tmp_path / "broken.cdx.json").write_text("{not json", encoding="utf-8")
    assert sbom_hunt.main([str(tmp_path), "nothing-here", "<1"]) == 2
    assert "UNREADABLE broken.cdx.json" in capsys.readouterr().err


def test_pins_from_sbom_skips_other_ecosystems_and_missing_versions():
    sbom = sbom_of(
        comp("Zeta", "1.0"), comp("alpha", "2.0"), comp("alpha", "2.0", ref="dup"),
        comp("left-pad", "1.3.0", purl="pkg:npm/[email protected]"), comp("no-version", None),
    )
    assert sbom_audit.pins_from_sbom(sbom) == ["alpha==2.0", "Zeta==1.0"]


def test_unique_advisories_merges_records_that_share_any_id():
    vulns = [
        {"id": "PYSEC-1", "aliases": ["GHSA-1", "CVE-1"]},
        {"id": "PYSEC-1", "aliases": ["GHSA-1", "CVE-1"]},  # same advisory from a second database
        {"id": "GHSA-2", "aliases": ["CVE-2"]},
        {"id": "PYSEC-3", "aliases": ["CVE-2"]},
        {"id": "PYSEC-4", "aliases": []},
    ]
    groups = advisories.unique_advisories(vulns)
    assert sorted(len(g) for g in groups) == [1, 3, 3]


class FakeRun:
    def __init__(self, stdout, stderr, returncode):
        self.stdout, self.stderr, self.returncode = stdout, stderr, returncode


def test_exit_code_one_with_json_is_success_but_no_json_is_failure(monkeypatch):
    payload = json.dumps({"dependencies": [{"name": "demo", "version": "1", "vulns": []}], "fixes": []})
    monkeypatch.setattr(subprocess, "run", lambda *a, **k: FakeRun(payload, "", 1))
    assert sbom_audit.run_pip_audit(["demo==1"])[0]["name"] == "demo"
    monkeypatch.setattr(subprocess, "run", lambda *a, **k: FakeRun("", "ERROR: network down", 1))
    with pytest.raises(RuntimeError, match="network down"):
        sbom_audit.run_pip_audit(["demo==1"])

python -m pytest -q

.................. [100%]
18 passed in 0.06s

Passing tests only prove something if they fail when the code is wrong. I made eight edits to the scripts, one at a time, in a scratch copy, and ran the suite after each. Every one made at least one test fail:

Edit	Test that fails
Compare names with `lower()` instead of `canonicalize_name`	`test_names_are_compared_in_canonical_form`
Ignore nested components	`test_iter_components_finds_nested_components`
Drop the ecosystem filter	`test_other_ecosystems_with_the_same_name_are_ignored`
Skip unreadable versions silently	`test_components_with_unreadable_versions_are_reported_not_skipped`
Skip unreadable SBOM files silently	`test_unreadable_sbom_file_fails_the_hunt_instead_of_being_skipped`
Forget which component pulled a package in	`test_hit_reports_which_component_pulled_it_in`
Treat pip-audit’s exit status 1 as a crash	`test_exit_code_one_with_json_is_success_but_no_json_is_failure`
Do not merge records that share an ID	`test_unique_advisories_merges_records_that_share_any_id`

You can try any row by hand. For example, in advisories.py change if g & ids to if False, run python -m pytest -q, and watch the merge test fail; then change it back. I automated the eight edits with a throwaway script, which I have not printed here.

Common mistakes and limits

Reading exit status 0 as “clean”

Audited as a pinned list against PyPI data, a skipped dependency still exits 0. Read the skip table, add --strict, or query OSV as well. The README’s own security model says to “explicitly state which ones it has skipped,” which is a promise about the report, not about the exit status.

Auditing the file instead of the environment

A pinned top-level requirements file is not the set of installed packages. In Step 5 the file-only audit saw 5 of the 20 advisories. Audit the installed environment, or resolve the file first and accept the extra time.

Archiving requirements SBOMs and calling it an inventory

Step 8 showed the cost: a service running a vulnerable urllib3 looked clean. Use requirements SBOMs only when the environment cannot exist, as with the gateway here, and say so in the file name or folder.

Counting rows

pip-audit printed 12 rows for 6 advisories. Report unique advisories, as advisories.py does.

Treating an SBOM as proof of safety

An SBOM lists what was installed, not what the code does. pip-audit’s README is blunt about its own limits: “you must not assume that pip-audit will defend you against malicious packages,” because it checks for known vulnerabilities in the dependency tree and is not a code analyzer. The LiteLLM payload shows why this matters twice. It ran from a .pth file, so a service that never imported litellm was still exposed and searching your code for import litellm would not have found it, and an SBOM only tells you the version was there. Also remember what this lab leaves out: packages installed at container start, vendored copies of libraries, operating system packages, other languages, and development tools you never put in the requirements file.

Trusting the dependency graph too much

The graph came from installed metadata and included edges the package metadata does not support, as the urllib3 example showed. Use PULLED IN BY to find where to look, then check the package’s own requirements.

Check that everything works end to end

From a clean folder with the active tools environment, this order reproduces everything above: python make_fleet.py builds the services, python gen_sboms.py archives their SBOMs, the five sbom_hunt.py commands in Step 8 print the same tables (apart from the BUILT times), python sbom_audit.py sboms/archive prints the audit with exit status 1, and python -m pytest -q prints 18 passed. If the audit shows fewer advisories than mine, suspect a network problem and look for a line that starts with FAILED; the script exits 2 when pip-audit cannot reach OSV, so a quiet run can never be mistaken for a clean one.

Where to go next

You now have an inventory you can search and a scheduled audit that reads it. A few ways to extend it:

Pin and verify what you install: our pylock.toml tutorial records every package, its exact version and a hash of the file, so nothing transitive is missing from your inventory.
Slow down fresh releases: a dependency cooldown keeps any release younger than your chosen delay out of a build, and this one lived for under two hours.
Go beyond one language and one package manager: container SBOMs with Syft and Cosign, and a Neo4j dependency graph for tracing vulnerable packages across many projects.
Read the wider checklist in our software supply chain security checklist and the incident that started this tutorial in our LiteLLM report.
Turn the hunt into a deny-list gate that runs on every build, and send the nightly audit report to the team that owns each service.

Tools used: pip-audit 2.10.1, cyclonedx-bom 7.5.0 (CycloneDX Python), packaging 26.3 and pytest 9.1.1, all on Python 3.13.14.

Tags:

Search the Site

Recent Posts

Categories

Articles 268 Posts

News 271 Posts

Learning Hub 241 Posts