A Poisoned LiteLLM 1.107.0 Sat on PyPI for 103 Minutes, Published by the Project’s Own Pipeline From a Repository One Hyphen Away From the Real One
LiteLLM says attackers used three engineers’ stolen GitHub tokens to rewrite its release workflow and publish version 1.107.0 from a lookalike repository; PyPI carried the credential stealer for 1...
On October 21, attackers published LiteLLM 1.107.0 to PyPI and to the project’s container registries with a credential stealer and a remote-code loader inside. LiteLLM is an open-source Python SDK and AI gateway that Socket puts at about 23 million downloads a week. By LiteLLM’s own timestamps the seven poisoned PyPI wheels were live for 1 hour 43 minutes, from 11:39:51 to 13:23 UTC, and ClickPy records 49,043 downloads of the version that day.
Table Of Content
- What was published, and for how long
- How the poisoned build got through
- The hardening that followed March did not cover this
- What the payload does, and what we can say about its second stage
- How to check a host without starting Python
- What LiteLLM says about the stolen tokens, and what it has not explained
- Where the accounts differ
- What is still unknown
- Sources and method
The route is the story. LiteLLM says the attacker held GitHub CLI OAuth tokens belonging to three engineers, used an organization owner’s token to disable branch protection and force-push a change to the private release workflow, and then ran that workflow “pointed at a lookalike repository (Berri-AI/litellm instead of BerriAI/litellm), which signed and published the poisoned artifacts.” The official v1.107.0 source tag, LiteLLM says, “does not contain the malicious files.” It is the project’s second compromise this year: in March, TeamPCP published two malicious versions, a campaign we covered in our report on Google’s TeamPCP mole.
The figures are as of 10:30 UTC on October 23. PyPI’s JSON API lists no 1.107.0 and names 1.106.2, uploaded at 10:44:51 UTC on October 21, as the latest release; on October 22, LiteLLM wrote that “Our registries remain locked to new releases, and regular LiteLLM releases remain paused.” How the attacker first got in is still unresolved. This report gives the exposure windows, a timeline with our own checks added, what the build records show, how to look for the payload without starting Python, and where the published accounts disagree.
What was published, and for how long
| Channel | What was affected | Live on October 21 (UTC) | Length (our arithmetic) |
|---|---|---|---|
| PyPI | litellm==1.107.0, all seven wheels (macOS x86-64 and arm64, manylinux and musllinux on x86-64 and aarch64, Windows amd64). The source distribution has the tampered build line but not the payload files. |
11:39:51 to 13:23 | 1 h 43 min |
| Docker Hub | Tags v1.107.0, latest, main-latest and main-stable, pointing at image index sha256:1bbcb1443bc9 (full digest below) |
11:46:56 to 14:20 | 2 h 33 min |
| GHCR | The same four tags on litellm, litellm-database and litellm-non_root |
11:46:56 until unavailable by 13:51 | up to 2 h 4 min |
LiteLLM says customers on its private enterprise registry were not affected. ClickPy’s public ClickHouse database records 49,043 downloads of 1.107.0 on October 21, which is 1.4 percent of all LiteLLM downloads that day, and 331 on October 22, after the removal. A download is not an infection: a caching proxy can turn one download into many installs, and a scanner can fetch a file without installing it. OSV lists the package as MAL-2026-2144 (“Malicious code in litellm (PyPI)”), so scanners that read OSV can flag it.
How the poisoned build got through
| When (UTC) | What happened | Source |
|---|---|---|
| Oct 13, 00:45:03 | The domain litellm.dev, later the exfiltration address, is registered. Google’s registry still lists it, with the status client transfer prohibited, at 10:30 UTC on October 23. |
Google Registry RDAP (our check) |
| Oct 21, 08:56:59 to 10:46:29 | Ten commits land in the lookalike repository. Nine carry GitHub’s “Verified” flag, which Wiz explains by their having been created on GitHub.com; the last, which adds copy lines to the Dockerfiles, is unsigned. They touch ten files, including a new cook/purelib/ folder with the three payload files. The repository is gone, but a fork shares objects with its parent, so BerriAI/litellm still serves the commits. |
GitHub compare API (our check); Wiz |
| 10:08:29 to 10:08:34 | Branch protection rejects an unauthorized change to the private release workflow. Five seconds later a credential tied to a privileged developer account overrides the protection and applies workflow commit 9cf97e9. |
LiteLLM post-mortem |
| 10:46:39 | The modified workflow starts a staging run from Berri-AI/litellm at commit 20535d8f, ten seconds after that commit’s timestamp. |
LiteLLM post-mortem; GitHub compare API |
| 11:11:18 | A second developer’s credential satisfies the production approval gate. | LiteLLM post-mortem |
| 11:39:51 to 11:40:13 | The seven wheels are uploaded to PyPI in 22 seconds. | LiteLLM post-mortem |
| 11:46:30 | Socket’s scanner raises malware alerts on the wheels, 6 minutes 39 seconds after the upload. | Socket |
| 11:53:49 to 11:54:01 | A third developer’s credential deletes the production and staging workflow run records, which limits the logs available. | LiteLLM post-mortem |
| 12:06:41 | An outside researcher, LidorMachluf, opens a public GitHub issue. LiteLLM’s post credits them as someone “who helped us to detect this incident.” | GitHub issue 47826; post-mortem |
| 13:23 and 13:50:33 | PyPI has removed 1.107.0 by 13:23. The first maintainer reply on the issue, at 13:50:33, says LiteLLM is “treating this as SEV-0” and that PyPI “quarantined 1.107.0.” | LiteLLM post-mortem; GitHub issue 47826 |
Three details stand out. LiteLLM says one of the tokens the attacker held belonged to one of its two GitHub organization owners, who could disable branch protection, so the control that fired at 10:08:29 was undone by an account allowed to undo it. The clean release 1.106.2 reached PyPI 1 minute 48 seconds before the staging run began and was the fourth upload of the day (PyPI’s JSON shows 1.104.5 at 05:43:16, 1.103.7 at 09:36:09 and 1.105.3 at 09:57:08), which fits LiteLLM’s statement that on Monday it “temporarily expanded release permissions to allow two internal engineers to publish daily bugfix backports for customers.” And detection came from outside: a vendor scanner at 11:46, a public issue at 12:06, and a maintainer reply at 13:50, 2 hours 10 minutes after the upload and after PyPI had already acted.
The hardening that followed March did not cover this
LiteLLM says that after March it “moved releases to a private repo,” used “Trusted GitHub Publishers,” “made our docker release tags immutable” and “are signing docker images.” StepSecurity’s reading is that none of it helped because the attacker changed “the trusted release workflow itself, so the malicious artifacts were built, signed, and published by the legitimate pipeline with valid provenance.” It lists the controls that failed as “branch protection that an owner token can disable, approval gates that any held developer credential can satisfy, and long lived developer tokens sitting in application logs.”
The build records still held a tell. StepSecurity, Socket and Endor Labs report that the provenance attached to the poisoned container images records the source as Berri-AI/litellm; StepSecurity and Endor give the commit as 20535d8f, and Endor adds that the workflow was dispatched with a source_owner of Berri-AI. We read the same record on the clean v1.106.2 image through Docker Hub’s registry API with the script below. It names https://github.com/BerriAI/litellm at commit 49943cd52ea46aacf8db2e5cb2b890d580fe9bb4, the commit GitHub’s v1.106.2 tag points to, and its dispatch inputs include no source_owner.
import json, urllib.request as u
repo, tag = "litellm/litellm", "v1.106.2"
token = json.load(u.urlopen(
f"https://auth.docker.io/token?service=registry.docker.io&scope=repository:{repo}:pull"))["token"]
OCI = "application/vnd.oci.image.index.v1+json,application/vnd.oci.image.manifest.v1+json"
def fetch(path, accept):
req = u.Request(f"https://registry-1.docker.io/v2/{repo}/{path}",
headers={"Authorization": "Bearer " + token, "Accept": accept})
return json.load(u.urlopen(req))
index = fetch(f"manifests/{tag}", OCI)
att = next(m for m in index["manifests"]
if m.get("annotations", {}).get("vnd.docker.reference.type") == "attestation-manifest")
layer = fetch(f"manifests/{att['digest']}", OCI)["layers"][0]["digest"]
build = fetch(f"blobs/{layer}", "application/json")["predicate"]["buildDefinition"]
args = build["externalParameters"]["request"]["root"]["request"]["args"]
print("source ", args["vcs:source"])
print("revision", args["vcs:revision"])
print("inputs ", sorted(build["internalParameters"]["github_event_payload"]["inputs"]))
source https://github.com/BerriAI/litellm
revision 49943cd52ea46aacf8db2e5cb2b890d580fe9bb4
inputs ['allow_republish', 'dry_run', 'git_ref', 'release_type', 'repo']
That output is from Python 3.13 on October 23 and applies only to images that carry BuildKit provenance. A policy that asks only whether provenance exists would have passed 1.107.0; one that asks whether the source is the repository you expect, and whether the revision is the commit of a release tag, would not. Our report on Tensorlake’s npm SDK had the same shape, a valid attestation on poisoned source, and a result that “passes any check that asks only who built the package.”
What the payload does, and what we can say about its second stage
Each wheel carries three extra files under litellm-1.107.0.data/purelib/ that pip unpacks into the top of site-packages: cook.pth (12 bytes, the text import cook), cook.py (22,252 bytes) and sitecustomize.py (25,509 bytes). StepSecurity, Endor Labs and the public issue agree that one added line, data = "cook" in the [tool.maturin] section of pyproject.toml, got them bundled. Python’s documentation explains the rest: .pth lines that start with import are executed at startup, and afterwards Python tries to import a module named sitecustomize, so the code runs on every interpreter start, LiteLLM imported or not, pip included. Python 3.15 deprecates import lines in .pth files but still executes them during the deprecation period, and sitecustomize is a separate mechanism; our tutorial on auditing .pth startup hooks shows the mechanism in a harmless lab.
The .pth file imports cook.py, a stealer that Wiz calls privcollect, with 16 modules. It takes browser data, AWS, GCP and Azure credentials, API keys for OpenAI, Anthropic, Cohere and others, SSH and GPG keys and shell history, and on a LiteLLM proxy it queries the table of stored provider credentials (SELECT * FROM "LiteLLM_CredentialsTable" LIMIT 500). The results are encrypted with AES-256-CBC, the key is wrapped with RSA-4096, and the bundle is posted to www.litellm[.]dev/healthcheck. Wiz judges from the code that it shows “strong indications of LLM-aided development.”
sitecustomize.py is the second payload and, per Wiz, runs on Linux and macOS only. StepSecurity describes it taking a lock at /tmp/remote_code_loader.lock, forking a daemon that outlives the Python process, and every hour reading a hostname from a file in a GitHub repository, then downloading http://<host>/codepy and running it; Socket notes that nothing verifies a signature or hash on that download. The channel was empty. StepSecurity’s history of the file shows localhost:18909 on October 18, test.test.com:18909 on October 19 and an empty file on October 20, and Wiz likewise says it never held more than test values. GitHub’s API returns 404 today for the account letters9813, for its repository and for the Berri-AI account (checked at 10:30 UTC). That cuts off the channel but not the daemons already running: StepSecurity warns that infected hosts keep polling the same address, and that anyone who later registers the name and recreates the file “would get code execution on every host that is still infected,” which is why LiteLLM’s guidance includes killing the daemon and not just deleting files.
Wiz saw the stolen credentials used about 4.5 hours after exfiltration. Between 16:20 and 17:01 UTC, AWS instance-role credentials harvested from compromised CI runners and Kubernetes nodes were checked with sts:GetCallerIdentity and then used to list Amazon Bedrock models, a pattern Wiz calls consistent with LLMjacking.
How to check a host without starting Python
LiteLLM’s post-mortem holds two instructions that pull in different directions. Under “How to check if you are affected” it says “You can check with pip show litellm.” Under “What to do if you are affected” it says “Don’t start with pip install. Any Python start in the affected environment runs cook.pth and sitecustomize.py again, and that includes running pip.” Socket and Endor Labs give the same warning, and Endor names find, sha256sum, ps and the container runtime as the safe tools. The safer rule is to read the filesystem first.
# Linux and macOS: look for the dropped files without starting Python
find / \( -path /proc -o -path /sys \) -prune -o \
\( -name cook.pth -o -name litellm-1.107.0.dist-info -o \( -name sitecustomize.py -size 25509c \) \) -print 2>/dev/null
# A running loader holds this lock file; the stealer leaves a run-once marker
ls -l /tmp/remote_code_loader.lock /tmp/.pc_once_* 2>/dev/null
# Windows PowerShell: the same search (slow on a large drive)
Get-ChildItem -Path C:\ -Recurse -Force -ErrorAction SilentlyContinue |
Where-Object { $_.Name -in 'cook.pth','litellm-1.107.0.dist-info' -or ($_.Name -eq 'sitecustomize.py' -and $_.Length -eq 25509) } |
Select-Object -ExpandProperty FullName
We ran both searches against dummy files with the same names and sizes (pointed at a test folder instead of the drive root), including a decoy sitecustomize.py of another size and a clean litellm-1.106.2.dist-info that they correctly skipped; we did not install or run the malicious package. The size test exists because sitecustomize.py is a legitimate hook name, and 25,509 bytes is the size the public issue lists for the payload file. Wiz, Endor Labs, StepSecurity and Socket all publish the same SHA-256 values for cook.pth, cook.py and the Linux and macOS sitecustomize.py. LiteLLM says a hit confirms exposure and a miss proves nothing; the same goes for the run-once marker, which proves exfiltration if it exists but is removed by container and temp-directory cleanup.
If you find the files, LiteLLM’s steps are: do not run pip first; delete cook.pth, cook.py, sitecustomize.py and the matching __pycache__ files from every affected site-packages; kill the detached Python daemon that holds the lock file; and reinstall pinned to litellm==1.106.2, or better, rebuild images and virtual environments. Socket and Endor add that you should isolate the host from the network and preserve process listings, filesystem state and egress logs first. If 1.107.0 was ever installed on a host with outbound internet, assume the exfiltration succeeded and rotate everything the host could reach, including the instance or task IAM role, every provider key in the LiteLLM proxy database, the master and virtual keys and the DATABASE_URL password, then check CloudTrail for GetCallerIdentity calls from the role around install time and for use of its credentials from outside your infrastructure.
For containers, LiteLLM’s test is whether a container executed the affected image, index digest sha256:1bbcb1443bc962e45f5aad53fe3d2c30600e0379715fb6c664cdd3ac126c339e; a pulled image that never ran does not start Python. StepSecurity also lists the amd64 manifest digest sha256:bd1aeee419ea36da9aa2b657288da4101887be468f020ac63d22ba6ed84a3ea5 and a kubectl command that prints every pod’s imageID for comparison. Endor recommends pinning images by digest and blocking litellm==1.107.0 in package proxies; ClickPy still logged 331 downloads of it the day after removal, so a proxy or mirror may be worth checking. Any dependency cooldown longer than the 1 hour 43 minutes the wheels were live would have kept 1.107.0 out of an unpinned install; our dependency-cooldown tutorial shows the pip and npm settings.
What LiteLLM says about the stolen tokens, and what it has not explained
LiteLLM’s explanation of how the tokens were stolen involves its own product and an AI coding agent. The post-mortem says: “we have observed specific situations in which Claude Code will embed a user’s GitHub CLI OAuth token into a response body, and in situations where response bodies are logged, these tokens may appear in LiteLLM logs.” It then says it found an “unexpected, innocuously named user account” in its internal LiteLLM deployment, “which was exposed to the public internet,” that it believes the attacker got into that deployment and “obtained the GitHub CLI OAuth tokens from its logs,” and that it has “not yet confirmed how the attacker gained access to our internal deployment, but currently suspect it was through a recently patched vulnerability.”
Several questions in that account are open. Which vulnerability, and in which LiteLLM versions? A reader asked on the public issue on October 22 and the thread has no reply. Another reader asked how Claude Code ends up putting a token in a response body, whether it runs gh auth token or reads GH_TOKEN, and that is also unanswered; we found no statement from Anthropic in the sources we read. LiteLLM’s update of October 21 at 18:30 UTC added that it had “not yet identified how the lateral movement occurred to the other affected engineers’ accounts.” What the account does establish is where the weak point sat: anything that appears in a model’s response lands wherever response bodies are logged, so a gateway’s logs work as a credential store whether or not anyone planned it that way. Internet-facing LiteLLM servers were already a target this month, as our report on PoeLLM describes.
Where the accounts differ
| Question | One account | What else the record shows |
|---|---|---|
| When the Docker Hub tags were cleaned | Wiz: the container tags “were removed ~13:53 UTC.” | LiteLLM’s timeline: at 13:51 the GHCR tags were unavailable but the Docker Hub tags “still resolved to compromised image index” until 14:20. The Docker Hub window therefore ends at 14:20. |
| How long the packages were live | SecurityWeek: “roughly three hours.” | LiteLLM’s timestamps give 1 h 43 min on PyPI, up to 2 h 4 min on GHCR and 2 h 33 min on Docker Hub; its own summary says “approximately 2 hours” for PyPI. |
| Whether the second stage ran | SecurityWeek: the remote-access component “was observed regularly checking with the attackers’ command-and-control (C&C) server.” | Wiz and StepSecurity describe an hourly poll of a GitHub file that was empty or held test values, and Wiz lists the second-stage payload as “never observed active.” |
| How fast it was detected | BleepingComputer: security firms detected it “within six minutes of publishing.” | Socket says its scanner flagged the wheels “seven minutes after publication”; its own record shows upload at 11:39:51 and alerts at 11:46:30, which is 6 minutes 39 seconds. |
| Whether the clean tag exists | The post-mortem and StepSecurity: the official v1.107.0 tag is clean. | GitHub’s API at 10:30 UTC lists no v1.107.0 tag in BerriAI/litellm, only v1.107.0-dev.1 to dev.3 and rc.1 and rc.2, and no GitHub release for it. We found no statement on whether the tag was removed. |
What is still unknown
- The point of entry. How the attacker reached the internal LiteLLM deployment, which vulnerability LiteLLM suspects, and which versions it affects.
- The Claude Code behavior. How a GitHub token gets into a response body, and under which settings. Neither LiteLLM nor Anthropic has explained it in anything we found.
- The actor. The post-mortem and the Wiz, StepSecurity, Socket and Endor Labs write-ups name none, and we found nothing tying October to the group behind March.
- The victim count. The 49,043 downloads are neither an upper nor a lower bound on installs or infections.
- What happens next. LiteLLM’s registries were locked to new releases as of its October 22 update, and its private registry carries an embargoed fix for a vulnerability whose disclosure date is October 27, a Tuesday. A maintainer comment dated October 22 says a town hall would follow the next day.
Sources and method
We read LiteLLM’s post-mortem (last updated October 21 at 20:30 UTC) and the public issue with its 28 comments, plus write-ups from Wiz, StepSecurity, Socket and Endor Labs and coverage from BleepingComputer and SecurityWeek. First-hand checks were run between 10:10 and 10:30 UTC on October 23: the PyPI JSON API (release list and upload times), GitHub’s REST API (compare view, tag list, account lookups), Docker Hub’s tag list and registry API, OSV, Google Registry’s RDAP record, and ClickPy through ClickHouse’s public SQL playground (the tables pypi_downloads_per_day_by_version and pypi_downloads_per_day). We did not download or run the package or any payload, did not contact litellm.dev or any other attacker-controlled address, and did not decode the stealer; the payload details are the vendors’. The durations and the 1.4 percent are our arithmetic, and times are UTC. The vendor pages are updated through the day, so the details above are as of the time we read them.








No Comment! Be the first one.