Learning Hub

How to Parse Untrusted XML Safely in Python and Stop XXE, Billion Laughs, and Entity Bombs

Reproduce XML entity bombs, file-reading XXE and internal requests against 11 Python parser setups, then build and test one safe parsing module that refuses them all.

October 17, 2026 38 Min Read
27

What you will build, and why XML can still hurt you

Plenty of programs accept XML from people they do not control. Single sign-on responses (SAML) are XML, SVG logos that users upload are XML, Word and Excel files are zip archives full of XML, and RSS feeds, SOAP calls and partner integrations speak it too. XML also has a feature that many applications never need: a document can define its own shortcuts, called entities, and some of those shortcuts can point at a file or a web address. If the parser is allowed to follow them, a stranger’s document can make your server read a local file, call a service that is only reachable from inside your network, or burn gigabytes of memory.

Table Of Content

In this tutorial you will reproduce those attacks safely, with harmless stand-ins: a text file that holds only a made-up canary string, and a tiny service on 127.0.0.1 that logs every request it receives. You will run 11 parser setups (Python’s own parsers, lxml and defusedxml) against 7 documents and see which combinations leak the canary, call the service, or expand into tens of millions of characters. Then you will build a small module called safexml that refuses all of them, still accepts ordinary XML, and is covered by 33 tests, and you will add a linter rule so nobody can quietly go around it.

Everything below was run, not copied: Windows 11, Python 3.13.14 (which reports Expat 2.8.1), lxml 6.1.3, defusedxml 0.7.1, pytest 9.1.1, Ruff 0.17.1 and Bandit 1.9.4. The standard-library steps were also run on Python 3.15.0 (Expat 2.8.5) and gave the same verdicts. Timings come from one ordinary machine and will differ on yours; the verdicts should not.

Is this still a live problem? XXE is catalogued as CWE-611, which describes a product that “processes an XML document that can contain XML entities with URIs that resolve to documents outside of the intended sphere of control, causing the product to embed incorrect documents into its output.” In CISA’s Known Exploited Vulnerabilities catalog, version 2026.10.16 with 1,748 entries, 8 entries list CWE-611. They include Adobe Commerce and Magento Open Source (CVE-2024-34102), two SysAid On-Prem flaws (CVE-2025-2775 and CVE-2025-2776) and GeoServer (CVE-2025-58360, added on 2025-12-11). Eight entries is fewer than 1 in 200, and the catalog only holds flaws with confirmed exploitation, so read it as evidence that attackers do use XXE against real enterprise products, not as a measure of how common it is. No entry lists CWE-776, the entity-expansion weakness behind the “billion laughs” attack.

Optional: count XXE in the catalog yourself

This script downloads the catalog from CISA’s JSON feed and counts the two weaknesses. The numbers grow as the catalog grows, so yours may be higher than the ones above.

# step00_kev.py - is XXE still exploited? Count CWE-611 in CISA's Known Exploited Vulnerabilities list
import requests

URL = "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json"
HEADERS = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130 Safari/537.36"}

catalog = requests.get(URL, headers=HEADERS, timeout=60).json()
entries = catalog["vulnerabilities"]
print("catalog", catalog["catalogVersion"], "with", len(entries), "entries")

for cwe in ("CWE-611", "CWE-776"):
    hits = sorted((e for e in entries if cwe in e.get("cwes", [])), key=lambda e: e["dateAdded"])
    since_2025 = [e for e in hits if e["dateAdded"] >= "2025-01-01"]
    print(f"{cwe}: {len(hits)} entries, {len(since_2025)} added since 2025-01-01")
    for e in hits[-5:]:
        print(f"   {e['dateAdded']}  {e['cveID']}  {e['vendorProject']} {e['product']}")
catalog 2026.10.16 with 1748 entries
CWE-611: 8 entries, 3 added since 2025-01-01
   2024-07-17  CVE-2024-34102  Adobe Commerce and Magento Open Source
   2024-12-03  CVE-2023-45727  North Grid Proself
   2025-07-22  CVE-2025-2775  SysAid SysAid On-Prem
   2025-07-22  CVE-2025-2776  SysAid SysAid On-Prem
   2025-12-11  CVE-2025-58360  OSGeo GeoServer
CWE-776: 0 entries, 0 added since 2025-01-01

A one-minute vocabulary

  • Parser: the library that turns XML text into objects your program can use. Python ships one called Expat; lxml brings another (libxml2).
  • Entity: a named shortcut written as &name;. Five are built in, such as & for an ampersand.
  • DTD / DOCTYPE: a block at the top of a document, between <!DOCTYPE and ]>, where the document can declare more entities and rules. Business XML usually needs none of this.
  • External entity: an entity whose value is not written in the document. Its declaration says SYSTEM followed by a file path or web address, and a parser that follows it fetches the value from there.

The attacks you will reproduce

Attack What the document asks the parser to do What it can cost you Where
Billion laughs Nest entities so a few hundred bytes expand to tens of millions of characters CPU and memory (denial of service) Step 4
Quadratic blowup Repeat one long entity thousands of times, with no nesting The same, without the deep nesting a parser can spot Step 4
External file entity (XXE) Use a file: address as an entity value File contents flow into your data Step 5
External HTTP entity Use an http: address as an entity value Your server calls an address the sender chose (SSRF) Step 5
External DTD and parameter entity Ask the parser to load more rules from elsewhere Extra fetches and extra entities Steps 5 and 6
Decompression bomb Arrive as a small gzip file that unpacks to 100 MB Memory Step 8
Large token One huge comment fed to the parser in small pieces Quadratic CPU time on a parser without the fix Step 8

The weakness behind the first two rows is CWE-776, which warns that “if the DTD contains a large number of nested or recursive entities, this can lead to explosive growth of data when parsed, causing a denial of service.” The remaining rows are external-resource and size problems, and OWASP’s XML External Entity Prevention Cheat Sheet discusses them for Python. Only run this lab on your own machine, and never point these documents at systems you do not own.

Prerequisites

  • System: Windows, macOS or Linux, and a terminal. The commands below work in PowerShell and in a bash-style shell unless a note says otherwise. We ran everything on Windows 11; matrix.py has a separate branch for stopping the helper service on other systems, which we did not run.
  • Python: 3.12 or newer. The examples use f-strings that need 3.12. We tested on 3.13.14 and, for the standard-library steps, on 3.15.0.
  • Knowledge: you can run a Python script and activate a virtual environment. No XML background is needed; the vocabulary above is enough.
  • Access: an ordinary user account. The only network traffic is pip and the optional catalog download; the lab itself talks to 127.0.0.1 only.
  • Time and space: about half an hour and roughly 100 MB of disk, most of it the virtual environment (lxml, Ruff and Bandit) and the three older lxml copies that Step 6 downloads.

Create a folder for the lab, a virtual environment, and install the packages. The version pins are the ones this tutorial was tested with.

mkdir xmllab
cd xmllab
python -m venv labvenv
# Windows PowerShell:   labvenv\Scripts\Activate.ps1
# macOS or Linux:       source labvenv/bin/activate
python -m pip install lxml==6.1.3 defusedxml==0.7.1 pytest==9.1.1 requests==2.34.2 ruff==0.17.1 bandit==1.9.4

Every file below goes into the xmllab folder, next to each other. Run every command from that folder with the environment active. Where a file grows over several steps, the later blocks begin with a comment such as # payloads.py (continued): paste them at the end of the same file.

Step 1: Create the lab folder and the files the attacks point at

Every attack needs something to aim at. A real attacker would aim at a password file or an admin page; we aim at a text file that contains a made-up marker, so a leak is easy to spot and nothing sensitive is ever read. The first file, payloads.py, builds the XML documents. In this step it gets the small builder that wraps text in an “order” document, one plain order, one order that uses a legitimate DTD shortcut, and a helper that writes the two data files.

# payloads.py - builders for the XML documents used in this tutorial
from pathlib import Path

LAB = Path(__file__).resolve().parent
DATA = LAB / "data"
CANARY_TEXT = "CANARY-5c1f-not-a-real-secret"
SERVICE_URL = "http://127.0.0.1:8099"
DTD_TEXT = '<!ENTITY dtdmarker "loaded-from-external-dtd">'


def ensure_data() -> None:
    """Create the two harmless files the attack documents point at."""
    DATA.mkdir(exist_ok=True)
    (DATA / "canary.txt").write_text(CANARY_TEXT, encoding="utf-8")
    (DATA / "param.dtd").write_text(DTD_TEXT, encoding="utf-8")


def wrap(dtd: str, note: str) -> bytes:
    """Put an optional DTD and a note inside a tiny order document."""
    text = '<?xml version="1.0" encoding="UTF-8"?>\n' + dtd
    text += "<order><note>" + note + "</note></order>"
    return text.encode("utf-8")


def benign() -> bytes:
    return wrap("", "2 x widgets &amp; 1 x gadget")


def internal_entity() -> bytes:
    """A legitimate use of a DTD: a text shortcut declared inside the document."""
    return wrap('<!DOCTYPE order [<!ENTITY company "Acme Corp">]>\n', "Ship to &company;")

Now run a short script that calls ensure_data() and shows what it created.

# step01_setup.py - create the canary and DTD files and show what the lab holds
import payloads

payloads.ensure_data()
for path in sorted(payloads.DATA.iterdir()):
    print(f"{path.name:<12}{path.stat().st_size:>4} bytes   {path.read_text(encoding='utf-8')}")
python step01_setup.py
canary.txt    29 bytes   CANARY-5c1f-not-a-real-secret
param.dtd     46 bytes   <!ENTITY dtdmarker "loaded-from-external-dtd">

Check that it worked: you should see a data folder holding canary.txt (29 bytes) and param.dtd (46 bytes). The canary string is the thing we will watch for in every later step. If it ever shows up in a parsed document, a parser read a file it should not have.

Step 2: See XML entities working as designed

Before attacking anything, watch the feature behave normally. The script below parses two orders with Python’s built-in ElementTree: a plain one, and one that declares a shortcut named company in its DTD and uses it in the note.

# step02_entities.py - XML entities are a normal feature, shown with ElementTree
import xml.etree.ElementTree as ET

import payloads

for label, data in (("a plain order", payloads.benign()),
                    ("an order with a DTD shortcut", payloads.internal_entity())):
    print(f"--- {label} ---")
    print(data.decode())
    print("note text ->", ET.fromstring(data).findtext("note"))
    print()
python step02_entities.py
--- a plain order ---
<?xml version="1.0" encoding="UTF-8"?>
<order><note>2 x widgets &amp; 1 x gadget</note></order>
note text -> 2 x widgets & 1 x gadget

--- an order with a DTD shortcut ---
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE order [<!ENTITY company "Acme Corp">]>
<order><note>Ship to &company;</note></order>
note text -> Ship to Acme Corp

The first order shows a built-in entity: &amp; became an ampersand. The second shows a custom one: the parser replaced &company; with the text from the DTD. Nothing is wrong here. Entities are a normal XML feature and ElementTree resolves this one by default. Everything that follows is a variation on this idea: what if the replacement text is enormous, or lives in a file, or at a web address?

Step 3: Find out which Expat your Python is using

Python’s built-in XML parsers rely on a C library called Expat. The Python documentation says that “the built-in XML parsers of Python rely on the library libexpat, commonly called Expat, for parsing XML.” It adds that “by default, Expat itself does not access local files or create network connections,” and that Expat versions lower than 2.7.2 may be vulnerable to the billion laughs, quadratic blowup and large-token attacks, or to disproportionate use of dynamic memory. The page continues: “Python bundles a copy of Expat, and whether Python uses the bundled or a system-wide Expat, depends on how the Python interpreter has been configured in your environment,” so you are told to “Check pyexpat.EXPAT_VERSION”. OWASP’s cheat sheet says the same from the defender’s side: “a Python version alone does not establish the linked Expat version.”

Expat 2.7.2 is the release whose changelog adds the memory limit (CVE-2025-59375): it describes a document of about 250 KiB that could previously make a parser allocate about 800 MiB, and says that now “once a threshold (that defaults to 64 MiB) is reached, a maximum amplification factor (that defaults to 100.0) is enforced.” Reading a version number is a start, but you can also test the behavior. The next file does both. It builds a small probe bomb (a 416 byte document that would expand to about 10 million characters), feeds it to a bare Expat parser, and reports whether the parser refused it.

# expatcheck.py - find out whether this Python's Expat refuses entity bombs
from xml.parsers import expat

MIN_EXPAT = (2, 7, 2)  # the floor named on Python's XML security page


def _probe_bomb(levels: int = 5, fan: int = 10) -> bytes:
    """About 10 million characters of output from a 416 byte document, harmless on any machine."""
    decls = ['<!ENTITY a0 "' + "x" * 100 + '">']
    for level in range(1, levels + 1):
        decls.append(f'<!ENTITY a{level} "' + f"&a{level - 1};" * fan + '">')
    return ("<!DOCTYPE r [" + "".join(decls) + f"]><r>&a{levels};</r>").encode()


def _expat_version() -> tuple:
    return tuple(int(part) for part in expat.EXPAT_VERSION.split("_")[1].split("."))


def check_expat(simulate_old: bool = False) -> dict:
    """Run the probe bomb through bare Expat. A protected build refuses it."""
    produced = 0

    def on_text(chunk):
        nonlocal produced
        produced += len(chunk)

    parser = expat.ParserCreate()
    parser.buffer_text = True
    parser.CharacterDataHandler = on_text
    if simulate_old:
        parser.SetBillionLaughsAttackProtectionActivationThreshold(2**63)
    refused = False
    try:
        parser.Parse(_probe_bomb(), True)
    except expat.ExpatError:
        refused = True
    return {"expat": expat.EXPAT_VERSION, "version_ok": _expat_version() >= MIN_EXPAT,
            "refused_probe": refused, "chars_produced": produced}


def require_protected_expat() -> None:
    """Call once at start-up: refuse to run on an Expat that expands bombs."""
    info = check_expat()
    if not (info["version_ok"] and info["refused_probe"]):
        raise RuntimeError(f"this Python's Expat is not protected: {info}")

The function has a second mode. With simulate_old=True it raises the parser’s activation threshold so far that the limit can never switch on, which approximates a build without the protection. The setter it uses, SetBillionLaughsAttackProtectionActivationThreshold, is documented in the pyexpat reference; it is the same call that defenders use to tune the limit. Now run it.

# step03_expat.py - which Expat does this Python use, and does it carry the protections?
import sys
from xml.parsers import expat

import expatcheck

print("python", sys.version.split()[0])
print("expat ", expat.EXPAT_VERSION)
parser = expat.ParserCreate()
for name in ("SetBillionLaughsAttackProtectionActivationThreshold",
             "SetBillionLaughsAttackProtectionMaximumAmplification",
             "SetAllocTrackerActivationThreshold",
             "SetAllocTrackerMaximumAmplification",
             "SetReparseDeferralEnabled"):
    print(f"{name:<56}{hasattr(parser, name)}")
print("minimum Expat named by the Python docs:", ".".join(map(str, expatcheck.MIN_EXPAT)))
print("probe on this build:        ", expatcheck.check_expat())
print("probe with protection off:  ", expatcheck.check_expat(simulate_old=True))
python step03_expat.py
python 3.13.14
expat  expat_2.8.1
SetBillionLaughsAttackProtectionActivationThreshold     True
SetBillionLaughsAttackProtectionMaximumAmplification    True
SetAllocTrackerActivationThreshold                      True
SetAllocTrackerMaximumAmplification                     True
SetReparseDeferralEnabled                               True
minimum Expat named by the Python docs: 2.7.2
probe on this build:         {'expat': 'expat_2.8.1', 'version_ok': True, 'refused_probe': True, 'chars_produced': 8027100}
probe with protection off:   {'expat': 'expat_2.8.1', 'version_ok': True, 'refused_probe': False, 'chars_produced': 10000000}

Read the output from the top. Python 3.13.14 reports expat_2.8.1, which is above the 2.7.2 floor, so version_ok is true. All five protection methods exist. The real test is the probe: refused_probe is true, and Expat stopped after 8,027,100 characters instead of producing all 10,000,000. With the protection switched off, the probe runs to the end and produces the full 10,000,000 characters.

Check that it worked, and what to do if it did not: on your machine both version_ok and refused_probe in the first probe line should be true. If either is false, your Python is linked to an older Expat (common when an interpreter uses a system library). Update Python or the system library before going further; the tests in Step 9 will fail on purpose until you do. This follows OWASP’s advice to “treat an unsupported security setting as a configuration failure” and not to “continue processing untrusted XML with partial hardening.”

Step 4: Reproduce the entity bombs

An entity bomb is a short document whose entities expand into a huge amount of text. It needs no network and no files, which makes it the simplest attack of the set. Two shapes matter. The billion laughs shape defines each entity as ten copies of the previous one, so the output grows ten times per level while the document grows by a few dozen bytes. The quadratic blowup shape defines one long entity and repeats it thousands of times; it avoids deep nesting, which a parser could use as a warning sign. Add the two builders to payloads.py.

# payloads.py (continued)
def billion_laughs(levels: int = 7, fan: int = 10) -> bytes:
    """Each level repeats the previous one `fan` times: 3 * fan**levels characters."""
    decls = ['<!ENTITY lol0 "lol">']
    for level in range(1, levels + 1):
        decls.append(f'<!ENTITY lol{level} "' + f"&lol{level - 1};" * fan + '">')
    dtd = "<!DOCTYPE order [\n" + "\n".join(decls) + "\n]>\n"
    return wrap(dtd, f"&lol{levels};")


def quadratic_blowup(entity_chars: int = 20_000, refs: int = 1_500) -> bytes:
    """One big entity repeated many times: no nesting, so no deep-nesting alarm."""
    dtd = f'<!DOCTYPE order [<!ENTITY big "{"A" * entity_chars}">]>\n'
    return wrap(dtd, "&big;" * refs)

The next script first prints a 3-level bomb in full, so you can see that there is no trick: lol0 is the text “lol”, lol1 is ten of those, lol2 is ten of lol1, and so on. Then it feeds bombs of growing size to a bare Expat parser twice: once with the default protection, and once with the protection switched off to approximate an older library. Finally it parses one large-but-allowed document with ElementTree and measures its cost. The parser option buffer_text = True only makes Expat hand text over in big pieces so that counting stays fast.

# step04_bombs.py - entity expansion: what Expat refuses by default, and what an old Expat would do
import time
import tracemalloc
import xml.etree.ElementTree as ET
from xml.parsers import expat

import payloads


def run_expat(data, protection=True):
    produced = 0

    def on_text(chunk):
        nonlocal produced
        produced += len(chunk)

    parser = expat.ParserCreate()
    parser.buffer_text = True
    parser.CharacterDataHandler = on_text
    if not protection:
        parser.SetBillionLaughsAttackProtectionActivationThreshold(2**63)
    started = time.perf_counter()
    try:
        parser.Parse(data, True)
        verdict = "parsed "
    except expat.ExpatError:
        verdict = "REFUSED"
    return f"{verdict} after {produced:>12,} chars, {time.perf_counter() - started:5.2f} s"


def table(title, builders):
    print(title)
    print(f"{'size':>8} {'document':>9} {'would expand to':>16}   {'this Python (default Expat)':<42}  protection off")
    for label, data, expands_to in builders:
        print(f"{label:>8} {len(data):>8,}B {expands_to:>15,}   "
              f"{run_expat(data):<42}  {run_expat(data, protection=False)}")
    print()


print("expat", expat.EXPAT_VERSION)
print()
print("A 3 level bomb, written out in full:")
print(payloads.billion_laughs(3).decode())
print()
table("Billion laughs (10 references per level, base text 'lol'):",
      [(f"{n} lvl", payloads.billion_laughs(n), 3 * 10**n) for n in range(3, 8)])
table("Quadratic blowup (one 20,000 character entity, repeated):",
      [(f"{n} refs", payloads.quadratic_blowup(20_000, n), 20_000 * n) for n in (50, 200, 400, 1500)])

# The protection only switches on after 8 MiB of output, so a smaller document is never refused.
doc = payloads.quadratic_blowup(25_000, 300)
tracemalloc.start()
started = time.perf_counter()
text = ET.fromstring(doc).findtext("note")
elapsed = time.perf_counter() - started
peak = tracemalloc.get_traced_memory()[1]
tracemalloc.stop()
print(f"Under the threshold: {len(doc):,} bytes in -> {len(text):,} characters out, "
      f"parsed by ElementTree in {elapsed:.3f} s, peak Python memory {peak / 1e6:.1f} MB")
print(f"8 MiB is {8 * 1024 * 1024:,} characters")
python step04_bombs.py
expat expat_2.8.1

A 3 level bomb, written out in full:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE order [
<!ENTITY lol0 "lol">
<!ENTITY lol1 "&lol0;&lol0;&lol0;&lol0;&lol0;&lol0;&lol0;&lol0;&lol0;&lol0;">
<!ENTITY lol2 "&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;">
<!ENTITY lol3 "&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;">
]>
<order><note>&lol3;</note></order>

Billion laughs (10 references per level, base text 'lol'):
    size  document  would expand to   this Python (default Expat)                 protection off
   3 lvl      349B           3,000   parsed  after        3,000 chars,  0.00 s   parsed  after        3,000 chars,  0.00 s
   4 lvl      427B          30,000   parsed  after       30,000 chars,  0.00 s   parsed  after       30,000 chars,  0.00 s
   5 lvl      505B         300,000   parsed  after      300,000 chars,  0.00 s   parsed  after      300,000 chars,  0.00 s
   6 lvl      583B       3,000,000   REFUSED after    2,596,230 chars,  0.04 s   parsed  after    3,000,000 chars,  0.05 s
   7 lvl      661B      30,000,000   REFUSED after    2,596,230 chars,  0.04 s   parsed  after   30,000,000 chars,  0.47 s

Quadratic blowup (one 20,000 character entity, repeated):
    size  document  would expand to   this Python (default Expat)                 protection off
 50 refs   20,353B       1,000,000   parsed  after    1,000,000 chars,  0.00 s   parsed  after    1,000,000 chars,  0.00 s
200 refs   21,103B       4,000,000   parsed  after    4,000,000 chars,  0.00 s   parsed  after    4,000,000 chars,  0.00 s
400 refs   22,103B       8,000,000   parsed  after    8,000,000 chars,  0.00 s   parsed  after    8,000,000 chars,  0.00 s
1500 refs   27,603B      30,000,000   REFUSED after    8,360,000 chars,  0.00 s   parsed  after   30,000,000 chars,  0.01 s

Under the threshold: 26,603 bytes in -> 7,500,000 characters out, parsed by ElementTree in 0.008 s, peak Python memory 15.0 MB
8 MiB is 8,388,608 characters

How to read the tables

  • Small bombs parse. Up to 5 levels (505 bytes, 300,000 characters) Expat just does the work. The document grows by 78 bytes per level, while the output grows ten times per level.
  • Bigger bombs are refused. At 6 and 7 levels this Python’s default Expat stops with an error after 2,596,230 characters, in 0.04 s. That is well below 8 MiB (8,388,608 characters), so Expat’s own byte count is clearly not the same number as the characters you finally receive. The Python reference says the count “includes amplification from entity expansion and reading DTD files,” and in this nested case it appears to include the entity text Expat walks through on the way.
  • Quadratic blowups are refused later. With no nesting the cut-off lands at 8,360,000 characters, right at the documented activation threshold: “Parser objects usually have a protection activation threshold of 8 MiB, but the actual default value depends on the underlying Expat library.” The same page gives the other half of the rule, a maximum amplification factor of 100, again “depends on the underlying Expat library.”
  • Without the limit, the cost is real. The last column parses the 30,000,000 character bomb in 0.47 s. Each extra level multiplies that work by ten, so 9 levels would be 3,000,000,000 characters (my arithmetic). We stopped at 7 on purpose.
  • Protected is not free. The final line parsed a 26,603 byte document into 7,500,000 characters with no error, in 0.008 s and 15.0 MB of peak Python memory, because it stays under the threshold. If 100 such documents were parsed at the same moment, that is about 1.5 GB of peak memory (my arithmetic, 100 times 15.0 MB).

You might think of lowering the threshold. The Python reference warns against it: “Activation thresholds below 4 MiB are known to break support for DITA 1.3 payload and are hence not recommended.” The more robust answer comes in Step 7: documents from strangers rarely need a DTD at all, so refuse it. Our tutorials on regular expression denial of service and path traversal follow the same pattern of reproducing the input problem first and then closing the door.

Step 5: Reproduce the external-entity attacks across every parser

Now the more dangerous half. An external entity does not carry its value; it says where to get it. Four documents cover the variations, and each one aims only at the lab. Add the builders and a cases() function that returns all seven documents of this tutorial, bombs included.

# payloads.py (continued)
def external_file_entity(path: Path) -> bytes:
    uri = path.resolve().as_uri()
    return wrap(f'<!DOCTYPE order [<!ENTITY secret SYSTEM "{uri}">]>\n', "&secret;")


def external_http_entity(url: str) -> bytes:
    return wrap(f'<!DOCTYPE order [<!ENTITY probe SYSTEM "{url}">]>\n', "&probe;")


def external_dtd(url: str) -> bytes:
    return wrap(f'<!DOCTYPE order SYSTEM "{url}">\n', "&dtdmarker;")


def parameter_entity(uri: str) -> bytes:
    dtd = f'<!DOCTYPE order [<!ENTITY % ext SYSTEM "{uri}"> %ext;]>\n'
    return wrap(dtd, "&dtdmarker;")


def cases() -> dict:
    """The seven documents every parser is tried against."""
    ensure_data()
    return {
        "internal": internal_entity(),
        "bomb": billion_laughs(),
        "quad": quadratic_blowup(),
        "file": external_file_entity(DATA / "canary.txt"),
        "http": external_http_entity(SERVICE_URL + "/admin"),
        "dtd": external_dtd(SERVICE_URL + "/evil.dtd"),
        "param": parameter_entity((DATA / "param.dtd").resolve().as_uri()),
    }

The file document declares secret as the canary file, the http document points at /admin on our pretend internal service, the dtd document asks the parser to fetch its rules from /evil.dtd on the same service, and the param document uses a parameter entity (the % form) to pull a DTD file into the document. The service itself is a short standard-library script. It records every request in a log file and answers with a recognizable string.

# internal_service.py - a pretend internal service that records every request it receives
import time
from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path

LOG = Path(__file__).resolve().parent / "out" / "service_hits.log"
BODIES = {"/evil.dtd": b'<!ENTITY dtdmarker "loaded-from-external-dtd">'}


class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        LOG.parent.mkdir(exist_ok=True)
        with LOG.open("a", encoding="utf-8") as log:
            log.write(f"{time.strftime('%H:%M:%S')} GET {self.path}\n")
        body = BODIES.get(self.path, b"INTERNAL-ADMIN-PAGE")
        self.send_response(200)
        self.send_header("Content-Type", "text/plain")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass


if __name__ == "__main__":
    HTTPServer(("127.0.0.1", 8099), Handler).serve_forever()

Next comes matrix.py, which runs every parser setup against every document and labels what happened. Each adapter returns the text the parser produced, so the classifier can look for the canary or the service’s answer. The first part imports the parsers; lxml and defusedxml are optional imports so that the standard-library rows still run on an interpreter without them (we used that for the Python 3.15.0 check).

# matrix.py - run every parser against every document and classify what happened
import contextlib
import io
import json
import os
import socket
import subprocess
import sys
import time
import xml.dom.minidom
import xml.dom.pulldom
import xml.etree.ElementTree as ET
import xml.sax
import xml.sax.handler

import payloads

try:  # the standard-library rows still run on a Python that lacks these two packages
    from lxml import etree as LET
except ImportError:
    LET = None
try:
    import defusedxml.ElementTree as DET
except ImportError:
    DET = None

OUT = payloads.LAB / "out"
HITS = OUT / "service_hits.log"


class Collect(xml.sax.ContentHandler):
    def __init__(self):
        super().__init__()
        self.parts = []

    def characters(self, content):
        self.parts.append(content)


def et_text(data):
    return ET.tostring(ET.fromstring(data), encoding="unicode")


def minidom_text(data):
    return xml.dom.minidom.parseString(data).documentElement.toxml()


def pulldom_text(data):
    parts = []
    for event, node in xml.dom.pulldom.parseString(data.decode("utf-8")):
        if event == xml.dom.pulldom.CHARACTERS:
            parts.append(node.data)
    return "".join(parts)


def sax_text(data, external=False):
    handler = Collect()
    parser = xml.sax.make_parser()
    parser.setContentHandler(handler)
    if external:
        parser.setFeature(xml.sax.handler.feature_external_ges, True)
    parser.parse(io.BytesIO(data))
    return "".join(handler.parts)

The lxml and defusedxml adapters come next. lxml_text takes parser options as keyword arguments, so one function creates every fromstring variant we want to test, and a separate adapter covers iterparse, the streaming API.

# matrix.py (continued)
def lxml_text(data, **options):
    parser = LET.XMLParser(**options)
    return LET.tostring(LET.fromstring(data, parser), encoding="unicode")


def lxml_default(data):
    return LET.tostring(LET.fromstring(data), encoding="unicode")


def lxml_iterparse(data):
    root = None
    for _event, element in LET.iterparse(io.BytesIO(data)):
        root = element
    return LET.tostring(root, encoding="unicode")


def defused_text(data, **options):
    return ET.tostring(DET.fromstring(data, **options), encoding="unicode")

Here are the 11 setups under test. They include the defaults, but also the settings that tutorials and old answers often recommend: feature_external_ges=True for the SAX reader and resolve_entities=True or no_network=False for lxml.

# matrix.py (continued)
STOCK_ROWS = [
    ("ElementTree.fromstring", et_text),
    ("minidom.parseString", minidom_text),
    ("pulldom.parseString", pulldom_text),
    ("sax defaults", sax_text),
    ("sax external_ges=True", lambda d: sax_text(d, external=True)),
]
if LET is not None:
    STOCK_ROWS += [
        ("lxml fromstring defaults", lxml_default),
        ("lxml resolve_entities=True", lambda d: lxml_text(d, resolve_entities=True)),
        ("lxml + no_network=False", lambda d: lxml_text(d, resolve_entities=True,
                                                        no_network=False, load_dtd=True)),
        ("lxml iterparse defaults", lxml_iterparse),
    ]
if DET is not None:
    STOCK_ROWS += [
        ("defusedxml defaults", defused_text),
        ("defusedxml forbid_dtd=True", lambda d: defused_text(d, forbid_dtd=True)),
    ]

The classifier turns each outcome into one word. REFUSED means the parser raised an error. LEAK means the canary text appears in the result, FETCH means the internal service’s answer appears, and LOADED means text from the external DTD appears. EXPAND means entity text was substituted (for the bombs, expanded in full), and SKIP means the parse succeeded but the entity text was silently left out.

# matrix.py (continued)
def classify(case, text, error):
    if error is not None:
        return "REFUSED"
    if payloads.CANARY_TEXT in text:
        return "LEAK"
    if "INTERNAL-ADMIN-PAGE" in text:
        return "FETCH"
    if "loaded-from-external-dtd" in text:
        return "LOADED"
    if (case in ("bomb", "quad") and len(text) > 10_000_000) or "Acme Corp" in text:
        return "EXPAND"
    return "SKIP"


def hit_count():
    return len(HITS.read_text(encoding="utf-8").splitlines()) if HITS.exists() else 0


def run_cell(func, data):
    before = hit_count()
    started = time.perf_counter()
    text, error = "", None
    try:
        text = func(data)
    except Exception as exc:
        error = exc
    elapsed = time.perf_counter() - started
    time.sleep(0.05)
    return text, error, elapsed, hit_count() - before


def run_matrix(label, rows, cases):
    names = list(cases)
    width = 28
    header = f"{'parser':<{width}}" + "".join(f"{name:<9}" for name in names)
    print(header)
    print("-" * len(header))
    results, details = {}, []
    for row_label, func in rows:
        cells = []
        for name in names:
            text, error, elapsed, hits = run_cell(func, cases[name])
            code = classify(name, text, error)
            cells.append(code + ("*" if hits else ""))
            note = f"{type(error).__name__}: {str(error)}" if error else f"{len(text)} chars out"
            note = note.replace(payloads.LAB.as_uri(), "file:///<lab>")[:120]
            results[f"{row_label}|{name}"] = {"code": code, "hits": hits,
                                              "seconds": round(elapsed, 3), "note": note}
            details.append(f"{row_label} | {name} | {code} | {elapsed:.3f}s | {note}")
        print(f"{row_label:<{width}}" + "".join(f"{cell:<9}" for cell in cells))
    OUT.mkdir(exist_ok=True)
    (OUT / f"{label}.json").write_text(json.dumps(results, indent=1), encoding="utf-8")
    (OUT / f"{label}_details.txt").write_text("\n".join(details) + "\n", encoding="utf-8")
    return results

The last part prints the request log and starts the internal service for the duration of a run. On Windows it stops the service with taskkill because the virtual environment’s python.exe is a launcher that starts a child process.

# matrix.py (continued)
def print_service_log():
    lines = HITS.read_text(encoding="utf-8").splitlines() if HITS.exists() else []
    print()
    print("* = the internal service logged a request")
    print(f"requests the internal service received: {len(lines)}")
    for line in lines:
        print("  ", line.split(" ", 1)[1])


@contextlib.contextmanager
def internal_service():
    OUT.mkdir(exist_ok=True)
    HITS.write_text("", encoding="utf-8")
    proc = subprocess.Popen([sys.executable, str(payloads.LAB / "internal_service.py")])
    try:
        for _ in range(60):
            try:
                socket.create_connection(("127.0.0.1", 8099), timeout=0.2).close()
                break
            except OSError:
                time.sleep(0.1)
        yield
    finally:
        if os.name == "nt":
            subprocess.run(["taskkill", "/PID", str(proc.pid), "/T", "/F"], capture_output=True)
        else:
            proc.terminate()

The script that drives it prints the two external-entity documents, runs the matrix, lists what the service received, and shows what a leak looks like.

# step05_matrix.py - run each stock parser against each document
import matrix
import payloads

cases = payloads.cases()
for name in ("file", "param"):
    print(f"--- the '{name}' document ---")
    print(cases[name].decode().replace(payloads.LAB.as_uri(), "file:///<lab>"))
    print()
with matrix.internal_service():
    matrix.run_matrix("matrix_stock", matrix.STOCK_ROWS, cases)
    matrix.print_service_log()

print()
print("What a LEAK looks like (the parsed order built from the 'file' document):")
for label, func in matrix.STOCK_ROWS:
    if label in ("sax external_ges=True", "lxml resolve_entities=True"):
        print(f"  {label:<28}{matrix.run_cell(func, cases['file'])[0]}")
python step05_matrix.py
--- the 'file' document ---
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE order [<!ENTITY secret SYSTEM "file:///<lab>/data/canary.txt">]>
<order><note>&secret;</note></order>

--- the 'param' document ---
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE order [<!ENTITY % ext SYSTEM "file:///<lab>/data/param.dtd"> %ext;]>
<order><note>&dtdmarker;</note></order>

parser                      internal bomb     quad     file     http     dtd      param    
-------------------------------------------------------------------------------------------
ElementTree.fromstring      EXPAND   REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  
minidom.parseString         EXPAND   REFUSED  REFUSED  SKIP     SKIP     SKIP     SKIP     
pulldom.parseString         EXPAND   REFUSED  REFUSED  SKIP     SKIP     SKIP     SKIP     
sax defaults                EXPAND   REFUSED  REFUSED  SKIP     SKIP     SKIP     SKIP     
sax external_ges=True       EXPAND   REFUSED  REFUSED  LEAK     FETCH*   LOADED*  LOADED   
lxml fromstring defaults    EXPAND   REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  
lxml resolve_entities=True  EXPAND   REFUSED  REFUSED  LEAK     SKIP     REFUSED  LOADED   
lxml + no_network=False     EXPAND   REFUSED  REFUSED  LEAK     FETCH*   LOADED*  LOADED   
lxml iterparse defaults     EXPAND   REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  
defusedxml defaults         REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  
defusedxml forbid_dtd=True  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  

* = the internal service logged a request
requests the internal service received: 4
   GET /admin
   GET /evil.dtd
   GET /admin
   GET /evil.dtd

What a LEAK looks like (the parsed order built from the 'file' document):
  sax external_ges=True       CANARY-5c1f-not-a-real-secret
  lxml resolve_entities=True  <order><note>CANARY-5c1f-not-a-real-secret</note></order>

What each row teaches

  • ElementTree.fromstring refuses all four external documents with an “undefined entity” error. It never fetched anything, which matches the documentation’s statement that Expat does not access files or the network by default.
  • minidom, pulldom and sax with default settings do something quieter: they parse successfully and simply drop the entity text (SKIP). No leak, but no error either, so an order whose note was supposed to come from an entity arrives with an empty note and your code carries on. Treat a silent skip as a bug in its own right.
  • sax with external_ges=True is the dangerous opt-in. It read the canary file (LEAK), fetched the page from the internal service (FETCH*), and loaded the external DTD (LOADED*). The star means the service logged a request, so this is a working server-side request from a document you did not write.
  • lxml with defaults refuses all four. That is the behavior you want from a current lxml: its changelog says the default for the normal XML and HTML parsers has been 'internal' since lxml 5.0, which resolves entities declared in the document but nothing external. (The options list on lxml’s parsing page still describes resolve_entities as “on by default”, so trust your tests over that line.)
  • lxml with resolve_entities=True, a setting you may find in older tutorials and answers, reads the canary file and loads the parameter entity. The HTTP document came back as SKIP without a request because no_network is on by default.
  • lxml with no_network=False as well is as open as the SAX reader: file, HTTP and DTD all work, and the service logged two requests.
  • lxml iterparse with defaults is refused on lxml 6.1.3. Step 6 shows that this was not always true.
  • defusedxml refuses everything, including the harmless &company; shortcut, because its default forbid_entities=True rejects any entity declaration.
  • The bombs were refused by every row, lxml included (its error reads Maximum entity amplification factor exceeded), so a current parser stack copes with these two bomb shapes. The Python 3.15.0 run matched this table on all 35 cells of the five standard-library rows.

Check that it worked: the log at the bottom of the output lists four requests, two for /admin and two for /evil.dtd, and they match the four starred cells. A mismatch would mean a parser made a request the table did not record. The “what a leak looks like” lines show the payoff for an attacker: the canary text arrives inside the parsed note, ready to be stored, logged or echoed back in a response. If a document makes your server fetch an address of the sender’s choosing, you are also in server-side request forgery territory, which our SSRF tutorial covers.

Step 6: See why the library version matters

The lxml row above deserves a second look, because lxml has shipped two relevant fixes this year. Version 6.1.0 (2026-04-17) says in its changelog that it “fixes a possible external entity injection (XXE) vulnerability” in iterparse() and ETCompatXMLParser: the resolve_entities option “was still set to True” there, “allowing for external entity injection (XXE) when using these parsers without setting this option explicitly.” The National Vulnerability Database record for CVE-2026-41066 describes it the same way: “Prior to 6.1.0, using either of the two parsers in the default configuration (with resolve_entities=True) allows untrusted XML input to read local files.” Version 6.1.3 (2026-09-02) fixed a second gap: “External parameter entity parsing was allowed by default.” Both entries are in lxml’s changelog.

You can check this yourself without trusting anyone’s summary. The first script runs two documents (the canary file and the parameter entity) through fromstring and iterparse using whichever lxml Python finds first. The second installs three lxml versions side by side with pip install --target and runs the first against each; the PYTHONPATH setting makes each folder win over the lxml in your environment.

# step06_lxml_versions.py - two XXE checks against whichever lxml comes first on PYTHONPATH
import io

from lxml import etree

import payloads

payloads.ensure_data()
FILE_DOC = payloads.external_file_entity(payloads.DATA / "canary.txt")
PARAM_DOC = payloads.parameter_entity((payloads.DATA / "param.dtd").resolve().as_uri())


def from_string(data):
    return etree.fromstring(data)


def from_iterparse(data):
    root = None
    for _event, element in etree.iterparse(io.BytesIO(data)):
        root = element
    return root


def outcome(parse, data):
    try:
        text = etree.tostring(parse(data), encoding="unicode")
    except Exception:
        return "refused"
    if payloads.CANARY_TEXT in text:
        return "LEAK"
    if "loaded-from-external-dtd" in text:
        return "LOADED"
    return "skipped"


version = ".".join(str(part) for part in etree.LXML_VERSION[:3])
print(f"lxml {version:<6}"
      f" fromstring: file={outcome(from_string, FILE_DOC):<8} param={outcome(from_string, PARAM_DOC):<8}"
      f" iterparse: file={outcome(from_iterparse, FILE_DOC):<8} param={outcome(from_iterparse, PARAM_DOC):<8}")
# step06_run.py - install three lxml versions side by side and run the same check against each
import os
import subprocess
import sys
from pathlib import Path

HERE = Path(__file__).resolve().parent
for version in ("6.0.4", "6.1.2", "6.1.3"):
    target = HERE / "lxold" / version
    if not target.exists():
        subprocess.run([sys.executable, "-m", "pip", "install", "-q", "--no-deps",
                        "--disable-pip-version-check", "--target", str(target),
                        f"lxml=={version}"], check=True)
    env = dict(os.environ, PYTHONPATH=str(target))  # this folder wins over site-packages
    subprocess.run([sys.executable, str(HERE / "step06_lxml_versions.py")], env=env, check=True)
python step06_run.py
lxml 6.0.4  fromstring: file=refused  param=LOADED   iterparse: file=LEAK     param=refused 
lxml 6.1.2  fromstring: file=refused  param=LOADED   iterparse: file=refused  param=refused 
lxml 6.1.3  fromstring: file=refused  param=refused  iterparse: file=refused  param=refused

Read it column by column. On lxml 6.0.4, iterparse with default settings returned the canary (LEAK), which is the CVE-2026-41066 behavior. On 6.0.4 and 6.1.2, plain fromstring with defaults loaded the external parameter entity from a file (LOADED), which is the gap fixed in 6.1.3. On 6.1.3 every cell is refused. The lesson is practical: pin lxml to 6.1.3 or newer, and keep a canary test in your suite (Step 9) so that an accidental downgrade or a new parser option is noticed by a failing test rather than by an incident report. We tested only these three releases, so do not read the table as a statement about versions in between.

Step 7: Build one safe door for untrusted XML

You now know which settings leak, and that the answer depends on library versions and options you do not control from one place. The fix is a policy enforced in exactly one function. OWASP’s general guidance is to “disable document type definitions (DTDs) whenever possible” and to “reject DOCTYPE declarations if the parser supports it.” Business documents rarely need a DTD, and the entity and external-resource attacks of Steps 4 and 5 all need one, so the policy is simple:

  1. Refuse documents above a size limit (1,000,000 bytes by default) before parsing anything.
  2. Refuse any document with a DOCTYPE declaration, found by Expat itself and not by searching the bytes.
  3. Only then hand the document to ElementTree, or to lxml with every risky option turned off.

Create safexml.py. The first part defines the error type and the size check.

# safexml.py - parse untrusted XML with a size cap and a DOCTYPE ban
import gzip
import io
import xml.etree.ElementTree as ET
from xml.parsers import expat

MAX_XML_BYTES = 1_000_000


class UnsafeXML(ValueError):
    """The document breaks the policy or is not well-formed."""


def check_size(data: bytes, max_bytes: int = MAX_XML_BYTES) -> None:
    if len(data) > max_bytes:
        raise UnsafeXML(f"document is {len(data)} bytes, the limit is {max_bytes}")

The second part is the heart of it. reject_doctype runs a bare Expat parser with a single handler, StartDoctypeDeclHandler. Expat calls that handler the moment it sees a DOCTYPE, before any entity has been declared, so this pass never expands an entity. The handler receives has_internal_subset, which is true when the document carries its own declarations in square brackets. Because Expat decodes the bytes itself, the check also works on UTF-16, as the next section shows.

# safexml.py (continued)
def reject_doctype(data: bytes, allow_plain_doctype: bool = False) -> None:
    """Pass 1: walk the document with a bare Expat parser that only watches for DOCTYPE.

    Expat decodes the bytes itself, so UTF-16 and other encodings cannot hide the keyword.
    The handler fires before any entity is declared, so nothing is ever expanded here.
    """

    def on_doctype(name, system_id, public_id, has_internal_subset):
        if has_internal_subset or not allow_plain_doctype:
            raise UnsafeXML("DOCTYPE declarations are not accepted")

    parser = expat.ParserCreate()
    parser.StartDoctypeDeclHandler = on_doctype
    try:
        parser.Parse(data, True)
    except expat.ExpatError as exc:
        raise UnsafeXML(f"not well-formed XML: {exc}") from exc

The third part is the public API. parse_untrusted returns an ElementTree element and parse_untrusted_lxml returns an lxml element, for code that needs XPath or other lxml features. Both run the same two checks first. The lxml version also switches off entity resolution, network access, DTD loading and the huge_tree override, so a future change to the policy cannot silently re-enable them.

# safexml.py (continued)
def parse_untrusted(data: bytes, max_bytes: int = MAX_XML_BYTES,
                    allow_plain_doctype: bool = False) -> ET.Element:
    """Pass 2 (ElementTree) only sees documents that passed the size and DOCTYPE checks."""
    check_size(data, max_bytes)
    reject_doctype(data, allow_plain_doctype)
    try:
        return ET.fromstring(data)
    except ET.ParseError as exc:
        raise UnsafeXML(f"not well-formed XML: {exc}") from exc


def parse_untrusted_lxml(data: bytes, max_bytes: int = MAX_XML_BYTES,
                         allow_plain_doctype: bool = False):
    """The same policy for code that needs lxml, with every risky parser option switched off."""
    from lxml import etree

    check_size(data, max_bytes)
    reject_doctype(data, allow_plain_doctype)
    parser = etree.XMLParser(resolve_entities=False, no_network=True, load_dtd=False,
                             dtd_validation=False, huge_tree=False)
    try:
        return etree.fromstring(data, parser)
    except etree.XMLSyntaxError as exc:
        raise UnsafeXML(f"not well-formed XML: {exc}") from exc

Why not simply search the bytes for DOCTYPE?

Because the bytes do not have to look like text you expect. The same attack document re-encoded as UTF-16 contains no <!DOCTYPE byte sequence at all, yet Expat decodes it and reports the declaration. The next part of payloads.py adds an SVG file with a plain, harmless DOCTYPE line (many real SVG files have one), a UTF-16 converter, and a few legitimate documents to prove that the door still opens for normal XML.

# payloads.py (continued)
SVG_PLAIN_DOCTYPE = (
    b'<?xml version="1.0"?>\n'
    b'<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" '
    b'"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">\n'
    b'<svg xmlns="http://www.w3.org/2000/svg" width="10" height="10">'
    b'<rect width="10" height="10"/></svg>'
)


def utf16(doc: bytes) -> bytes:
    """The same document re-encoded as UTF-16, which a byte-level search cannot see through."""
    text = doc.decode("utf-8").replace('encoding="UTF-8"', 'encoding="UTF-16"')
    return text.encode("utf-16")


def legit_documents() -> dict:
    return {
        "plain order": benign(),
        "namespaced": wrap("", "x").replace(b"<order>", b'<order xmlns="urn:example:orders">'),
        "cdata and char refs": wrap("", "<![CDATA[a < b]]> &#169; 2026"),
        "comment and pi": wrap("", "ok").replace(b"<order>", b"<?app v=1?><!-- note --><order>"),
    }

Now run the same seven documents through the three safe parsers, then through the legitimate documents, then through the UTF-16 comparison.

# step07_fixed.py - the same documents through the hardened parsers, then the legitimate documents
import xml.etree.ElementTree as ET

from lxml import etree

import matrix
import payloads
import safexml

FIXED_ROWS = [
    ("safexml (ElementTree)", lambda d: ET.tostring(safexml.parse_untrusted(d), encoding="unicode")),
    ("safexml (lxml hardened)",
     lambda d: etree.tostring(safexml.parse_untrusted_lxml(d), encoding="unicode")),
    ("defusedxml forbid_dtd=True", lambda d: matrix.defused_text(d, forbid_dtd=True)),
]

with matrix.internal_service():
    matrix.run_matrix("matrix_fixed", FIXED_ROWS, payloads.cases())
    matrix.print_service_log()

print()
print("Legitimate documents through safexml.parse_untrusted:")
documents = dict(payloads.legit_documents())
documents["svg with a plain DOCTYPE"] = payloads.SVG_PLAIN_DOCTYPE
for label, data in documents.items():
    try:
        root = safexml.parse_untrusted(data)
        result = f"accepted <{root.tag.split('}')[-1]}>"
    except safexml.UnsafeXML as exc:
        result = f"refused ({exc})"
    print(f"  {label:<26}{result}")
root = safexml.parse_untrusted(payloads.SVG_PLAIN_DOCTYPE, allow_plain_doctype=True)
print(f"  {'svg, allow_plain_doctype':<26}accepted <{root.tag.split('}')[-1]}>")

print()
print("Why not just search the bytes for <!DOCTYPE ?")
attack = payloads.cases()["file"]
for label, doc in (("UTF-8", attack), ("UTF-16", payloads.utf16(attack))):
    passes = b"<!DOCTYPE" not in doc
    try:
        safexml.parse_untrusted(doc)
        verdict = "accepted"
    except safexml.UnsafeXML as exc:
        verdict = f"refused ({exc})"
    print(f"  {label}: byte search says safe = {passes!s:<5}  safexml: {verdict}")
python step07_fixed.py
parser                      internal bomb     quad     file     http     dtd      param    
-------------------------------------------------------------------------------------------
safexml (ElementTree)       REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  
safexml (lxml hardened)     REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  
defusedxml forbid_dtd=True  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  REFUSED  

* = the internal service logged a request
requests the internal service received: 0

Legitimate documents through safexml.parse_untrusted:
  plain order               accepted <order>
  namespaced                accepted <order>
  cdata and char refs       accepted <order>
  comment and pi            accepted <order>
  svg with a plain DOCTYPE  refused (DOCTYPE declarations are not accepted)
  svg, allow_plain_doctype  accepted <svg>

Why not just search the bytes for <!DOCTYPE ?
  UTF-8: byte search says safe = False  safexml: refused (DOCTYPE declarations are not accepted)
  UTF-16: byte search says safe = True   safexml: refused (DOCTYPE declarations are not accepted)

All three safe setups refuse all seven documents, and the internal service received no request at all. The four ordinary orders are accepted. The SVG file with a plain DOCTYPE is refused by default and accepted when you opt in with allow_plain_doctype=True, which is the option to use if you accept SVG uploads (an internal subset is never accepted, as the tests in Step 9 check). The last two lines are the reason for asking Expat instead of searching the bytes: for the UTF-16 version of the attack, the byte search says the document is safe, while safexml refuses it.

What about defusedxml?

The last row of the table shows that defusedxml with forbid_dtd=True also refuses all seven documents, and OWASP’s Python guidance recommends it: use “the defusedxml parsing interfaces in place of the corresponding standard-library parsing functions,” keep entity and external-resource rejection on, and set forbid_dtd=True “when DTDs are unnecessary.” OWASP adds a caveat that applies to every option here: these interfaces “reject prohibited constructs; they do not sanitize XML or replace application resource limits.” So either door is a fine choice; what matters is that there is one. We built safexml because it is short enough to read in one sitting, it covers lxml with the same policy, and its behavior is pinned by tests. For context, defusedxml’s latest stable release on PyPI is 0.7.1 from March 2021, with 0.8.0 release candidates also listed, and it worked on Python 3.13.14 in every run here.

Wire it in

In your application, check the Expat build once at start-up and send every untrusted document through the door. This excerpt is not part of the lab; it shows the shape. Turn UnsafeXML into a client error such as HTTP 400.

# app.py (excerpt, not part of the lab)
import expatcheck
import safexml

expatcheck.require_protected_expat()  # refuse to start on an unprotected Expat


def read_order(body: bytes):
    root = safexml.parse_untrusted(body)  # raises UnsafeXML for anything unsafe
    return root.findtext("note")

Step 8: Bound the unzipped size and the slow-fed token

Two attacks need no DTD, so the DOCTYPE ban does not stop them. The first is the decompression bomb: a size limit on the incoming body is not enough if the body is gzip, because a small file can unpack to something huge. The Expat 2.7.2 changelog makes the same point: “a layer of compression around XML can significantly reduce the minimum attack payload size.” Add the bounded readers to safexml.py. read_limited asks the stream for one byte more than the limit and refuses if it gets it, so a decompressing stream is cut off after about 1 MB instead of after 100 MB.

# safexml.py (continued)
def read_limited(stream, max_bytes: int = MAX_XML_BYTES) -> bytes:
    """Read at most max_bytes from a stream, even when the stream decompresses on the fly."""
    data = stream.read(max_bytes + 1)
    if len(data) > max_bytes:
        raise UnsafeXML(f"input is larger than {max_bytes} bytes")
    return data


def gunzip_limited(blob: bytes, max_bytes: int = MAX_XML_BYTES) -> bytes:
    with gzip.GzipFile(fileobj=io.BytesIO(blob)) as stream:
        return read_limited(stream, max_bytes)
# step08_gzip.py - a 100 MB document that travels as a small gzip file
import gzip
import time

import safexml

blob = gzip.compress(b"<a>" + b"A" * 100_000_000 + b"</a>")
print(f"compressed size: {len(blob):,} bytes")

started = time.perf_counter()
naive = gzip.decompress(blob)
print(f"gzip.decompress(blob): {len(naive):,} bytes in memory after {time.perf_counter() - started:.2f} s")
del naive

try:
    safexml.gunzip_limited(blob)
except safexml.UnsafeXML as exc:
    print("safexml.gunzip_limited:", exc)
python step08_gzip.py
compressed size: 97,234 bytes
gzip.decompress(blob): 100,000,007 bytes in memory after 0.10 s
safexml.gunzip_limited: input is larger than 1000000 bytes

A 100,000,007 byte document travelled as 97,234 bytes. The naive gzip.decompress inflated it fully in memory. The bounded reader stopped at the limit and raised UnsafeXML. The same idea applies to any format that wraps XML in compression. OWASP’s Python note says to “independently bound input size and decompressed data,” and mentions that the standard-library xmlrpc module is vulnerable to decompression bombs.

The second attack is the large token. If a single comment or text run is huge and the data arrives in small pieces (as it does when you read from a socket), an unprotected parser re-reads the unfinished token from its start every time a piece arrives, which makes the total time grow with the square of the size. Expat 2.6.0 added reparse deferral to prevent this; the Python reference warns that “Calling SetReparseDeferralEnabled(False) has security implications.” We switch it off here only to measure what the protection saves.

# step09_large_tokens.py - the third protection: Expat defers re-parsing of a huge unfinished token
import time
from xml.parsers import expat


def feed_comment(size, deferral, chunk=512):
    parser = expat.ParserCreate()
    parser.SetReparseDeferralEnabled(deferral)
    data = b"<r><!--" + b"A" * size + b"--></r>"
    started = time.perf_counter()
    for start in range(0, len(data), chunk):
        parser.Parse(data[start:start + chunk], False)
    parser.Parse(b"", True)
    return time.perf_counter() - started


print(f"one comment fed to Expat in 512 byte pieces ({expat.EXPAT_VERSION})")
print(f"{'comment':>8} {'deferral on (default)':>22} {'deferral off':>14}")
for size in (500_000, 1_000_000, 2_000_000):
    on = feed_comment(size, True)
    off = feed_comment(size, False)
    print(f"{size:>8,} B {on:>18.3f} s {off:>12.3f} s")
python step09_large_tokens.py
one comment fed to Expat in 512 byte pieces (expat_2.8.1)
 comment  deferral on (default)   deferral off
 500,000 B              0.000 s        0.052 s
1,000,000 B              0.001 s        0.206 s
2,000,000 B              0.002 s        0.827 s

With deferral on (the default) a 2,000,000 byte comment fed in 512 byte pieces takes 0.002 s. With it off, the time is 0.052 s, 0.206 s and 0.827 s for 0.5, 1 and 2 million bytes: each doubling of the size makes the run about four times slower, which is the quadratic growth in numbers. On a current Expat you do not need to do anything for this attack except leave the default alone, which is one more reason to check the build in Step 3.

Step 9: Lock the behavior in with tests

A policy that lives only in your head decays. These tests run the seven attack documents and the ordinary documents through both safe parsers, and also check the UTF-16 case, the size limit, the plain DOCTYPE option, the gzip bound and the Expat self-test from Step 3. The first part covers the parsers.

# test_safexml.py - the policy as executable tests
import gzip

import pytest

import expatcheck
import payloads
import safexml
from safexml import UnsafeXML

PARSERS = [safexml.parse_untrusted, safexml.parse_untrusted_lxml]
ATTACKS = payloads.cases()
LEGIT = payloads.legit_documents()
by_name = lambda func: func.__name__  # noqa: E731


@pytest.mark.parametrize("parse", PARSERS, ids=by_name)
@pytest.mark.parametrize("name", sorted(ATTACKS))
def test_documents_with_a_dtd_are_refused(parse, name):
    with pytest.raises(UnsafeXML):
        parse(ATTACKS[name])


@pytest.mark.parametrize("parse", PARSERS, ids=by_name)
@pytest.mark.parametrize("name", sorted(LEGIT))
def test_ordinary_documents_still_parse(parse, name):
    assert parse(LEGIT[name]).tag.endswith("order")


@pytest.mark.parametrize("parse", PARSERS, ids=by_name)
def test_utf16_encoding_does_not_hide_the_doctype(parse):
    with pytest.raises(UnsafeXML):
        parse(payloads.utf16(ATTACKS["file"]))


@pytest.mark.parametrize("parse", PARSERS, ids=by_name)
def test_oversized_document_is_refused_and_the_cap_can_be_raised(parse):
    big = payloads.wrap("", "x" * 1_200_000)
    with pytest.raises(UnsafeXML):
        parse(big)
    assert parse(big, max_bytes=2_000_000).tag == "order"


@pytest.mark.parametrize("parse", PARSERS, ids=by_name)
def test_plain_doctype_is_opt_in_and_an_internal_subset_never_passes(parse):
    with pytest.raises(UnsafeXML):
        parse(payloads.SVG_PLAIN_DOCTYPE)
    assert parse(payloads.SVG_PLAIN_DOCTYPE, allow_plain_doctype=True).tag.endswith("svg")
    with pytest.raises(UnsafeXML):
        parse(ATTACKS["internal"], allow_plain_doctype=True)

The second part covers malformed input, the gzip bound and the Expat checks. The last test replaces check_expat with a fake that reports an unprotected build and confirms that start-up would refuse to continue.

# test_safexml.py (continued)
def test_malformed_xml_is_reported_as_unsafe():
    with pytest.raises(UnsafeXML):
        safexml.parse_untrusted(b"<order><note></order>")


def test_gzip_bomb_is_cut_off_and_a_small_file_passes():
    bomb = gzip.compress(b"<a>" + b"A" * 20_000_000 + b"</a>")
    assert len(bomb) < 100_000
    with pytest.raises(UnsafeXML):
        safexml.gunzip_limited(bomb)
    assert safexml.gunzip_limited(gzip.compress(payloads.benign())) == payloads.benign()


def test_this_expat_refuses_the_probe_bomb():
    info = expatcheck.check_expat()
    assert info["version_ok"] and info["refused_probe"]


def test_self_test_notices_an_unprotected_parser():
    info = expatcheck.check_expat(simulate_old=True)
    assert not info["refused_probe"]
    assert info["chars_produced"] > 9_000_000


def test_require_protected_expat_raises_on_a_bad_build(monkeypatch):
    monkeypatch.setattr(expatcheck, "check_expat", lambda: {"version_ok": True, "refused_probe": False})
    with pytest.raises(RuntimeError):
        expatcheck.require_protected_expat()
python -m pytest -q test_safexml.py
................................. [100%]
33 passed in 0.13s

That is 33 tests: 14 for the seven attack documents on two parsers, 8 for ordinary documents, 2 each for UTF-16, the size limit and the plain DOCTYPE option, then one for malformed input, one for gzip and three for Expat. A passing suite is only evidence if it can fail, so the next script switches the DOCTYPE ban off and runs the same tests.

# step10_break_it.py - prove the tests notice a broken policy
import pytest

import safexml

safexml.reject_doctype = lambda *args, **kwargs: None  # switch the DOCTYPE ban off
raise SystemExit(pytest.main(["-q", "--tb=no", "-p", "no:cacheprovider", "test_safexml.py"]))
python step10_break_it.py
...F.F.FFF.............F..FF..... [100%]
=== short test summary info ===
FAILED test_safexml.py::test_documents_with_a_dtd_are_refused[dtd-parse_untrusted_lxml] - Failed: DID NOT RAISE UnsafeXML
FAILED test_safexml.py::test_documents_with_a_dtd_are_refused[file-parse_untrusted_lxml] - Failed: DID NOT RAISE UnsafeXML
FAILED test_safexml.py::test_documents_with_a_dtd_are_refused[http-parse_untrusted_lxml] - Failed: DID NOT RAISE UnsafeXML
FAILED test_safexml.py::test_documents_with_a_dtd_are_refused[internal-parse_untrusted] - Failed: DID NOT RAISE UnsafeXML
FAILED test_safexml.py::test_documents_with_a_dtd_are_refused[internal-parse_untrusted_lxml] - Failed: DID NOT RAISE UnsafeXML
FAILED test_safexml.py::test_utf16_encoding_does_not_hide_the_doctype[parse_untrusted_lxml] - Failed: DID NOT RAISE UnsafeXML
FAILED test_safexml.py::test_plain_doctype_is_opt_in_and_an_internal_subset_never_passes[parse_untrusted] - Failed: DID NOT RAISE UnsafeXML
FAILED test_safexml.py::test_plain_doctype_is_opt_in_and_an_internal_subset_never_passes[parse_untrusted_lxml] - Failed: DID NOT RAISE UnsafeXML
8 failed, 25 passed in 0.21s

Eight tests fail, which is the point. The failures are instructive. With the ban removed, ElementTree still refuses the bombs and the external documents, but only because Expat’s limits and its “undefined entity” error happen to catch them; the legitimate-looking company entity and the plain SVG DOCTYPE get through, and those tests fail. The hardened lxml parser is the surprise: with resolve_entities=False it does not raise on the file, http and dtd documents, because it leaves the unresolved references in the tree (the file document serializes as <order><note>&secret;</note></order>). That is quiet rather than safe, which is why the policy refuses the DOCTYPE instead of relying on parser flags. The patch exists only inside that script, so a normal run of the suite still gives 33 passed.

Step 10: Make the safe door the only door

New code will eventually call ET.fromstring directly, because it is the first thing anyone finds. Two checks in CI can stop that. First, see what the usual security linters notice. risky_calls.py contains the calls from this tutorial, including the two dangerous lxml and SAX configurations, plus one call that goes through safexml. It is only ever scanned, never run.

# risky_calls.py - parser calls a linter might notice (this file is only ever scanned, never run)
import xml.dom.minidom
import xml.dom.pulldom
import xml.etree.ElementTree as ET
import xml.sax
import xml.sax.handler

from lxml import etree

import safexml


def with_etree(data):
    return ET.fromstring(data)


def with_minidom(data):
    return xml.dom.minidom.parseString(data)


def with_pulldom(data):
    return xml.dom.pulldom.parseString(data)


def with_sax(data):
    return xml.sax.parseString(data, xml.sax.handler.ContentHandler())


def with_lxml(data):
    return etree.fromstring(data)


def with_lxml_entities_on(data):
    return etree.fromstring(data, etree.XMLParser(resolve_entities=True, no_network=False))


def with_sax_external_entities(data):
    parser = xml.sax.make_parser()
    parser.setFeature(xml.sax.handler.feature_external_ges, True)
    return parser


def with_safexml(data):
    return safexml.parse_untrusted(data)

The script below runs Ruff’s suspicious-call rules (the S3 group) and Bandit over that file and prints which function each rule fired in.

# step11_lint.py - which of the risky calls do Ruff and Bandit notice?
import re
import subprocess
import sys

SOURCE = "risky_calls.py"
FUNCTION = re.compile(r"^def (\w+)")
names = {}
for number, line in enumerate(open(SOURCE, encoding="utf-8"), start=1):
    match = FUNCTION.match(line)
    if match:
        names[number] = match.group(1)


def owner(line_number):
    earlier = [n for n in names if n <= line_number]
    return names[max(earlier)] if earlier else "(import lines)"


def run(command):
    done = subprocess.run([sys.executable, "-m"] + command, capture_output=True, text=True, encoding="utf-8")
    return done.stdout + done.stderr


def report(title, text, pattern):
    print(title)
    flagged = {}
    for found in re.finditer(pattern, text, re.MULTILINE):
        flagged.setdefault(owner(int(found.group("line"))), []).append(found.group("rule"))
    for name in ["(import lines)"] + list(names.values()):
        if name in flagged or name in names.values():
            print(f"  {name:<28}{', '.join(sorted(set(flagged.get(name, [])))) or '-'}")


ruff = run(["ruff", "check", "--select", "S3", "--no-cache", "--output-format", "concise", SOURCE])
report("Ruff, rules S3xx:", ruff, rf"^{SOURCE}:(?P<line>\d+):\d+: (?P<rule>S\d+)")
bandit = run(["bandit", "-q", "-f", "custom", "--msg-template", "{line} {test_id}", SOURCE])
report("Bandit:", bandit, r"^(?P<line>\d+) (?P<rule>B\d+)")
python step11_lint.py
Ruff, rules S3xx:
  with_etree                  S314
  with_minidom                S318
  with_pulldom                S319
  with_sax                    S317
  with_lxml                   -
  with_lxml_entities_on       -
  with_sax_external_entities  S317
  with_safexml                -
Bandit:
  (import lines)              B405, B406, B408, B409
  with_etree                  B314
  with_minidom                B318
  with_pulldom                B319
  with_sax                    B317
  with_lxml                   -
  with_lxml_entities_on       -
  with_sax_external_entities  B317
  with_safexml                -

Both tools flag the standard-library parsers (Ruff S314 and its siblings, and Bandit’s B314 family plus import-level B405, B406, B408 and B409). Neither flags any of the lxml lines, including resolve_entities=True with no_network=False, which is the most dangerous configuration in this tutorial, and neither flags the call that goes through safexml. The standard-library findings are warnings about a safe default that depends on the Expat build (Step 3), not proof of a bug, and the lxml silence is a real gap.

The second check closes the gap with a different kind of rule. Ruff’s banned-api rule (TID251) bans importing the parsers anywhere except the one file that is allowed to. Create ruff.toml:

# ruff.toml - one door for untrusted XML
[lint.flake8-tidy-imports.banned-api]
"xml.etree.ElementTree".msg = "Parse untrusted XML with safexml.parse_untrusted."
"xml.dom.minidom".msg = "Parse untrusted XML with safexml.parse_untrusted."
"xml.dom.pulldom".msg = "Parse untrusted XML with safexml.parse_untrusted."
"xml.sax".msg = "Parse untrusted XML with safexml.parse_untrusted."
"lxml.etree".msg = "Parse untrusted XML with safexml.parse_untrusted_lxml."

[lint.per-file-ignores]
"safexml.py" = ["TID251"]
python -m ruff check --select TID251 --no-cache --output-format concise risky_calls.py safexml.py
risky_calls.py:2:8: TID251 `xml.dom.minidom` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:3:8: TID251 `xml.dom.pulldom` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:4:8: TID251 `xml.etree.ElementTree` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:5:8: TID251 `xml.sax` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:6:8: TID251 `xml.sax` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:8:18: TID251 `lxml.etree` is banned: Parse untrusted XML with safexml.parse_untrusted_lxml.
risky_calls.py:18:12: TID251 `xml.dom.minidom` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:22:12: TID251 `xml.dom.pulldom` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:26:12: TID251 `xml.sax` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:26:38: TID251 `xml.sax` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:38:14: TID251 `xml.sax` is banned: Parse untrusted XML with safexml.parse_untrusted.
risky_calls.py:39:23: TID251 `xml.sax` is banned: Parse untrusted XML with safexml.parse_untrusted.
Found 12 errors.

Twelve findings, all in risky_calls.py, including the lxml.etree import that the S3 rules never noticed, and none in safexml.py because the per-file ignore exempts the door itself. Run the same command over your whole source tree in CI. The exit code is 1 when there are findings, which fails the build.

Confirm everything works end to end

Open a fresh terminal, activate the environment, and run these four checks. Together they confirm the whole chain:

  1. python step03_expat.py: the first probe line shows version_ok and refused_probe both true.
  2. python step05_matrix.py: at least one row shows LEAK (the external_ges=True and resolve_entities=True rows), and the service log lists four requests.
  3. python step07_fixed.py: every cell in the fixed table says REFUSED and the service received 0 requests.
  4. python -m pytest -q test_safexml.py: 33 passed.

If check 2 shows no LEAK at all, something is wrong with the lab (for example, the canary file is missing); if check 3 shows any cell other than REFUSED, the door has a hole, and the failing tests will tell you which document got through.

Common mistakes

  • Copying resolve_entities=True or feature_external_ges=True from an old answer. Both turned the lab into a file reader. A parser default that is safe today only helps if nobody overrides it.
  • Treating a silent skip as success. minidom, pulldom and sax defaults drop external entity text without an error.
  • Filtering the bytes for DOCTYPE. A UTF-16 document walks straight past a byte search.
  • Trusting the Python version number. The Expat build decides; check pyexpat.EXPAT_VERSION and run the probe.
  • Lowering Expat’s activation threshold below 4 MiB or disabling reparse deferral. The documentation warns against both.
  • Letting the linter be the policy. Ruff and Bandit flagged the standard-library calls but stayed silent on the riskiest lxml setting.

Next steps

  • Find the places in your own code base where XML enters: SAML libraries, SVG upload handlers, Office or ODF readers, feed importers. Many call a parser internally, so check each library’s documentation and pin its version.
  • OWASP’s guidance also says to keep XInclude disabled and to restrict external access during schema validation and style sheet processing. This tutorial did not test XInclude, XSLT or schema validation, so verify them separately if you use them.
  • Add resource limits outside the parser: request size limits at the web server, memory and time limits on the worker. The 15.0 MB peak in Step 4 is what a document under the threshold costs, multiplied by your concurrency.
  • Read the sibling tutorials on pickle deserialization, mass assignment and webhook signatures: each is another place where a parser trusts what an outsider sends.

Sources and method

Method: every output block above was produced by the files shown, on Windows 11 with Python 3.13.14, and a clean-folder rebuild from this page’s code blocks reproduced the same output lines. Steps 3, 4, 5 and 9 were repeated on the python.org Python 3.15.0 build; the five standard-library rows of the matrix matched on all 35 cells. Figures marked “my arithmetic” are computed from the printed outputs, and the lab uses a canary string and a loopback service in place of real data.

Tags:

Application SecurityOWASPpytestPythonXMLXXE

Share

No Comment! Be the first one.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts