CNCF’s Dragonfly Shows Why Kubernetes Image Distribution Doesn’t Need a Database

Every Kubernetes node that pulls the same container image from a central registry at the same moment is a small bet against that registry’s bandwidth. Roll a large image out to a hundred nodes at once, during a deployment, a cluster autoscale event, or a CI pipeline spinning up fresh runners, and the registry becomes the bottleneck every one of those nodes is waiting on. Dragonfly, a peer-to-peer file and image distribution project that reached CNCF’s Graduated maturity level on October 28, 2025, exists to break that dependency: nodes that already have an image’s layers pass them to nodes that need them, so only a fraction of pulls ever have to touch the origin registry at all.

A post published on the CNCF blog on August 13, 2026 by Dragonfly maintainer Wenbo Qi (Gaius) addresses a different problem: getting that benefit has, until recently, meant taking on real infrastructure to earn it.

What the Full Deployment Actually Requires

A standard Dragonfly install runs a Scheduler, a Seed Client, and a Client on every node to move data, plus a Manager that handles dynamic configuration, backed by MySQL for state and Redis for caching and asynchronous job distribution. The Manager is Dragonfly’s control plane: it hosts a web console, exposes APIs for integrations like registry-triggered preheating, and coordinates relationships across multiple P2P clusters at once.

That is a reasonable cost for a platform team running Dragonfly as shared infrastructure across many clusters. It is a much harder sell for one cluster, an edge location, or a CI/CD pipeline that just wants faster image pulls without operating two more stateful services. As the post puts it, a Manager-backed setup "fits platform teams running Dragonfly across large multi-cluster fleets," but it "can be heavy for a single cluster focused primarily on resolving registry overload during image pulls." The key architectural fact that makes an alternative possible: the Scheduler and Client can run without any Manager or database at all, simply by leaving the Manager’s address unconfigured.

Two Kubernetes Primitives Instead of a Control Plane

The lightweight model replaces the Manager with two things Kubernetes already provides: a ConfigMap and a headless Service.

Configuration Through a ConfigMap

When no Manager address is set, the Scheduler and Client load their dynamic configuration from a local /etc/dragonfly/dynconfig.yaml file, which the Helm chart mounts as a ConfigMap. If the file isn’t present, default values are generated at startup. The Scheduler’s copy defines cluster-level scheduling parameters directly as YAML, for example a seed-peer concurrent upload limit (loadLimit: 2000), a candidate-parent scheduling limit (candidateParentLimit: 3), a filter-parent limit (filterParentLimit: 15), and a per-client peer concurrent upload limit (loadLimit: 200).

The Scheduler and Client reload this configuration periodically, on an interval controlled by refreshInterval that defaults to one minute, and updates to the ConfigMap reach running pods within that window without a pod restart. The same parameters are exposed as Helm values (scheduler.dynconfig, seedClient.dynconfig, client.dynconfig), so the whole thing stays declarative and GitOps-friendly rather than living behind an API call to a Manager.

Finding the Scheduler Without an API Call

In a Manager-based deployment, Clients query the Manager to find available Schedulers. In the lightweight model, the Client’s configuration points straight at the Scheduler’s headless Service, something like dragonfly-scheduler.dragonfly-system.svc.cluster.local:8002. The Client process, called dfdaemon, resolves that hostname over DNS to discover every Scheduler pod IP, health-checks each one, and filters out anything unhealthy. Scale the Scheduler StatefulSet up or down and the discovered endpoints update automatically. For Dragonfly peers running outside Kubernetes entirely, a static list of addresses can override DNS discovery instead.

What Actually Runs in the Cluster

Three workloads make up the lightweight footprint: a Scheduler (StatefulSet) that coordinates task scheduling and piece distribution across peers, a Seed Client (StatefulSet) that acts as the root peer and downloads content directly from the origin repository, and a Client (DaemonSet) on every node, where a component called dfinit configures containerd to route image pulls through the local peer client. None of that needs database backups or migration scripts during upgrades, since state lives in local disk caches that can be rebuilt on demand from the origin source if lost.

Three Tiers, Not One Right Answer

Because the Helm chart disables the Manager, MySQL, and Redis by default, the lightweight setup is the baseline rather than a stripped-down alternative. Dragonfly actually ships three deployment options. The base lightweight tier (Scheduler, Seed Client, and Client only) is described as ideal for single clusters, edge locations, or CI/CD pipelines focused on P2P distribution, and it still supports blocklists, task distribution, and preheating triggered through Dragonfly’s dfctl CLI. Adding Redis on top gains persistent task and cache-task tracking that survives restarts instead of relying purely on rebuild-from-origin caches. Adding the full Manager, with MySQL and Redis both enabled, is what unlocks preheating triggered through the OpenAPI or the web console, the web console itself, broader OpenAPI integration, and personal access tokens.

Framed that way, the lightweight tier isn’t giving up P2P distribution or scheduling logic, the part actually cutting registry load. It’s giving up state durability across restarts and a management API surface that a single cluster often doesn’t need in the first place.

Trying the Lightweight Model

The post walks through standing up the lightweight deployment on a local kind cluster (Kubernetes-in-Docker, a common tool for spinning up a disposable multi-node cluster for testing). The shape of it: define a three-node cluster, one control-plane and two workers, in a small YAML file and create it with kind create cluster --config kind-config.yaml; pre-load the dragonflyoss/scheduler, dragonflyoss/client, and dragonflyoss/dfinit images into the cluster with kind load docker-image so the nodes don’t need to pull them from a registry first; write a Helm values file naming those images and enabling metrics; then install with helm install --wait --create-namespace --namespace dragonfly-system dragonfly dragonfly/dragonfly -f charts-config.yaml against the dragonflyoss.github.io/helm-charts repository. A successful install produces one Client pod per worker node plus one Scheduler and one Seed Client, all visible with kubectl get po -n dragonfly-system.

What Still Needs the Manager

Preheating, pre-downloading an image or file onto Seed Clients or node Clients before it’s actually requested, works in any of the three tiers through the dfctl CLI, which talks directly to the Scheduler’s gRPC endpoint. A command like dfctl task preheat oci://docker.io/library/alpine:3.19 --scheduler-endpoint http://dragonfly-scheduler.dragonfly-system.svc.cluster.local:8002 --scope all_seed_peers works without a Manager in the picture at all. What doesn’t work without one is triggering that same preheating through the OpenAPI or the web console, or using the web console for anything else.

Separately from all three tiers, a mutating admission webhook called dragonfly-injector (its own project, with its own GitHub repository) can inject Dragonfly’s client tooling into arbitrary application pods that need to pull secondary assets, model weights, datasets, build artifacts, through the P2P network without modifying the application’s own container image. On pod creation, the webhook adds an init container that copies the necessary binaries into a shared volume and mounts the node’s dfdaemon socket, letting the application run dfget commands natively.

The Migration Path Is the Real Point

The Manager architecture is still the right call for a platform team running Dragonfly as centralized infrastructure across multiple clusters, or for anyone who specifically wants the web console or OpenAPI-triggered automation; enabling it is a matter of setting manager.enable, mysql.enable, and redis.enable to true in the Helm configuration. But because the underlying Scheduler, Seed Client, and Client components, and the protocol they speak, don’t change between tiers, teams can start with a lightweight deployment on one cluster and add the Manager later without redesigning anything.

That migration path is arguably the more durable change here. Dragonfly graduated within CNCF less than a year ago, a tier reserved for projects CNCF considers stable, widely adopted, and production-ready. A P2P distribution project at that maturity level defaulting to a database-free install, rather than treating a lightweight mode as an afterthought, makes the case for Dragonfly on a single three-node CI cluster look a lot different than it did when the only on-ramp was MySQL and Redis.

Sources: CNCF’s Lightweight Dragonfly Deployment: P2P Distribution Without the Database Stack, Dragonfly’s own Lightweight Deployment documentation, CNCF’s Dragonfly project page, and the Dragonfly GitHub repository.

Featured image: Emperor dragonfly (Anax imperator) male in flight, Malta by Charles J. Sharp, Sharp Photography, released under CC BY-SA 4.0 via Wikimedia Commons; cropped, resized, and converted to WebP.