CNCF’s Platform Readiness Blueprint Turns Kubernetes-for-AI Adoption Into a Maturity Gap

Kubernetes has become the default place to run AI inference. It has not become the default place to run AI well. That is the argument at the center of a CNCF blog post published August 28, and the numbers behind it are more specific than the usual “AI is eating infrastructure” framing.

The post was written by Kasia Hilborne, a product marketing manager at the GPU cloud provider Vultr, and published through CNCF’s member-contributor program. It opens with a statistic from CNCF’s own 2025 Annual Cloud Native Survey: 66% of organizations hosting generative AI models use Kubernetes for some or all of their inference workloads. The next number is the catch. Only 7% of those organizations deploy AI models daily; 47% deploy only occasionally. CNCF’s own announcement of the same survey adds a second data point that didn’t make it into Hilborne’s post: 44% of respondents overall said they don’t yet run AI/ML workloads on Kubernetes at all, even as Kubernetes production adoption among container users overall reached 82%.

Read together, the numbers describe a familiar shape: broad platform adoption, thin operational maturity. Most organizations that want to run generative AI have already put Kubernetes underneath it. Very few are running it the way they run any other production system, with routine, repeatable deployment rather than sporadic, hand-managed pushes.

A second survey finds the same gap from the platform team’s side

Hilborne’s post pairs the CNCF numbers with a separate study: the 2025 State of AI in Platform Engineering report, co-published by Vultr and the community site Platform Engineering, written by analyst Luca Galante of Weave Intelligence and Sam Barlien of Platform Engineering, and based on survey data from 204 platform engineers combined with additional insights from platform engineering leaders. Its headline finding: 35% of platform teams still don’t orchestrate AI workloads at all.

That statistic locates the readiness problem specifically inside platform teams, the group whose job is normally to make infrastructure boring and repeatable for everyone else. If a third of them haven’t brought AI workloads into that discipline yet, the gap CNCF’s survey measures at the infrastructure level (deployed daily versus deployed occasionally) has a plausible organizational cause sitting one layer down.

Five gaps between a container platform and an AI platform

The post’s actual argument isn’t that platform teams need to abandon cloud native practice and build something new for AI. It’s that closing the gap means extending five things platform teams already do for containers so they also cover models.

A resource model that accounts for more than CPU and memory

Production AI pipelines aren’t uniformly GPU-bound. Data preparation, retrieval, and orchestration logic typically run on CPUs, while training or inference calls out to GPUs or other accelerators, often within the same pipeline. That mix changes what a scheduler has to reason about: not just CPU and memory, but accelerator type, availability, and topology.

Kubernetes has been building toward this. Dynamic Resource Allocation (DRA) lets device drivers and cluster admins define device classes that workloads can claim declaratively, instead of through the older device-plugin model that mostly just exposed a device count. DRA’s core reached General Availability in Kubernetes v1.34, released in September 2025; in the following release, v1.35, its feature gate was locked in (no longer possible to disable). The mechanism exists and ships by default. Hilborne’s point is that most platform teams haven’t yet built the scheduling and quota policy around it that they already have for CPU and memory.

CI/CD that tracks the model, not just the code

Cloud native teams made application delivery repeatable through CI/CD and GitOps: code, build, test, deploy. AI adds a new artifact partway through that chain. The post frames the extended pipeline as code plus model plus configuration, evaluated, then deployed, then observed, then updated, and argues platform teams need a straightforward answer to a simple question: which application, model, and configuration combination is actually running right now, and can it be reproduced. Models are larger than most build artifacts, often tied to specific runtimes or hardware, and typically need evaluation before they ship in a way application code usually doesn’t.

Observability that follows the workload, not just the cluster

CPU, memory, and request latency don’t describe what’s happening inside an AI workload. The post lists accelerator utilization, scheduling and queue times, inference latency, model load time, throughput, and endpoint health as the metrics that actually explain where an AI workload is spending its time. The more interesting claim is about correlation, not collection: the goal isn’t a separate AI monitoring stack, but extending the existing cloud native observability layer so infrastructure, application, and AI-specific telemetry line up against the same request. That’s what makes it possible to answer a question GPU utilization alone can’t: where is the workload actually waiting.

A golden path that understands models and accelerators

“Golden path” is platform engineering’s term for a paved, self-service route that lets developers deploy without becoming infrastructure experts themselves. Kubernetes platforms have built these for containers for years: specify what the app needs, get a repeatable deployment back. The post argues the same pattern needs to extend to models, so a developer can go from model to resources to deployment to endpoint to observability to policy without personally reasoning about GPU topology or DRA device classes.

Treating AI as another production workload, not a parallel stack

The thread running through all four changes is the same: don’t build AI infrastructure next to the Kubernetes platform, build it into the platform. The post’s closing argument is that the operational patterns cloud native teams already have, orchestration, declarative infrastructure, automated delivery, observability, policy, developer self-service, are the right foundation for AI too. What’s missing in most organizations isn’t a new platform. It’s the extension work.

The messenger has a stake in the message

It’s worth being direct about where this argument comes from. Vultr is a GPU cloud provider. A post arguing that most companies’ Kubernetes platforms aren’t actually ready to run AI in production is also, not coincidentally, an argument for buying GPU infrastructure and platform expertise from a company that says it has already solved the problem. CNCF’s blog explicitly labels the post a “Member Post,” distinct from staff or Ambassador content, which is exactly the category CNCF uses so member organizations can publish this kind of material under the CNCF banner. Readers should weigh it accordingly.

What doesn’t depend on Vultr’s motives is the underlying data. The 66/7/47/44/82 figures come from CNCF’s own survey, run independently of any single vendor. The DRA timeline is documented in Kubernetes’ own release notes. And the 35% orchestration figure comes from a named report with a disclosed sample size and named authors, not an anonymous vendor claim. Strip away the pitch and what’s left is a reasonably well-supported observation: a lot of organizations have Kubernetes running AI workloads somewhere, and comparatively few of them have finished the unglamorous platform engineering work, resource models, CI/CD, observability, golden paths, that would let them run AI the way they already run everything else.

That gap is also why coverage of platform engineering on this site keeps circling back to the same theme from different angles: Red Hat’s Metal to Agents framework made a similar argument about AI token growth outrunning infrastructure planning, and CNCF’s own write-up on Kyverno made a parallel case, in a non-AI context, for treating platform automation as infrastructure rather than a bolt-on gate. The specific vendor and the specific workload change. The underlying claim, that operational maturity lags the adoption curve, keeps showing up in the data.