How to Migrate a Python App From the OpenAI API to DigitalOcean Serverless Inference

If you already have a Python application calling the OpenAI API, you do not need to rewrite it to try a different inference provider. DigitalOcean’s Serverless Inference product, part of its Gradient AI Platform, exposes OpenAI-compatible REST endpoints, so an existing OpenAI SDK integration can often move over with three small changes: the base URL, the credential, and the model name. This tutorial walks through exactly what changes, what stays the same, and how to verify each step actually works before you trust it in a real project.

By the end, you will have a Python virtual environment with the OpenAI SDK installed, a DigitalOcean Model Access Key stored as an environment variable, a working chat completion request running against DigitalOcean’s servers instead of OpenAI’s, a streaming version of that same request, and a raw HTTP fallback that does not depend on the SDK at all. You will also know which OpenAI endpoints DigitalOcean supports today, and which ones do not carry over cleanly.

This tutorial is based on DigitalOcean’s own migration guide, “How to Switch from the OpenAI API to DigitalOcean’s Serverless Inference” (published July 22, 2026). Every code sample below was independently tested against DigitalOcean’s live endpoint while preparing this guide: with an invalid credential, the request reaches DigitalOcean’s real server and comes back with a genuine 401 Unauthorized response, which confirms the request shape (URL, headers, JSON body) is correct even without a paid account. That test, and what it tells you, is covered in Step 5.

What “OpenAI-Compatible” Actually Means

Before changing any code, it helps to be precise about a few terms this tutorial uses throughout.

• An OpenAI-compatible API is a service built by a different company that accepts the same request format (URLs, JSON fields, authentication style) as OpenAI’s API, so software written against the OpenAI SDK can talk to it by changing configuration instead of rewriting request logic.
• The base URL is the server address the SDK sends requests to. The official OpenAI SDK defaults to OpenAI’s own servers; pointing it at a different base URL is what redirects your traffic to another provider entirely.
• A Model Access Key is the credential DigitalOcean issues specifically for Serverless Inference. It is not your OpenAI API key, and it is not automatically the same as a general DigitalOcean personal access token, though DigitalOcean’s documentation notes that a personal access token with the right scope can also authenticate these requests.
• A model ID is the string you pass in the model field of a request. DigitalOcean’s model IDs (for example, llama3.3-70b-instruct) are catalog identifiers specific to DigitalOcean. They are not OpenAI model names that happen to also work here, and the catalog changes over time, which is why Step 6 shows how to look one up instead of guessing.

Here is the complete before-and-after picture, drawn directly from DigitalOcean’s migration guide:

Three things change. Everything else, including the SDK you already know, stays the same.

Prerequisites

• A DigitalOcean account with access to Serverless Inference under the Gradient AI Platform. This is a paid, usage-based product, so make sure billing is set up on your account before you start; an account alone is not enough to make real requests against it.
• Python 3.9 or newer. This is the openai Python package’s own published minimum, listed as Requires-Python: >=3.9 in its package metadata. This tutorial was written and tested with Python 3.11 and openai SDK version 2.48.0.
• Comfort running commands in a terminal, editing a plain text file, and setting an environment variable. No prior DigitalOcean experience is assumed.
• An existing Python script that calls the OpenAI API, or willingness to type a short one from scratch. This tutorial provides a complete one.
• Optional: curl, if you want to follow Step 8, which tests the API without any SDK at all.

Step 1: Create a DigitalOcean Model Access Key

Log in to the DigitalOcean control panel. In the sidebar, open Inference, then Serverless Inference, and create a new Model Access Key. DigitalOcean’s documentation is explicit that this is a distinct credential from your OpenAI dashboard key: the HTTP request still authenticates with a Bearer token, exactly like OpenAI, but the token value itself is a different secret issued by DigitalOcean.

Copy the key somewhere safe as soon as it is generated. Like most API credentials, it is normal for control panels to show a secret only once at creation time.

Why you store it as an environment variable instead of pasting it into code

Every example in this tutorial reads the key from an environment variable rather than hardcoding it into a Python file. This matters for a practical reason, not just a stylistic one: source files get committed to git, pasted into chat messages, and shared as gists far more often than anyone intends. An environment variable keeps the secret out of any file that might end up somewhere you did not plan for.

On Linux or macOS, set it for your current terminal session:

# Context: bash or zsh terminal, after generating a Model Access Key in the control panel.
# Purpose: make the key available to Python as an environment variable, without writing it into a file.
export MODEL_ACCESS_KEY="paste-your-real-key-here"

On Windows PowerShell, the equivalent is $env:MODEL_ACCESS_KEY = "paste-your-real-key-here". Either way, this setting only lasts for the current terminal session; closing the terminal clears it, which is expected and fine for following along with this tutorial.

Step 2: Set Up a Local Python Environment

Isolate this project’s dependencies in a virtual environment so they do not conflict with anything else on your machine:

# Context: a new, empty project directory.
# Purpose: create and activate an isolated Python environment.
python3 -m venv venv
source venv/bin/activate

On Windows PowerShell, activate it with venv\Scripts\Activate.ps1 instead of the source line above.

Install the OpenAI Python SDK. DigitalOcean Serverless Inference does not require its own SDK; the same official openai package you would use against OpenAI directly is what you point at DigitalOcean’s servers:

# Context: virtual environment activated (your prompt now starts with "(venv)").
# Purpose: install the SDK used for both OpenAI and DigitalOcean Serverless Inference requests.
pip install openai

Expected output: pip resolves and installs openai along with its dependencies (httpx, pydantic, anyio, and a few others), ending with a line like Successfully installed openai-2.48.0 .... Confirm it worked:

python3 -c "import openai; print(openai.__version__)"

Expected output: a version string, for example 2.48.0. Any recent 1.x or 2.x release works the same way for everything in this tutorial; the client construction pattern used below has been stable across that range.

Step 3: Write the Before Code (Plain OpenAI)

To make the migration concrete, start with an ordinary OpenAI SDK call. Save this as before.py:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("OPENAI_API_KEY"),
)

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a fun fact about octopuses."},
    ],
)
print(resp.choices[0].message.content)

Notice what this code does not specify: there is no base_url argument. When you omit it, the SDK defaults to OpenAI’s own servers. That default is exactly what Step 4 overrides.

Step 4: Write the After Code (DigitalOcean Serverless Inference)

Save this as after.py. Compare it line by line with before.py:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.do-ai.run/v1",
    api_key=os.getenv("MODEL_ACCESS_KEY"),
)

resp = client.chat.completions.create(
    model="llama3.3-70b-instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a fun fact about octopuses."},
    ],
)
print(resp.choices[0].message.content)

Three lines changed: a base_url was added, the environment variable read by api_key changed from OPENAI_API_KEY to MODEL_ACCESS_KEY, and the model string changed from an OpenAI name to a DigitalOcean catalog ID. The method call, the message list structure, and the way you read the response (resp.choices[0].message.content) are all identical to the OpenAI version, because DigitalOcean’s endpoint returns the same response shape the OpenAI SDK already knows how to parse.

With MODEL_ACCESS_KEY set to a real, valid key from Step 1, run it:

python3 after.py

Expected output: a few sentences of generated text answering the prompt. The exact wording will differ each time you run it (language models are not deterministic by default), but you should get back readable prose, not an error.

Step 5: Confirm You Are Actually Talking to DigitalOcean

Before trusting any output, it is worth knowing what failure looks like, so you can tell a credential problem apart from a code problem. While preparing this tutorial, the exact code from Step 4 was run against DigitalOcean’s live endpoint with a deliberately invalid key. The SDK raised a real exception from a real server response:

openai.AuthenticationError: Error code: 401 - {'id': 'Unauthorized', 'message': 'Unable to authenticate you'}

This is a useful signal, not just an error to dismiss. Getting back a structured 401 with that exact JSON body means your request successfully reached https://inference.do-ai.run/v1, DigitalOcean’s server parsed it correctly, and the only problem is the credential. If you see this exact error, do not start debugging your message format or your model string; go back and check that MODEL_ACCESS_KEY is set, spelled correctly, and holds a real key copied from the control panel. A typo in the environment variable name is the single most common cause: if you set MODEL_ACESS_KEY by mistake, os.getenv() silently returns None instead of raising an error, and you will see this exact 401 with no other clue.

By contrast, a TypeError or AttributeError at this point would mean something in the request construction itself is wrong, which is a different kind of bug to chase.

Step 6: List Available Models Instead of Guessing a Model ID

DigitalOcean’s model catalog changes over time, and llama3.3-70b-instruct from the examples above will not necessarily be the right (or only) choice for your use case. Rather than guessing a name, ask the API what is actually available:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.do-ai.run/v1",
    api_key=os.getenv("MODEL_ACCESS_KEY"),
)

for model in client.models.list():
    print(model.id)

Expected output: one model ID per line, matching the DigitalOcean catalog available to your account at that moment. This same request was tested against the live endpoint with an invalid key while preparing this tutorial and returned the identical structured 401 shown in Step 5, which confirms client.models.list() is a real, reachable endpoint and not something invented for this guide. You can also browse the same catalog visually under Model Catalog in the DigitalOcean control panel. Changing the model ID keeps your code’s structure the same, but it does not guarantee the new model behaves like the old one: test output quality, context length, and tool-calling support for any model before relying on it in production.

Step 7: Stream Responses Token by Token

For anything user-facing, printing the whole answer only after the model finishes generating it feels slow. Streaming prints each piece of text as it arrives instead:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.do-ai.run/v1",
    api_key=os.getenv("MODEL_ACCESS_KEY"),
)

stream = client.chat.completions.create(
    model="llama3.3-70b-instruct",
    messages=[{"role": "user", "content": "Write a haiku about Kubernetes."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Expected output: the haiku text appears incrementally in your terminal instead of all at once. The only change from Step 4’s request is stream=True and iterating over the response instead of reading .choices[0].message.content directly; everything else about the client is identical. This exact call was also verified against the live endpoint with an invalid key, and failed at the same authentication step with the same structured error as Steps 5 and 6, confirming the streaming request reaches the server correctly before any token is generated.

Step 8: Test With Raw HTTP (No SDK Required)

Sometimes you want to confirm the API itself is working independent of any SDK or programming language, for debugging or for use from a non-Python stack. Since the underlying protocol is plain HTTPS with a Bearer token, curl works directly:

curl -X POST https://inference.do-ai.run/v1/chat/completions \
  -H "Authorization: Bearer $MODEL_ACCESS_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.3-70b-instruct",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "temperature": 0.7,
    "max_completion_tokens": 256
  }'

Expected output: a JSON object with a choices array containing the generated answer, in the same shape OpenAI’s Chat Completions endpoint returns. This exact command was run against the live endpoint while preparing this tutorial with an invalid key in place of $MODEL_ACCESS_KEY, and returned HTTP 401 with the body {"id": "Unauthorized", "message": "Unable to authenticate you"}, the same structured error the SDK examples raised as an exception. That consistency is itself useful: it confirms the SDK is not doing anything hidden, it is a thin wrapper over exactly this HTTP request.

Know What Does Not Migrate One to One

Chat Completions is the closest thing to a drop-in swap, but DigitalOcean’s own endpoint support table shows the picture is more nuanced across the rest of the OpenAI-shaped API surface:

A few of these are worth expanding on:

• Chat Completions: tool-calling support depends on which specific model you choose from the catalog, not just on the endpoint. Check the Model Catalog entry for a model before assuming it can call tools.
• Responses API: text responses, multimodal responses, prompt caching, and reasoning are supported in some configurations, but not every OpenAI Responses feature has a matching DigitalOcean equivalent. Verify the specific feature you need before migrating a Responses-based integration wholesale.
• Embeddings: useful as-is for semantic search and retrieval-augmented generation pipelines. If you want to build one next, sxz.io’s local RAG agent tutorial covers the retrieval side of that pattern, though it runs entirely on a local Ollama model rather than a hosted API.
• Batch inference: this is not a real-time endpoint at all. You submit a job and poll for results, and it runs on separate rate limits from real-time Serverless Inference. It accepts input files formatted for either the native OpenAI Batch API or the Anthropic Message Batches API, using each provider’s own file format rather than translating one into the other. If you already run OpenAI or Anthropic batch jobs, moving them to DigitalOcean is described as an endpoint and authentication change, not a file-format rewrite, though current limits restrict it to text prompts on commercial OpenAI and Anthropic models, with no open-weight models, multimodal input, or image generation.

The practical takeaway: request parameters can vary by model and by endpoint. Do not assume every OpenAI parameter behaves identically across every DigitalOcean-hosted model; check current documentation before relying on a less common one in production.

Common Mistakes and Gotchas

• Forgetting to set base_url. If you only swap the API key but leave base_url unset, your DigitalOcean Model Access Key gets sent to OpenAI’s own servers instead, where it is not a valid credential. You will get an authentication error from the wrong company, which is a confusing thing to debug if you are not looking for it specifically.
• Reusing an OpenAI model name. Passing "gpt-4o" as the model value against DigitalOcean’s endpoint will not work; DigitalOcean’s catalog uses its own IDs like llama3.3-70b-instruct. Use Step 6’s client.models.list() to see what is actually available rather than guessing.
• Typing the environment variable name wrong. As covered in Step 5, os.getenv() returns None for a misspelled variable name instead of raising an error, which produces the same generic 401 as an outright missing key. Print the variable (or at least confirm it is non-empty) if authentication fails unexpectedly.
• Treating Batch inference as a real-time call. It uses the same base URL and Model Access Key as everything else in this tutorial, but it is a submit-and-poll workflow on its own rate limits, not a synchronous request you can drop into Step 4’s pattern.
• Assuming feature parity everywhere. Chat Completions is close to a drop-in replacement; the Responses API and some model-specific features are not guaranteed to behave identically. Test the specific feature your application depends on, not just the general endpoint.

How to Verify the Whole Migration Works End to End

Work through this checklist in order. Each item builds on the one before it:

1. Confirm echo $MODEL_ACCESS_KEY (or the PowerShell equivalent) prints a real key, not an empty line.
2. Run Step 6’s client.models.list() and confirm it returns a non-empty list of model IDs rather than raising an exception.
3. Run Step 4’s after.py and confirm it prints readable generated text, not an error.
4. Run Step 7’s streaming example and confirm the text appears incrementally rather than as one final block.
5. Run Step 8’s curl command and confirm it returns a JSON body with a choices array, matching what the SDK examples produced.
6. As a deliberate negative test, temporarily set MODEL_ACCESS_KEY to an obviously wrong string and rerun after.py. You should see the same structured 401 shown in Step 5. If you instead see a TypeError or the request silently hangs, something in the request construction itself needs another look, separate from the credential.

If all six checks pass, your application is genuinely talking to DigitalOcean Serverless Inference, not silently falling back to OpenAI or failing in a way that happens to look like success.

Next Steps

From here, a few natural directions to keep exploring:

• Swap the Embeddings endpoint into a semantic search or retrieval-augmented generation project, using the same base_url and credential pattern from this tutorial.
• Try the image generation endpoint at /v1/images/generations and inspect the base64-encoded response it returns.
• Browse the Model Catalog in the control panel for models beyond llama3.3-70b-instruct, and compare context length and tool-calling support before standardizing on one.
• If you have an existing batch of OpenAI or Anthropic offline jobs, look at DigitalOcean’s Batch inference documentation before migrating them, since it is a distinct workflow from everything covered in this tutorial.