EN ▾
Get API key

API Migration Guide: Switching LLM Providers Safely

This API migration guide walks through moving an app that already talks to an OpenAI-style chat endpoint onto a different provider. Most of the work is not the two lines you change, it is finding what depended on the old provider by accident.

Last updated

Valve diverting part of a stream of light cubes from one pipeline to another, showing a gradual migration

Summary

  • For an OpenAI-compatible target you change base URL, key and model ID, then verify behaviour.
  • Inventory features first: embeddings, images and fine-tunes will not move to a text-only API.
  • Roll out by percentage with a flag so rollback takes one config change.
  • Estimate the new bill from your own token counts before the cutover.

The plan in six steps

  1. Inventory every API feature your app calls.
  2. Move provider settings into configuration.
  3. Map parameters and model names.
  4. Add error and rate-limit handling that matches the new limits.
  5. Route a small share of traffic to the new provider and compare.
  6. Check the cost, cut over, and keep the old route for a while.

The target in the examples is https://api.apimigration.com/v1 with model ID uncensored. The same steps apply to any OpenAI-compatible provider, so read them as a checklist rather than a vendor recipe. If something breaks along the way, the troubleshooting article on common LLM integration errors covers the usual causes.

Step 1: Inventory what your app uses

Search your code for every call to the old provider and list the features as well as the endpoints. A single chat call often hides several dependencies: a system prompt that relies on a particular style, a tool schema, a JSON response format, a large context.

Feature in your appOn this API
Chat completions, /v1/responsesSupported
Streaming over SSESupported, usage sent at the end
Tool callingSupported, including streamed tool_calls
JSON modeSupported via response_format
Context100,000 tokens, prompt plus completion
Embeddings, images, audio, fine-tuningNot offered, keep these on another provider

If the old provider also handled embeddings or image generation, split the migration: move only the text calls and leave the rest where they are.

Step 2: Put provider settings in configuration

Hard-coded URLs are what turn a migration into a code change. Read the base URL, key and model from the environment so you can switch without a deploy.

export LLM_BASE_URL="https://api.apimigration.com/v1"
export LLM_API_KEY="YOUR_KEY"
export LLM_MODEL="uncensored"
import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ["LLM_API_KEY"],
)
MODEL = os.environ.get("LLM_MODEL", "uncensored")

Auth is the standard Authorization: Bearer header, which the SDK sets for you. To confirm the key and see what the server offers, call GET /v1/models.

curl https://api.apimigration.com/v1/models \
  -H "Authorization: Bearer $LLM_API_KEY"

Step 3: Map parameters and prompts

Parameters that pass through on this API: temperature, top_p, stop, seed and the frequency and presence penalties. max_tokens is optional. If your old code sets it to protect against runaway cost, keep it.

Three things deserve a deliberate check.

  • Vendor-specific fields. Anything outside the OpenAI request format may be ignored or rejected. Send one request with your exact production payload and read the response.
  • Model behaviour. A different model reads the same prompt differently. System prompts tuned for the old model often need small edits. Test them, do not assume.
  • Refusals. If your old prompts carried workarounds to avoid refusals, you can usually delete them here, which shortens the prompt.

The model also differs in what it will answer. This one is tuned for lawful adult use without content refusals, and sexual content involving minors is blocked.

Step 4: Match error handling to the new limits

A full key allows 8 concurrent requests and 300 per minute. A trial key allows 2 and 60. Over the limit you get HTTP 429 with error code rate_limited or concurrency. On temporary overload you also get a 429, with Retry-After: 5.

import time
from openai import RateLimitError

def chat(messages, retries=4):
    for attempt in range(retries):
        try:
            return client.chat.completions.create(model=MODEL, messages=messages)
        except RateLimitError:
            time.sleep(5 * (attempt + 1))
    raise RuntimeError("still rate limited after retries")

Streamed requests keep their concurrency slot until the stream ends, so a stuck client can starve the rest. Close streams in a finally block. If your old provider allowed more parallel calls, add a semaphore sized to the new limit. Failed requests are not charged, which makes retries safe for your budget.

Step 5: Roll out by percentage

Do not flip everything at once. Send a small, stable share of users to the new provider and keep the old one as a fallback. Hashing the user ID keeps each user on one side, so conversations do not change voice halfway through.

import hashlib

def use_new(user_id: str, percent: int) -> bool:
    bucket = int(hashlib.sha256(user_id.encode()).hexdigest(), 16) % 100
    return bucket < percent

# 5 percent, then 25, 50, 100 as metrics hold
new_client = OpenAI(base_url="https://api.apimigration.com/v1", api_key=os.environ["NEW_KEY"])
old_client = OpenAI(base_url=os.environ["OLD_BASE_URL"], api_key=os.environ["OLD_KEY"])

def pick(user_id):
    return new_client if use_new(user_id, int(os.environ.get("ROLLOUT_PERCENT", "0"))) else old_client

Raise the percentage only after the stage before it has run clean for a day or so. Rollback is then one environment variable set back to 0.

What usually surprises teams after the switch

The base URL change works on the first try. The surprises come later.

  • Tone and format drift. The same prompt gives a differently worded answer. Features that parse free text, such as a regex on "Answer:" prefixes, are the first to break. Prefer JSON mode or tool calls for anything a program reads.
  • Different token counts. Each model tokenizes text its own way. Your cost estimate from the old provider's token counts will be off by some amount, so re-measure with the usage field.
  • Output length. Without max_tokens the completion may use the rest of the 100,000-token window. If your old setup had a default cap, set one on purpose.
  • Timeouts. Your client timeout was tuned for the old provider. Long answers on a new one can run past it. Stream where you can and raise the timeout for non-streamed calls.
  • Silent refusals changing. If your product used to catch refusal messages and show a fallback, that code may stop firing. Decide whether you still want it.

A short checklist before cutover

  1. Every feature in your inventory has a tested equivalent or a documented gap.
  2. Base URL, key and model come from configuration in every service that calls the API.
  3. Rate limit and concurrency caps in your code match the new limits.
  4. Streams are closed on client disconnect.
  5. Logs include the X-Request-Id header, so you can quote a request when asking for help.
  6. The balance is monitored through X-Balance-USD or GET /v1/balance, with an alert before it runs low.
  7. The old provider is still funded and configured for rollback.
  8. A named person owns the decision to roll back.

That last item sounds silly until an incident at 2 a.m. Someone has to be allowed to flip the flag.

Step 6: Check the cost, then cut over

Work out the new bill from your own traffic. Prices are $0.15 per 1M input tokens and $0.50 per 1M output tokens.

Example: 10,000 requests a day, each with 1,500 input tokens and 400 output tokens.

  • Input: 15,000,000 tokens x $0.15 / 1,000,000 = $2.25.
  • Output: 4,000,000 tokens x $0.50 / 1,000,000 = $2.00.
  • Total: $4.25 per day.

Compare that with the old invoice, using the old provider's current pricing page. Billing here is prepaid credit with no subscription, and credit does not expire. Top-ups run from $20 to $500 in whole amounts, with a bonus of 5% from $50, 10% from $100 and 15% from $200, paid in crypto (USDT on TRC20 or USDC on Base). The balance shows in the X-Balance-USD response header, so you can alert on it.

New accounts can claim $0.50 of trial credit for the first tests. See pricing, then get an API key, and keep the docs open while you map parameters.

Common questions

Can I migrate by changing only the base URL?

For an OpenAI-compatible API, the base URL, key and model ID are the required changes. You should still test prompts, since a different model answers differently.

Will my tool calls and JSON mode still work?

Tool calling and JSON mode (response_format json_object) are supported. Validate your schemas with real prompts before cutover.

What if my app uses embeddings?

This API is text only and offers no embeddings, image generation or fine-tuning. Keep those calls on your current provider.

How do I roll back quickly?

Route by an environment flag such as a rollout percentage, and keep the old client configured until the new route has run cleanly for a few days.

Get your API key

Register, copy the key, update the base URL. Setup is complete.

Get API key