Short answer: create one Roo Code configuration profile per model, all using the OpenAI Compatible provider with the same Base URL (https://cloudgpu.app/v1) and API key, and change only the Model field: deepseek-v4.1-flash, kimi-k2.6, kimi-k3 or minimax-m3. Then link a profile to each Roo Code mode, and run one tool-call request per model ID before you trust it with a task.

Disclosure: I work on CloudGPU, and the endpoint below is ours. The Roo Code steps follow the official Roo Code docs (API Configuration Profiles and Using OpenAI Compatible Providers). We haven't run Roo Code end to end against this endpoint, so the checks below are for you to run, not results we measured.

Why profiles

According to the Roo Code docs, a configuration profile holds its own provider, API key, model selection, temperature, thinking budget and rate limit setting, and you can switch profiles from Settings or from the API Configuration dropdown in the chat. The docs also say you can link a profile to each mode, and that each task remembers the profile it started with, so a resumed task keeps its original model.

That fits a gateway well: the provider, URL and key never change, so each profile differs only in its model ID.

What you need

• An API key (cgw-sk-...) from the API console. It is shown once, so copy it right away. The quick start is on the API docs page.
• Roo Code installed in VS Code.
• The key in an environment variable for the checks below:

export CLOUDGPU_API_KEY="cgw-sk-your-key"

Step 1: Pick the model IDs

These chat model IDs are in the model table on the CloudGPU API page (checked 2026-10-28). IDs are case-sensitive.

GET /v1/models is public: it lists model IDs without a key. Other endpoints still need your key.

curl -s https://cloudgpu.app/v1/models

Step 2: Create one profile per model

Following the Roo Code docs: open Settings (gear icon), go to Providers, click + next to the profile selector and name the profile, for example cgw-deepseek. Then fill in:

Repeat for cgw-kimi-k2.6, cgw-kimi-k3 and cgw-minimax, changing only the Model field.

The docs list a per-profile Rate Limit (minimum seconds between requests, default 0, meaning off). Leave it at 0 unless you hit 429 rate_limit_exceeded; the API docs list 100 requests per minute per key and 10,000 requests per day per account. Every profile here uses the same key, so they share the per-key limit.

Model Configuration (Max Output Tokens, Context Window and so on) is optional. We don't publish values for these models, so leave the defaults unless you know the limits you need.

Step 3: Link profiles to modes

The Roo Code docs say you can associate a profile with each mode in the Prompts tab, and Roo Code also remembers the last profile you used in each mode. Which model goes with which mode is your call; one way to try it is a Kimi profile for Architect and a DeepSeek profile for Code, then swap and compare on the same small task.

Orchestrator subtasks inherit the parent task's profile, according to the docs, so a subtask won't switch models on you mid-task.

Step 4: Tool-call check for every profile

The Roo Code docs say it "uses native tool calling exclusively" and that a model without native tool calling "cannot be used with Roo Code". We haven't confirmed tool calls for any of these four IDs inside Roo Code, so the way to verify is to run one request with a tools array for each model ID. This uses the openai Python package (pip install openai):

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://cloudgpu.app/v1",
    api_key=os.environ["CLOUDGPU_API_KEY"],
)

tools = [{
    "type": "function",
    "function": {
        "name": "read_file",
        "description": "Read a file from the workspace",
        "parameters": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
        },
    },
}]

for model in ["deepseek-v4.1-flash", "kimi-k2.6", "kimi-k3", "minimax-m3"]:
    try:
        resp = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": "Open README.md using the tool."}],
            tools=tools,
            max_tokens=2048,  # leave room: some models think before they answer
        )
    except Exception as e:
        print(f"{model}: error -> {e}")
        continue
    msg = resp.choices[0].message
    if msg.tool_calls:
        call = msg.tool_calls[0].function
        print(f"{model}: tool call -> {call.name}({call.arguments})")
    else:
        print(f"{model}: no tool call, text -> {msg.content!r}")

How to read it:

• tool_calls present (a read_file call with {"path": "README.md"} or similar): try that profile in Roo Code on a small task.
• Plain text or an error: don't use that profile with Roo Code; try another ID.
• Empty content and no tool call: some models spend output tokens on thinking first. Raise max_tokens here (and Max Output Tokens in that profile's Model Configuration) and run it again before you decide.

Troubleshooting

• HTTP 401 invalid_api_key: the key is wrong or revoked. Create a new one in the console and update it in every profile.
• HTTP 401 missing_authorization: no Authorization: Bearer header was sent (/v1/models is public, but chat requests need the key). In a shell, check that echo $CLOUDGPU_API_KEY isn't empty.
• HTTP 400 model_not_found: check the Model field against /v1/models; IDs are case-sensitive.
• Connection errors: the Base URL must be exactly https://cloudgpu.app/v1.
• HTTP 429 rate_limit_exceeded: all profiles share the same key's limit; set a small per-profile Rate Limit or slow down parallel tasks.
• A resumed task uses the "wrong" model: per the docs, tasks keep the profile they started with. Start a new task to use another profile.

FAQ

Can Roo Code use several models through one OpenAI-compatible endpoint? Yes. Create one configuration profile per model ID with the same Base URL and API key, and switch profiles from Settings or the chat dropdown.

Can each Roo Code mode use a different model? Yes. The Roo Code docs say you can link a configuration profile to each mode in the Prompts tab.

Do deepseek-v4.1-flash, the Kimi IDs and minimax-m3 support the tool calling Roo Code needs? We haven't confirmed tool calls on any of them inside Roo Code. Run the Step 4 Python check for each ID before you rely on it.

Do all profiles share one rate limit? They share the limits of the key they use: 100 requests per minute per key and 10,000 requests per day per account, per the API docs.

Where are the rates? Per-model rates are in the price table on the CloudGPU API page.

Facts checked 2026-10-28.

Everything below is quoted from the issue or from official documentation. We have not reproduced the runs ourselves.

1. The symptom and the conditions it was reported under

From InvokeAI issue #9831 (opened 2026-10-12, open, labelled bug, no comments when read on 2026-10-28):

• Machine: Windows 10, RTX 4090 with 24 GB VRAM, manual development checkout of InvokeAI main at 8111b5067e (7.0.0-alpha.1), Python 3.12, torch 2.13.0+cu130.
• Model: Wan 2.2 T2V A14B, GGUF Q4_K_M, both experts (High Noise and Low Noise), plus the Wan2.2-T2V-A14B-Diffusers components and the Lightning 4-step LoRAs (V1.1, high and low).
• Job: 1280×720, 81 frames, 4 steps, guidance 1.0, default invokeai.yaml.
• Error: torch.OutOfMemoryError in diffusers/models/transformers/transformer_wan.py:502.

The issue's results table (settings and outcome as reported; we list the memory outcome only):

Two details matter when you compare this with your own machine:

• The run with partial loading off did not really "fit". The issue says the card still overflowed and Windows moved memory to shared system memory instead of raising an OOM.
• This is a Windows machine on a 7.0.0 alpha development build. Linux, or a release build, may behave differently; the issue does not test them.

What to check

• Your error matches: torch.OutOfMemoryError in the Wan transformer, at the first denoise block.
• Your invokeai.yaml still has the defaults (enable_partial_loading: true, device_working_mem_gb: 3, or neither line set).
• On Windows, watch "Shared GPU memory" in Task Manager. If it climbs during generation, you are in the spill case, not the OOM case.

2. Why 24 GB is not enough here

Two experts, one at a time. The official Wan2.2 README (read 2026-10-28) describes the A14B models as a two-expert design: "a high-noise expert for the early stages, focusing on overall layout; and a low-noise expert for the later stages, refining video details. Each expert model has about 14B parameters, resulting in a total of 27B parameters but only 14B active parameters per step". So you download and load two diffusion models, but each denoise step uses one.

The official script's own guidance. For its single-GPU 1280*720 A14B command (with --offload_model True --convert_model_dtype), the README says: "This command can run on a GPU with at least 80GB VRAM." That statement is about the Wan repo's generate.py, not about InvokeAI or GGUF weights, but it tells you 720p A14B is not a small job.

What the issue found inside InvokeAI. According to #9831:

• Wan denoise has no working-memory estimate. _get_wan_transformer_working_mem_bytes returns None unless wan_memory_optimization is on, so the expert is loaded with the floor, device_working_mem_gb (3 GB), as its working memory.
• At 720p with 81 frames the transformer sees 75,600 tokens, "and the activations need far more than 3 GB."
• With partial loading on, the cache frees only 2,003 MB of the 10.8 GB UMT5 text encoder before loading the 9.2 GB expert, so the first block runs out of memory with 21.3 GiB allocated.

In short: model weights plus text encoder plus the attention activations for 75,600 tokens do not fit in 24 GB, and InvokeAI reserves only 3 GB for the activations by default.

What to check

• The InvokeAI log around model loading: is the text encoder still partly in VRAM when the expert loads?
• Your resolution and frame count. The 75,600-token figure is for 1280×720 with 81 frames only.

3. Local settings to change

3a. Raise the working memory

InvokeAI's Low-VRAM mode docs (read 2026-10-28): "Invoke reserves 3GB VRAM as working memory by default", and you can raise it in invokeai.yaml to prevent OOMs (the docs' example sets 4).

The issue's workaround for this exact case on 24 GB is 10:

# invokeai.yaml — workaround reported in InvokeAI issue #9831 for 24 GB
device_working_mem_gb: 10

With that, partial loading on and SDPA, the issue reports the 720p, 81-frame job completing. Invoke needs a restart after you edit the file.

What to check

• The OOM is gone and Task Manager's shared GPU memory stays flat during denoise.
• If you also set max_cache_vram_gb, the docs say to keep it at or below your VRAM minus device_working_mem_gb minus reserve_vram_gb.

3b. Keep partial loading on, and turn off the Windows sysmem fallback

The docs say Low-VRAM mode is enabled by default via enable_partial_loading: true. In the issue, turning it off did not prevent the overflow: the encoder was unloaded, but the run spilled into shared memory.

The same docs ask Windows users to disable the Nvidia sysmem fallback: NVIDIA Control Panel → 3D Settings → Manage 3D Settings → "CUDA - Sysmem Fallback Policy" → "Prefer No Sysmem Fallback". With it disabled, an overflow shows up as an OOM you can act on instead of silently spilling into shared system memory.

What to check

• enable_partial_loading is true (or absent) in invokeai.yaml.
• On Windows, the NVIDIA Control Panel setting reads "Prefer No Sysmem Fallback".

3c. Try wan_memory_optimization (opt-in)

From the same docs page: setting wan_memory_optimization: true makes partial-load Wan transformer weights "target about 2 GiB resident and stream remaining layers from RAM". It "reduces peak VRAM during both denoise and decode, but generation can be substantially slower and requires enough system RAM for offloaded weights." The docs also note same-seed output may differ slightly from the baseline path.

The issue explains that this flag is also what turns on Wan's working-memory estimate. It does not report a run with the flag on, so we have no figures for it.

wan_memory_optimization: true

What to check

• Free system RAM during generation; the docs say offloaded weights need it.
• Compare a short test clip against your baseline before committing to long runs.

3d. Lower the resolution, and treat frame count with care

The issue's only other resolution is 832×480. With default settings it also ran out of memory. With partial loading on and device_working_mem_gb: 10, it fit with both attention backends.

The issue tests 81 frames only. Fewer frames means a shorter clip, but the issue gives no figures for it, so try it on your own setup rather than relying on a number from us.

What to check

• At 832×480, the default settings still fail in the issue. Raise device_working_mem_gb as well; do not rely on resolution alone.

3e. Quantization, and the official script's flags

The issue already used GGUF Q4_K_M weights for both experts and still ran out of memory at default settings. Lower-bit weights alone did not fix it there; working memory did.

If you use the Wan repo's own generate.py instead of InvokeAI, the README says: "If you encounter OOM (Out-of-Memory) issues, you can use the --offload_model True, --convert_model_dtype and --t5_cpu options to reduce GPU memory usage."

On attention, these are figures reported in InvokeAI issue #9831 (one RTX 4090 24GB on Windows), not measured by us and not a claim about CloudGPU cards: the issue reports SageAttention held about 1.2 GB more than SDPA at 75,600 tokens, which pushed that card over the edge at 720p. At 832×480, the issue reports both backends fit.

What to check

• If you switched to SageAttention and shared GPU memory now climbs at 720p, switch back to SDPA and compare.
• The Wan2.2 model weights are licensed Apache-2.0 (per the README and the model card); check the license of any community GGUF build you download separately.

4. If local settings are still not enough: run the same workflow on a card with more VRAM

If the slower settings are not an option for you, the other route is more VRAM. On CloudGPU you can rent a card with more VRAM, billed per minute, and set up the same InvokeAI workflow on it. Cards listed on /pricing when we read it on 2026-10-28 include:

To be clear about what this does and does not mean:

• We have not tested Wan 2.2 A14B, InvokeAI or this workflow on any of these cards. The only difference we can state is more VRAM than a 24 GB card.
• VRAM need depends on model, precision and workflow. See How much VRAM do I need to run an LLM?
• Billing is per minute, rounded up, with a 1-minute minimum, and stops when you destroy the instance (terms). Rates per card are on /pricing.

What to check

• Before renting, compare the VRAM column above with what your settings needed locally (the InvokeAI log and Task Manager tell you where it ran out).
• Destroy the instance when the job is done, so billing stops.

FAQ

Why does Wan 2.2 A14B run out of memory at 720p on a 24 GB GPU in InvokeAI? According to InvokeAI issue #9831, Wan denoise has no working-memory estimate unless wan_memory_optimization is on, so InvokeAI reserves only the default 3 GB for activations. At 1280×720 with 81 frames the transformer sees 75,600 tokens, and the activations need far more than that.

What setting fixed it in the issue?device_working_mem_gb: 10 in invokeai.yaml, with partial loading left on. The issue reports the 720p, 81-frame job then completed on an RTX 4090 24GB (Windows 10, InvokeAI 7.0.0-alpha.1).

Does turning off partial loading help? Not in the issue. With enable_partial_loading: false the card still overflowed, and Windows moved memory to shared system memory instead of raising an OOM.

Does lowering the resolution to 832×480 avoid the OOM? Not on its own. The issue reports an OOM at 832×480 with default settings too; with device_working_mem_gb: 10 it fit.

What does the official Wan2.2 README say about VRAM for A14B at 720p? For its own single-GPU generate.py command at 1280*720, it says the command "can run on a GPU with at least 80GB VRAM", and suggests --offload_model True, --convert_model_dtype and --t5_cpu for OOM issues.

Sources

• InvokeAI issue #9831, https://github.com/invoke-ai/InvokeAI/issues/9831 (opened 2026-10-12; read 2026-10-28)
• Wan2.2 README (GitHub), https://github.com/Wan-Video/Wan2.2 (read 2026-10-28)
• Wan2.2-T2V-A14B model card, https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B (read 2026-10-28)
• InvokeAI docs, Low-VRAM mode, https://invoke.ai/configuration/low-vram-mode/ (read 2026-10-28)
• CloudGPU /pricing and /terms (read 2026-10-28)

If a tool already speaks the OpenAI-compatible chat format, switching it to Kimi comes down to three values: the base URL, the API key and the model ID. This post shows those values on the CloudGPU gateway, then the same setup in curl, the openai Python package, Cline and Aider.

Disclosure: I work on CloudGPU.

The three values

Both IDs are in the model table on the live API page (checked 2026-10-27), which also lists kimi-k2.5. The original post suggests kimi-k2.6 for everyday coding agents and kimi-k3 for harder reasoning tasks.

A team test on 2026-10-26 with an internal key found both IDs in /v1/models, a working chat request, and a single tool call (the tools parameter, returning tool_calls) for each.

Step 1: Get a key

1. Sign up at cloudgpu.app with an email address or a Google account.
2. Top up the prepaid balance. Payment options are card or USDT.
3. In the API console, create a key. It starts with cgw-sk- and is shown only once, so copy it straight away.
4. Keep it in an environment variable:

export CGW_API_KEY="cgw-sk-..."

The same balance pays for the API per token and for rented GPUs by the minute.

Step 2: Check the key with curl

curl https://cloudgpu.app/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $CGW_API_KEY" \
  -d '{
    "model": "kimi-k2.6",
    "messages": [{"role": "user", "content": "Refactor this function to use early returns."}]
  }'

If something is off, the error codes from the API docs tell you what:

• 401 missing_authorization: no Authorization header was sent.
• 401 invalid_api_key: the key is unknown or revoked.
• 400 model_not_found: the model ID is not in the catalog. Copy it from the model table.
• 429 rate_limit_exceeded: you hit the per-minute or per-day limit. The public beta defaults are 100 requests per minute per key and 10,000 requests per day per account.

Step 3: The OpenAI Python SDK

Only base_url and api_key change:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://cloudgpu.app/v1",
    api_key=os.environ["CGW_API_KEY"],
)

resp = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Explain this stack trace in two sentences."}],
    max_tokens=2048,
)
print(resp.choices[0].message.content)

For streaming, pass stream=True. The API docs say the response is Server-Sent Events in OpenAI format and the SDK handles it for you.

If a reply comes back empty with finish_reason set to "length", the model ran out of output budget. Raise max_tokens and try again.

Step 4: Cline

According to the official Cline docs ("OpenAI Compatible" provider page, read 2026-10-27), the settings are in the Cline settings panel (the gear icon):

1. API Provider: select "OpenAI Compatible".
2. Base URL: https://cloudgpu.app/v1
3. API Key: your cgw-sk-... key.
4. Model: enter kimi-k2.6 (or kimi-k3).

The same docs list the usual fixes: "Invalid API Key" means the key or provider is wrong, and "Model Not Found" means the model ID isn't available at that base URL.

Step 5: Aider

According to the official Aider docs ("OpenAI compatible APIs", read 2026-10-27), set two environment variables and prefix the model name with openai/:

export OPENAI_API_BASE=https://cloudgpu.app/v1
export OPENAI_API_KEY=$CGW_API_KEY

aider --model openai/kimi-k2.6

On Windows the docs use setx OPENAI_API_BASE ... and setx OPENAI_API_KEY ..., then restart the shell.

What the gateway does and doesn't do

It speaks the OpenAI chat-completions format, streams tokens, and bills per token from a prepaid balance with no subscription. It doesn't store your prompts or train on them, and there is no automatic failover. Per-token rates for every model are listed on the API page and can change, so this post doesn't repeat them.

Model IDs, rates and the full docs: cloudgpu.app/api

FAQ

Which base URL do I use?https://cloudgpu.app/v1 for any OpenAI-compatible client.

Which model ID should I start with? The original post suggests kimi-k2.6 for everyday coding agents and kimi-k3 for harder reasoning. Both are in the live model table.

Does it work with the OpenAI SDK? Yes. Set base_url to https://cloudgpu.app/v1 and pass your cgw-sk- key as api_key.

How do I pay? From a prepaid balance, topped up by card or USDT.

How do you use DeepSeek V4.1 Flash and Kimi K3 in opencode through a custom OpenAI-compatible provider?

You add a provider block to opencode.json that uses the @ai-sdk/openai-compatible package, set its baseURL to https://cloudgpu.app/v1, read the key from {env:CGW_API_KEY}, and list deepseek-v4.1-flash and kimi-k3 under models; opencode then offers them as cloudgpu/deepseek-v4.1-flash and cloudgpu/kimi-k3.

The rest of this post is the copy-paste version of that sentence, with each opencode behaviour traced to the opencode docs.

Disclosure: I work on CloudGPU.

What this is based on

We ran this provider block in opencode on 2026-10-26 and both headline models replied. Everything else below about opencode itself (the /models screen, /connect, the limit block, the default model) follows the opencode docs, so read "according to the opencode docs" literally.

The four values at a glance

The model IDs were checked against the model table on the live API page on 2026-10-27.

Step 1: Export the key

According to the opencode docs, config files support {env:VARIABLE_NAME} substitution, so the key can stay out of the JSON file:

export CGW_API_KEY="cgw-sk-..."   # put this in ~/.bashrc or ~/.zshrc to keep it

Step 2: Add the custom provider to opencode.json

Put this in opencode.json in your project root. According to the opencode docs, the global file lives at ~/.config/opencode/opencode.json, the project file overrides it, and "configuration files are merged together, not replaced", so the same block also works globally.

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "cloudgpu": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "CloudGPU",
      "options": {
        "baseURL": "https://cloudgpu.app/v1",
        "apiKey": "{env:CGW_API_KEY}"
      },
      "models": {
        "deepseek-v4.1-flash": { "name": "DeepSeek V4.1 Flash" },
        "kimi-k3": { "name": "Kimi K3" },
        "kimi-k2.6": { "name": "Kimi K2.6" },
        "glm-5.3-flash": { "name": "GLM-5.3 Flash" }
      }
    }
  }
}

Field guide, according to the opencode docs (Providers → Custom provider):

• cloudgpu is the provider ID. You choose it, and it becomes the prefix in cloudgpu/<model>.
• npm: "@ai-sdk/openai-compatible for OpenAI-compatible providers (for /v1/chat/completions)".
• options.baseURL is the API endpoint; options.apiKey is optional "if not using auth".
• models: each key must be the exact ID the endpoint serves. name is only the label in the UI.

Only want the two headline models? Delete the kimi-k2.6 and glm-5.3-flash lines.

Alternative: /connect instead of apiKey

According to the opencode docs, you can run /connect, scroll to Other, enter the provider ID (cloudgpu), and paste the key. The docs warn that this "only stores a credential", so you still need the provider block, just without the apiKey line.

Step 3: Choose a model

Check from the command line that opencode sees the provider. According to the opencode CLI docs, opencode models [provider] lists models in provider/model form:

opencode models cloudgpu

The output should look like this (illustrative, not a captured run):

cloudgpu/deepseek-v4.1-flash
cloudgpu/glm-5.3-flash
cloudgpu/kimi-k2.6
cloudgpu/kimi-k3

Then pick one with -m in provider/model form. The dot in v4.1 is part of the ID:

opencode -m cloudgpu/deepseek-v4.1-flash
opencode run -m cloudgpu/kimi-k3 "Explain what this repo's main entry point does"

Inside the TUI, according to the opencode docs, you "run the /models command and your custom provider and models will appear in the selection list". To make one the default, the opencode models docs use a top-level "model" key in provider_id/model_id form, here "model": "cloudgpu/deepseek-v4.1-flash".

Which one for what? There are no benchmarks here, so try them on your own repo. All four are in the config, so comparing them takes one switch.

Step 4: Sanity-check the endpoint outside opencode

If opencode misbehaves, take it out of the loop. First, list the IDs your key can use:

curl https://cloudgpu.app/v1/models \
  -H "Authorization: Bearer $CGW_API_KEY"

Without the header this returns 401 with missing_authorization, which is expected. The API docs list it as "No Authorization header." (As of 2026-10-27, see /api/docs.)

Then send one chat request with the openai Python package:

# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://cloudgpu.app/v1",
    api_key=os.environ["CGW_API_KEY"],
)

for model in ["deepseek-v4.1-flash", "kimi-k3"]:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Reply with: ok"}],
    )
    print(model, resp.choices[0].finish_reason, repr(resp.choices[0].message.content))

If a model comes back with empty or None content and finish_reason is "length", the reply ran out of output tokens: leave max_tokens unset or set it higher.

Troubleshooting

1. The model doesn't show up, or you get 400 model_not_found. The keys under models must match the IDs from GET /v1/models character for character. Watch for short aliases (deepseek-flash), a missing version segment (deepseek-v4-flash vs deepseek-v4.1-flash) and capital letters. Also check that the provider ID is the same in all three places: the provider key, the ID typed into /connect (if you used it), and the cloudgpu/ prefix. According to the opencode docs, opencode auth list shows which credentials are stored.

2. The base URL is wrong. The AI SDK docs describe baseURL as the URL prefix for API calls, and their example sends chat requests to <baseURL>/chat/completions:

https://cloudgpu.app/v1                     correct
https://cloudgpu.app                        missing /v1
https://cloudgpu.app/v1/chat/completions    too specific

3. An error asking for an Authorization: Bearer header. According to the opencode docs, an unset {env:...} variable "will be replaced with an empty string", so the request goes out without a key. Check the variable in the shell you launch opencode from:

echo "${CGW_API_KEY:+set}"   # prints "set" if the variable is present

After editing ~/.bashrc or ~/.zshrc, open a new terminal or source the file. IDE terminals and desktop launchers may not read your shell profile.

4. Empty replies. Check finish_reason as in Step 4. In opencode, the per-model knob the docs describe is the optional limit block (limit.context = "maximum input tokens the model accepts", limit.output = "maximum tokens the model can generate"). If you add it, use real values from the model provider's docs, not guesses.

5. 401 invalid_api_key, 402 insufficient_balance or 429 rate_limit_exceeded. The CloudGPU API docs describe them like this (As of 2026-10-27, see /api/docs):

• 401 invalid_api_key: "Key unknown or revoked."
• 402 insufficient_balance: "Top up."
• 429 rate_limit_exceeded: "Per-minute or per-day limit hit."

The documented rate limits are "Per API key, per minute: 100 requests" and "Per account, per day: 10,000 requests", and the docs add: "These defaults exist to stop abuse during the public beta. Contact us if you need them raised — happy to do it for any genuine workload, just need to know the use case."

Paying for it

Usage is billed per token from a prepaid balance (card, Alipay, Google Pay, Apple Pay or USDT). Keys, the model table and per-model rates are on the CloudGPU API page.

FAQ

Which npm package does an OpenAI-compatible custom provider use in opencode? @ai-sdk/openai-compatible. According to the opencode docs, it's the package for providers that use /v1/chat/completions.

What base URL goes in options.baseURL? https://cloudgpu.app/v1, with /v1 and nothing after it.

Can the model ID contain a dot, like deepseek-v4.1-flash? Yes. Use it exactly as listed, in provider/model form: -m cloudgpu/deepseek-v4.1-flash.

Why does a model sometimes return an empty answer? Check finish_reason. If it is "length", the reply ran out of output tokens; raise max_tokens or leave it unset.

Do I have to put the API key in opencode.json? No. Use "apiKey": "{env:CGW_API_KEY}" so the file only holds a reference. According to the opencode docs, /connect can store the credential instead.

Are there rate limits? Yes. The API docs list "Per API key, per minute: 100 requests" and "Per account, per day: 10,000 requests", and say: "These defaults exist to stop abuse during the public beta. Contact us if you need them raised — happy to do it for any genuine workload, just need to know the use case." As of 2026-10-27, see /api/docs.

Most coding agents let you swap the model behind them by changing one setting: the OpenAI-compatible base URL. This post shows how to put DeepSeek V4 behind Cursor, Continue, Cline and aider using the CloudGPU gateway, paid from one prepaid balance. Switching back later is the same one-field change.

Disclosure: I work on CloudGPU.

The three values every tool needs

The model IDs are copied from the model table on the live API page (checked 2026-10-27). Type them exactly: a wrong case or a missing dot returns 400 model_not_found.

Step 1: Get a key

1. Sign up at cloudgpu.app with an email address or a Google account.
2. Open the API console, create a key (name it after the tool, e.g. cursor) and copy it. It starts with cgw-sk- and is shown only once, so keep it in a password manager.
3. Put it in an environment variable so it stays out of your code and dotfiles:

export CGW_API_KEY="cgw-sk-..."

Step 2: Check the key with curl

curl https://cloudgpu.app/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $CGW_API_KEY" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"hello"}]}'

A JSON reply with choices[0].message.content means the key and balance work. To list the model IDs your key can use, call GET https://cloudgpu.app/v1/models with the same Authorization header. Without a key it returns 401 with missing_authorization, which is expected.

The same check from the openai Python package, if your agent framework is built on it:

# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://cloudgpu.app/v1",
    api_key=os.environ["CGW_API_KEY"],
)

resp = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Explain what this regex matches: ^\\d{3}-\\d{4}$"}],
)
print(resp.choices[0].message.content)

Step 3: Cursor

Per Cursor's help page on using your own API key, open Cursor Settings > Models and paste your cgw-sk-... key into the API key field. That page doesn't cover the base URL. As of 2026-10-27, per Cursor staff on the forum, using your own key through the base URL override "is an officially supported setup" (forum thread). Turn on the option to override the base URL and set it to https://cloudgpu.app/v1. Then add deepseek-v4-flash (or deepseek-v4.1-flash / deepseek-v4-pro) as a custom model and select it in Chat. Setting names change between Cursor versions, so follow what your build shows.

Two things to know, per Cursor: custom API keys only work with chat models, and Tab completion keeps using Cursor's built-in models. As of 2026-10-27, per Cursor staff on the forum (same thread), while the base URL override is on, requests for Cursor's own models go to the custom endpoint too, so turn the override off when you switch back to them.

Step 4: Continue

Per the Continue docs, the current config format is config.yaml (~/.continue/config.yaml; the older config.json is deprecated). It uses the openai provider for any OpenAI-compatible endpoint; you change apiBase.

The Continue docs say the VS Code and JetBrains extensions can't read variables exported in your shell, so put the key in ~/.continue/.env (plain NAME=value, no quotes):

CGW_API_KEY=cgw-sk-...

Then reference it from config.yaml:

name: CloudGPU
version: 0.0.1
schema: v1

models:
  - name: DeepSeek V4 Flash (CloudGPU)
    provider: openai
    model: deepseek-v4-flash
    apiBase: https://cloudgpu.app/v1
    apiKey: ${{ secrets.CGW_API_KEY }}
    roles: [chat, edit, apply]

If your config.yaml already has name, version and schema, add only the model entry. Restart the IDE after editing .env, and don't commit a .env file that contains the key. Swap the model: line for deepseek-v4.1-flash or deepseek-v4-pro to try the others.

Step 5: Cline

Per the Cline docs, open the Cline settings panel, set API Provider to "OpenAI Compatible" and fill in three fields: Base URL https://cloudgpu.app/v1, your API key, and model ID deepseek-v4-flash, deepseek-v4.1-flash or deepseek-v4-pro. Run a small task first to check the model handles your workflow before you hand it a long one.

Step 6: aider

The aider docs connect to any OpenAI-compatible endpoint through OPENAI_API_BASE and OPENAI_API_KEY, with the model name prefixed by openai/. Reuse the key you already exported:

export OPENAI_API_KEY="$CGW_API_KEY"
export OPENAI_API_BASE=https://cloudgpu.app/v1
aider --model openai/deepseek-v4-flash

Any other framework that accepts an OpenAI-compatible base URL takes the same two values.

Which DeepSeek ID to pick

All three IDs are in the model table on the API page, each with its own rate. deepseek-v4-flash is the model used in the API docs examples, so start there. Then try deepseek-v4.1-flash or deepseek-v4-pro on a real task in your agent and keep the one that fits. Check DeepSeek's own docs for each model's context length and reasoning behaviour.

If a reply comes back with an empty content and finish_reason="length", raise max_tokens or leave it unset.

Paying for it

Usage is paid per token from a prepaid balance; there's no subscription, and the balance doesn't expire. Top up on the billing page by card, Alipay, Google Pay, Apple Pay, or USDT on TRC-20. Rates change, so this post doesn't repeat them: the per-token rate for each model is in the table on the CloudGPU API page. GPU rental rates are on /pricing, and the same balance pays for both.

Troubleshooting

• Cursor rejects the key: check the base URL is exactly https://cloudgpu.app/v1 and the key has no leading or trailing spaces.
• 429 rate_limit_exceeded: the public-beta default is 100 requests per minute per API key and 10,000 requests per day per account. Use a second key for parallel agents, or email support to raise the limit.
• 402 insufficient_balance: top up on the billing page.
• Streaming stops mid-answer: check whether the tool has a request timeout setting and raise it.

CloudGPU is in public beta with no SLA. If a model returns 503 upstream_not_configured, the API docs say it is temporarily unavailable: retry later or use another model.

FAQ

Which base URL do I use in Cursor, Continue or Cline?https://cloudgpu.app/v1, with a cloudgpu.app API key that starts with cgw-sk-.

Which DeepSeek model IDs can I use?deepseek-v4-flash, deepseek-v4.1-flash and deepseek-v4-pro, as listed on the API page. Add the ID as a custom model in the tool and select it.

Does this change Cursor tab completion? No. Per Cursor's docs, custom API keys only work with chat models, and Tab completion keeps using Cursor's built-in models.

Are embeddings supported? Not yet, per the API docs. If your tool needs an embeddings model for codebase indexing, configure that separately.

The formula

api_cost_per_hour  = (input_tokens_per_hour × input_price + output_tokens_per_hour × output_price) / 1,000,000
gpu_cost_per_hour  = hourly_gpu_price                     # you pay it whether the GPU is busy or idle

break_even_tokens_per_hour = hourly_gpu_price / blended_price_per_token
blended_price_per_token    = (r × input_price + output_price) / ((r + 1) × 1,000,000)    # r = input:output ratio

Then correct for utilisation: if your GPU is busy only 40% of the hours you rent it, the throughput you need while busy is break-even ÷ 0.4.

Worked example: Qwen3 32B (4-bit) on an RTX 5090 32G

Qwen3 32B is an open-weight model (Apache-2.0) that is not offered on the CloudGPU API, so self-hosting is the way to run it here. We use the 4-bit AWQ build because it fits a 32 GB card.

Model size (estimate). Hugging Face metadata for Qwen/Qwen3-32B lists 32,762,123,264 parameters. In Qwen/Qwen3-32B-AWQ, 31.21B of them are stored at 4 bits and 1.56B stay in BF16:

31.21B × 0.5 byte           ≈ 15.6 GB   (4-bit weights)
 1.56B × 2 bytes            ≈  3.1 GB   (BF16 embeddings, norms, output layer)
AWQ scales/zeros, group 128 ≈  0.6 GB
total                       ≈ 19.3 GB   (the repository's files add up to 19.33 GB)

The FP8 build (Qwen/Qwen3-32B-FP8, about 34.3 GB of files) does not fit in 32 GB of VRAM; it needs a 48 GB card such as the RTX 4090 48G. Figures read from the Hugging Face API on 2026-10-23 21:12 Beijing time; see also Qwen3 32B VRAM.

Context headroom (estimate). With an FP16 KV cache, Qwen3 32B stores 2 × 64 layers × 8 KV heads × 128 × 2 bytes = 262,144 bytes (about 0.26 MB) per token: about 2.1 GB at 8k tokens and 8.6 GB at 32k. After about 19.3 GB of weights, a 32 GB card has roughly 12.7 GB for the KV cache and runtime overhead, so at most about 48k cached tokens shared by all concurrent requests, fewer once the overhead is counted.

Inputs (2026-10-23 21:13 Beijing time):

What a self-hosted token costs. At a fixed $0.59 per hour, the cost per million tokens depends only on the throughput you sustain. The throughputs below are inputs to the arithmetic, not measurements:

Break-even against per-token prices (3 input : 1 output):

These are different models from Qwen3 32B, so the table compares cost only: decide on quality first, then use the arithmetic. How to read it: on one card you would need to sustain several hundred tokens per second, aggregated across concurrent requests, for every hour you pay for. That needs batching and steady traffic, and you should verify it with a benchmark on the card. For spiky or low-volume traffic, a per-token API is usually the lower-cost choice.

We do not publish a measured tokens-per-second figure for this card here; measure your own with your prompt lengths and concurrency.

When renting the GPU wins anyway

Price per token is not the only reason to rent:

When the API wins

For that last case, the same CloudGPU balance pays for the API. Our per-1M-token rate for each Claude model is shown next to the official list price on the API page; official list prices were checked 2026-10-24 on the official Claude pricing page and can change after that date. Rates for the other models are on the same page.

What an always-on GPU costs

If you keep a card running 24/7 to serve traffic, the bill is fixed regardless of tokens (list prices, 2026-10-23 21:13 Beijing time):

“Fixed cost” does not mean unlimited output: the card’s throughput caps how many tokens you can serve. Also budget for restarts — destroying an instance deletes its disk, so a redeploy re-downloads weights. Stock changes during the day; check Deploy.

A script for your own numbers

def break_even(gpu_price_h, in_price_m, out_price_m, in_out_ratio=3.0, utilisation=1.0):
    blended = (in_out_ratio * in_price_m + out_price_m) / (in_out_ratio + 1) / 1e6   # $ per token
    tokens_per_hour = gpu_price_h / blended
    return tokens_per_hour, tokens_per_hour / 3600 / utilisation                       # tokens/h, tokens/s while busy

def monthly(gpu_price_h, hours_per_day=24, days=30):
    return gpu_price_h * hours_per_day * days

def cost_per_million(gpu_price_h, tokens_per_second):
    return gpu_price_h / (tokens_per_second * 3600 / 1e6)                              # $ per 1M tokens

# Qwen3 32B AWQ on an RTX 5090 32G at $0.59/h vs a $0.16 in / $0.55 out per-token reference (2026-10-23 21:13)
tph, tps = break_even(0.59, 0.16, 0.55, in_out_ratio=3, utilisation=0.5)
print(f"break-even: {tph/1e6:.2f}M tokens/h, {tps:.0f} tokens/s while busy")
print(f"self-hosted cost at 500 tokens/s: ${cost_per_million(0.59, 500):.2f} per 1M tokens")
print(f"always-on RTX 5090 32G for 30 days: ${monthly(0.59):.2f}")

Replace the prices with the live values from /pricing and /api on the day you decide, and replace utilisation and the input:output ratio with your logs.

FAQ

Is self-hosting an LLM cheaper than using an API?

Only if you keep the GPU busy enough. Divide the GPU’s hourly price by the API’s blended price per token; if your measured sustained throughput is above that number, self-hosting is cheaper.

How do I calculate the break-even between GPU rental and API pricing?

break-even tokens per hour = hourly GPU price ÷ blended API price per token, where the blended price weights input and output by your traffic ratio. Divide by utilisation to get the throughput you need while busy.

What utilisation do I need for self-hosting to pay off?

It depends on the model and card; compute break-even from the formula, then compare with the throughput you measure under realistic concurrency.

Why rent a GPU if the API is cheaper per token?

For models with no API, fine-tunes and LoRA adapters, custom serving stacks, ComfyUI pipelines, or steady batch jobs that keep the card busy.

Can I use the same balance for GPUs and the API?

Yes. One prepaid balance covers per-minute GPU rental and per-token API calls.

If you want to check the hourly side of the formula, the per-minute GPU rental page explains how a job is billed. Originally published at cloudgpu.app.

Break-even figures are arithmetic on list prices, not performance measurements. Prices were read on 2026-10-23 21:13 Beijing time (UTC+8); model sizes are estimates from Hugging Face metadata read at 21:12; current prices on /deploy and /api take precedence.

This post gives the formula, the per-token numbers for popular models (read from their config.json), and how much context fits on 24, 32, 48 and 80 GB cards.

The formula

kv_bytes_per_token = 2 × num_hidden_layers × num_key_value_heads × head_dim × bytes_per_value
kv_total_bytes     = kv_bytes_per_token × context_tokens × concurrent_sequences

• The 2 is for keys and values.
• num_key_value_heads is smaller than the number of attention heads in models with grouped-query attention (GQA), which is why modern models have much smaller caches than older ones.
• bytes_per_value is 2 for FP16/BF16, 1 for 8-bit KV cache, about 0.5 for 4-bit.
• Concurrent sequences multiply the cache. If your server keeps four parallel slots, budget four times the context.

Per-token KV cache for popular models (FP16)

Architecture values are from each model’s config.json on Hugging Face (read 2026-10-23). GB here means GiB (1024³ bytes).

* Beyond the model’s native window. Qwen3 dense models are documented with a native 32,768-token context, extendable with YaRN rope scaling; check the model card before going past the native length.

The formula does not apply as-is to models with sliding-window layers (for example Gemma 3) or latent attention (DeepSeek V3-style MLA); their caches are smaller than this formula suggests.

How much context fits on each card

Total memory ≈ weights + KV cache + overhead. Weights are the Q4_K_M sizes on our /run pages; overhead is an assumed 1.5 GB for CUDA context, buffers and the runtime. The result is a ceiling for a single sequence; runtimes reserve extra buffers, so plan below it.

Qwen3 32B at Q4_K_M (≈ 19.5 GB weights, 256 KiB/token):

Llama 3.3 70B at Q4_K_M (≈ 41.5 GB weights, 320 KiB/token):

The “practical guidance” column matches the per-model notes on our Qwen3 32B page and Llama 3.3 70B page.

A small calculator

def kv_gb(layers, kv_heads, head_dim, ctx, bytes_per_value=2, seqs=1):
    return 2 * layers * kv_heads * head_dim * bytes_per_value * ctx * seqs / 1024**3

def max_ctx(vram_gb, weights_gb, layers, kv_heads, head_dim,
            bytes_per_value=2, overhead_gb=1.5, seqs=1):
    free = vram_gb - weights_gb - overhead_gb
    per_tok = 2 * layers * kv_heads * head_dim * bytes_per_value / 1024**3
    return max(0, int(free / (per_tok * seqs)))

# Qwen3 32B (64 layers, 8 KV heads, head_dim 128), Q4 weights ≈ 19.5 GB
print(kv_gb(64, 8, 128, 32768))          # 8.0
print(max_ctx(32, 19.5, 64, 8, 128))     # ≈ 45k (native limit is 32k)
print(max_ctx(24, 19.5, 64, 8, 128, bytes_per_value=1))  # 8-bit KV: ≈ 24k

Read num_hidden_layers, num_key_value_heads and head_dim (or hidden_size / num_attention_heads) from the model’s config.json.

Three ways to fit more context

1. Quantise the KV cache. Halving bytes per value halves the cache.

# Ollama: 8-bit KV cache (requires flash attention)
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
ollama serve

# vLLM: FP8 KV cache
vllm serve Qwen/Qwen3-32B-AWQ --kv-cache-dtype fp8 --max-model-len 32768

Quality impact is usually small at 8-bit and more noticeable at 4-bit; test on your own prompts.

2. Set the context you need, not the maximum. In Ollama, pass "options": {"num_ctx": 16384} per request (or set OLLAMA_CONTEXT_LENGTH). In vLLM, --max-model-len caps the context and --gpu-memory-utilization sets how much VRAM the cache may take.

3. Watch parallelism. Ollama’s OLLAMA_NUM_PARALLEL and vLLM’s batching both allocate cache per sequence. If memory runs out under load but not in single tests, this is usually why.

Troubleshooting order when you run out of memory

Say you run Qwen3 32B (Q4_K_M) with Ollama on a 24 GB RTX 4090, set num_ctx to 32768, and get out-of-memory errors or a sudden slowdown (layers spilling to CPU). The formula explains it: about 19.5 GB of weights plus about 8 GB of FP16 KV cache is more than 24 GB. Try these, least effort first:

1. Lower the context. Back to 8192 and the KV cache is about 2 GB; total about 23 GB — it runs, but tight.
2. Quantise the KV cache. With OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0, the same memory holds roughly twice the context — about 24k by the formula (a ceiling; leave headroom).
3. Check parallelism. Set OLLAMA_NUM_PARALLEL=1 to rule out several slots sharing the memory.
4. Change cards. Still short? Move to a 32 GB RTX 5090 or a 48 GB RTX 4090 48G. Switching means starting a new instance and downloading the model again.

After each change, check real usage with nvidia-smi and note the number; next time you size a card, you will have your own data.

Picking a card from the numbers

Once you have weights + KV + overhead, round up to the next VRAM class and check Deploy for live stock. GPUs are billed per minute; current rates are on the pricing page. For a side-by-side of those cards, see RTX 4090 vs RTX 5090 vs A100.

FAQ

How do I calculate the KV cache size for a model?

Multiply 2 × layers × KV heads × head dimension × bytes per value (2 for FP16) to get bytes per token, then multiply by context length and the number of parallel sequences. The values are in the model’s config.json.

How much VRAM does a 32k context need for Qwen3 32B?

About 8 GB of FP16 KV cache (256 KiB per token × 32,768 tokens), on top of about 19.5 GB of Q4_K_M weights, so a 32 GB card is the comfortable choice.

Why does my model run out of memory only at long context?

The weights are fixed, but the KV cache grows with every token of context and with each parallel request. Lower num_ctx / max-model-len, reduce parallel slots, or quantise the KV cache.

Does an 8-bit KV cache halve memory?

Yes, the cache itself is halved compared with FP16. In Ollama set OLLAMA_KV_CACHE_TYPE=q8_0 with flash attention on; in vLLM use --kv-cache-dtype fp8.

Does the formula work for every model?

It works for standard and grouped-query attention. Models with sliding-window layers or latent attention use less memory than it predicts.

Once you have your own numbers, the RTX 4090 vs RTX 5090 vs A100 comparison lines up the cards by VRAM class. Originally published at cloudgpu.app.

CloudGPU (cloudgpu.app) has no partnership with, authorization from, or endorsement by the model vendor. Model names are trademarks of their owners. Context ceilings are estimates under the stated assumptions, not measurements. Live prices are on /pricing and live stock on /deploy.