Models & API

How to connect

The gateway is OpenAI-compatible. Point any OpenAI SDK at it with the API key issued from your Dashboard.

Primary models

Always-on GPU lanes (live status). Fastest first token.

On-demand models (e.g. qwen3-vl-235b-awq) are spun up when needed and are otherwise down — a call returns an error until an operator brings them up.

Examples

Python (OpenAI SDK)

from openai import OpenAI
client = OpenAI(base_url="http://192.168.1.199:14000/v1", api_key="YOUR_KEY")
r = client.chat.completions.create(
    model="qwen3-coder-30b-fp8",
    messages=[{"role": "user", "content": "Write a haiku about GPUs."}],
)
print(r.choices[0].message.content)

curl

curl http://192.168.1.199:14000/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gemma4-26b-fp8","messages":[{"role":"user","content":"hello"}]}'

Vision — qwen3-vl-235b-awq (on-demand)

client.chat.completions.create(model="qwen3-vl-235b-awq", messages=[{"role":"user","content":[
    {"type":"image_url","image_url":{"url":"data:image/png;base64,..."}},
    {"type":"text","text":"What is in this image?"}]}])

All models on the gateway (36)

Everything callable with your key. Non-primary models are Ollama-backed and load on demand (slower first token). Embeddings: embeddinggemma, bge-m3.

Advanced — Anthropic-format router

For Claude Code / Anthropic-format clients, the router at http://192.168.1.199:14100 speaks the Anthropic Messages API (/v1/messages) for the primary models. Reachable on the LAN without a key.