Elastic Inference API

Serverless OpenAI-compatible chat completions and embeddings. You send requests; we route them across our global inference network. No infrastructure on your part.

We maintain a focused catalog of production-ready models — not every model available, only the ones that work reliably at scale. See Supported Models and Code Examples.

To let Hypervize pick the catalog model for you, enable Olympus and send "model": "auto". See Olympus. prefer_savings and avoid_flagship on that request change the pick. See Host modes.

Endpoint

POST https://hypervize.tech/api/chat/completions

All requests must include a valid Authorization: Bearer hvz_live_... header (or be made from the dashboard while logged in with at least one active key).

For most applications use the standard endpoint above. For model control at the API key level (agents and tools), see The /b/ Endpoint (Model Pinning).

The /b/ Endpoint (Model Pinning)

In addition to the primary POST /api/chat/completions endpoint, Hypervize offers:

POST https://hypervize.tech/api/b/chat/completions

This path is for cases where you need to manage or swap the underlying model at the API key level, without changing calling code.

How Model Pinning Works

When you call /api/b/chat/completions:

1. We authenticate with the provided Authorization: Bearer hvz_live_... key (same as the standard endpoint).
2. We look up the pinned model configured for that API key.
3. If a model is pinned, we override the model field from your request body with the pinned value.
4. The request continues as a normal Elastic Inference call (including tools, usage tracking, and billing).

If no model is pinned to the key, /b/ behaves identically to the standard endpoint.

Pin a catalog display name to force that model. Pin auto (or olympus / olympus-auto) to run Olympus on every /b/ call.

Pinning is applied as soon as the request is accepted, so the rest of the pipeline always sees the pinned model — not whatever the client sent.

Primary Use Case: Tools, Agents, and Platforms like OpenClaw

Agent frameworks and workflow tools often:

• Hardcode a model name in configuration or source.
• Generate inference calls programmatically across many agents or workers.
• Need different models per environment or customer.
• Want to upgrade, experiment, or roll back models centrally.

With the /b/ path you can:

• Create a dedicated API key for the integration (e.g. "OpenClaw Production").
• Pin the desired model (e.g. claude-sonnet-5 or mistral-large-3) to that key.
• Point the deployment at https://hypervize.tech/api/b/chat/completions.
• The client can send any model value (or none) — Hypervize forces the pinned model.
• When you switch models, update the pin on the key in the dashboard. No code changes or restarts on the client side.

This pattern is useful for:

• Centralized model governance — Enforce models for cost, latency, or compliance.
• Agent fleets — Same calling code, different pinned models via separate keys.
• Third-party or embedded integrations — Calling code outside your control; you still control the model.
• Rapid iteration — A/B test, canary, or revert by updating the pin.

All normal Elastic features remain available: server-managed tools (when streaming), platform tools, vision, usage reporting, and prepaid billing.

Configuring Pinned Models

Set or change the pinned model for any of your API keys in Settings → Keys (Access Keys). The value accepts any display name or internal identifier from the Supported Models catalog.

Updates take effect on the next request using that key.

Choosing Between Endpoints

Standard and /b/ endpoints share the same API keys, authentication, rate limiting, and billing semantics.

Request Format

The API is OpenAI Chat Completions compatible. Supported fields:

{
  "model": "claude-sonnet-5",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello!" }
  ],
  "max_tokens": 1024,
  "temperature": 0.7,
  "top_p": 0.9,
  "stream": true,
  "stop": ["\n\n"]
}

Important Model Name Handling

Prefer human-friendly display names from the catalog (e.g., claude-sonnet-5, gemini-3.7-flash, gemini-3.1-pro, llama-3-3-70b-instruct, grok-4.3, kimi-k3).

Hypervize maps display names to backend identifiers. You may also pass the raw provider identifier if needed.

See Supported Models for the full list with pricing and context windows.

Provider rate limits and model fallbacks

For some catalog models, if the primary upstream provider is rate-limited or temporarily unavailable, Hypervize may automatically serve an alternate catalog model so the request can still complete. That platform behavior is the same for every key. It is separate from a list you store on one key, and from Olympus Auto.

• Fallbacks are not guaranteed on every model or every error type.
• When a fallback serves, the response model field (and metered usage) reflect the model that actually ran, not only the name you requested. You are billed at that model’s catalog rates. See Billing & Payments.
• A model we serve from Amazon Bedrock may still be answered, and billed, as Grok (xai.grok-4.6) before any list on your key is tried.
• Gemma 4 31B stays on Gemma: if the primary host fails, we retry the same model on a second host. When that retry starts an answer, a list on your key stays unused. Other Elastic models may fall back to a different catalog model.
• Calls to Anthropic, OpenAI, Gemini, xAI, and Moonshot do not receive a second platform swap.
• If no fallback is available or all attempts fail, you get a normal error (often 429 or 5xx). Retry with exponential backoff. See Errors.

Clients should treat the returned model (and usage) as authoritative for what ran.

Fallback models on an API key

You can store up to three catalog chat models on an inference key, in the order they should be tried. The list runs only after the platform attempt failed to start an answer. Hypervize then tries the next model when that attempt was rate-limited or unavailable, when the prompt was refused for context length or a content policy, or when two minutes passed with no answer started.

Three choices stay separate:

• Platform fallbacks above. They apply on their own. You do not configure them per key.
• This ordered list on the key.
• Olympus Auto, plus prefer_savings and avoid_flagship on that request. Those choose the model that is called. They do not reorder this list. A pin on the key is still the model that is called. model: "auto" is still the automatic pick.

Set the list with the keys API while you are signed in to the dashboard. Send display names or catalog ids. The stored value and the response are catalog ids. auto is rejected. The completion request itself has no fallback list.

PATCH /api/keys/{keyId}
Content-Type: application/json

{ "fallbackModels": ["claude-haiku-4.5", "gemma-4-31b"] }

POST /api/keys accepts the same field. Omit fallbackModels to leave the stored list. Send null or [] to clear it. A key with no list keeps platform behavior only. GET /api/keys returns fallbackModels as catalog ids, or null.

The list runs when this key is the Bearer key on chat completions, Responses, or legacy completions, including the /api/b/… paths. Hypervize Chat, the dashboard playground, embeddings, images, and a dedicated endpoint leave the list unused. A Chronos job uses a list only when the job names a key id. See Chronos → Fallback key.

A 400 whose text only talks about quota stays on that error. So do a 504, a 401, a 400 with no context-length or content-policy refusal, and an unknown model name. Two minutes with no answer started tries the next model. The first words of a stream, or a tool call, keep that model. Browsing after the tool call stays on it, including a reservation that runs longer than two minutes. A finished non-streaming document is kept even when it takes longer than two minutes. A failure that arrives after two minutes on that finished document is returned as that failure.

An error you still receive can mean the failure was outside this list, the model had already started, the finished document failed after two minutes, or the key had no list. Retry that error on your side. See Errors.

The completion model and the charge follow the model that answered. On a live Responses stream, response.created may still name the model you sent. response.completed and the stored response follow the model that ran. When a model from your list answers before the response has started, the response includes x-hypervize-fallback-from set to the catalog id you requested. If the response has already started, that header is absent and the completion model plus the usage properties are the record of what ran. See Billing & Payments.

Streaming Responses (Recommended)

Hypervize Inference strongly encourages streaming for production use cases.

Responses are standard Server-Sent Events:

data: {"id":"...","choices":[{"delta":{"content":"Hello"},"index":0}],"model":"...","created":...}

data: {"id":"...","choices":[{"delta":{"content":" world"},"index":0}],"model":"...","created":...}

...

data: {"usage":{"prompt_tokens":12,"completion_tokens":48,"total_tokens":60}}

data: [DONE]

The final usage object (when present) contains the token counts used for metering.

Non-Streaming Responses

You may set "stream": false (or omit the field). You receive a single JSON response with the full assistant message and usage statistics.

Streaming is often nicer for UIs (tokens appear as they are generated). Stream and non-stream are both supported for normal completions and plain client tools.

Request wall clock (important for agents)

Each elastic completion request is subject to a platform wall-clock limit of about 13 minutes (serverless function max duration). That covers the whole turn: model time, tool waits, and managed re-invokes when platform/webhook tools run on our side.

If a request hits the wall clock, the call fails; retry or redesign the loop rather than extending a single request indefinitely.

Supported Features

• Chat messages (system / user / assistant, including multi-turn)
• Vision / image inputs (on models that support them)
• Streaming and non-streaming
• max_tokens, temperature, top_p, stop
• Token usage reporting in the final chunk
• Full tool / function calling (plain OpenAI schemas, webhooks, and platform tools)
  • Mode A plain client-driven tools: streaming and non-streaming (chat.completion with tool_calls). Plain tools in the request do not auto-attach platform tools unless hybrid is enabled (see Tool Calling).
  • Mode B server-managed tools (webhooks + platform tools such as Athena, Desk, Vesper, Herald, Pandora, Iris): stream or non-stream; non-stream returns one aggregated chat.completion after the managed loop. Whole loop still counts against the ~13 minute wall clock. Chronos is a scheduling product flag — it is not injected as a model-callable tool.

See the Tool Calling guide and Platform Tools for details.

Limited / Not Yet Supported

• JSON mode / response format enforcement (limited support)
• Logprobs
• Seed / deterministic sampling guarantees
• Audio / speech inputs (supported on certain models; availability varies)

Embeddings

Hypervize supports the standard OpenAI Embeddings API for catalog models designed for vector embeddings (e.g. Cohere).

Endpoint

POST https://hypervize.tech/api/embeddings

Request

{
  "model": "cohere.embed-v4",
  "input": "The quick brown fox jumps over the lazy dog.",
  "encoding_format": "float"
}

encoding_format is optional: "float" (default) or "base64".

input can be a single string or an array of strings.

Response

Standard OpenAI embeddings response:

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.1, -0.2]
    }
  ],
  "model": "cohere.embed-v4",
  "usage": {
    "prompt_tokens": 8,
    "total_tokens": 8
  }
}

Using the OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="https://hypervize.tech/api",
    api_key="hvz_live_..."
)

resp = client.embeddings.create(
    model="cohere.embed-v4",
    input=["text one", "text two"]
)

for item in resp.data:
    print(item.embedding[:5])  # first few dimensions

Embeddings are billed by prompt tokens (input only) against your prepaid balance. See the model entry in Supported Models for the per-million-token rate.

Pricing (Elastic)

Pricing is transparent and usage-based. You are charged per million input and output tokens (or per image for vision models).

Current pricing for each model is visible in:

• The model selector in the dashboard
• The Supported Models page
• The final usage chunk of streaming responses

Error Handling

All errors return a JSON body with an error field:

{
  "error": "Invalid API key"
}

Common status codes:

• 401 — Missing or invalid Authorization header / key
• 403 — Valid key but no inference keys on account, or other permission issue
• 500 — Internal error (rare — check status page or contact support)

Full error reference: Errors & Troubleshooting

Best Practices

1. Keep each request under the ~13 minute wall clock; multi-step agents should multi-call.
2. Handle the usage object for cost attribution.
3. Set reasonable max_tokens values (many models have different defaults).
4. Use catalog display names rather than raw provider IDs for future-proofing.
5. Implement exponential backoff on 429/5xx responses.

Hypervize Elastic is deliberately opinionated. Our catalog focuses on the highest-quality, most reliable models rather than maximizing the number of options. For any model not in the catalog, use Dedicated Inference.

Billing

Elastic inference (tokens) and Hypervize-managed tool calls are charged against your prepaid balance. New accounts start with 3 free queries. Opening Hypervize Chat for the first time may grant additional free messages (shared with API usage).

Tool calls are metered per execution (see published per-call prices) and apply whether you are using elastic or dedicated inference. See the Tool Calls section in Billing & Payments.

See Billing & Payments for top-ups, auto top-up, balance enforcement, and how elastic billing differs from dedicated weekly post-pay.

Related

• Quickstart — First Inference Call
• Supported Models
• Code Examples
• Platform Tools