Skip to main content
API7.ai Logo

Fireworks AI

Fireworks AI provides hosted inference for a catalog of generative AI models. Applications call selected models by stable AISIX aliases while the gateway holds the Fireworks credential.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • A Fireworks API key from the Fireworks console.
  • curl and jq.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the Fireworks-backed chat-completions route.

Because Fireworks AI exposes an OpenAI-compatible API, AISIX connects through the openai adapter and uses the Fireworks API root as api_base.

Create a Provider Key​

Create the provider key that stores the Fireworks credential and API root, and allow it into the environment:

# Replace with your value
export FIREWORKS_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "fireworks-prod",
    "provider": "fireworks-ai",
    "api_key": "'"${FIREWORKS_API_KEY}"'",
    "api_base": "https://api.fireworks.ai/inference/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is fireworks-ai, the catalog provider ID. The hyphen is part of the identifier; fireworks alone is rejected with a 400 INVALID_REQUEST.

❷ api_key stores the Fireworks API key. It follows the credential-handling behavior in Provider Keys.

❸ api_base points at the Fireworks inference API. The /inference segment is significant: Fireworks serves its OpenAI-compatible inference routes under https://api.fireworks.ai/inference/v1, while its account-management REST API — the one that lists and creates Fireworks API keys — is served under https://api.fireworks.ai/v1. Dropping /inference points the provider key at the wrong API surface.

AISIX appends the endpoint path to api_base, so use the API root without a trailing /chat/completions. A trailing slash is trimmed before the endpoint path is appended, so https://api.fireworks.ai/inference/v1/ and https://api.fireworks.ai/inference/v1 resolve identically.

api_base is optional for this provider. When it is omitted, the AISIX Cloud Admin API fills in https://api.fireworks.ai/inference/v1. Setting it explicitly keeps the upstream root visible on the resource. A Fireworks dedicated deployment uses the same inference root; point model_name at accounts/<ACCOUNT_ID>/deployments/<DEPLOYMENT_ID> to change the target.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Create a Model​

Fireworks text-model resource names are fully qualified account paths rather than bare names. A serverless text model published by Fireworks uses the form accounts/fireworks/models/<name>, for example accounts/fireworks/models/gpt-oss-120b. A flat OpenAI-style name such as gpt-oss-120b is not valid for this chat model.

Copy the full identifier from the Fireworks models overview rather than assembling it by hand. Some catalog entries replace the models segment with routers, and models you deploy yourself use your own account name in the first segment.

Other Fireworks inference APIs can publish different model-ID forms. For example, the embeddings guide uses fireworks/qwen3-embedding-8b. Use the exact identifier documented for the endpoint and model instead of forcing every identifier into the accounts/... form.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "fireworks-gptoss-prod",
    "model_name": "accounts/fireworks/models/gpt-oss-120b",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model. Callers never see the account path.

❷ model_name is the Fireworks model ID, for example accounts/fireworks/models/gpt-oss-120b.

❸ provider_key_id attaches the alias to the Fireworks provider key.

Create a Caller API Key​

Create the caller API key that can access the model alias. The API generates the key value and returns the plaintext once:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "fireworks-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value references the model by its ID. The plaintext key is returned only in this response, so store it securely.

The new resources project to the attached gateway automatically.

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export FIREWORKS_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "fireworks-prod"
    provider: "fireworks-ai"
    adapter: "openai"
    api_key: ${FIREWORKS_API_KEY}
    api_base: "https://api.fireworks.ai/inference/v1"
    request:
      param_renames:
        max_completion_tokens: max_tokens

models:
  - display_name: "fireworks-gptoss-prod"
    provider: "fireworks-ai"
    model_name: "accounts/fireworks/models/gpt-oss-120b"
    provider_key: "fireworks-prod"

api_keys:
  - display_name: "fireworks-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "fireworks-gptoss-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fireworks-gptoss-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Fireworks AI."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias fireworks-gptoss-prod. If the request fails, check the provider key api_key, the /inference/v1 path in api_base, and the full accounts/... model ID in model_name.

Understand the Token-Limit Parameter Rewrite​

Fireworks documents max_tokens as the output-limit parameter on its chat-completions route, while current OpenAI SDKs and many agent frameworks send max_completion_tokens. AISIX Cloud resolves the difference through a built-in request override on its fireworks-ai catalog entry. The open-source AISIX gateway does not add that catalog override itself, so the resources.yaml example configures the same rename explicitly.

When configured, the rewrite has three consequences worth knowing:

  • Callers do not need a Fireworks-specific code path. A client that sends max_completion_tokens reaches Fireworks with max_tokens set to the same value.
  • If a single request carries both fields, the max_completion_tokens value replaces max_tokens. The newer field wins because it is the one the caller most likely set deliberately.
  • The rename applies to the outbound body on every normalized route this provider key serves, not to chat completions alone. A passthrough route is the exception, because it relays the body verbatim, so send the field name Fireworks expects on that route.

Fireworks applies its own default output limit to requests that omit the field, so set an explicit limit for long generations. See Querying text models for the current parameter reference.

caution

A request block supplied on the provider key replaces the built-in block rather than merging with it. If you add your own request overrides for this provider key, re-declare param_renames alongside them, or the rewrite stops applying:

{
  "request": {
    "param_renames": {
      "max_completion_tokens": "max_tokens"
    },
    "default_headers": {
      "X-Team": "platform"
    }
  }
}

For the open-source AISIX gateway, retain the rename when adding other request settings:

    request:
      param_renames:
        max_completion_tokens: max_tokens
      default_headers:
        X-Team: platform

See Provider-Specific Overrides for the full override schema.

Use Reasoning Models​

Fireworks documents two mutually exclusive reasoning controls, and support for each is model-specific:

  • reasoning_effort, whose accepted effort levels are model-specific and include low, medium, and high.
  • thinking, an object with type set to enabled and budget_tokens of at least 1024.

A request must not set both. AISIX forwards top-level request fields it does not model verbatim to the upstream, so either control reaches Fireworks unchanged:

{
  "model": "fireworks-gptoss-prod",
  "messages": [{ "role": "user", "content": "Plan a cache invalidation strategy." }],
  "reasoning_effort": "medium"
}

Fireworks normally returns reasoning output in reasoning_content, which is already the canonical AISIX field, although some models return it in content instead. This provider therefore needs no response.reasoning_field override: AISIX preserves reasoning_content as choices[0].message.reasoning_content on non-streaming responses and as delta.reasoning_content on streaming responses.

For models that interleave reasoning with tool calls, retain the complete assistant reasoning_content and include it when sending the assistant message back in the next tool-call request. AISIX preserves the field but does not manage or replay application conversation state. Check Reasoning for the controls and replay requirements of the selected model.

Choose Translated or Fireworks-Native Formats​

Fireworks publishes a native Responses API and an Anthropic-compatible Messages API. The fireworks-ai catalog provider still uses the openai adapter, so the normalized AISIX routes translate through chat completions instead of calling those native APIs.

Configuration and routeUpstream behavior
Catalog alias on /v1/responsesUses the AISIX chat-based Responses bridge. Fireworks-native state, including previous_response_id, stored responses, and server-executed tools, is not available through the bridge.
/passthrough/fireworks-ai/responsesCalls Fireworks' native Responses API. Send the Fireworks model ID rather than the AISIX alias. Fireworks stores native responses by default; send store: false when persistence and continuation are not needed.
Catalog alias on /v1/messagesAISIX translates the Anthropic-shaped request to chat completions. It does not call Fireworks' native Messages API.
/passthrough/fireworks-ai/messagesCalls Fireworks' native Anthropic-compatible Messages API with the request and response bodies unchanged. Send the Fireworks model ID rather than the AISIX alias.
Separate BYO provider key with adapter: anthropic and api_base: https://api.fireworks.ai/inferenceLets normalized /v1/messages traffic call Fireworks' native Messages route. Fireworks' documented Anthropic compatibility limits still apply.

Use a separate bring-your-own endpoint provider key for the Anthropic adapter because a catalog provider key cannot override its adapter. The /passthrough/fireworks-ai paths on this page assume a passthrough route claiming that prefix with the Fireworks inference root as its target_url; grant the route name on the caller key's allowed_routes. A passthrough route does not rewrite AISIX model aliases. AISIX detects chat, completions, and Responses envelopes from each request and records supported usage fields. Requests without a recognized carrier field remain opaque: buffered responses record zero tokens, while opaque SSE can still record top-level supported usage fields.

Review Endpoint Support​

A Fireworks-backed alias works on the routes below. See Provider Compatibility for the full endpoint matrix.

RouteBehavior with a fireworks-ai provider key
/v1/chat/completionsSupported, including stream: true.
/v1/completionsSupported for Fireworks models that accept the legacy prompt-based completions format.
/v1/embeddingsSupported when the target is a Fireworks embedding model. Use the endpoint-specific model ID, such as fireworks/qwen3-embedding-8b, from the embeddings guide.
/v1/responsesSupported through the chat-based Responses bridge, not Fireworks' native Responses API. Fields without a chat equivalent are ignored. Use /passthrough/fireworks-ai/responses when native Responses semantics are required.
/v1/messagesSupported through translation to chat completions, not Fireworks' native Messages API. /v1/messages/count_tokens is unavailable because it requires an anthropic provider.
/v1/rerankNot supported for a fireworks-ai alias because the route's provider allowlist excludes it. Call Fireworks' native reranking API at /passthrough/fireworks-ai/rerank and send its endpoint-specific model ID.
/v1/audio/*Not supported. Fireworks does not publish matching OpenAI-compatible speech or transcription routes under this API base; supported audio and video inputs are sent through multimodal chat models.
/v1/images/generationsNot supported. The route requires a model whose configured provider is openai. Fireworks' native image-generation workflow endpoints are available only through /passthrough/fireworks-ai/workflows/....
/v1/videosNot supported. The route has its own provider allowlist, which does not include fireworks-ai.
/passthrough/fireworks-ai/*Available through a configured passthrough route for provider-native endpoints that AISIX does not model.

On a prefix-matched passthrough route, AISIX joins the remaining path onto the route's target_url. Because a target of https://api.fireworks.ai/inference/v1 ends with an API-version segment, a passthrough path that also begins with v1/ is deduplicated, so /passthrough/fireworks-ai/v1/<path> and /passthrough/fireworks-ai/<path> both resolve to https://api.fireworks.ai/inference/v1/<path>. A route with that target stays under the inference API and cannot reach Fireworks account or deployment management routes at https://api.fireworks.ai/v1/accounts/....

Passthrough authorization comes from the caller API key's allowed_routes list, which must grant the route name; model allowlists do not gate these paths. On a raw route, token counts and token-based costs remain zero, but caller-key request-count limits still apply. See Passthrough Routes.

Next Steps​

You have now connected AISIX to Fireworks AI and verified the model alias. Continue with these guides: