Semantic Routing Decisions and Session Affinity
A semantic router classifies one user turn before dispatching to a target. Session affinity can then keep later requests on the same route without repeating classification. Configure the router in Semantic Routing; the rules below explain exactly which text and session ID AISIX uses.
What the Router Classifies
The same rule applies on every endpoint a semantic router serves. AISIX walks the request's user turns from the newest to the oldest and classifies the first one that carries text the user wrote:
- Only the turn's text parts are read. When a turn has several, AISIX joins them with a space.
- A turn that holds only tool results, images, or reminders has no such text, so AISIX moves on to the turn before it.
- The system prompt, the Responses API
instructions, and assistant turns are never classified.
Before classification, AISIX removes every complete <system-reminder>…</system-reminder> block and keeps the text around it. For a Codex client, identified by its User-Agent as described in Where the Session ID Comes From, AISIX also removes these separate tag-delimited blocks:
<environment_context>...</environment_context>
<recommended_plugins>...</recommended_plugins>
<user_instructions>...</user_instructions>
<environments_instructions>...</environments_instructions>
AISIX also removes one AGENTS.md instructions span. The span starts at the heading and ends at the closing instructions tag:
# AGENTS.md instructions for ...
<instructions>
...
</instructions>
The delimiters are matched case-insensitively. An opening delimiter without its closing delimiter remains part of the text. For clients other than Codex, only complete <system-reminder> blocks are removed from the text before classification.
A request with no such text goes to the default target without an embedding call, unless its session is pinned to a route.
During a Claude Code or Codex tool-call loop, AISIX classifies the user's most recent question rather than a tool result. It recognizes a user turn in these forms:
- A
usermessage on/v1/chat/completions. - A
usermessage with string content ortextblocks on/v1/messagesand/v1/messages/count_tokens. - A string
input, or aninput_textpart of ausermessage item, on/v1/responses.
On /v1/chat/completions, a request whose last user message holds only an image is classified on the newest earlier user message that has text. Gateways before 1.10.0 sent such a request to the default target.
Tune and Protect Routing
Cross-lingual matching depends on the embedding model. Tune thresholds with representative prompts and examples from every language in the intended workload.
Adversarial prompts can influence routing decisions. When the destination has security or compliance consequences, apply input guardrails to the semantic router before dispatch. A guardrail attached only to a destination model does not run when the caller requests the router. See Guardrail Scope Follows the Addressed Model.
Model Group Targets
After a semantic route selects a model group, the group's strategy chooses the member. The group's failover, cooldown, and target-model limits apply normally.
If the group has no available member, or its client-IP allowlist excludes the caller, the router uses its default target. The access log records semantic_fallback="target_unavailable".
| Mechanism | Applies to |
|---|---|
| Guardrails, model access control, and response cache | The semantic router addressed by the caller |
| Rate limits | The router once per request; the group once when selected; the dispatched member once per upstream attempt. See API Key and Model Rate Limits. |
| Cost and usage | The member that served the request. Billing uses the group's sale price, then the member's sale price, then the member's platform price. See Bill a Semantic Router's Requests. |
| Metrics | The semantic router in model; the dispatched member in upstream_model |
Pin the Route and Group Member
Session affinity pins a session to the route and group, not to an individual member. To pin both layers, configure the group for consistent hashing with session headers, as in Pin Sessions with Pool Failover.
This example supports Claude Code and clients that send the AISIX session header. The first non-empty source wins:
{
"routing": {
"strategy": "consistent_hash",
"hash_on": [
{"type": "header", "name": "x-claude-code-session-id"},
{"type": "header", "name": "x-aisix-session-id"},
{"type": "api_key"}
],
"targets": []
}
}
How Pins Behave
Pin Lifecycle
A pin belongs to one semantic router, caller API key, and session ID. Two callers or routers never share a pin, even when they receive the same session ID.
| Event | Behavior |
|---|---|
| First request that matches a route | Creates the pin when the route is chosen. |
| Later request in the session | Uses the pinned route without classification and refreshes the TTL. |
| Request served by the default or failure target | Creates no pin; the next request is classified normally. |
| Pinned target is temporarily unavailable | Uses the default target and keeps the pin. |
| Pinned target no longer belongs to a route | Drops the pin and classifies the request again. |
/v1/messages/count_tokens | Uses and refreshes an existing pin but never creates one. |
Coding-Agent Side Requests
Coding agents also send side requests outside the main conversation loop, such as title generation. A side request never creates a pin. If the session already has one, the side request follows and refreshes it. AISIX recognizes these requests from the x-claude-code-request-class: auxiliary header or known Claude Code and Codex prompts.
Recognizing side requests by their prompts works only for prompts AISIX knows, so a client release that changes them can make a side request pin its session. To keep such requests out of classification entirely, point the client's background or small model at a direct model rather than at the semantic router. For Claude Code, that is the small, fast model set by ANTHROPIC_DEFAULT_HAIKU_MODEL; see Configure Claude Code.
Pin Storage
In AISIX Enterprise and AISIX Cloud, gateways in an environment share pins through the control plane. A session therefore stays on its route whichever gateway receives the request. An environment holds about 1,000,000 shared pins; beyond that, only the gateway that creates a new pin keeps it.
When a gateway cannot reach the shared pins, it keeps local pins and continues serving requests. Different gateways can then pin the same session to different routes until connectivity returns. An AISIX OSS gateway always keeps its pins in its own process.
Where the Session ID Comes From
AISIX uses the first valid session ID in this order. Values over 256 bytes are ignored.
| Priority | Source |
|---|---|
| 1 | x-aisix-session-id header |
| 2 | Any x-<vendor>-session-id header, such as x-claude-code-session-id. If several exist, the header whose name sorts first wins. |
| 3 | For a Codex client, the session-id, session_id, thread-id, or conversation_id header. The User-Agent must start with codex followed by -, _, /, or a space. |
| 4 | x-session-id header |
| 5 | metadata.user_id in the request body: the session_id of an object or encoded JSON object, or the text after the last _session_ in a string |
| 6 | metadata.session_id in the request body |
| 7 | session.id in the W3C baggage header |
The headers in sources 2, 3, and 4 count only when their value is at least 8 letters, digits, _, or -. A request without a session ID is classified as usual and pins nothing.
Claude Code and Codex already send a session ID from this list, so their sessions are pinned with no client configuration. For another client, send x-aisix-session-id with an ID that stays the same for the whole conversation:
curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
-H "Authorization: Bearer $AISIX_API_KEY" \
-H "x-aisix-session-id: conversation-1a2b3c" \
-H "Content-Type: application/json" \
-d '{
"model": "topic-router",
"messages": [
{"role": "user", "content": "Review this contract for liability risk."}
]
}'
Observe Pinned Requests
A request that went to its pinned route carries semantic_pinned=true and the route in semantic_route on its access-log line. It has no semantic_score, because nothing was classified. When the pinned target was unavailable, the line carries semantic_pinned=true and semantic_fallback="target_unavailable" and no semantic_route.