Running Agentic API in front of SGLang

Point Agentic API at SGLang's /v1/responses endpoint using --llm-api-base. The gateway manages stored responses and rehydrates item history for subsequent turns.

Launch configuration

The conformance recorder targets SGLang 0.5.18 and Qwen/Qwen3-8B. The model uses the qwen3 reasoning parser and qwen tool-call parser. See the upstream installation guide and DGX Spark guide.

docker run --name agentic-sglang --gpus all --shm-size 8g \
  -p 127.0.0.1:30000:30000 \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  lmsysorg/sglang@sha256:9e148f5ac788e856a06166bd6347a831831eb9fcfab4d1770874823a7c29a1a1 \
  python3 -m sglang.launch_server \
  --model-path Qwen/Qwen3-8B --host 0.0.0.0 --port 30000 \
  --revision b968826d9c46dd6066d109eabc6255188de91218 \
  --reasoning-parser qwen3 --tool-call-parser qwen \
  --mem-fraction-static 0.3 --context-length 8192 --cuda-graph-max-bs 4

Once http://127.0.0.1:30000/health succeeds, start the gateway:

cargo run -p agentic-server -- --llm-api-base http://127.0.0.1:30000

Record and replay

The SGLang suite uses the same executor replay assertions as Dynamo: blocking and streaming text, gateway-managed two-turn continuation, and a client-executed function call. It compares each provider's output with its own captured traffic. The CI job replays fixtures without a GPU or a running model.

From the repository root, install the recorder dependencies into a virtual environment and run (Rust is also required for replay validation):

uv venv .venv
uv pip install --python .venv/bin/python click fastapi httpx uvicorn 'PyYAML==6.0.3'

PYTHON=.venv/bin/python SGLANG_URL=http://127.0.0.1:30000 \
  SGLANG_VERSION=0.5.18 MODEL=Qwen/Qwen3-8B \
  bash crates/agentic-server-core/tests/cassettes/record_sglang_cassettes.sh

cargo test -p agentic-server-core --test sglang_cassette_test

The script records through record_cassette.py into a temporary staging directory. It builds the second turn from the captured first-turn assistant message, removes request headers and unstable identifiers/timestamps, and adds provider provenance. Structural validation and executor replay must pass before fixtures are copied into the repository. Staging files remain available for diagnosing a failed refresh. Review the resulting diff and commit intentional updates; never fabricate captured YAML.

Negative integer sequence_number values are treated as unspecified during event normalization. The normal streaming delivery path assigns client sequence numbers; text, function-call payloads, and terminal usage still pass through ingestion.

The initial recording set does not establish support for parallel function calls, structured text, reasoning summaries, or upstream WebSocket transport. These are listed as unverified in provenance. The broader live-engine matrix and scenario expansion remain tracked in issue #211.

GPT-OSS limitation in SGLang 0.5.18

A live GPT-OSS 20B probe completed blocking continuation, but streaming changed reasoning item IDs/output indexes before completion and returned an empty terminal output array. The gateway rejects this malformed lifecycle. Do not infer GPT-OSS streaming support from the Qwen3 baseline.

GPT-OSS also requires Harmony vocabularies. A failed vocabulary download can disable SGLang's optional Responses handler while /health still succeeds. Check startup logs for OpenAI Responses API ... disabled; the upstream DGX Spark guide explains mounting the vocabulary files with TIKTOKEN_ENCODINGS_BASE.