CodexProbe

CodexProbe is a local reverse proxy and LLM-call recorder for the Codex agent harness. It sits at Codex's configured model-provider base_url, captures each Responses request and response, relays streaming responses as they arrive, and can route the same traffic to a configured OpenAI-compatible backend.

Codex CLI  ->  CodexProbe  ->  configured backend
              record +     (OpenAI-compatible or local vLLM/Qwen)
              backend swap

The engineering result for Task 7 is positive: transparent passthrough, complete capture, live streaming, semantic reconstruction, session traces, and configuration-only backend selection are implemented. The local Qwen backend was reached through the production compatibility adapter. The controlled calculator qualification did not complete; the tested Qwen models did not reliably satisfy Codex's JavaScript/V8 custom exec contract. This is the honest alternate outcome allowed by the task, and is distinct from the successful backend swap.

Install

From the repository root:

python -m pip install -e .

The package declares FastAPI, Uvicorn, HTTPX, and Zstandard as runtime dependencies. Install test dependencies when needed:

python -m pip install -e ".[test]"

The supported Python range starts at 3.10. The importable package is codex_probe.

Public API

The assignment-facing API accepts a plain dictionary or a JSON object string. The internal RecorderConfig and typed call objects remain available for callers that need them.

import json

from codex_probe import ProxyRecorder

config = {
    "base_url": "https://api.openai.com",
    "wire_api": "responses",
    "auth": None,
    "listen_host": "127.0.0.1",
    "listen_port": 8135,
    "log_dir": "logs/openai-passthrough",
    "seed": 0,
}

recorder = ProxyRecorder(config)
endpoint = recorder.start()
print(endpoint)

# Point Codex at endpoint, then run Codex normally.
calls = recorder.stop()
assert isinstance(calls, list)
assert all(isinstance(call, dict) for call in calls)
json.dumps(calls)  # every returned call is JSON-compatible

start() returns the local URL to use as the provider base_url. stop() returns an ordered list[dict]. Each dictionary includes sequence, session identity (session_id, thread_id, turn_id), prompt_cache_key, the full parsed request, the fully reassembled/parsed response (events, output items, and tool calls), transport details, artifact_paths, and a non-secret backend descriptor (base_url, wire_api, profile, model, and seed).

For typed access, use an isolated snapshot:

typed_calls = recorder.semantic_calls
# Equivalent compatibility method:
typed_calls = recorder.get_calls_typed()
sessions = recorder.sessions
unclassified = recorder.unclassified_calls

The seed is retained as reproducibility metadata. CodexProbe does not use it to alter or inject fields into a passthrough request body.

Configuration boundary

The accepted public names are:

Public name	Internal field	Meaning
base_url or upstream	upstream	Absolute HTTP(S) backend URL
listen_host or host	host	Local bind host
listen_port or port	port	Local bind port, 1 through 65535
wire_api	wire_api	Currently only responses
auth	auth	None or an environment-variable reference
log_dir	log_dir	Capture directory
seed	seed	Optional integer metadata

The internal compatibility fields compatibility_profile, tool_choice_policy, target_model, and thinking_mode are also accepted for the local adapter. Unknown keys and conflicting aliases are rejected. URLs cannot contain query strings, fragments, user-info, or control characters.

Auth is deliberately reference-only; a literal secret is not accepted or stored:

"auth": {"env_var": "OPENAI_API_KEY", "scheme": "Bearer"}

When configured, the environment value is resolved only while constructing a live upstream request. It replaces every incoming Authorization header with one header. With auth=None, incoming authorization is passed through for the request but is not written to logs, metadata, returned calls, or repr output.

Default/OpenAI-compatible walkthrough

1. Install the package with python -m pip install -e ..
2. Start a recorder using the dictionary above (or run python examples/quickstart.py).
3. Put the printed endpoint in Codex's provider configuration. Use the endpoint exactly as printed; do not add a second proxy or hard-code a machine path.

model_provider = "codexprobe"

[model_providers.codexprobe]
name = "CodexProbe"
base_url = "http://127.0.0.1:8135"
wire_api = "responses"
env_key = "OPENAI_API_KEY"

The env_key line tells Codex where its normal credential comes from. It is not a credential value. Run Codex normally, for example:

codex exec "summarize this repository"

While Codex runs, CodexProbe forwards each request to the configured backend and writes artifacts below logs/openai-passthrough/. After Codex exits, call recorder.stop() and inspect the returned dictionaries or the JSON files. The complete request body includes the system/developer instructions, running message/tool history, and tool schema; the response contains the complete captured stream after semantic reassembly.

The same code can target another OpenAI-compatible server by changing only base_url (and, when needed, the opt-in compatibility profile). No proxy implementation or Codex source change is required.

Architecture and streaming

Passthrough path

In the default passthrough profile, request body bytes are captured and sent unchanged to the configured backend. Response chunks are yielded directly to Codex while a second in-memory copy is retained for the recorder. Raw wire bytes and readable parsed artifacts are written separately. Hop-by-hop headers are filtered, duplicate ordinary headers are retained, and the query string is preserved for upstream routing but never copied into the safe metadata URL.

Codex request
    -> capture exact body
    -> forward to configured backend
    <- relay each SSE chunk immediately
    -> retain all chunks
    -> parse/reassemble after stream completion
    -> write semantic + session artifacts

This means the recorder observes transport rather than changing model semantics. Compressed passthrough responses retain their original wire bytes and encoding. Adapted responses are decoded incrementally before translation, while the original encoded bytes remain available in the adapter artifacts.

Local Qwen compatibility path

The active local profile is qwen3_5_exec_vllm. Codex exposes its main execution capability as a custom exec tool, while the tested vLLM/Qwen path expects a standard function tool. The narrow adapter translates the request declaration to function(exec), translates the streamed function lifecycle back to Codex custom-tool events, and correlates continuation tool output by call identity. It does not execute, repair, or rewrite generated JavaScript.

Codex Responses + custom exec
    -> CodexProbe qwen3_5_exec_vllm adapter
    -> Responses function(exec)
    -> local vLLM/Qwen
    <- function_call(exec) SSE
    -> Codex custom_tool_call SSE

Responses API and Chat Completions

Codex's current provider path in this project uses the OpenAI Responses wire format: requests are sent to /responses, and streaming responses are SSE events such as response.created, output-item deltas, tool-call events, and response.completed. Chat Completions uses a different endpoint and message shape (/chat/completions) with choices deltas and conventional function calls.

The public recorder configuration accepts only wire_api = "responses"; it does not claim to implement a Chat Completions-to-Responses conversion. Local vLLM commonly exposes Chat Completions, so the Qwen profile adapts the Responses request/tool shape at the same proxy boundary and records both the original Codex request and the translated upstream request. This distinction is why the compatibility profile is explicit rather than silently changing the default passthrough behavior.

Session logs and schema

For a run configured with log_dir = "logs/example", the flat capture files and session hierarchy look like this:

logs/example/
  0001-request.bin                 exact client request bytes
  0001-request.json                readable request, when JSON parses
  0001-response.bin                exact client-facing response bytes
  0001-response.json               parsed/reassembled Responses response
  0001-meta.json                   transport metadata and backend/seed
  0001-upstream-request.bin        adapter-only translated request bytes
  0001-upstream-request.json       adapter-only translated request JSON
  0001-upstream-response.bin       adapter-only raw backend response bytes
  0001-adapter-manifest.json       adapter decisions and identity evidence
  runs/
    run-0001/
      trace-index.json
      sessions/
        session-.../
          session.json
          threads/
            thread-.../thread.json
            turns/turn-.../turn.json
              calls/0001.json
      unclassified/calls/0002.json

Every inference call has a stable sequence number. Metadata includes method, path, safe upstream URL, status, byte counts, latency, content type/errors, the selected backend descriptor, and seed. No authorization header, auth reference, or auth value is written to these artifacts. The session trace index groups calls by Codex session, thread, and turn and records tool-call interactions for turn-by-turn replay.

Local Qwen3.5 backend

The maintained local checkpoint and served alias are exactly:

checkpoint: drawais/Qwen3.5-9B-AWQ-INT4
served model: Qwen/Qwen3.5-9B

The tested machine was approximately an NVIDIA RTX 5070 Ti Laptop GPU with 11.94 GiB VRAM, 32 GiB system RAM, and Windows with WSL2. The active profile uses the Qwen3.5 architecture, compressed-tensors/WNA16 quantization, Marlin linear kernel, 16,384-token context, one sequence, and eager execution.

In WSL, set the maintained environment variables:

export VLLM_WSL2_ENABLE_PIN_MEMORY=1
export HF_HUB_DISABLE_XET=1
export VLLM_USE_FLASHINFER_SAMPLER=0

Then start vLLM with the current profile command:

vllm serve drawais/Qwen3.5-9B-AWQ-INT4 \
  --host 0.0.0.0 \
  --port 8000 \
  --served-model-name Qwen/Qwen3.5-9B \
  --task generate \
  --dtype auto \
  --trust-remote-code \
  --quantization compressed-tensors \
  --gpu-memory-utilization 0.88 \
  --max-model-len 16384 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --enforce-eager \
  --enable-auto-tool-choice \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Point the same recorder at http://127.0.0.1:8000/v1 and opt into the profile. The profile pins the served target model without mutating the original Codex request:

config = {
    "base_url": "http://127.0.0.1:8000/v1",
    "wire_api": "responses",
    "auth": None,
    "listen_host": "127.0.0.1",
    "listen_port": 8135,
    "log_dir": "logs/qwen35",
    "seed": 0,
    "compatibility_profile": "qwen3_5_exec_vllm",
    "target_model": "Qwen/Qwen3.5-9B",
    "thinking_mode": "disabled",
}

Point Codex at the recorder with the maintained local-provider fragment:

model_provider = "codexprobe_qwen"
model = "Qwen/Qwen3.5-9B"

[model_providers.codexprobe_qwen]
name = "CodexProbe Qwen"
base_url = "http://127.0.0.1:8135"
wire_api = "responses"
requires_openai_auth = false
supports_websockets = false
request_max_retries = 0
stream_max_retries = 0
stream_idle_timeout_ms = 600000

With vLLM and the recorder running, invoke Codex normally, for example:

codex exec "inspect the repository and report whether the calculator tests pass"

Changing between OpenAI-compatible and local backends is therefore a config change. The recorder, capture schema, streaming path, and returned API stay the same.

Research result and acceptance outcome

The controlled local experiment reached the real Qwen/Qwen3.5-9B backend through the production adapter. The final run launched Codex once, produced 12 Qwen generations and 11 continuations, used no hosted fallback, and did no model-output repair. The calculator remained unchanged because the first meaningful divergence was Python emitted inside Codex's JavaScript/V8 exec contract; later tool behavior was malformed and the run timed out.

The precise supported conclusion is narrow: the Qwen models tested here did not reliably satisfy the Codex JavaScript/V8 custom-exec contract used by this experiment. This does not establish that Qwen cannot write code, that every Qwen/vLLM tool path is broken, or that another integration using a different Codex tool path would fail.

In Task 7 terms:

• PASS: the recorder captures complete ordered default-backend calls, relays streaming traffic live, reconstructs semantic Responses/tool events, organizes session logs, and swaps the configured backend to local Qwen.
• PASS WITH DOCUMENTED MODEL DIVERGENCE: the local open model was reached and recorded end-to-end, but the controlled calculator task did not complete; the exact JavaScript/V8 tool-calling/runtime divergence is documented.

Detailed immutable evidence remains under experiments/output/**. Those artifacts are historical evidence, not generated source, and are not required to run the recorder or the quickstart.

Tests and validation

Focused recorder/API tests use a local fake OpenAI-compatible server and cover config normalization, aliases, invalid inputs, auth forwarding/sanitization, dict result serialization, typed snapshots, seed metadata, passthrough, and stream reconstruction. Existing adapter, protocol, transport, response, and trace tests cover the production path.

Typical commands are:

python -m pytest tests/test_task7_api.py -q
python -m pytest tests/test_recorder.py tests/test_proxy.py -q
python -m pip check
git diff --check

Do not execute generated historical model code and do not modify experiments/output/** during validation.

Quickstart file

examples/quickstart.py is intentionally manual: it starts the recorder, prints a provider fragment, waits while the user runs Codex, then stops and prints the returned list[dict]. It contains no machine-specific paths or credentials and does not auto-launch Codex.