Files

T

thanhnvandClaude Opus 4.8 4918012199 docs(plans): sync roadmap status + paths to post-restructure state

- Plan-01 marked ✅ DONE (INDEX table + P3 tier + BACKLOG row + plan header).
- Plan-06 / Plan-12 dependency on 01 satisfied → 🔓 unblocked (headers + BACKLOG rows).
- Repoint command/path refs in all plans (except Plan-01's migration narrative):
  .specify/{scripts,tests,security,config,templates,governance} -> packages/casan-harness/...;
  golden-runs/traceability-map/docs-input -> apps/okr/domain/...; `cd AINative_OKR_CASAN5`
  -> `cd $(git rev-parse --show-toplevel)`; fix relative links + Plan-13 control-plane location.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-08 14:57:01 +09:00

4.6 KiB

Raw Blame History

CASAN PLAN 03 — Cloud Provider Patch (OpenAI/Anthropic)

Status 2026-07-06: MVP implemented + deterministic tests added. The router can call OpenAI/Anthropic when real API keys are present, while local tests verify endpoint pinning, provider usage parsing, and fail-closed malformed payload handling without requiring paid keys.

Delivered

Capability	Where	Verification
Model spec prefixes `openai:<model>` and `anthropic:<model>`	`packages/casan-harness/scripts/bash/model-call.py`	`phase3-model-router-tests.sh`
Endpoint allowlist/SSRF posture	Cloud hosts are hard-pinned to `api.openai.com` and `api.anthropic.com`; Ollama remains `127.0.0.1:11434` only	router SSRF test
Honest unavailable state	Missing `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` exits non-zero; no fake PASS or fake token usage	router cloud-unavailable test
Real provider token usage path	OpenAI `usage.prompt_tokens/completion_tokens`; Anthropic `usage.input_tokens/output_tokens`	deterministic monkeypatched cloud parser test
Malformed provider payload fail-closed	Missing usage/schema exits non-zero via `provider_usage_invalid`	deterministic parser test
Provider telemetry source tags	Usage rows write `openai_api_real_tokens` or `anthropic_api_real_tokens`	`model-call.py` provider log path

How to run live when keys exist

cd "$(git rev-parse --show-toplevel)"  # repo root (app promoted here)
export CASAN_MODEL_PRIMARY=openai:gpt-4o-mini
export OPENAI_API_KEY=...
bash packages/casan-harness/scripts/bash/model-router.sh /tmp/prompt.txt /tmp/out.json --role classify

export CASAN_MODEL_PRIMARY=anthropic:claude-sonnet-4-5
export ANTHROPIC_API_KEY=...
bash packages/casan-harness/scripts/bash/model-router.sh /tmp/prompt.txt /tmp/out.json --role judge

Expected evidence:

command exits 0 for well-formed provider output;
/tmp/out.json has non-zero input_tokens and output_tokens;
.specify/logs/level5/provider-usage.jsonl gets a row with provider openai or anthropic and cost source *_api_real_tokens;
malformed or unreachable provider exits non-zero and does not fabricate a successful verdict.

Remaining production work

Priority	Work	Done when
P1	Run live smoke with real org keys on the target Mac/CI	Evidence file records real provider response, token counts, and no secret leakage
P2	Add provider cost lookup table for selected models	`cost_usd` is computed from current provider pricing instead of `0.0`
P3	Wire real provider billing usage API, not only per-call response usage	H6 reconcile uses OpenAI/Anthropic ground truth APIs with schema-versioned fetchers
P4	Add model allowlist policy for approved cloud models	Unknown cloud model names require approval or block
P5	Decide default failover order for local -> cloud or cloud -> local	`model-fallback.sh` policy is explicit per role

Điều kiện gỡ blocker API key (để P02 đợt B–D không treo vô thời hạn)

Live smoke (P1 ở trên) và Plan-02 đợt B–D đều chờ cùng một điều kiện: có API key provider thật. Ghi rõ để bất kỳ AI/người kế tiếp gỡ được:

Hạng mục	Quyết định cần chốt
Ai cấp key	Org owner cấp `OPENAI_API_KEY` và/hoặc `ANTHROPIC_API_KEY` (ít nhất 1 provider đủ để mở đường).
Nạp ở đâu	Chỉ qua biến môi trường / secret store của runner CI — không commit vào repo, không ghi vào audit dạng plaintext (H5 mask).
Model tối thiểu	`openai:gpt-4o-mini` hoặc `anthropic:claude-*-mini` là đủ cho smoke (rẻ, đủ chứng minh đường sống).
Evidence phải lưu	1 file smoke ghi: exit `0`, token thật > 0 trong `/tmp/out.json`, 1 dòng `provider-usage.jsonl` với source `*_api_real_tokens`, và xác nhận không rò secret.
Khi CHƯA có key	Giữ nguyên trạng thái: deterministic offline test là bằng chứng hợp lệ; không claim live provider. Không tạo fake PASS/fake token.

Chốt: P02 đợt B–D có thể phát triển & test offline bằng deterministic mock; chỉ bước xác nhận live mới cần key. Vì vậy thiếu key không chặn việc mở rộng source-gen — chỉ chặn claim "đã chạy live provider thật".

Notes for Plan-02

Plan-02 source generation should call model-router.sh --role generate, not provider SDKs directly. This keeps H4 prompt handling, H6 telemetry, endpoint allowlisting, and provider usage logging in one path.

4.6 KiB Raw Blame History Unescape Escape