Complete cloud provider patch MVP

2026-07-06 12:32:54 +09:00
parent c7a1c743cb
commit d0bde2a8b3
8 changed files with 168 additions and 24 deletions
@@ -0,0 +1,55 @@
+# CASAN PLAN 03 — Cloud Provider Patch (OpenAI/Anthropic)
+
+> Status 2026-07-06: **MVP implemented + deterministic tests added**. The router
+> can call OpenAI/Anthropic when real API keys are present, while local tests
+> verify endpoint pinning, provider usage parsing, and fail-closed malformed
+> payload handling without requiring paid keys.
+
+## Delivered
+
+| Capability | Where | Verification |
+|---|---|---|
+| Model spec prefixes `openai:<model>` and `anthropic:<model>` | `.specify/scripts/bash/model-call.py` | `phase3-model-router-tests.sh` |
+| Endpoint allowlist/SSRF posture | Cloud hosts are hard-pinned to `api.openai.com` and `api.anthropic.com`; Ollama remains `127.0.0.1:11434` only | router SSRF test |
+| Honest unavailable state | Missing `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` exits non-zero; no fake PASS or fake token usage | router cloud-unavailable test |
+| Real provider token usage path | OpenAI `usage.prompt_tokens/completion_tokens`; Anthropic `usage.input_tokens/output_tokens` | deterministic monkeypatched cloud parser test |
+| Malformed provider payload fail-closed | Missing usage/schema exits non-zero via `provider_usage_invalid` | deterministic parser test |
+| Provider telemetry source tags | Usage rows write `openai_api_real_tokens` or `anthropic_api_real_tokens` | `model-call.py` provider log path |
+
+## How to run live when keys exist
+
+```bash
+cd AINative_OKR_CASAN5
+export CASAN_MODEL_PRIMARY=openai:gpt-4o-mini
+export OPENAI_API_KEY=...
+bash .specify/scripts/bash/model-router.sh /tmp/prompt.txt /tmp/out.json --role classify
+
+export CASAN_MODEL_PRIMARY=anthropic:claude-sonnet-4-5
+export ANTHROPIC_API_KEY=...
+bash .specify/scripts/bash/model-router.sh /tmp/prompt.txt /tmp/out.json --role judge
+```
+
+Expected evidence:
+
+- command exits `0` for well-formed provider output;
+- `/tmp/out.json` has non-zero `input_tokens` and `output_tokens`;
+- `.specify/logs/level5/provider-usage.jsonl` gets a row with provider
+  `openai` or `anthropic` and cost source `*_api_real_tokens`;
+- malformed or unreachable provider exits non-zero and does not fabricate a
+  successful verdict.
+
+## Remaining production work
+
+| Priority | Work | Done when |
+|---|---|---|
+| P1 | Run live smoke with real org keys on the target Mac/CI | Evidence file records real provider response, token counts, and no secret leakage |
+| P2 | Add provider cost lookup table for selected models | `cost_usd` is computed from current provider pricing instead of `0.0` |
+| P3 | Wire real provider billing usage API, not only per-call response usage | H6 reconcile uses OpenAI/Anthropic ground truth APIs with schema-versioned fetchers |
+| P4 | Add model allowlist policy for approved cloud models | Unknown cloud model names require approval or block |
+| P5 | Decide default failover order for local -> cloud or cloud -> local | `model-fallback.sh` policy is explicit per role |
+
+## Notes for Plan-02
+
+Plan-02 source generation should call `model-router.sh --role generate`, not
+provider SDKs directly. This keeps H4 prompt handling, H6 telemetry, endpoint
+allowlisting, and provider usage logging in one path.