# Prompt For Claude — CASAN Phase 3 Wave 1 Real Core Improvements

You are working inside this repo:

`/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5`

Read these files first:

- `docs/output/casan/phase3-push-to-90-plan.md`
- `docs/output/casan/phase3-security-hardening-plan.md`
- `docs/output/casan/phase3-preflight.md`
- `docs/output/casan/app-evidence/MANIFEST.md`
- `docs/output/output_logs/001-okr-web-app/pipeline-context.yaml`

Execute Phase 3 Wave 1 as real engineering work. Do not produce demo-grade controls. Do not claim Phase 3 is complete.

## Preflight Facts To Respect

- This is a git repo on branch `main`.
- Current preflight has untracked Phase 3 plan/evidence files.
- Mac runtime: Node v24.12.0, npm 11.7.0, Python 3.9.0.
- Linux server runtime: Ubuntu 26.04, Node v22.23.1, npm 11.17.0, Python 3.14.4, Ollama 0.30.11.
- Ollama runs on Linux server `192.168.1.5`, not directly on Mac.
- Mac accesses Linux Ollama through SSH tunnel: `http://127.0.0.1:11434`.
- Primary local model is `ollama:ornith:9b`.
- `ornith:9b` digest: `a75697c145891910e312c95e4a9fc1ccb8653e5ef543b23b0403a4665b82fd91`.
- `gemma4:12b` is also available but is not the declared primary.
- Cloud env is unavailable: `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` are unset.
- Current captured `npm test` and `npm run build` failed because `prisma: command not found`.
- Current `npm audit --workspaces --audit-level=high` failed with 5 high and 7 moderate vulnerabilities.

Before implementing, resolve or clearly document the local dependency issue:

```bash
npm install
npm test
npm run build
```

Capture refreshed logs under:

`docs/output/casan/phase3-evidence/wave1/`

If tests/build still fail, stop implementation and write a blocker report. Do not build Phase 3 on a broken baseline unless the failure is explicitly scoped and accepted.

## Absolute Integrity Rules

- Do not fake PASS, APPROVED, token usage, model verdicts, cost, drift, rollback, or audit results.
- Do not hardcode expected PASS results.
- Do not recycle sample token/cost numbers.
- Every test you add must be able to fail.
- Do not use `.specify/tests/generate-casan-demo-context.py`.
- Do not call regex-only logic semantic or model-based.
- Do not claim cloud/hybrid judging; current run is local-only unless real cloud keys are present.
- Do not claim H5 WORM/KMS; AWS/KMS/Object Lock is not configured.
- Do not claim target scores as achieved. Only write achieved scores after a real audit.
- Keep evidence under `docs/output/casan/phase3-evidence/wave1/`.
- If infrastructure is missing, write `BLOCKED` or `SKIPPED` with a reason. Do not mark it pass.

## Core Modification Policy

For Phase 3, you are explicitly allowed and expected to modify core CASAN harness files when that is the correct engineering path.

This includes files under:

- `.specify/scripts/bash/*`
- `.specify/tests/*`
- `scripts/run-casan-pipeline.mjs`
- `scripts/casan-step.mjs`

Do not avoid core changes by adding superficial wrappers if the real control belongs in the core harness.

This is not a demo-grade improvement task. Implement real controls in the actual execution path. If a claimed H1-H7 improvement is not wired into the path that the harness/pipeline actually uses, it does not count.

Rules for core changes:

- Do not weaken existing security, governance, tool-audit, rollback, drift, or agent-metrics checks.
- Do not edit tests to hide failures.
- Do not loosen adversarial tests.
- Do not replace real command execution with canned output.
- Do not hardcode PASS/APPROVED/verdict/token/cost values.
- Every core behavior change must include fail-able evidence:
  1. demonstrate the old weakness or failing case,
  2. implement the fix,
  3. demonstrate the case now fails closed or passes correctly.
- If a core script changes, run relevant existing harness checks afterward:
  - `bash .specify/tests/run-casan4-harness-tests.sh`
  - `bash .specify/tests/adversarial-harness-tests.sh`
  - `bash .specify/scripts/bash/verify-audit-chain.sh`
  - `bash .specify/scripts/bash/verify-tool-audit.sh`
- If a test cannot run because of environment constraints, record that as a blocker or limitation, not a pass.

For every core harness modification, document in `docs/output/casan/phase3-wave1-results.md`:

- file changed,
- behavior changed,
- threat/control improved,
- tests/evidence proving the change,
- compatibility result with existing harness tests.

## Required Environment

Assume operator has opened this tunnel in another terminal:

```bash
ssh -N -L 11434:127.0.0.1:11434 thanhnv@192.168.1.5
```

Use:

```bash
export CASAN_MODEL_BACKEND=local
export CASAN_MODEL_PRIMARY=ollama:ornith:9b
export CASAN_MODEL_SECONDARY=ollama:ornith:9b
```

Verify before model work:

```bash
curl -sS http://127.0.0.1:11434/api/tags
curl -sS http://127.0.0.1:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{"model":"ornith:9b","prompt":"Return exactly: OK","stream":false}'
```

## Wave 1 Scope

Implement the foundation and minimum defensible security layer, wired into real execution paths.

### A. Model Router Foundation

Create or modify as needed:

- `.specify/scripts/bash/model-router.sh`
- `.specify/scripts/bash/model-call.py`
- any core script that must call the router for the control to be real

Interface:

```bash
.specify/scripts/bash/model-router.sh <prompt-file> <out-json> [--role classify|judge|generate]
```

Required behavior:

- Support `ollama:<model>` via `POST http://127.0.0.1:11434/api/generate`.
- Use `ornith:9b` as the primary local model.
- Cloud backends may be implemented only if real env keys exist; otherwise report unavailable honestly.
- Output JSON must include:
  - `text`
  - `model_id`
  - `role`
  - `input_tokens`
  - `output_tokens`
  - `latency_ms`
  - `temperature`
  - `route`
  - `verdict` when role is `classify` or `judge`
- For Ollama, use real `prompt_eval_count` and `eval_count`.
- Append real usage to `.specify/logs/level5/provider-usage.jsonl`.
- Do not recycle sample values.
- On backend/model failure, return nonzero with a clear error. Do not emit a fake fallback success.

### B. WP-S1 Router Hardening

Implement hardening in the real model path:

- Delimit untrusted content:
  `<<<UNTRUSTED>>> ... <<<END_UNTRUSTED>>>`
- `classify` output must be exactly `INJECTION` or `SAFE`.
- `judge` output must be exactly `APPROVED` or `REJECTED`.
- Any malformed model output must fail closed.
- Never log API keys, secrets, raw Authorization headers, or `.env` contents.
- For future cloud egress, mask PII before sending. Since cloud keys are unset now, test this path as unavailable or with a local dry-run payload capture.
- Enforce endpoint allowlist:
  - `api.anthropic.com`
  - `api.openai.com`
  - `127.0.0.1:11434`
- Reject arbitrary model URLs and metadata IP attempts.
- Use `temperature=0` for classify/judge.
- Write router audit/usage records without leaking prompt secrets.

### C. Wire The Router Into Real Harness Paths

Do not leave the router as an unused utility.

For Wave 1, wire at least one real path:

1. H6 usage path:
   - Ensure a real model-router call can feed `.specify/logs/level5/provider-usage.jsonl`.
   - Ensure `agent-metrics.sh` can consume real provider telemetry for a matching step where available.

2. H7 fallback path:
   - Replace fake fallback triggers like `bash -c "exit 9"` in `scripts/run-casan-pipeline.mjs` with a real failing model-router primary route, then fallback to `ollama:ornith:9b` only if the model is available.
   - If the model is unavailable, record H7 fallback as blocked, not passed.

3. Optional H4/H3 path if feasible in Wave 1:
   - Wire classify/judge into `security-check.sh` or `casan-step.mjs` only if you can prove fail-before/fix/pass in this turn.
   - If not feasible, document it as Wave 2 work. Do not claim H4/H3 model-based improvement.

### D. Fail-able Router Tests

Create or modify:

- `.specify/tests/phase3-model-router-tests.sh`

Tests must cover:

1. `ollama:ornith:9b` generate works through tunnel, or fails with clear reason if tunnel unavailable.
2. Router records real `input_tokens` and `output_tokens` from Ollama.
3. Classifier/judge malformed output fails closed.
4. SSRF-like endpoint/model config is rejected.
5. Fake API key pattern does not appear in `.specify/logs`.
6. Cloud backend reports unavailable honestly while keys are unset.
7. A deliberate failing primary route causes fallback only through the real router path, not `exit 9`.

No hardcoded PASS.

### E. Context Validation

Create or modify:

- `.specify/scripts/bash/context-validate.sh`
- `scripts/run-casan-pipeline.mjs` if needed to add `generated_at` or invoke validation in the real pipeline

Behavior:

- Default context:
  `docs/output/output_logs/001-okr-web-app/pipeline-context.yaml`
- Accept optional context path arg.
- Fail if any `artifact:` path is missing.
- Fail if any `trace_file:` path is missing.
- Warn if entries have no `generated_at`.
- If adding `generated_at`, do it in the real `appendContext` path.

Create fail/pass evidence:

- copy context to temp,
- inject one missing artifact path,
- show validator exits nonzero,
- run validator on real context and show pass.

### F. Supply Chain Baseline

Current `npm audit --workspaces --audit-level=high` fails with 5 high vulnerabilities.

For Wave 1:

- Do not claim supply-chain gate green.
- Either fix high vulnerabilities without breaking app/test/build, or record them as a blocker for WP-S3.
- If you run `npm audit fix` or upgrade major dependencies, run full app tests/build afterward and capture logs.

### G. Wave 1 Results

Create:

- `docs/output/casan/phase3-wave1-results.md`

Include:

- what was implemented,
- what was not implemented,
- infrastructure used,
- exact commands run,
- exact evidence files,
- real test results,
- core files modified and why,
- current blockers,
- remaining work for WP-A/B/C/D/E/F/G and WP-S2..S8,
- explicit statement that no average >90 score is claimed yet.

## Required Commands And Evidence

Capture logs under:

`docs/output/casan/phase3-evidence/wave1/`

Run and capture:

```bash
npm install
npm test
npm run build
bash .specify/tests/phase3-model-router-tests.sh
bash .specify/scripts/bash/context-validate.sh docs/output/output_logs/001-okr-web-app/pipeline-context.yaml
```

If any core harness script changed, also run and capture:

```bash
bash .specify/tests/run-casan4-harness-tests.sh
bash .specify/tests/adversarial-harness-tests.sh
bash .specify/scripts/bash/verify-audit-chain.sh
bash .specify/scripts/bash/verify-tool-audit.sh
```

If baseline is green and model tunnel is available, run and capture:

```bash
node scripts/run-casan-pipeline.mjs
```

## Deliverables

- Working Wave 1 code.
- Real evidence logs under `docs/output/casan/phase3-evidence/wave1/`.
- `docs/output/casan/phase3-wave1-results.md`.
- No fabricated evidence.
- No target score claimed as achieved.

If blocked, stop and write a precise blocker report instead of manufacturing evidence.