Self-contained, full-screen-projectable HTML visualizing the base → competition
→ now journey: OLED-glass design, 3-milestone timeline, 7 harness bento cards
(color-coded H1–H7 with each file's role), 5 upgrade-track tables (Track A /
C-MVP / Evidence / H5+ / H6+) with gap-closed columns, honest-maturity panel +
one-line flow diagram. UTF-8 standalone doc (opens directly in a browser),
IntersectionObserver scroll reveals, custom cubic-bezier motion, mobile fallback.
Verified rendering in preview. Cross-linked from both architecture md files.
Adds .claude/launch.json (static server for local preview of optimize-docs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two companion docs tracing the .specify/scripts evolution from the 5-file
spec-kit scaffold shown in the file tree:
- CASAN_ARCHITECTURE_BEFORE.md — state brought to the competition (freeze
fbcef96): base 5 → 37 scripts implementing all 7 harnesses, each file's
purpose grouped by H1–H7, + the 8 competition test suites. Honest maturity:
demo/PoC (~3.0/5).
- CASAN_ARCHITECTURE_AFTER.md — the feat/plan07-track-a-hardening upgrades:
37 → 60 scripts (+23) grouped by Track A / Track C-MVP / Evidence Pack /
H5+ / H6+, each new file's purpose + the gap it closes, notes on in-place
modifications (strict fail-closed, cost caps, KMS rotate, window breaker),
+ the 6 new test suites (+96 checks). Honest maturity: Level 4 proven by
attack (~4.0/5), not full production.
File counts verified against git (ls-tree fbcef96 vs HEAD).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deep-dive defense ammo tied to the video scenes so a presenter can field
follow-up questions live. For each of the 26 attacks across the 3 videos:
plain-language what-it-is, why-dangerous, how-CASAN-blocks (flagging which
layer is deterministic vs the 3-4 AI scenes), likely challenge question +
answer, and the honest limit. Opens with 3 "mantra" lines that cover most
hard questions and closes with a 10-toughest-questions cheat sheet.
Link it from CASAN_SCRIPT_2VIDEO.md. Numbers kept consistent: V1 = 23 vectors,
V2 = 140 checks, V3 = 175 checks; ~90% of controls are deterministic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cross-layer showpiece stops ONE attack at 4 sequential checkpoints
(injection block → tool-input schema reject → runaway timeout → signed audit),
which span H4 + the Tool layer (H2) + H5 — not "four harness layers". That
contradicted the intro's "this video focuses on 3 layers (H4/H5/H6)". Reframe
the close as one attack caught at each successive stage, listing the 4 concrete
checkpoints, so it's accurate to the demo without mislabeling them as harnesses.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add an explicit difficulty/depth progression the presenter can say out loud:
- Video 1 = basic & classic attacks (everyone must block these)
- Video 2 = advanced evasion + proof-you-can-trust (not just harder attacks —
half of it is production maturity: fail-closed, evidence pack, FP budget)
- Video 3 = deepest gaps — real governance & ops hardening
Update each video's subtitle and the quick-reference table with a Level column.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Rewrite Video 1 (attack battery) and Video 2 (Track A + C-MVP + Evidence Pack)
in plain, presentation-friendly language: each scene now has a 【what's on screen】
cue + 🎙️ spoken line, with everyday analogies (sealed ledger, fresh signature,
key-in-a-vault) so a non-technical viewer follows while the terminal video plays.
- Add Video 3 covering the feat/plan07-track-a-hardening work that was NOT in the
recorded videos: H5+ governance (approval-identity, KMS-in-vault, external WORM)
and H6+ AgentOps (live alerting + dead-letter, provider-API cost reconciliation,
stale-aware hosted dashboard, sliding-window breaker). Framed as the
"lift the weakest link" arc, ending on the honest 175/0 · Level 4 · not-yet-full-
production close.
- Add a quick-reference table + 60-second highlight cut for presenters.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Close the three gaps the scoring report itself flagged for H6 plus V15,
each as a real MVP + fail-able adversarial test (same pattern that lifted H5):
- D1 alert-dispatch.sh: alerts POST to a real HTTP webhook (severity routing,
dedup window, retry) + dead-letter queue with redelivery; fail-loud in strict.
Wired into agent-metrics.sh so a failing step pages live end-to-end.
- D2 provider-usage-fetch.sh + telemetry-reconcile.sh: pull usage from a provider
usage HTTP API (all-or-nothing schema gate, fail-loud) + reconcile local vs
provider ground truth — token under-reporting/hidden runs => TELEMETRY_DISCREPANCY.
- D3 dashboard-serve.sh + dashboard-server.py: serve the dashboard over HTTP with
a stale-aware /healthz probe (fresh=200 ok, telemetry silent-death=503 stale).
- D4 circuit-breaker-check.sh: sliding-window failure-rate breaker (V15) — interleaved
successes no longer evade the consecutive-failure breaker (CIRCUIT_OPEN_WINDOW).
New suite phase-h6-agentops-tests.sh: 20/20, all live against local HTTP endpoints
(webhook sink, mock provider API, dashboard server) — deterministic, no model needed.
Also fix sign-policy-bundle.sh key-sync invariant: the local-fallback branch only
exported policy-public.pem when generating a NEW key, so a Vault-DOWN run after a
Vault-signed run verified a local-key signature against the Vault pubkey (RSA padding
error, run-casan4 died mid-suite). Now always re-exports the pubkey before signing —
same fix class as tool-audit-lib.sh / governance-check.sh.
Full battery re-run sequentially: 175/175 PASS, 0 FAIL across 8 suites
(KMS SKIP this run — Vault down; validated live 2026-07-04). Docs synced:
scoring-run-report (H6 79→80, no harness below 80, 155→175), CASAN_HARDENING_STATUS
(Phase 5 D1–D4), Plan-07, submission README, and run-hardening.sh (H6+ scenes HO1–HO4).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One plain-language closing paragraph per video for time-constrained delivery:
Video 1 (attacks blocked, proven by exit codes) and Video 2 (production
hardening + signed evidence pack). Jargon-light, keeps both closers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CASAN_SCRIPT_2VIDEO.md — tight summary voiceover for just the two demo videos,
one screen each: Video 1 (battery: H4/H5/H6 + chain) and Video 2 (hardening
Track A + C-MVP + Evidence Pack). Strips the per-scene cues/notes/soundbites of
the full narration; keeps the spoken beats and both closers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CASAN_VIDEO_NARRATION.md — spoken lines timed to each on-screen scene of
run-all.sh (Video 1) and run-hardening.sh (Video 2). Golden rule: terminal says
WHAT, narrator says WHY — never read the screen. Per-scene ▶cue / 🎙️line / ⏸pause
markers, hero-scene emphasis (A5, B1, B6, D1, chain, HA1, HA4, HE3 money-shot),
clustered narration for fast scenes, delivery notes (tone/pace/verdict timing),
a soundbite bank for judge Q&A, and a duration table (~13-16 min). Aligned with
existing claims: Level 4 proven, AI as optional escalation, sandbox scaffold
honesty, "casan-old wins a demo; casan5 survives production".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- CASAN_ATTACK_CATALOG.md: one-page visual map of ALL ~38 attack scenes from
run-all.sh (A1-A9, B1-B8, D1-D6, chain) + run-hardening.sh (HA1-7, HC1-4,
HE1-4). Legend for 12 attack directions (input/output/artifact/tool-out/audit/
telemetry/cost/action/supply/exfil/runtime/evidence), a defense-in-depth
diagram, per-scene matrix with control + verdict + AI marker.
- CASAN_ATTACK_PLAYBOOK.md: internal deep-dive — per attack: scenario, why
dangerous, exact blocking mechanism, AI-or-deterministic, verify command +
expected result, threat-model ref. Prominent AI-usage answer: only A3/A8/D4
invoke the model; everything else is deterministic. Semantic AI is an optional
escalation that only ADDS a block, never removes one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cloud branch of model-call.py was a stub (cloud_backend_not_implemented,
failed even with a key set); casan-step.mjs gated the judge on a hard-coded
Ollama ping. Wire up the real cloud path so a model can run without Ollama.
- model-call.py: add call_openai() and call_anthropic() (raw urllib, no new
dependency — matches the existing call_ollama). Endpoints hard-pinned to the
SSRF allowlist; keys read from env, never logged. Anthropic sends no
temperature/thinking (rejected as 400 on Opus 4.8/4.7; omitting thinking
keeps the terse one-word classify/judge answer). main() routes by
ollama:/openai:/anthropic: prefix; key-unset still fails closed honestly.
provider-usage.jsonl cost_source is per-backend, keeping ollama's exact
"ollama_local_real_tokens" tag that evidence/tests key on.
- casan-step.mjs: ollamaAvailable() -> modelAvailable() — when
CASAN_MODEL_PRIMARY is a cloud spec with its key set, the judge runs through
the cloud path; otherwise it pings local Ollama as before. Default
(unset CASAN_MODEL_PRIMARY) is unchanged.
- CASAN_MASTER_RUNBOOK.md: update sections 0/1/4/7/8 — cloud is now
implemented (not a stub); keep the honest "untested with a real key" +
CA-cert caveats.
Not verified against a live API key (none available); confirmed key-set makes
a real HTTPS call and key-unset fails closed. Gates unchanged:
security-gate PASS=11 FAIL=0, adversarial PASS=44 FAIL=0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add REAL=1 to the video-steps demo so attack vectors flow through the real
production entry-point instead of calling sub-scripts directly.
- run-all.sh: REAL=1 feeds each H4 vector (A1/A2/A4/A6/A7 + cross-layer step
1) as the INPUT of an agent step run through casan-harness.sh, so the
BLOCK/PASS verdict is produced by the wrapper itself (H4-in -> H5 -> H6 ->
exec -> H4-out) exactly as when the real pipeline meets malicious input.
After the battery it runs a real pipeline slice (STEP1 okr.srs via
casan-harness.sh -- node casan-step.mjs) and shows audit.jsonl growing by a
real record. An inline inventory documents which vectors intentionally keep
calling a single control directly (artifact-scan, audit tamper/re-forge,
detectors on synthetic telemetry) and why. Default mode (no REAL) unchanged.
- map-live.sh: show the PIPELINE (STEP1) row only under REAL=1, driven by a
mode sidecar file written by run-all.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hướng A — scorecard.sh (video demo):
- h5_1 approval workflow: hardcode 0 → governance-check deploy live (approval_required)
- h6_2 hallucination rate: hardcode 0 → hallucination-scan phân biệt dirty>clean live
- "N/5 mục" chuyển từ text cứng sang đếm động
- H4/H5/H6 → 100/100 (5/5 gate live), Average 57.9 → 90.0
Hướng B — run-casan-pipeline.mjs:
- fallback: stub 'exit 9' → 'cat /nonexistent' (real failure, nhất quán adversarial T3)
- drift: giữ so fallback-output vs golden (clean run=1.0); năng lực phát hiện
drift thật chứng minh ở adversarial suite
- Full 12-step run verify: H1 CONTEXT_VALID=24, H2 tool-audit records=25 signed,
H5 audit-chain records=22 signed, H6 provider_telemetry per-step thật, H7 rollback real
phase3-real-run-scoring.md: giải thích vì sao scorecard cũ cho H5=60/H6=80
(hardcode), phân biệt scorecard-90 vs re-score-84 (2 mục đích khác nhau).
Verify: adversarial 44/0, security-gate 11/0/0, pipeline 12 steps OK.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
thanhnv