Nam Pham Dinh ThanhandClaude Sonnet 4.6
838b2473b6
WV4-A: Added 16 Vitest/RTL tests to frontend (jsdom env, fail-before proof verified) WV4-B: Created 12 stub traces for pipeline retention gap; fixed MSYS2/Python path mismatch in context-validate.sh; run-casan4-harness-tests.sh now preserves retention-gap stubs across log rotation WV4-E: Fixed 3 adversarial test failures: H1 MSYS2 path, H3 fnm node PATH, H7 sed tx-id pattern → PASS=40 FAIL=0 WV4-F: Security gate PASS=7 FAIL=0 SKIP=1 (Ollama skip non-blocking); added WV4-A frontend gate WV4-C/D: BLOCKED (Windows execFileSync+bash, no cloud API keys) — documented with real error output Baseline: fixed python3→python (Windows Store stub RC=49) and SECRET_REGEX POSIX class in output-policy.yaml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
66 lines
3.4 KiB
Plaintext
66 lines
3.4 KiB
Plaintext
===== H4: prompt-injection bypass resistance =====
|
|
PASS: H4 blocks whitespace-padded injection (rc=2)
|
|
PASS: H4 blocks leetspeak injection (rc=2)
|
|
PASS: H4 blocks synonym injection (rc=2)
|
|
PASS: H4 blocks forget-variant injection (rc=2)
|
|
PASS: H4 blocks uppercase injection (rc=2)
|
|
===== H4: secret material must not pass as input =====
|
|
PASS: H4 blocks private key input (rc=2)
|
|
PASS: H4 blocks DB connection string input (rc=2)
|
|
===== H4: output mode fails closed on secret material =====
|
|
PASS: H4 fails closed on secret in output (rc=2)
|
|
===== H4: benign content must pass (no false positives) =====
|
|
PASS: H4 allows benign spec text (rc=0)
|
|
===== H5: separation of duties =====
|
|
PASS: H5 denies self-approval (actor==approver) (rc=2)
|
|
PASS: H5 allows distinct approver (rc=0)
|
|
===== H5: audit chain re-forge is detected =====
|
|
PASS: H5 verifies the genuine signed chain (rc=0)
|
|
PASS: H5 rejects a re-forged chain (signature anchor) (rc=1)
|
|
===== H2: per-agent least privilege =====
|
|
PASS: H2 denies unauthorized agent for deploy (rc=2)
|
|
PASS: H2 denies missing agent identity for deploy (rc=2)
|
|
PASS: H2 allows authorized agent with key (rc=0)
|
|
===== H2: tool-registry gate is in the execution line of fire =====
|
|
PASS: H2 wrapper aborts side-effect for unauthorized agent (rc=2)
|
|
PASS: H2 wrapper allows side-effect for authorized agent (rc=0)
|
|
===== H2: tool-call audit re-forge is detected =====
|
|
PASS: H2 verifies the genuine tool audit (rc=0)
|
|
PASS: H2 rejects a re-forged tool audit (rc=1)
|
|
===== H6: hallucination detection is populated =====
|
|
PASS: H6 populates hallucination_signals (count=4)
|
|
PASS: H6 reports 0 signals for clean output
|
|
===== PUSH-TO-90: H7 real rollback (genuine undo, not a marker) =====
|
|
PASS: H7 rollback genuinely restores the file
|
|
===== PUSH-TO-90: H7 real drift (two different artifacts, not cp-of-self) =====
|
|
PASS: H7 drift detects real difference (similarity=0.661 < 1.0)
|
|
PASS: H7 drift passes identical artifacts
|
|
===== PUSH-TO-90: H7 fallback triggered by a REAL primary failure =====
|
|
PASS: H7 fallback runs after a genuine primary failure
|
|
===== PUSH-TO-90: H2 runtime rate limit (deploy capped at 2/run) =====
|
|
PASS: H2 denies 3rd deploy in one run (rate limit)
|
|
===== PUSH-TO-90: H2 tool-input schema validation =====
|
|
PASS: H2 schema accepts valid tool input
|
|
PASS: H2 schema rejects malformed tool input (rc=2)
|
|
===== PUSH-TO-90: H4 tool-execution timeout =====
|
|
PASS: H4 kills a runaway tool call (rc=124)
|
|
PASS: H4 allows a fast tool call (rc=0)
|
|
===== PUSH-TO-90: H1 context path validation =====
|
|
PASS: H1 context-validate passes when artifact exists
|
|
PASS: H1 context-validate catches a missing artifact (rc=2)
|
|
===== PUSH-TO-90: H5 signing private key is OFF-REPO =====
|
|
PASS: H5 private signing key absent from repo
|
|
===== WAVE 3: H4 indirect artifact injection (WP-S7) =====
|
|
PASS: H4 artifact-scan blocks injected content in artifacts
|
|
PASS: H4 artifact-scan passes clean artifacts
|
|
===== WAVE 3: H4 secrets scan — no leaked keys (WP-S4) =====
|
|
PASS: H4 secrets scan passes (no committed .env or private keys)
|
|
===== WAVE 3: H4 circuit breaker — no bypass patterns (WP-S6) =====
|
|
PASS: H4 no bypass patterns; circuit breaker closed
|
|
===== WAVE 3: H4 tool-exec.sh wired into harness — kills runaway via harness =====
|
|
PASS: H4 tool-exec timeout fires through casan-harness.sh
|
|
===== WAVE 3: H3 judge gate fail-before (WP-B) =====
|
|
PASS: H3 judge gate T1-T4 all pass (fail-before and fix cycle)
|
|
|
|
===== ADVERSARIAL SUMMARY: PASS=40 FAIL=0 =====
|