Files

T

thanhnvandClaude Opus 4.8 36a4812ef3 refactor(structure): promote app to repo root + remove redundant workspace cruft

Standard production layout: the OKR app (was nested under AINative_OKR_CASAN5/) is now
the repository root. No more wrapper directory.

- Promote AINative_OKR_CASAN5/* -> repo root (backend/ frontend/ packages/ apps/
  .specify/ docs/ infra/ nginx/ scripts/ + configs). Merge tool dirs: .gitea (kept the
  active deploy ci.yml, added harness-ci.yml + runbooks), .claude (agents/commands +
  launch.json), .github moved up.
- Remove redundant: 00_SUBMISSION_PACKAGE, scattered root notes (FPT_CASAN_Full.md,
  tu-tuong-casan.md, casan-tu-sinh..., casan_harness_assessment.md, source-review...,
  README_CASAN5_REFINED.md), casan-next-plans/ and optimize-docs/ (competition/planning
  artifacts — roadmap + design history preserved in git log / commit messages).
- Update all references to the old layout:
  - .gitea/workflows/{ci,harness-ci}.yml, .github/workflows/{ci,deploy}.yml:
    working-directory .; drop AINative_OKR_CASAN5/ prefix; .specify/{tests,scripts}
    -> packages/casan-harness/... (.specify/logs state kept)
  - .claude/launch.json, .gitea/*-runbook.md: path prefixes
  - CLAUDE.md, README.md: docs/input -> apps/okr/domain/input
  - policy-bundle.yaml: 8 policy paths -> packages/casan-harness/...; manifest re-signed
- secrets-scan.sh: fixture excludes -> new package/domain paths.

Full gate from the new root: PASS=64 FAIL=0 SKIP=3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-08 13:26:36 +09:00

3.1 KiB

Raw Blame History

CASAN Refined Re-Assessment - AINative_OKR_CASAN5

Generated: 2026-06-28

Summary

This package merges the stable CASAN4 submission with the strongest additions from AINative_OKR_Claude_GHCP, then fixes the regressions found during review.

Positioning:

CASAN Level 4: achieved.
Level 5: stronger local demonstration, not full enterprise production.
Evidence score: 88.7 / 100.

Scorecard

Harness	CASAN4	Refined	Delta	Reason
H1 Context	90	91	+1	Context schema now exposes Level 5 paths for tool registry, drift, rollback, and AgentOps evidence.
H2 Tool	84	88	+4	Tool registry expanded, side-effect idempotency tested, per-call audit added, cache hit no longer bypasses CASAN trace generation.
H3 Evaluation	85	86	+1	Hallucination checklist added for review gates.
H4 Security	88	90	+2	Jailbreak and private key tests added; private key leakage bug fixed with explicit regex and `grep --`.
H5 Governance	87	88	+1	Risk policy and registry auto-update guidance added; audit chain remains verified.
H6 AgentOps	89	90	+1	JSON alert format and tool-call audit evidence added.
H7 Orchestration	87	88	+1	Refined wrapper preserves evidence on cache hit and keeps rollback/drift/fallback evidence.
Average	87.1	88.7	+1.6	Stronger Level 5 evidence path with verified regression fixes.

Verified Improvements

The test harness now checks:

Prompt injection block.
PII masking.
Jailbreak block.
Private key material block.
High-risk deny-by-default.
Explicit approval path.
Tool-call audit log.
Cache hit still records fresh CASAN traces.
Audit hash-chain verification.
Drift, fallback, tool registry, rollback, KPI, signed policy, provider telemetry, harness reuse, and central dashboard evidence.

Regressions Fixed from AINative_OKR_Claude_GHCP

Issue	Fix
Bash scripts were not executable in the submitted folder	New folder is copied from CASAN4 baseline with executable bits preserved.
Private key regex caused `grep` option parsing and leaked private key text	`security-check.sh` now uses `grep --` and explicit private key, DB connection string, and AWS key regexes.
Idempotency cache bypassed H4/H5/H6 on cache hit	`casan-harness.sh` now runs H4 input and H5 before cache use, records H6 cache-copy metrics, and runs H4 output.
Tool registry did not write central tool-call audit	`tool-registry-gate.sh` now writes `.specify/logs/audit/tool-calls.jsonl`.
Submission package still referenced CASAN4 paths	Paths were updated to `AINative_OKR_CASAN5`.

Claim Boundary

This package is stronger than CASAN4 for hackathon evidence. It should still be described as:

Level 4 achieved; Level 5 demonstrated locally with verifiable controls.

Do not claim full production Level 5 until these are connected to enterprise services:

IdP-backed approval and separation of duties.
WORM or external signed log storage.
Live provider telemetry APIs.
Hosted dashboard with real-time ingestion.
Shared harness package reused across independent production projects.

3.1 KiB Raw Blame History