Files

T

thanhnvandClaude Opus 4.8 36a4812ef3 refactor(structure): promote app to repo root + remove redundant workspace cruft

Standard production layout: the OKR app (was nested under AINative_OKR_CASAN5/) is now
the repository root. No more wrapper directory.

- Promote AINative_OKR_CASAN5/* -> repo root (backend/ frontend/ packages/ apps/
  .specify/ docs/ infra/ nginx/ scripts/ + configs). Merge tool dirs: .gitea (kept the
  active deploy ci.yml, added harness-ci.yml + runbooks), .claude (agents/commands +
  launch.json), .github moved up.
- Remove redundant: 00_SUBMISSION_PACKAGE, scattered root notes (FPT_CASAN_Full.md,
  tu-tuong-casan.md, casan-tu-sinh..., casan_harness_assessment.md, source-review...,
  README_CASAN5_REFINED.md), casan-next-plans/ and optimize-docs/ (competition/planning
  artifacts — roadmap + design history preserved in git log / commit messages).
- Update all references to the old layout:
  - .gitea/workflows/{ci,harness-ci}.yml, .github/workflows/{ci,deploy}.yml:
    working-directory .; drop AINative_OKR_CASAN5/ prefix; .specify/{tests,scripts}
    -> packages/casan-harness/... (.specify/logs state kept)
  - .claude/launch.json, .gitea/*-runbook.md: path prefixes
  - CLAUDE.md, README.md: docs/input -> apps/okr/domain/input
  - policy-bundle.yaml: 8 policy paths -> packages/casan-harness/...; manifest re-signed
- secrets-scan.sh: fixture excludes -> new package/domain paths.

Full gate from the new root: PASS=64 FAIL=0 SKIP=3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-08 13:26:36 +09:00

2.9 KiB

Raw Blame History

CASAN Before/After Scorecard

Baseline

Baseline is taken from the original Input/Review/casan_harness_assessment.md.

Harness	Baseline score	Key baseline gap
H1 Context	90	Already strong
H2 Tool	75	No idempotency key / formal registry
H3 Evaluation	85	Already strong
H4 Security	20	No prompt injection scan, secret scan, sandbox
H5 Governance	25	No approval workflow, immutable audit, risk registry enforcement
H6 AgentOps	30	No cost/token/latency tracking, alerting, drift detection
H7 Orchestration	80	Strong DAG, missing full fallback / rollback

Baseline average: 57.9 / 100.

Improved CASAN4 Submission

Harness	Improved score	Evidence
H1 Context	90	Pipeline context protocol retained and extended with CASAN fields
H2 Tool	78	Unified wrapper, deterministic scripts, side-effect action taxonomy
H3 Evaluation	85	Existing review/test gates retained
H4 Security	84	Policy-backed prompt injection block, PII masking, secret redaction, output filter
H5 Governance	82	Non-interactive approval, high-risk deny-by-default, audit hash-chain verifier
H6 AgentOps	83	Trace JSON, metrics JSONL, alert log, failure preservation
H7 Orchestration	81	Boss protocol mandates H4/H5/H6 around every step; demo context covers Steps 0-13

Improved average: 88.7 / 100 for Level 4.

CASAN result: Level 4 - Automated, with remaining production-hardening work listed below.

Level 5 Readiness Add-on

Level 5 mechanism	Status
Drift detection against golden runs	Prototype implemented
Model fallback routing	Prototype implemented
Tool registry with idempotency gate	Prototype implemented
Rollback transaction record/execute	Prototype implemented
Business KPI feedback loop	Prototype implemented
Centralized cross-project governance	Local central bundle implemented
Signed policy bundle / identity provider approval	Signed policy bundle implemented; IdP integration pending
Shared harness package used by multiple projects	Registry evidence implemented
Real provider telemetry path	Importer implemented with sample provider record
Central dashboard	Local central dashboard implemented

Level 5 demonstration score: 86 / 100.

Interpretation: Level 5 is demonstrable in this local package. Full enterprise production Level 5 still requires external service integration: IdP-backed approvals, WORM/signed log storage, provider API telemetry, and hosted dashboard.

Remaining Hardening

Replace file-based audit with WORM storage or signed log store.
Replace word-count token estimate with actual provider usage.
Add tool registry with per-tool schema, idempotency keys, and permission policy.
Execute a real full OKR generation pipeline and retain per-step CASAN traces.
Add rollback transaction boundaries for deploy/database/write actions.

2.9 KiB Raw Blame History