thanhnvandClaude Opus 4.8
36a4812ef3
Standard production layout: the OKR app (was nested under AINative_OKR_CASAN5/) is now
the repository root. No more wrapper directory.
- Promote AINative_OKR_CASAN5/* -> repo root (backend/ frontend/ packages/ apps/
.specify/ docs/ infra/ nginx/ scripts/ + configs). Merge tool dirs: .gitea (kept the
active deploy ci.yml, added harness-ci.yml + runbooks), .claude (agents/commands +
launch.json), .github moved up.
- Remove redundant: 00_SUBMISSION_PACKAGE, scattered root notes (FPT_CASAN_Full.md,
tu-tuong-casan.md, casan-tu-sinh..., casan_harness_assessment.md, source-review...,
README_CASAN5_REFINED.md), casan-next-plans/ and optimize-docs/ (competition/planning
artifacts — roadmap + design history preserved in git log / commit messages).
- Update all references to the old layout:
- .gitea/workflows/{ci,harness-ci}.yml, .github/workflows/{ci,deploy}.yml:
working-directory .; drop AINative_OKR_CASAN5/ prefix; .specify/{tests,scripts}
-> packages/casan-harness/... (.specify/logs state kept)
- .claude/launch.json, .gitea/*-runbook.md: path prefixes
- CLAUDE.md, README.md: docs/input -> apps/okr/domain/input
- policy-bundle.yaml: 8 policy paths -> packages/casan-harness/...; manifest re-signed
- secrets-scan.sh: fixture excludes -> new package/domain paths.
Full gate from the new root: PASS=64 FAIL=0 SKIP=3.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
65 lines
1.8 KiB
Python
Executable File
65 lines
1.8 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""CASAN H6 hallucination signal detector.
|
|
|
|
Reads the quoted keyword markers from hallucination-tracking.yaml plus a set of
|
|
generic uncertainty markers, scans the agent output, and reports how many
|
|
hallucination signals were found. This turns hallucination-tracking.yaml from
|
|
dead config into a real, populated metric written to metrics.jsonl.
|
|
|
|
Usage: hallucination-scan.py <hallucination-tracking.yaml> <output-file>
|
|
Output (stdout): line 1 = integer signal count, line 2 = JSON list of matches.
|
|
"""
|
|
import json
|
|
import re
|
|
import sys
|
|
|
|
GENERIC_MARKERS = [
|
|
"maybe", "might be incorrect", "i am not sure", "uncertain",
|
|
"i think", "probably", "as far as i know",
|
|
]
|
|
|
|
|
|
def load_keywords(path):
|
|
keywords = []
|
|
try:
|
|
with open(path, encoding="utf-8") as fh:
|
|
for line in fh:
|
|
# Quoted list items are the hallucination keyword markers
|
|
# (unquoted list items are structured signal names, not text).
|
|
m = re.match(r'\s*-\s*"(.+)"\s*$', line)
|
|
if m:
|
|
keywords.append(m.group(1))
|
|
except OSError:
|
|
pass
|
|
return keywords
|
|
|
|
|
|
def main():
|
|
if len(sys.argv) < 3:
|
|
print(0)
|
|
print("[]")
|
|
return
|
|
keywords = load_keywords(sys.argv[1]) + GENERIC_MARKERS
|
|
try:
|
|
with open(sys.argv[2], encoding="utf-8") as fh:
|
|
text = fh.read().lower()
|
|
except OSError:
|
|
print(0)
|
|
print("[]")
|
|
return
|
|
matched = []
|
|
for kw in keywords:
|
|
k = kw.lower()
|
|
if not k:
|
|
continue
|
|
count = text.count(k)
|
|
if count:
|
|
matched.append({"marker": kw, "count": count})
|
|
total = sum(m["count"] for m in matched)
|
|
print(total)
|
|
print(json.dumps(matched))
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|