feat(R05): tool capability registry, unified policy gateway, MCP lifecycle manager

EPIC R05 (Team Hoa) - one security/approval path for every tool call.

R05-T01 domain/tools/{tool_descriptor,tool_registry}.py
  ToolCapability (READ/WRITE/EXECUTE/NETWORK, composable) + ToolDescriptor +
  ToolRegistry, replacing three independently-maintained gating lists
  (core/tools.py::WRITE_TOOLS, code_agent.py's WRITE_TOOLS|MS365_WRITE_TOOLS,
  chat_agent.py's literal ("run_command","install_package") tuple) with one
  capability lookup.

R05-T02 infrastructure/filesystem/{file_tools,command_tools,fetch_tools,tool_context}.py
  core/tools.py's execute_tool if/elif chain split into per-concern modules.
  core/tools.py is now a strangler-fig shim: re-exports ToolContext/ToolError,
  dispatches through a {name: handler} dict built from the split modules.
  core/tools.py: 566 -> 291 lines.

R05-T03 application/conversations/tool_policy_gateway.py
  ToolPolicyGateway.allow(name, gate, payload) - capability-driven ALLOW vs
  ask-the-gate decision. Wired into both chat_agent.py::run_cowork and
  code_agent.py::run_code, replacing their separate hand-rolled checks.
  Verified equivalent to the old hardcoded sets by test.

R05-T04 (behavior change, not just refactor)
  MCP/connector tools (core/mcp_client.py, core/ext_connectors.py) reached
  chat_agent.py via extra_executor(name, args) with NO permission check at
  all. They are now tagged with a conservative default capability
  (WRITE|EXECUTE|NETWORK - no MCP tool self-declares risk) and routed through
  the SAME ToolPolicyGateway as built-ins. When "confirm before running
  commands" is on, MCP/connector calls now prompt like run_command already
  did - a real gap closed, and a user-visible change worth calling out.

R05-T05 infrastructure/mcp/mcp_source_manager.py
  McpToolSourceManager extracts the connection cache/lock/start-or-skip
  lifecycle out of state.py::AppContext (_mcp_connections/_conn_lock) into a
  standalone, directly-testable class. AppContext.build_mcp_tools and
  _ms365_builtin_connection now call ensure()/stop(); _ext_connections
  (unified Connectors) is out of scope for this task and keeps its own lock.

New tests: tests/unit/test_tool_registry_and_policy.py,
test_code_agent_tool_policy.py, test_cowork_extra_tool_policy.py,
test_mcp_source_manager.py (26 new tests).

Suite: 254 passed, 4 pre-existing failures unrelated to R05 (2 EPIC R02
config-security, 2 environment-dependent routing tests - see checklist).
check_imports: PASS. All new files < 400 LOC.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

infrastructure/filesystem/__init__.py

@@ -0,0 +1,6 @@
+"""Filesystem/process/network tool adapters split out of ``core/tools.py``
+(EPIC R05) and the sandbox execution context they share."""
+
+from .tool_context import CancelFn, ToolContext, ToolError
+
+__all__ = ["CancelFn", "ToolContext", "ToolError"]

infrastructure/filesystem/command_tools.py

@@ -0,0 +1,110 @@
+"""Command tools - run_command, install_package (R05-T02).
+
+Moved verbatim out of ``core/tools.py`` (see ``file_tools.py`` for why). These
+two are the ones today's hand-written permission gate in
+``core/chat_agent.py`` singles out by literal name
+(``name in ("run_command", "install_package")``) — R05-T03 replaces that
+tuple with a capability lookup, but the tools themselves are unchanged here.
+"""
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from typing import Any, Dict, Optional
+
+from .tool_context import CancelFn, ToolContext
+
+COMMAND_TIMEOUT = 120  # seconds
+
+_SNAPSHOT_SKIP = {".git", "__pycache__", "node_modules", ".scratch", ".venv",
+                  ".idea", ".mypy_cache", ".pytest_cache"}
+
+
+def _snapshot(workdir: Path) -> Dict[str, Any]:
+    """Map of file path -> (mtime, size) under the workdir (noise dirs skipped)."""
+    snap: Dict[str, Any] = {}
+    try:
+        for dirpath, dirnames, filenames in os.walk(str(workdir)):
+            dirnames[:] = [d for d in dirnames if d not in _SNAPSHOT_SKIP]
+            for fn in filenames:
+                full = os.path.join(dirpath, fn)
+                try:
+                    st = os.stat(full)
+                    snap[full] = (st.st_mtime_ns, st.st_size)
+                except OSError:
+                    pass
+            if len(snap) > 5000:
+                return snap
+    except OSError:
+        pass
+    return snap
+
+
+def _sandbox_python(ctx: ToolContext, cancel: Optional[CancelFn] = None,
+                     on_output=None) -> Optional[str]:
+    """Lazily create/reuse this ctx's project sandbox venv (Code tab only —
+    ``ctx.sandbox``); returns its python path, or None to use the app's own."""
+    if not ctx.sandbox:
+        return None
+    from cowork_local.core.deps import ensure_project_venv
+
+    py = ensure_project_venv(ctx.workdir, cancel=cancel, on_output=on_output)
+    return str(py) if py else None
+
+
+def run_command(ctx: ToolContext, args: Dict[str, Any],
+                 cancel: Optional[CancelFn] = None,
+                 on_output=None) -> Dict[str, Any]:
+    from cowork_local.core.deps import network_blocked_env, run_cancellable, sandbox_env
+    from cowork_local.core.sandbox_manager import ExecutionConfig, SandboxManager
+    from cowork_local.security.command_risk_classifier import classify_command
+
+    command = str(args.get("command", "")).strip()
+    if not command:
+        return {"ok": False, "output": "Empty command."}
+
+    # --- Security validation pipeline ---
+    risk = classify_command(command, is_cowork_mode=ctx.flatten_writes)
+    if risk.blocked:
+        denial = "Command blocked by security policy: " + "; ".join(risk.reasons)
+        return {"ok": False, "output": denial}
+
+    # Route through SandboxManager for risk-based isolation
+    mgr = SandboxManager(ExecutionConfig(
+        enabled=True,
+        block_network_by_default=ctx.block_network,
+        is_cowork_mode=ctx.flatten_writes,
+    ))
+    sandbox_result = mgr.run(
+        command=command,
+        workdir=str(ctx.workdir),
+        block_network=ctx.block_network,
+        timeout_sec=COMMAND_TIMEOUT,
+        cancel=cancel,
+    )
+    # Sandbox ALWAYS executes (never double-run). Return its result directly.
+    if sandbox_result.get("sandbox") == "blocked":
+        return {"ok": False, "output": sandbox_result.get("stderr", "Command blocked")}
+    out = sandbox_result.get("stdout", "").strip() or "(no output)"
+    err = sandbox_result.get("stderr", "")
+    rc = sandbox_result.get("returncode", -1)
+    if err:
+        out = f"{out}\n{err}" if out else err
+    return {"ok": sandbox_result.get("ok", False), "output": f"[exit {rc}]\n{out}"}
+
+
+def install_package(ctx: ToolContext, args: Dict[str, Any],
+                     cancel: Optional[CancelFn] = None,
+                     on_output=None) -> Dict[str, Any]:
+    from cowork_local.core.deps import pip_install
+
+    package = str(args.get("package", "")).strip()
+    if not package:
+        return {"ok": False, "output": "No package specified."}
+    python = _sandbox_python(ctx, cancel, on_output)
+    ok, detail = pip_install(package, cancel=cancel, on_output=on_output, python=python)
+    head = f"Installed {package}." if ok else f"Could not install {package}."
+    return {"ok": ok, "output": f"{head}\n{detail}"}
+
+
+__all__ = ["COMMAND_TIMEOUT", "run_command", "install_package", "_snapshot"]

infrastructure/filesystem/fetch_tools.py

@@ -0,0 +1,55 @@
+"""Fetch tools - fetch_url, jira_search, jira_get_issue (R05-T02).
+
+Moved verbatim out of ``core/tools.py`` (see ``file_tools.py`` for why). The
+network access these three carry is exactly what the ``ToolCapability.NETWORK``
+tag added in R05-T01/domain/tools/tool_registry.py describes.
+"""
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from .tool_context import ToolContext
+
+
+def fetch_url(ctx: ToolContext, args: Dict[str, Any]) -> Dict[str, Any]:
+    """Fetch a URL's text content (web page / online document / SharePoint-
+    OneDrive share link) via link_fetch — the same parser task-link attachments
+    use. Honors the Sandbox Security Layer's "Block network" policy."""
+    url = str(args.get("url", "")).strip()
+    if not url:
+        return {"ok": False, "output": "fetch_url: 'url' is required."}
+    if not url.lower().startswith(("http://", "https://")):
+        return {"ok": False, "output": f"fetch_url: not an http(s) URL: {url}"}
+    if not ctx.allow_url_fetch:
+        return {"ok": False,
+                "output": ("fetch_url: URL fetching is turned off in Settings → Security "
+                           "(\"Allow the agent to fetch URLs\").")}
+    # A pasted Jira issue link on the CONNECTED Jira host is read via the
+    # authenticated API (so private issues resolve, not a login page). Public
+    # links / any other URL fall through to the normal fetcher below.
+    from cowork_local.core import jira_tool
+    if jira_tool.is_jira_issue_url(ctx.jira, url):
+        return {"ok": True, "output": jira_tool.get_issue_by_url(ctx.jira, url)}
+    from cowork_local.core.link_fetch import fetch_link_preview
+
+    return {"ok": True, "output": fetch_link_preview(url)}
+
+
+def jira_search(ctx: ToolContext, args: Dict[str, Any]) -> Dict[str, Any]:
+    from cowork_local.core import jira_tool
+
+    out = jira_tool.search(ctx.jira, str(args.get("jql", "")),
+                           int(args.get("max_results", 25) or 25))
+    return {"ok": not out.lower().startswith(("jira is not configured", "jira search failed")),
+            "output": out}
+
+
+def jira_get_issue(ctx: ToolContext, args: Dict[str, Any]) -> Dict[str, Any]:
+    from cowork_local.core import jira_tool
+
+    out = jira_tool.get_issue(ctx.jira, str(args.get("key", "")))
+    return {"ok": not out.lower().startswith(("jira is not configured", "could not fetch")),
+            "output": out}
+
+
+__all__ = ["fetch_url", "jira_search", "jira_get_issue"]

infrastructure/filesystem/file_tools.py

@@ -0,0 +1,136 @@
+"""File tools - read_file, list_dir, write_file, edit_file (R05-T02).
+
+Moved verbatim out of ``core/tools.py``, whose ``execute_tool`` used to
+dispatch to these via a hand-written if/elif chain over every tool name it
+knew about. Splitting the built-in handlers into per-concern modules
+(this one, ``command_tools.py``, ``fetch_tools.py``) means adding a tool no
+longer means growing that one function; ``core/tools.py::execute_tool`` now
+looks the name up in a dict built from these modules instead.
+
+Behavior is unchanged from before the split - this is a pure move, not a
+rewrite. Every existing characterization/contract test that exercises
+read_file/write_file/edit_file/list_dir through ``core.tools.execute_tool``
+still exercises the exact same code, just imported from here.
+"""
+from __future__ import annotations
+
+import ast
+from pathlib import Path
+from typing import Any, Dict
+
+from .tool_context import ToolContext
+
+MAX_READ_BYTES = 200_000
+
+
+def _flatten_rel(rel: str) -> str:
+    """Collapse a sub-folder path down to a bare filename so the file lands in the
+    workdir root — EXCEPT the ``.scratch`` sandbox subtree, which is preserved.
+
+    Used by the Cowork agent (flatten_writes=True) so it can never create a
+    per-session / per-chat / per-task output sub-folder: every deliverable stays
+    directly in the single configured Output folder."""
+    parts = Path(rel).parts
+    if parts and parts[0] == ".scratch":
+        return rel  # temporary sandbox is allowed (and cleaned up afterwards)
+    return Path(rel).name or rel
+
+
+def _check_python_syntax(target: Path, content: str) -> str:
+    """Return a short warning if ``content`` is invalid Python, else ''.
+
+    Catches syntax errors the instant a .py file is written/edited — before the
+    agent wastes a whole run_command round-trip just to get the same error back
+    from a traceback."""
+    if target.suffix.lower() not in (".py", ".pyw"):
+        return ""
+    try:
+        ast.parse(content, filename=str(target))
+        return ""
+    except SyntaxError as exc:
+        return f"\n⚠ Syntax error at line {exc.lineno}: {exc.msg} — fix this before running the file."
+
+
+def read_file(ctx: ToolContext, args: Dict[str, Any]) -> Dict[str, Any]:
+    target = ctx.resolve(str(args.get("path", "")))
+    if not target.exists():
+        return {"ok": False, "output": f"File not found: {args.get('path')}"}
+    data = target.read_bytes()[:MAX_READ_BYTES]
+    text = data.decode("utf-8", errors="replace")
+    return {"ok": True, "output": text}
+
+
+def list_dir(ctx: ToolContext, args: Dict[str, Any]) -> Dict[str, Any]:
+    rel = str(args.get("path", ".") or ".")
+    target = ctx.resolve(rel)
+    # A missing/not-yet-created path is NOT a tool failure — report it as an
+    # ordinary result so the agent can create it or pick another path and keep
+    # going. Returning ok=False here surfaced a false "tool failed: list_dir" in
+    # Co4E flows and could stall a step on a recoverable situation.
+    if not target.exists():
+        return {"ok": True, "output": f"(path '{rel}' does not exist yet — create it or use another path)"}
+    if target.is_file():
+        return {"ok": True, "output": f"('{rel}' is a file, not a directory)"}
+    entries = []
+    for child in sorted(target.iterdir(), key=lambda p: (p.is_file(), p.name.lower())):
+        marker = "/" if child.is_dir() else ""
+        entries.append(f"{child.name}{marker}")
+    return {"ok": True, "output": "\n".join(entries) or "(empty folder)"}
+
+
+def write_file(ctx: ToolContext, args: Dict[str, Any]) -> Dict[str, Any]:
+    rel = str(args.get("path", ""))
+    if ctx.flatten_writes:
+        rel = _flatten_rel(rel)
+    target = ctx.resolve(rel)
+    content = str(args.get("content", ""))
+    target.parent.mkdir(parents=True, exist_ok=True)
+    # A .xlsx is a binary package — build a REAL workbook from the content
+    # (CSV/TSV/Markdown-table/JSON) rather than writing raw text (which corrupts it).
+    if target.suffix.lower() in (".xlsx", ".xlsm"):
+        from cowork_local.core import xlsx_write
+        if xlsx_write.build_xlsx_from_text(target, content):
+            return {"ok": True, "path": str(target),
+                    "output": f"Wrote spreadsheet {rel} ({target.name})."}
+        return {"ok": False, "output": "Could not build the .xlsx (openpyxl unavailable) — "
+                "write a .csv instead, or use a generator script."}
+    target.write_text(content, encoding="utf-8")
+    warning = _check_python_syntax(target, content)
+    return {"ok": True, "path": str(target),
+            "output": f"Wrote {len(content)} chars to {rel}.{warning}"}
+
+
+def edit_file(ctx: ToolContext, args: Dict[str, Any]) -> Dict[str, Any]:
+    """Replace an exact snippet inside an existing file (precise patch edit)."""
+    rel = str(args.get("path", ""))
+    if ctx.flatten_writes:
+        rel = _flatten_rel(rel)
+    target = ctx.resolve(rel)
+    if not target.exists():
+        return {"ok": False,
+                "output": f"File not found: {rel} — use write_file to create it."}
+    old = str(args.get("old_string", ""))
+    new = str(args.get("new_string", ""))
+    replace_all = bool(args.get("replace_all", False))
+    if not old:
+        return {"ok": False, "output": "old_string is empty — provide the exact text to replace."}
+    try:
+        text = target.read_text(encoding="utf-8", errors="replace")
+    except OSError as exc:
+        return {"ok": False, "output": f"Could not read file: {exc}"}
+    count = text.count(old)
+    if count == 0:
+        return {"ok": False, "output": ("old_string not found. Read the file and copy the exact "
+                                        "text to replace, including indentation/whitespace.")}
+    if count > 1 and not replace_all:
+        return {"ok": False, "output": (f"old_string appears {count} times — add surrounding "
+                                        "context to make it unique, or set replace_all=true.")}
+    updated = text.replace(old, new) if replace_all else text.replace(old, new, 1)
+    target.write_text(updated, encoding="utf-8")
+    n = count if replace_all else 1
+    warning = _check_python_syntax(target, updated)
+    return {"ok": True,
+            "output": f"Edited {args.get('path')} ({n} replacement{'' if n == 1 else 's'}).{warning}"}
+
+
+__all__ = ["MAX_READ_BYTES", "read_file", "list_dir", "write_file", "edit_file"]

infrastructure/filesystem/tool_context.py

@@ -0,0 +1,62 @@
+"""ToolContext / ToolError / CancelFn - the sandboxed execution context every
+built-in tool runs against (moved out of ``core/tools.py`` in R05-T02).
+
+Kept as its own leaf module (no dependency on any sibling in this package) so
+``file_tools.py``, ``command_tools.py`` and ``fetch_tools.py`` can each import
+it without creating an import cycle back through ``core/tools.py``, which
+itself re-exports ``ToolContext``/``ToolError`` from here for the existing
+callers (``core/chat_agent.py``, ``core/code_agent.py``,
+``core/task_executors.py``) that do ``from .tools import ToolContext``.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Callable, Dict, Optional
+
+CancelFn = Callable[[], bool]
+
+
+class ToolError(Exception):
+    pass
+
+
+@dataclass
+class ToolContext:
+    workdir: Path
+    flatten_writes: bool = False  # Cowork: force every write into the workdir root
+    sandbox: bool = False  # Code tab: isolate run_command/install_package into <workdir>/.venv
+    # Sandbox Security Layer — Settings' "Resource Limits" (cpu_percent/memory_mb/
+    # disk_mb), applied to every run_command/install_package this context runs.
+    # None (default) = no limits, matching pre-existing behavior.
+    resource_limits: Optional[Dict[str, float]] = None
+    # Sandbox Security Layer — Settings' "Block network for agent commands"
+    # (policy-level, see deps.py::network_blocked_env). False (default) =
+    # unrestricted, matching pre-existing behavior.
+    block_network: bool = False
+    # Whether the fetch_url tool may read URLs — SEPARATE from block_network
+    # (reading a web page/share link for info is safe; running networked shell
+    # commands is the risk). Defaults True; set from agent_security.allow_url_fetch.
+    allow_url_fetch: bool = True
+    # Jira read connector config (base_url/email/api_token) — None disables the
+    # jira_* tools' ability to connect. Populated from config.data["jira"].
+    jira: Optional[Dict[str, Any]] = None
+
+    def resolve(self, rel: str) -> Path:
+        """Resolve ``rel`` inside the workdir, rejecting escapes."""
+        if rel in ("", "."):
+            return self.workdir
+        candidate = (self.workdir / rel).expanduser()
+        try:
+            resolved = candidate.resolve()
+        except OSError as exc:
+            raise ToolError(f"Invalid path: {rel} ({exc})")
+        root = self.workdir.resolve()
+        if resolved != root and root not in resolved.parents:
+            raise ToolError(
+                f"Refused: '{rel}' is outside the working folder ({root})."
+            )
+        return resolved
+
+
+__all__ = ["CancelFn", "ToolError", "ToolContext"]

infrastructure/mcp/__init__.py

@@ -0,0 +1,5 @@
+"""MCP server connection lifecycle management (EPIC R05)."""
+
+from .mcp_source_manager import McpToolSourceManager
+
+__all__ = ["McpToolSourceManager"]

infrastructure/mcp/mcp_source_manager.py

@@ -0,0 +1,113 @@
+"""McpToolSourceManager - the MCP server connection lifecycle, extracted out
+of ``state.py::AppContext`` (R05-T05).
+
+Today ``AppContext.build_mcp_tools`` inlines all of this: a ``_mcp_connections``
+dict, a ``_conn_lock`` guarding check-then-create against concurrent turns (a
+Cowork tab, a Co4E flow and a Scheduled Task can all call it at once), and a
+"start it, cache it, skip it on failure" loop repeated for both the
+admin-configured servers AND the built-in MS365 server
+(``_ms365_builtin_connection``). None of that logic touches Qt; it was only
+ever inline because ``AppContext`` is where the config lived.
+
+This class owns the SAME cache/lock/start-or-skip behavior as a standalone,
+directly testable object — ``AppContext`` becomes a thin caller (one instance
+per app, same as it holds one ``RoutingApplicationService``).
+
+Pure Python: no Qt. It DOES touch the network/filesystem via
+``core.mcp_client.McpServerConnection`` (a subprocess + asyncio loop), which is
+exactly what makes it infrastructure rather than domain.
+"""
+from __future__ import annotations
+
+import threading
+from typing import Dict, List, Optional
+
+from cowork_local.core.mcp_client import McpServerConnection
+
+
+class McpToolSourceManager:
+    """Caches and supervises one :class:`McpServerConnection` per server name.
+
+    ``connection_factory`` defaults to ``McpServerConnection`` itself; tests
+    substitute a fake so no real subprocess is spawned (see
+    ``tests/unit/test_mcp_source_manager.py``).
+    """
+
+    def __init__(self, connection_factory=McpServerConnection) -> None:
+        self._connections: Dict[str, McpServerConnection] = {}
+        self._lock = threading.Lock()
+        self._connection_factory = connection_factory
+
+    def ensure(self, name: str, command: str, args: Optional[List[str]] = None,
+               env: Optional[Dict[str, str]] = None) -> Optional[McpServerConnection]:
+        """Return a live connection for ``name``, starting one if there is
+        none cached or the cached one's subprocess has died.
+
+        Serialized under one lock so two turns racing to build their tool
+        list at the same moment share one subprocess per server instead of
+        each spawning their own (the bug this replaces:
+        ``AppContext._conn_lock``'s original docstring). Returns ``None`` -
+        never raises - when the server fails to start, matching the existing
+        "one broken server must not block the turn" behavior.
+        """
+        with self._lock:
+            existing = self._connections.get(name)
+            if existing is not None and existing.is_alive():
+                return existing
+            if existing is not None:
+                self._connections.pop(name, None)
+            connection = self._connection_factory(name, command, args or [], env)
+            try:
+                connection.start()
+            except Exception:  # noqa: BLE001 - one broken server must not block the turn
+                return None
+            self._connections[name] = connection
+            return connection
+
+    def get(self, name: str) -> Optional[McpServerConnection]:
+        """The cached connection for ``name``, without starting one."""
+        return self._connections.get(name)
+
+    def is_alive(self, name: str) -> bool:
+        connection = self._connections.get(name)
+        return connection is not None and connection.is_alive()
+
+    def restart(self, name: str, command: str, args: Optional[List[str]] = None,
+                env: Optional[Dict[str, str]] = None) -> Optional[McpServerConnection]:
+        """Force a fresh connection for ``name`` even if the cached one still
+        looks alive - for a server the caller knows is misbehaving."""
+        with self._lock:
+            self._connections.pop(name, None)
+        return self.ensure(name, command, args, env)
+
+    def stop(self, name: str) -> None:
+        """Stop and forget one connection - used when a server becomes
+        unavailable by configuration (e.g. MS365 signed out) rather than by
+        crashing."""
+        with self._lock:
+            connection = self._connections.pop(name, None)
+        if connection is not None:
+            try:
+                connection.stop()
+            except Exception:  # noqa: BLE001 - shutdown must never raise into the caller
+                pass
+
+    def active(self) -> List[McpServerConnection]:
+        """Every currently cached connection - what
+        ``core/mcp_client.py::build_mcp_tools`` merges tool specs from."""
+        return list(self._connections.values())
+
+    def stop_all(self) -> None:
+        """Terminate every connection's subprocess - called on app shutdown
+        so none of them linger as orphan processes."""
+        with self._lock:
+            connections = list(self._connections.values())
+            self._connections.clear()
+        for connection in connections:
+            try:
+                connection.stop()
+            except Exception:  # noqa: BLE001
+                pass
+
+
+__all__ = ["McpToolSourceManager"]