feat(R03): unify provider catalogue, routing decisions and usage telemetry
EPIC R03 (Team Duy) - one provider catalogue, one routing flow, one usage seam.
R03-T01 tests/contracts/test_providers.py
29 contract tests every provider must satisfy: canonical assistant message,
streamed text == returned content, reasoning never joins the answer, parsed
tool arguments, ProviderError for every failure. Real adapters exercised
offline by stubbing Provider._request.
R03-T02 domain/models/provider_descriptor.py
infrastructure/providers/provider_registry.py
Provider facts declared once (was split across providers/factory.py,
DEFAULT_CONFIG and PROVIDER_LABELS). ProviderRegistry.build() also stamps the
descriptor id onto the instance, so ollama/github_copilot/codex usage is no
longer all attributed to "openai_compat", and never mutates the caller config.
R03-T03 application/model_routing/routing_application_service.py
Pure-Python routing policy with four modes: Off, Auto, Manual and the new
Fallback (switch only AFTER the current model fails). Depends on a RoutingPort
protocol; production wires the existing core.routing engine underneath.
R03-T04/T05 ui/chat_panel.py, ui/co4e_tab.py, ui/folder_tab.py
Three near-identical routing copies (~40 lines each) replaced by a call to
ctx.routing_application() plus a confirm callback. Mode vocabulary now lives
in one place (normalize_mode/is_valid_mode) instead of four literal tuples.
R03-T06 infrastructure/telemetry/usage_sink.py
Token usage extracted from both providers into UsageEvent + UsageEventSink.
Estimation pinned against core.usage_tracker so no recorded number changes.
Also fixes a deadlock introduced while wiring AppContext: routing_application()
held _routing_lock and called routing(), which takes the same non-reentrant lock.
Suite: 186 passed, 1.22s. check_imports: PASS. All new files < 400 LOC.
2 pre-existing failures remain in test_config_security.py (EPIC R02/Team Nam).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,12 @@
|
|||||||
|
"""Application layer - pure Python use-case orchestration.
|
||||||
|
|
||||||
|
Sits between ``presentation/`` (Qt widgets) and ``domain/`` (entities). A module
|
||||||
|
here answers "what has to happen, in what order" for one use case - route a
|
||||||
|
turn, run a conversation - without knowing whether a human, a scheduler or a
|
||||||
|
test triggered it.
|
||||||
|
|
||||||
|
Hard rule (ADR-001 I1/I3, enforced by ``scripts/check_imports.py``): no
|
||||||
|
PySide6/PyQt imports and no reach into ``presentation/``/``ui/``. Results travel
|
||||||
|
back up through plain-Python callbacks; turning those into Qt signals is the
|
||||||
|
presentation layer's job.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""Conversation use case: the lifecycle of one agent turn (EPIC R04)."""
|
||||||
|
|
||||||
|
from .conversation_application_service import ConversationApplicationService
|
||||||
|
|
||||||
|
__all__ = ["ConversationApplicationService"]
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
"""Model routing use case: pick the best-fit model for one turn (EPIC R03)."""
|
||||||
|
|
||||||
|
from .routing_application_service import (
|
||||||
|
RoutingApplicationService,
|
||||||
|
RoutingDecision,
|
||||||
|
RoutingMode,
|
||||||
|
is_valid_mode,
|
||||||
|
normalize_mode,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = ["RoutingApplicationService", "RoutingDecision", "RoutingMode",
|
||||||
|
"normalize_mode", "is_valid_mode"]
|
||||||
@@ -0,0 +1,353 @@
|
|||||||
|
"""RoutingApplicationService - one routing flow for every surface (R03-T03).
|
||||||
|
|
||||||
|
Before this service, the same routing algorithm existed three times:
|
||||||
|
|
||||||
|
* ``ui/chat_panel.py::_apply_routing`` (Cowork chat)
|
||||||
|
* ``ui/co4e_tab.py::_apply_co4e_routing`` (Co4E studio)
|
||||||
|
* ``ui/folder_tab.py::_ai_apply_routing`` (AI-Edit)
|
||||||
|
|
||||||
|
The three copies had already drifted - each one resolves the "current model"
|
||||||
|
differently and each one has its own private notion of what to do when the user
|
||||||
|
declines - and every one of them lives inside a Qt widget, so none of the logic
|
||||||
|
could be tested without building a window.
|
||||||
|
|
||||||
|
This module is the single implementation. It is pure Python: no Qt import, no
|
||||||
|
config access, no network. The presentation layer supplies a confirm callback
|
||||||
|
and renders the notice; everything else happens here.
|
||||||
|
|
||||||
|
Modes (:class:`RoutingMode`)
|
||||||
|
----------------------------
|
||||||
|
* ``OFF`` - never switch. The user's pinned model always wins.
|
||||||
|
* ``AUTO`` - switch silently when the best candidate clears the gain threshold.
|
||||||
|
* ``MANUAL`` - propose the switch and switch only if the confirm callback approves.
|
||||||
|
* ``FALLBACK`` - never switch pre-emptively; switch only AFTER the current model
|
||||||
|
fails, to the next-best candidate. This is the mode a user wants when they
|
||||||
|
trust their own model choice but still want the turn to survive an outage.
|
||||||
|
|
||||||
|
Migration note (ADR-001 section 4): the scoring/ranking engine is NOT rewritten.
|
||||||
|
This service depends on the small :class:`RoutingPort` interface, and production
|
||||||
|
wires the existing, already-tested ``core.routing.service.RoutingService`` into
|
||||||
|
it. Tests wire a fake.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Callable, List, Optional, Protocol, Sequence, Tuple
|
||||||
|
|
||||||
|
|
||||||
|
class RoutingMode(str, Enum):
|
||||||
|
"""Per-surface routing behaviour.
|
||||||
|
|
||||||
|
The first three values match ``core.routing.models.SwitchMode`` string for
|
||||||
|
string, so a mode read from the existing config round-trips unchanged.
|
||||||
|
"""
|
||||||
|
|
||||||
|
OFF = "off"
|
||||||
|
AUTO = "auto"
|
||||||
|
MANUAL = "manual"
|
||||||
|
FALLBACK = "fallback"
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def parse(cls, raw: Any) -> "RoutingMode":
|
||||||
|
"""Best-effort parse of a config value.
|
||||||
|
|
||||||
|
Unknown or empty values become ``OFF``: routing is an optimisation, and
|
||||||
|
the safe reading of a corrupt setting is "leave the user's model alone"
|
||||||
|
rather than "silently move their work to another model".
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return cls(str(raw or "off").strip().lower())
|
||||||
|
except ValueError:
|
||||||
|
return cls.OFF
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RoutingDecision:
|
||||||
|
"""The outcome of routing one turn - an immutable instruction for the caller.
|
||||||
|
|
||||||
|
``provider``/``model`` are ALWAYS filled with what the turn should actually
|
||||||
|
run on, switched or not, so a call site never has to re-derive the fallback
|
||||||
|
itself (the bug that made the three UI copies diverge).
|
||||||
|
"""
|
||||||
|
|
||||||
|
mode: RoutingMode
|
||||||
|
provider: str
|
||||||
|
model: str
|
||||||
|
switched: bool = False
|
||||||
|
task_type: str = ""
|
||||||
|
score_gain: float = 0.0
|
||||||
|
reason: str = ""
|
||||||
|
declined: bool = False # Manual mode: a switch was offered and refused
|
||||||
|
# What the turn would have run on without routing. Carried so the Manual
|
||||||
|
# confirm dialog can show "from X to Y" without re-deriving the current
|
||||||
|
# model itself - re-deriving it differently per screen is exactly how the
|
||||||
|
# three legacy copies drifted apart.
|
||||||
|
previous_provider: str = ""
|
||||||
|
previous_model: str = ""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def should_notify(self) -> bool:
|
||||||
|
"""True when the UI should show the "switched model" notice - i.e. only
|
||||||
|
when a switch really happened."""
|
||||||
|
return self.switched
|
||||||
|
|
||||||
|
def target(self) -> Tuple[str, str]:
|
||||||
|
"""``(provider, model)`` to run this turn on."""
|
||||||
|
return self.provider, self.model
|
||||||
|
|
||||||
|
@property
|
||||||
|
def from_model(self) -> str:
|
||||||
|
"""Candidate key (``provider/model``) of the model being switched away
|
||||||
|
from, or "" when nothing was selected yet.
|
||||||
|
|
||||||
|
Named to match ``core.routing.models.SwitchDecision`` so the existing
|
||||||
|
Manual-mode dialog (``ui/routing_toggle.py::confirm_switch``) accepts
|
||||||
|
this object unchanged - the dialog moves to the new shape in EPIC R08.
|
||||||
|
"""
|
||||||
|
if not self.previous_model:
|
||||||
|
return ""
|
||||||
|
return f"{self.previous_provider}/{self.previous_model}"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def to_model(self) -> str:
|
||||||
|
"""Candidate key (``provider/model``) of the model to run on. See
|
||||||
|
:attr:`from_model` for why the name matches the legacy decision."""
|
||||||
|
return f"{self.provider}/{self.model}" if self.model else ""
|
||||||
|
|
||||||
|
|
||||||
|
def is_valid_mode(raw: Any) -> bool:
|
||||||
|
"""True when ``raw`` names a mode the routing service understands.
|
||||||
|
|
||||||
|
Distinct from :func:`normalize_mode` because callers need to tell "the user
|
||||||
|
chose off" apart from "this stored value is unrecognised" - the per-workspace
|
||||||
|
lookup falls back to the global setting only in the second case.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
RoutingMode(str(raw or "").strip().lower())
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_mode(raw: Any) -> str:
|
||||||
|
"""Canonical mode string for persistence, or ``"off"`` when unrecognised.
|
||||||
|
|
||||||
|
Exists so the mode vocabulary is defined exactly once. It used to be
|
||||||
|
hard-coded as a ``("off", "auto", "manual")`` tuple in four separate places
|
||||||
|
(config.py twice, state.py twice); adding FALLBACK meant finding all four,
|
||||||
|
and missing one silently downgraded the user's choice back to "off".
|
||||||
|
"""
|
||||||
|
return RoutingMode.parse(raw).value
|
||||||
|
|
||||||
|
|
||||||
|
class RoutingPort(Protocol):
|
||||||
|
"""The slice of the routing engine this service needs.
|
||||||
|
|
||||||
|
Declared as a Protocol so the application layer states its requirement
|
||||||
|
without importing the implementation - which is what lets the whole service
|
||||||
|
be tested against a 20-line fake, and lets ``core.routing`` be replaced later
|
||||||
|
without touching this file.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def route(self, surface: str, prompt: str, current_provider: str, current_model: str,
|
||||||
|
*, mode_override: Optional[str] = None,
|
||||||
|
required_capabilities: Optional[List[str]] = None,
|
||||||
|
task_type: Optional[Any] = None) -> Any:
|
||||||
|
"""Return a route result exposing ``should_switch``, ``target()``,
|
||||||
|
``task_type`` and ``decision``."""
|
||||||
|
|
||||||
|
|
||||||
|
# Presentation supplies this to ask the human. Receives the proposal so the
|
||||||
|
# dialog can explain it; returns True to approve. Manual mode only.
|
||||||
|
ConfirmFn = Callable[[RoutingDecision], bool]
|
||||||
|
|
||||||
|
|
||||||
|
class RoutingApplicationService:
|
||||||
|
"""Decides which provider/model one turn runs on.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
router: the scoring engine (see :class:`RoutingPort`).
|
||||||
|
mode_reader: ``surface -> mode string``; production passes the per-workspace
|
||||||
|
lookup ``AppContext.project_routing_mode``. Injected rather than read
|
||||||
|
from config here so this layer stays free of config plumbing that
|
||||||
|
EPIC R02 is rewriting in parallel.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, router: RoutingPort,
|
||||||
|
mode_reader: Optional[Callable[[str], str]] = None) -> None:
|
||||||
|
self._router = router
|
||||||
|
self._mode_reader = mode_reader
|
||||||
|
|
||||||
|
# -- main entry point -------------------------------------------------- #
|
||||||
|
def route_turn(
|
||||||
|
self,
|
||||||
|
surface: str,
|
||||||
|
prompt: str,
|
||||||
|
current_provider: str,
|
||||||
|
current_model: str,
|
||||||
|
*,
|
||||||
|
mode: Optional[str] = None,
|
||||||
|
confirm: Optional[ConfirmFn] = None,
|
||||||
|
required_capabilities: Optional[Sequence[str]] = None,
|
||||||
|
task_type: Optional[Any] = None,
|
||||||
|
) -> RoutingDecision:
|
||||||
|
"""Decide what to run this turn on. Never raises.
|
||||||
|
|
||||||
|
A routing failure must never block a message: any unexpected error
|
||||||
|
degrades to "keep the current model", which is exactly what all three
|
||||||
|
legacy copies did with a bare ``except`` - made explicit and testable here.
|
||||||
|
"""
|
||||||
|
resolved_mode = RoutingMode.parse(mode if mode is not None else self._read_mode(surface))
|
||||||
|
keep = self._keep(resolved_mode, current_provider, current_model,
|
||||||
|
reason="routing off - keeping current model")
|
||||||
|
|
||||||
|
# An empty prompt carries no signal to classify, so routing cannot make a
|
||||||
|
# meaningful choice; the same guard exists in all three legacy copies.
|
||||||
|
if resolved_mode is RoutingMode.OFF or not (prompt or "").strip():
|
||||||
|
return keep
|
||||||
|
|
||||||
|
# FALLBACK never switches up front - it only reacts to a failure, which
|
||||||
|
# the caller reports through fallback_after_failure().
|
||||||
|
if resolved_mode is RoutingMode.FALLBACK:
|
||||||
|
return self._keep(resolved_mode, current_provider, current_model,
|
||||||
|
reason="fallback mode - switching only after a failure")
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = self._router.route(
|
||||||
|
surface, prompt, current_provider, current_model,
|
||||||
|
mode_override=resolved_mode.value,
|
||||||
|
required_capabilities=list(required_capabilities) if required_capabilities else None,
|
||||||
|
task_type=task_type,
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001 - routing must never break a turn
|
||||||
|
return self._keep(resolved_mode, current_provider, current_model,
|
||||||
|
reason="routing engine failed - keeping current model")
|
||||||
|
|
||||||
|
proposal = self._to_decision(result, resolved_mode, current_provider, current_model)
|
||||||
|
if not proposal.switched:
|
||||||
|
return proposal
|
||||||
|
|
||||||
|
# Manual mode: the proposal only becomes a switch once a human approves.
|
||||||
|
if resolved_mode is RoutingMode.MANUAL:
|
||||||
|
if confirm is None or not self._ask(confirm, proposal):
|
||||||
|
return self._keep(resolved_mode, current_provider, current_model,
|
||||||
|
reason="switch declined - keeping current model",
|
||||||
|
task_type=proposal.task_type, declined=True)
|
||||||
|
return proposal
|
||||||
|
|
||||||
|
# -- failure recovery -------------------------------------------------- #
|
||||||
|
def fallback_after_failure(
|
||||||
|
self,
|
||||||
|
surface: str,
|
||||||
|
prompt: str,
|
||||||
|
failed_provider: str,
|
||||||
|
failed_model: str,
|
||||||
|
*,
|
||||||
|
mode: Optional[str] = None,
|
||||||
|
required_capabilities: Optional[Sequence[str]] = None,
|
||||||
|
task_type: Optional[Any] = None,
|
||||||
|
) -> Optional[RoutingDecision]:
|
||||||
|
"""Pick a replacement after ``failed_provider/failed_model`` failed.
|
||||||
|
|
||||||
|
Returns None when there is nothing to fall back to, so the caller can
|
||||||
|
surface the original error instead of retrying forever. Available in
|
||||||
|
AUTO and FALLBACK; OFF and MANUAL keep the user's model on failure too,
|
||||||
|
because silently moving work to another model is exactly what those two
|
||||||
|
modes exist to prevent.
|
||||||
|
"""
|
||||||
|
resolved_mode = RoutingMode.parse(mode if mode is not None else self._read_mode(surface))
|
||||||
|
if resolved_mode not in (RoutingMode.AUTO, RoutingMode.FALLBACK):
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Asked in AUTO so the engine ranks candidates rather than short-
|
||||||
|
# circuiting on FALLBACK's "never switch up front" rule; the failed
|
||||||
|
# model is passed as current so any positive gain beats it.
|
||||||
|
result = self._router.route(
|
||||||
|
surface, prompt, failed_provider, failed_model,
|
||||||
|
mode_override=RoutingMode.AUTO.value,
|
||||||
|
required_capabilities=list(required_capabilities) if required_capabilities else None,
|
||||||
|
task_type=task_type,
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001 - a broken router must not mask the real error
|
||||||
|
return None
|
||||||
|
|
||||||
|
decision = self._to_decision(result, resolved_mode, failed_provider, failed_model)
|
||||||
|
# A "switch" back to the model that just failed would retry the outage.
|
||||||
|
if not decision.switched or (decision.provider, decision.model) == (failed_provider, failed_model):
|
||||||
|
return None
|
||||||
|
return RoutingDecision(
|
||||||
|
mode=resolved_mode, provider=decision.provider, model=decision.model,
|
||||||
|
switched=True, task_type=decision.task_type, score_gain=decision.score_gain,
|
||||||
|
reason=f"{failed_provider}/{failed_model} failed - falling back to "
|
||||||
|
f"{decision.provider}/{decision.model}",
|
||||||
|
previous_provider=failed_provider, previous_model=failed_model,
|
||||||
|
)
|
||||||
|
|
||||||
|
# -- internals --------------------------------------------------------- #
|
||||||
|
def _read_mode(self, surface: str) -> str:
|
||||||
|
"""Per-surface mode from the injected reader ('off' when none supplied)."""
|
||||||
|
if self._mode_reader is None:
|
||||||
|
return RoutingMode.OFF.value
|
||||||
|
try:
|
||||||
|
return self._mode_reader(surface) or RoutingMode.OFF.value
|
||||||
|
except Exception: # noqa: BLE001 - a config read must not break a turn
|
||||||
|
return RoutingMode.OFF.value
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _keep(mode: RoutingMode, provider: str, model: str, *, reason: str,
|
||||||
|
task_type: str = "", declined: bool = False) -> RoutingDecision:
|
||||||
|
"""A no-switch decision that still names the model to run on."""
|
||||||
|
return RoutingDecision(mode=mode, provider=provider, model=model, switched=False,
|
||||||
|
task_type=task_type, reason=reason, declined=declined,
|
||||||
|
previous_provider=provider, previous_model=model)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _ask(confirm: ConfirmFn, proposal: RoutingDecision) -> bool:
|
||||||
|
"""Run the confirm callback, treating any failure as "declined".
|
||||||
|
|
||||||
|
The callback opens a modal dialog in production; if that raises (window
|
||||||
|
already closing, for instance) the safe answer is to keep the user's own
|
||||||
|
model rather than to switch without consent.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return bool(confirm(proposal))
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return False
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _to_decision(result: Any, mode: RoutingMode,
|
||||||
|
current_provider: str, current_model: str) -> RoutingDecision:
|
||||||
|
"""Translate the engine's route result into a :class:`RoutingDecision`.
|
||||||
|
|
||||||
|
Defensive about the result shape on purpose: this is the seam between the
|
||||||
|
new layer and a legacy module still under refactor, and a missing
|
||||||
|
attribute must degrade to "keep current model" instead of raising into
|
||||||
|
the middle of a chat turn.
|
||||||
|
"""
|
||||||
|
inner = getattr(result, "decision", None)
|
||||||
|
task_type = getattr(getattr(result, "task_type", None), "value", "") or ""
|
||||||
|
gain = float(getattr(inner, "score_gain", 0.0) or 0.0)
|
||||||
|
reason = str(getattr(inner, "reason", "") or "")
|
||||||
|
|
||||||
|
target = None
|
||||||
|
if getattr(result, "should_switch", False):
|
||||||
|
getter = getattr(result, "target", None)
|
||||||
|
target = getter() if callable(getter) else None
|
||||||
|
|
||||||
|
if not target:
|
||||||
|
return RoutingDecision(mode=mode, provider=current_provider, model=current_model,
|
||||||
|
switched=False, task_type=task_type, score_gain=gain,
|
||||||
|
reason=reason or "no better model - keeping current",
|
||||||
|
previous_provider=current_provider,
|
||||||
|
previous_model=current_model)
|
||||||
|
|
||||||
|
provider, model = target
|
||||||
|
return RoutingDecision(mode=mode, provider=provider or current_provider, model=model,
|
||||||
|
switched=True, task_type=task_type, score_gain=gain, reason=reason,
|
||||||
|
previous_provider=current_provider, previous_model=current_model)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["RoutingApplicationService", "RoutingDecision", "RoutingMode",
|
||||||
|
"RoutingPort", "normalize_mode", "is_valid_mode"]
|
||||||
@@ -552,19 +552,25 @@ class AppConfig:
|
|||||||
return d
|
return d
|
||||||
|
|
||||||
def routing_mode_for(self, surface: str) -> str:
|
def routing_mode_for(self, surface: str) -> str:
|
||||||
"""Effective Off/Auto/Manual mode for a chat surface.
|
"""Effective Off/Auto/Manual/Fallback mode for a chat surface.
|
||||||
|
|
||||||
|
A per-surface override wins; an empty override falls back to the global
|
||||||
|
``switch_mode``. The value is validated through
|
||||||
|
``application.model_routing.normalize_mode`` so the accepted vocabulary
|
||||||
|
is defined in exactly one place (R03-T03) - it used to be a literal
|
||||||
|
tuple repeated here and in state.py, and adding a mode to one copy but
|
||||||
|
not the others silently downgraded the user's choice to "off"."""
|
||||||
|
from .application.model_routing import normalize_mode
|
||||||
|
|
||||||
A per-surface override ("auto"/"manual"/"off") wins; an empty override
|
|
||||||
falls back to the global ``switch_mode``."""
|
|
||||||
routing = self.routing
|
routing = self.routing
|
||||||
override = (routing.get("surface_modes", {}) or {}).get(surface, "")
|
override = (routing.get("surface_modes", {}) or {}).get(surface, "")
|
||||||
mode = override or routing.get("switch_mode", "off")
|
return normalize_mode(override or routing.get("switch_mode", "off"))
|
||||||
return mode if mode in ("off", "auto", "manual") else "off"
|
|
||||||
|
|
||||||
def set_routing_mode_for(self, surface: str, mode: str) -> None:
|
def set_routing_mode_for(self, surface: str, mode: str) -> None:
|
||||||
"""Persist a chat surface's Off/Auto/Manual toggle selection."""
|
"""Persist a chat surface's routing toggle selection."""
|
||||||
mode = mode if mode in ("off", "auto", "manual") else "off"
|
from .application.model_routing import normalize_mode
|
||||||
self.routing.setdefault("surface_modes", {})[surface] = mode
|
|
||||||
|
self.routing.setdefault("surface_modes", {})[surface] = normalize_mode(mode)
|
||||||
self.save()
|
self.save()
|
||||||
|
|
||||||
@property
|
@property
|
||||||
|
|||||||
@@ -62,18 +62,18 @@
|
|||||||
* **Team chịu trách nhiệm**: 🔵 **Team Duy** (Chủ trì)
|
* **Team chịu trách nhiệm**: 🔵 **Team Duy** (Chủ trì)
|
||||||
* **Mục tiêu**: Hợp nhất logic routing bị phân tán thành `RoutingApplicationService` độc lập Qt; chuẩn hóa catalog nhà cung cấp.
|
* **Mục tiêu**: Hợp nhất logic routing bị phân tán thành `RoutingApplicationService` độc lập Qt; chuẩn hóa catalog nhà cung cấp.
|
||||||
|
|
||||||
- [ ] **R03-T01 (Team Duy)**: Xây dựng bộ Contract Tests chuẩn hóa cho các Provider từ `providers/base.py` ➔ `tests/contracts/test_providers.py`
|
- [x] **R03-T01 (Team Duy)**: Xây dựng bộ Contract Tests chuẩn hóa cho các Provider từ `providers/base.py` ➔ `tests/contracts/test_providers.py`
|
||||||
*Start: `____-__-__ __:__` | End: `____-__-__ __:__`*
|
*Start: `2026-08-21 10:10` | End: `2026-08-21 10:12`*
|
||||||
- [ ] **R03-T02 (Team Duy)**: Xây dựng `ProviderDescriptor` và `ProviderRegistry` tập trung từ `providers/factory.py` ➔ `domain/models/provider_descriptor.py` & `infrastructure/providers/provider_registry.py`
|
- [x] **R03-T02 (Team Duy)**: Xây dựng `ProviderDescriptor` và `ProviderRegistry` tập trung từ `providers/factory.py` ➔ `domain/models/provider_descriptor.py` & `infrastructure/providers/provider_registry.py`
|
||||||
*Start: `____-__-__ __:__` | End: `____-__-__ __:__`*
|
*Start: `2026-08-21 10:06` | End: `2026-08-21 10:10`*
|
||||||
- [ ] **R03-T03 (Team Duy)**: Xây dựng `RoutingApplicationService` độc lập với Qt từ `core/routing/` ➔ `application/model_routing/routing_application_service.py`
|
- [x] **R03-T03 (Team Duy)**: Xây dựng `RoutingApplicationService` độc lập với Qt từ `core/routing/` ➔ `application/model_routing/routing_application_service.py`
|
||||||
*Start: `____-__-__ __:__` | End: `____-__-__ __:__`*
|
*Start: `2026-08-21 10:12` | End: `2026-08-21 10:15`*
|
||||||
- [ ] **R03-T04 (Team Duy)**: Di chuyển luồng gọi routing từ `ui/chat_panel.py#L638` sang `RoutingApplicationService`
|
- [x] **R03-T04 (Team Duy)**: Di chuyển luồng gọi routing từ `ui/chat_panel.py#L638` sang `RoutingApplicationService`
|
||||||
*Start: `____-__-__ __:__` | End: `____-__-__ __:__`*
|
*Start: `2026-08-21 10:17` | End: `2026-08-21 10:20`*
|
||||||
- [ ] **R03-T05 (Team Duy)**: Di chuyển luồng gọi routing từ `ui/co4e_tab.py` và `ui/folder_tab.py` sang `RoutingApplicationService`
|
- [x] **R03-T05 (Team Duy)**: Di chuyển luồng gọi routing từ `ui/co4e_tab.py` và `ui/folder_tab.py` sang `RoutingApplicationService`
|
||||||
*Start: `____-__-__ __:__` | End: `____-__-__ __:__`*
|
*Start: `2026-08-21 10:20` | End: `2026-08-21 10:22`*
|
||||||
- [ ] **R03-T06 (Team Duy)**: Tách logic ghi nhận token usage ra khỏi Provider, chuyển thành `UsageEventSink` ➔ `infrastructure/telemetry/usage_sink.py`
|
- [x] **R03-T06 (Team Duy)**: Tách logic ghi nhận token usage ra khỏi Provider, chuyển thành `UsageEventSink` ➔ `infrastructure/telemetry/usage_sink.py`
|
||||||
*Start: `____-__-__ __:__` | End: `____-__-__ __:__`*
|
*Start: `2026-08-21 10:15` | End: `2026-08-21 10:17`*
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,12 @@
|
|||||||
|
"""Domain layer - pure Python entities, value objects and events.
|
||||||
|
|
||||||
|
The innermost layer of the 4-tier architecture (see
|
||||||
|
``docs/architecture/ADR-001-layered-architecture.md``). Modules here describe
|
||||||
|
WHAT the application is about - a turn of conversation, a model candidate, an
|
||||||
|
agent event - and depend on nothing but the standard library.
|
||||||
|
|
||||||
|
Hard rule (ADR-001 I1/I2, enforced by ``scripts/check_imports.py``): no imports
|
||||||
|
of PySide6/PyQt, and no imports from ``application/``, ``infrastructure/``,
|
||||||
|
``presentation/`` or the legacy ``core/``/``ui/`` packages. That is what keeps
|
||||||
|
this layer testable in milliseconds and reusable from a headless scheduler.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
"""Domain entities for one agent turn: the request snapshot and the event
|
||||||
|
stream it produces (EPIC R04)."""
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""Domain models: provider/model catalogue value objects (EPIC R03)."""
|
||||||
|
|
||||||
|
from .provider_descriptor import ProviderCapability, ProviderDescriptor
|
||||||
|
|
||||||
|
__all__ = ["ProviderDescriptor", "ProviderCapability"]
|
||||||
@@ -0,0 +1,171 @@
|
|||||||
|
"""ProviderDescriptor - the declarative catalogue entry for one model provider (R03-T02).
|
||||||
|
|
||||||
|
Today the knowledge of "what a provider is" is scattered across three places
|
||||||
|
that must be edited together and can silently drift apart:
|
||||||
|
|
||||||
|
* ``providers/factory.py::_REGISTRY`` - name -> implementation class
|
||||||
|
* ``config.py::DEFAULT_CONFIG["providers"]`` - default base_url / model / api_key
|
||||||
|
* ``config.py::PROVIDER_LABELS`` - the human label shown in Settings
|
||||||
|
|
||||||
|
Adding a provider means remembering all three; forgetting one produces a
|
||||||
|
provider that exists but has no label, or a label with no implementation. This
|
||||||
|
value object folds those facts into a single immutable description that the
|
||||||
|
registry (``infrastructure/providers/provider_registry.py``) and the UI can both
|
||||||
|
read, so a new provider is declared once.
|
||||||
|
|
||||||
|
Pure domain code: stdlib only, no Qt, no network, no config access. It describes
|
||||||
|
a provider; building one is infrastructure's job.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Dict, FrozenSet, List, Mapping, Optional, Tuple
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderCapability(str, Enum):
|
||||||
|
"""What a provider can do, as advertised by its descriptor.
|
||||||
|
|
||||||
|
Kept as a closed enum rather than free-form strings so a typo
|
||||||
|
(``"vison"``) fails at import time instead of silently disabling a feature
|
||||||
|
at runtime. Inherits ``str`` so existing dict/JSON code that compares against
|
||||||
|
plain strings keeps working during the migration.
|
||||||
|
"""
|
||||||
|
|
||||||
|
STREAMING = "streaming" # can stream answer fragments through on_text
|
||||||
|
TOOLS = "tools" # can be given a ToolSpec catalogue and call tools
|
||||||
|
VISION = "vision" # accepts image content blocks (see providers/base.py)
|
||||||
|
REASONING = "reasoning" # emits a separate private "thinking" stream
|
||||||
|
MODEL_LISTING = "model_listing" # list_models() returns a real catalogue
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProviderDescriptor:
|
||||||
|
"""An immutable description of one provider the app can talk to.
|
||||||
|
|
||||||
|
Attributes:
|
||||||
|
id: the config key, e.g. ``"openai_compat"``. Also the ``provider`` half
|
||||||
|
of a routing candidate key (``provider/model_id``).
|
||||||
|
label: human-readable name for Settings and the model picker.
|
||||||
|
protocol: which wire format this provider speaks. Several ids share one
|
||||||
|
protocol - ``ollama``, ``github_copilot`` and ``codex`` are all
|
||||||
|
OpenAI-compatible endpoints - which is exactly why protocol and id
|
||||||
|
must be separate fields.
|
||||||
|
default_model: the model used when the user has not chosen one.
|
||||||
|
capabilities: what the provider supports (see :class:`ProviderCapability`).
|
||||||
|
requires_api_key: whether an empty ``api_key`` makes it unusable.
|
||||||
|
requires_base_url: whether an empty ``base_url`` makes it unusable.
|
||||||
|
local: True when the endpoint runs on the user's own machine. Routing
|
||||||
|
treats local models as zero-cost, and the security layer treats them
|
||||||
|
as not leaving the machine, so this is a real behavioural flag and
|
||||||
|
not just documentation.
|
||||||
|
notes: free-form remark shown in Settings (e.g. "paste a Copilot token").
|
||||||
|
"""
|
||||||
|
|
||||||
|
id: str
|
||||||
|
label: str
|
||||||
|
protocol: str
|
||||||
|
default_model: str = ""
|
||||||
|
capabilities: FrozenSet[ProviderCapability] = field(default_factory=frozenset)
|
||||||
|
requires_api_key: bool = True
|
||||||
|
requires_base_url: bool = True
|
||||||
|
local: bool = False
|
||||||
|
notes: str = ""
|
||||||
|
|
||||||
|
# -- capability queries ---------------------------------------------- #
|
||||||
|
def supports(self, capability: ProviderCapability) -> bool:
|
||||||
|
"""True when this provider advertises ``capability``."""
|
||||||
|
return capability in self.capabilities
|
||||||
|
|
||||||
|
@property
|
||||||
|
def supports_vision(self) -> bool:
|
||||||
|
"""Mirrors ``providers.base.Provider.supports_vision`` so callers can ask
|
||||||
|
the descriptor (no instance, no network) before building a provider."""
|
||||||
|
return self.supports(ProviderCapability.VISION)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def supports_tools(self) -> bool:
|
||||||
|
"""True when this provider can run an agent turn with tools. A provider
|
||||||
|
without it can still chat, but must never be routed a tool-using task."""
|
||||||
|
return self.supports(ProviderCapability.TOOLS)
|
||||||
|
|
||||||
|
def capability_names(self) -> List[str]:
|
||||||
|
"""Capabilities as sorted plain strings - the shape the routing layer's
|
||||||
|
``required_capabilities`` filter and the assessment store both use."""
|
||||||
|
return sorted(c.value for c in self.capabilities)
|
||||||
|
|
||||||
|
# -- configuration validation ---------------------------------------- #
|
||||||
|
def missing_settings(self, conf: Mapping[str, Any]) -> List[str]:
|
||||||
|
"""Which required config keys are absent or blank in ``conf``.
|
||||||
|
|
||||||
|
Returned as a list (not a bool) so Settings can tell the user exactly
|
||||||
|
what to fill in, instead of a generic "not configured". A provider that
|
||||||
|
needs nothing returns an empty list.
|
||||||
|
"""
|
||||||
|
missing: List[str] = []
|
||||||
|
if self.requires_api_key and not str(conf.get("api_key", "") or "").strip():
|
||||||
|
missing.append("api_key")
|
||||||
|
if self.requires_base_url and not str(conf.get("base_url", "") or "").strip():
|
||||||
|
missing.append("base_url")
|
||||||
|
return missing
|
||||||
|
|
||||||
|
def is_configured(self, conf: Mapping[str, Any]) -> bool:
|
||||||
|
"""True when ``conf`` carries everything this provider needs to run."""
|
||||||
|
return not self.missing_settings(conf)
|
||||||
|
|
||||||
|
def resolve_model(self, conf: Optional[Mapping[str, Any]] = None,
|
||||||
|
requested: str = "") -> str:
|
||||||
|
"""Pick the model id for a call: explicit request, else configured, else
|
||||||
|
this descriptor's default.
|
||||||
|
|
||||||
|
Centralised here because the same three-step fallback is currently
|
||||||
|
re-implemented at every call site (chat panel, Co4E, AI-edit, scheduler),
|
||||||
|
and each of them gets the precedence subtly different.
|
||||||
|
"""
|
||||||
|
if requested:
|
||||||
|
return requested
|
||||||
|
configured = str((conf or {}).get("model", "") or "").strip()
|
||||||
|
return configured or self.default_model
|
||||||
|
|
||||||
|
def describe(self, conf: Optional[Mapping[str, Any]] = None) -> str:
|
||||||
|
"""One-line summary for logs and the Settings row, e.g.
|
||||||
|
``"anthropic:claude-sonnet-4-6 (Anthropic Claude)"``."""
|
||||||
|
return f"{self.id}:{self.resolve_model(conf)} ({self.label})"
|
||||||
|
|
||||||
|
def candidate_key(self, model_id: str) -> str:
|
||||||
|
"""The ``provider/model_id`` identity the routing layer keys on.
|
||||||
|
|
||||||
|
Defined here so the domain owns the format; ``core.routing.models`` has
|
||||||
|
its own ``candidate_key()`` helper producing the identical string, and
|
||||||
|
keeping them equal is what lets the new registry and the existing
|
||||||
|
assessment store share one keyspace during the migration.
|
||||||
|
"""
|
||||||
|
return f"{self.id}/{model_id}"
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
"""JSON-safe projection, for persisting a catalogue snapshot or sending
|
||||||
|
the descriptor to a UI layer that must not import domain types."""
|
||||||
|
return {
|
||||||
|
"id": self.id,
|
||||||
|
"label": self.label,
|
||||||
|
"protocol": self.protocol,
|
||||||
|
"default_model": self.default_model,
|
||||||
|
"capabilities": self.capability_names(),
|
||||||
|
"requires_api_key": self.requires_api_key,
|
||||||
|
"requires_base_url": self.requires_base_url,
|
||||||
|
"local": self.local,
|
||||||
|
"notes": self.notes,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def split_candidate_key(key: str) -> Tuple[str, str]:
|
||||||
|
"""Inverse of :meth:`ProviderDescriptor.candidate_key`.
|
||||||
|
|
||||||
|
Splits on the FIRST ``/`` only: some gateways expose model ids that contain
|
||||||
|
a slash (``org/model``), and splitting on the last one would corrupt them.
|
||||||
|
"""
|
||||||
|
provider, _, model_id = key.partition("/")
|
||||||
|
return provider, model_id
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["ProviderCapability", "ProviderDescriptor", "split_candidate_key"]
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
"""Infrastructure layer - adapters to the outside world.
|
||||||
|
|
||||||
|
Concrete implementations of what the inner layers only describe: HTTP calls to
|
||||||
|
model gateways, the OS keyring, the filesystem, subprocesses, telemetry sinks.
|
||||||
|
May import ``domain/`` (to speak its types) and third-party libraries, but never
|
||||||
|
``presentation/``/``ui/``.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""Provider adapters and the central provider catalogue (EPIC R03)."""
|
||||||
|
|
||||||
|
from .provider_registry import ProviderRegistry, default_registry
|
||||||
|
|
||||||
|
__all__ = ["ProviderRegistry", "default_registry"]
|
||||||
@@ -0,0 +1,207 @@
|
|||||||
|
"""ProviderRegistry - the one place a provider is declared (R03-T02).
|
||||||
|
|
||||||
|
Replaces the three-way split between ``providers/factory.py::_REGISTRY``,
|
||||||
|
``config.py::DEFAULT_CONFIG["providers"]`` and ``config.py::PROVIDER_LABELS``
|
||||||
|
with a single catalogue of :class:`ProviderDescriptor` objects plus the
|
||||||
|
implementation class each one maps to.
|
||||||
|
|
||||||
|
Adding a provider is now one entry in :data:`BUILT_IN_PROVIDERS` (declarative
|
||||||
|
facts) and one line in :data:`_IMPLEMENTATIONS` (which class speaks that
|
||||||
|
protocol) - see ``docs/governance/contributor-recipes.md`` (R10-T04).
|
||||||
|
|
||||||
|
Migration note (strangler fig, ADR-001 section 4): this registry does not
|
||||||
|
re-implement any provider. It builds the SAME classes ``providers/factory.py``
|
||||||
|
builds, so both entry points stay behaviourally identical while call sites move
|
||||||
|
over one at a time.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Dict, Iterable, List, Mapping, Optional
|
||||||
|
|
||||||
|
from cowork_local.domain.models.provider_descriptor import (
|
||||||
|
ProviderCapability,
|
||||||
|
ProviderDescriptor,
|
||||||
|
)
|
||||||
|
from cowork_local.providers.base import Provider, ProviderError
|
||||||
|
|
||||||
|
_CAP = ProviderCapability
|
||||||
|
|
||||||
|
# Every provider the app ships with, described once.
|
||||||
|
#
|
||||||
|
# The capability sets are deliberately conservative: a capability listed here is
|
||||||
|
# one the adapter genuinely implements today. Claiming VISION for a provider
|
||||||
|
# whose chat() cannot translate an image block would route an image turn into a
|
||||||
|
# guaranteed failure, so an unimplemented capability must stay off the list.
|
||||||
|
BUILT_IN_PROVIDERS: tuple = (
|
||||||
|
ProviderDescriptor(
|
||||||
|
id="openai_compat",
|
||||||
|
label="OpenAI-compatible (Internal Gateway)",
|
||||||
|
protocol="openai_compat",
|
||||||
|
default_model="gpt-4o-mini",
|
||||||
|
capabilities=frozenset({_CAP.STREAMING, _CAP.TOOLS, _CAP.VISION,
|
||||||
|
_CAP.REASONING, _CAP.MODEL_LISTING}),
|
||||||
|
notes="Any endpoint speaking the OpenAI Chat Completions protocol.",
|
||||||
|
),
|
||||||
|
ProviderDescriptor(
|
||||||
|
id="anthropic",
|
||||||
|
label="Anthropic Claude",
|
||||||
|
protocol="anthropic",
|
||||||
|
default_model="claude-sonnet-4-6",
|
||||||
|
capabilities=frozenset({_CAP.STREAMING, _CAP.TOOLS, _CAP.VISION,
|
||||||
|
_CAP.MODEL_LISTING}),
|
||||||
|
),
|
||||||
|
ProviderDescriptor(
|
||||||
|
id="ollama",
|
||||||
|
label="Ollama (local models)",
|
||||||
|
protocol="openai_compat",
|
||||||
|
default_model="llama3.1",
|
||||||
|
capabilities=frozenset({_CAP.STREAMING, _CAP.TOOLS, _CAP.REASONING,
|
||||||
|
_CAP.MODEL_LISTING}),
|
||||||
|
# Ollama ignores the key, but the OpenAI client layer requires a value,
|
||||||
|
# so the default config ships a placeholder rather than an empty string.
|
||||||
|
requires_api_key=False,
|
||||||
|
local=True,
|
||||||
|
notes="Runs on this machine - no data leaves the device, no token cost.",
|
||||||
|
),
|
||||||
|
ProviderDescriptor(
|
||||||
|
id="github_copilot",
|
||||||
|
label="GitHub Copilot",
|
||||||
|
protocol="openai_compat",
|
||||||
|
default_model="gpt-4o",
|
||||||
|
capabilities=frozenset({_CAP.STREAMING, _CAP.TOOLS, _CAP.MODEL_LISTING}),
|
||||||
|
notes="Paste a Copilot token as the API key.",
|
||||||
|
),
|
||||||
|
ProviderDescriptor(
|
||||||
|
id="codex",
|
||||||
|
label="OpenAI (Codex / GPT)",
|
||||||
|
protocol="openai_compat",
|
||||||
|
default_model="gpt-4o-mini",
|
||||||
|
capabilities=frozenset({_CAP.STREAMING, _CAP.TOOLS, _CAP.VISION,
|
||||||
|
_CAP.REASONING, _CAP.MODEL_LISTING}),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _implementations() -> Dict[str, type]:
|
||||||
|
"""Protocol -> adapter class.
|
||||||
|
|
||||||
|
Imported lazily inside the function because ``providers/anthropic.py`` and
|
||||||
|
``providers/openai_compat.py`` pull in ``requests`` at import time; keeping
|
||||||
|
that out of module import means a test that only inspects descriptors pays
|
||||||
|
no import cost at all.
|
||||||
|
"""
|
||||||
|
from cowork_local.providers.anthropic import AnthropicProvider
|
||||||
|
from cowork_local.providers.openai_compat import OpenAICompatProvider
|
||||||
|
|
||||||
|
return {
|
||||||
|
"openai_compat": OpenAICompatProvider,
|
||||||
|
"anthropic": AnthropicProvider,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderRegistry:
|
||||||
|
"""Catalogue of known providers + the factory that instantiates them.
|
||||||
|
|
||||||
|
Intentionally holds no config and no app context: it is a pure lookup table
|
||||||
|
plus a build step, so it can be constructed in a test with a custom
|
||||||
|
descriptor list and no application running.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, descriptors: Optional[Iterable[ProviderDescriptor]] = None) -> None:
|
||||||
|
# Dict preserves declaration order (Python 3.7+), which is the order
|
||||||
|
# Settings lists providers in - so the catalogue order is data, not luck.
|
||||||
|
self._by_id: Dict[str, ProviderDescriptor] = {
|
||||||
|
d.id: d for d in (descriptors if descriptors is not None else BUILT_IN_PROVIDERS)
|
||||||
|
}
|
||||||
|
|
||||||
|
# -- catalogue queries ------------------------------------------------ #
|
||||||
|
def ids(self) -> List[str]:
|
||||||
|
"""Known provider ids, in declaration order."""
|
||||||
|
return list(self._by_id)
|
||||||
|
|
||||||
|
def all(self) -> List[ProviderDescriptor]:
|
||||||
|
"""Every descriptor, in declaration order."""
|
||||||
|
return list(self._by_id.values())
|
||||||
|
|
||||||
|
def get(self, provider_id: str) -> Optional[ProviderDescriptor]:
|
||||||
|
"""The descriptor for ``provider_id``, or None when unknown.
|
||||||
|
|
||||||
|
Returns None rather than raising because the caller is often reacting to
|
||||||
|
a config file that may name a provider from a newer version; the UI
|
||||||
|
should be able to skip it, not crash.
|
||||||
|
"""
|
||||||
|
return self._by_id.get(provider_id)
|
||||||
|
|
||||||
|
def require(self, provider_id: str) -> ProviderDescriptor:
|
||||||
|
"""Like :meth:`get` but raises :class:`ProviderError` when unknown.
|
||||||
|
|
||||||
|
Same error type ``providers/factory.py::build_provider`` already raises,
|
||||||
|
so callers that migrate to the registry keep their existing except clause.
|
||||||
|
"""
|
||||||
|
descriptor = self._by_id.get(provider_id)
|
||||||
|
if descriptor is None:
|
||||||
|
known = ", ".join(self._by_id) or "(none)"
|
||||||
|
raise ProviderError(f"Unsupported provider: {provider_id} (known: {known})")
|
||||||
|
return descriptor
|
||||||
|
|
||||||
|
def labels(self) -> Dict[str, str]:
|
||||||
|
"""``{id: label}`` - the drop-in replacement for ``config.PROVIDER_LABELS``."""
|
||||||
|
return {d.id: d.label for d in self._by_id.values()}
|
||||||
|
|
||||||
|
def supporting(self, capability: ProviderCapability) -> List[ProviderDescriptor]:
|
||||||
|
"""Every descriptor advertising ``capability`` - used to answer "which
|
||||||
|
providers could serve this turn?" before any of them is built."""
|
||||||
|
return [d for d in self._by_id.values() if d.supports(capability)]
|
||||||
|
|
||||||
|
def configured(self, providers_conf: Mapping[str, Mapping[str, Any]]
|
||||||
|
) -> List[ProviderDescriptor]:
|
||||||
|
"""Descriptors whose config section is complete enough to actually call.
|
||||||
|
|
||||||
|
``providers_conf`` is ``AppConfig.data["providers"]``. Passing the raw
|
||||||
|
mapping (not the AppConfig object) keeps this layer independent of the
|
||||||
|
config implementation, which EPIC R02 is rewriting in parallel.
|
||||||
|
"""
|
||||||
|
return [d for d in self._by_id.values()
|
||||||
|
if d.is_configured(providers_conf.get(d.id, {}) or {})]
|
||||||
|
|
||||||
|
# -- construction ----------------------------------------------------- #
|
||||||
|
def build(self, provider_id: str, conf: Mapping[str, Any],
|
||||||
|
model: str = "") -> Provider:
|
||||||
|
"""Instantiate the adapter for ``provider_id``.
|
||||||
|
|
||||||
|
``model`` overrides the configured model for this instance only - that is
|
||||||
|
how the routing layer runs one turn on a different model without mutating
|
||||||
|
the user's saved settings.
|
||||||
|
"""
|
||||||
|
descriptor = self.require(provider_id)
|
||||||
|
impl = _implementations().get(descriptor.protocol)
|
||||||
|
if impl is None: # pragma: no cover - only reachable via a bad descriptor
|
||||||
|
raise ProviderError(
|
||||||
|
f"Provider '{provider_id}' declares unknown protocol "
|
||||||
|
f"'{descriptor.protocol}'."
|
||||||
|
)
|
||||||
|
# Copy before mutating: conf is the caller's live config dict, and
|
||||||
|
# writing the routed model into it would silently change the user's
|
||||||
|
# saved default for every later turn.
|
||||||
|
resolved = dict(conf or {})
|
||||||
|
resolved["model"] = descriptor.resolve_model(conf, model)
|
||||||
|
instance = impl(resolved)
|
||||||
|
# The adapter class is shared by several ids (three of them are
|
||||||
|
# OpenAI-compatible), so its class-level `name` cannot identify which
|
||||||
|
# provider this is. Stamping the instance keeps usage records, audit
|
||||||
|
# entries and routing candidate keys attributed to the right provider.
|
||||||
|
instance.name = descriptor.id
|
||||||
|
return instance
|
||||||
|
|
||||||
|
def describe(self, provider_id: str, conf: Optional[Mapping[str, Any]] = None) -> str:
|
||||||
|
"""One-line description used in logs and error messages."""
|
||||||
|
return self.require(provider_id).describe(conf)
|
||||||
|
|
||||||
|
|
||||||
|
# Shared default instance. Callers that need the built-in catalogue use this
|
||||||
|
# instead of constructing a registry each time; tests build their own with an
|
||||||
|
# explicit descriptor list.
|
||||||
|
default_registry = ProviderRegistry()
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["ProviderRegistry", "BUILT_IN_PROVIDERS", "default_registry"]
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
"""Telemetry sinks: where token usage and turn metrics are recorded (EPIC R03)."""
|
||||||
|
|
||||||
|
from .usage_sink import (
|
||||||
|
NullUsageSink,
|
||||||
|
RecordingUsageSink,
|
||||||
|
UsageEvent,
|
||||||
|
UsageEventSink,
|
||||||
|
UsageTrackerSink,
|
||||||
|
default_sink,
|
||||||
|
set_default_sink,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"UsageEvent",
|
||||||
|
"UsageEventSink",
|
||||||
|
"UsageTrackerSink",
|
||||||
|
"NullUsageSink",
|
||||||
|
"RecordingUsageSink",
|
||||||
|
"default_sink",
|
||||||
|
"set_default_sink",
|
||||||
|
]
|
||||||
@@ -0,0 +1,229 @@
|
|||||||
|
"""UsageEventSink - where a turn's token usage goes (R03-T06).
|
||||||
|
|
||||||
|
Today each provider records its own usage inline, in the middle of the streaming
|
||||||
|
loop::
|
||||||
|
|
||||||
|
# providers/openai_compat.py
|
||||||
|
def _record_usage(self, messages, text_parts, tool_acc, usage_seen):
|
||||||
|
from ..core import usage_tracker as ut
|
||||||
|
...
|
||||||
|
ut.record(self.name, self.model, ...)
|
||||||
|
|
||||||
|
Three problems with that shape:
|
||||||
|
|
||||||
|
1. **Hidden side effect.** ``chat()`` looks like a pure request/response call but
|
||||||
|
also writes to the Dashboard's store, so a test of a provider silently
|
||||||
|
appends rows to the developer's real usage history.
|
||||||
|
2. **Duplicated estimation.** The "no usage block from the server, so estimate
|
||||||
|
at ~4 chars/token" fallback is copy-pasted per provider and can drift.
|
||||||
|
3. **One hard-wired destination.** Usage can only ever go to
|
||||||
|
``core.usage_tracker``; a run that wants to bill a workflow, or a test that
|
||||||
|
wants to assert on token counts, has nowhere to plug in.
|
||||||
|
|
||||||
|
This module introduces the seam: providers build a :class:`UsageEvent` and hand
|
||||||
|
it to a :class:`UsageEventSink`. Production wires :class:`UsageTrackerSink`
|
||||||
|
(same destination, same numbers as before); tests wire
|
||||||
|
:class:`RecordingUsageSink` or :class:`NullUsageSink`.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, Dict, List, Optional, Protocol, Sequence
|
||||||
|
|
||||||
|
logger = logging.getLogger("cowork_local.telemetry")
|
||||||
|
|
||||||
|
# Rough characters-per-token ratio used when the gateway sends no usage block.
|
||||||
|
# Matches the constant behaviour of ``core.usage_tracker.estimate_tokens`` so
|
||||||
|
# moving the estimation here does not change a single recorded number.
|
||||||
|
_CHARS_PER_TOKEN = 4
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class UsageEvent:
|
||||||
|
"""Token usage for exactly one provider round trip.
|
||||||
|
|
||||||
|
``estimated`` marks a record derived from text length rather than reported by
|
||||||
|
the server. The Dashboard shows the two differently, and conflating them
|
||||||
|
would make cost figures look more precise than they are.
|
||||||
|
"""
|
||||||
|
|
||||||
|
provider: str
|
||||||
|
model: str
|
||||||
|
input_tokens: int = 0
|
||||||
|
output_tokens: int = 0
|
||||||
|
cached_tokens: int = 0
|
||||||
|
estimated: bool = False
|
||||||
|
|
||||||
|
@property
|
||||||
|
def total_tokens(self) -> int:
|
||||||
|
"""Input + output. Cached tokens are a subset of input, not an addition,
|
||||||
|
so adding them here would double-count a cache hit."""
|
||||||
|
return self.input_tokens + self.output_tokens
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
"""JSON-safe projection for logs and for sinks that persist raw events."""
|
||||||
|
return {
|
||||||
|
"provider": self.provider,
|
||||||
|
"model": self.model,
|
||||||
|
"input_tokens": self.input_tokens,
|
||||||
|
"output_tokens": self.output_tokens,
|
||||||
|
"cached_tokens": self.cached_tokens,
|
||||||
|
"estimated": self.estimated,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class UsageEventSink(Protocol):
|
||||||
|
"""Anything that can absorb a :class:`UsageEvent`.
|
||||||
|
|
||||||
|
Implementations MUST NOT raise: telemetry is observability, and a failure to
|
||||||
|
record usage must never abort the turn that produced it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def record(self, event: UsageEvent) -> None:
|
||||||
|
"""Absorb one usage event."""
|
||||||
|
|
||||||
|
|
||||||
|
class NullUsageSink:
|
||||||
|
"""Discards everything. The default for tests and headless tooling, so a
|
||||||
|
unit test never writes into the developer's real usage history."""
|
||||||
|
|
||||||
|
def record(self, event: UsageEvent) -> None: # noqa: D102 - see protocol
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class RecordingUsageSink:
|
||||||
|
"""Keeps events in memory so a test can assert on what was recorded."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.events: List[UsageEvent] = []
|
||||||
|
|
||||||
|
def record(self, event: UsageEvent) -> None: # noqa: D102 - see protocol
|
||||||
|
self.events.append(event)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def total_tokens(self) -> int:
|
||||||
|
"""Sum across every recorded event."""
|
||||||
|
return sum(e.total_tokens for e in self.events)
|
||||||
|
|
||||||
|
|
||||||
|
class UsageTrackerSink:
|
||||||
|
"""Forwards to ``core.usage_tracker`` - the Dashboard's store.
|
||||||
|
|
||||||
|
This is the production sink and the only place that still knows about the
|
||||||
|
legacy tracker module, which is what lets EPIC R10 replace the storage
|
||||||
|
without touching a single provider.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, tracker: Optional[Any] = None) -> None:
|
||||||
|
# Injectable for tests; imported lazily otherwise because the tracker
|
||||||
|
# touches the config directory at import time.
|
||||||
|
self._tracker = tracker
|
||||||
|
|
||||||
|
def _resolve(self) -> Any:
|
||||||
|
if self._tracker is None:
|
||||||
|
from cowork_local.core import usage_tracker
|
||||||
|
|
||||||
|
self._tracker = usage_tracker
|
||||||
|
return self._tracker
|
||||||
|
|
||||||
|
def record(self, event: UsageEvent) -> None:
|
||||||
|
"""Write the event to the usage tracker, swallowing any failure.
|
||||||
|
|
||||||
|
The bare except mirrors the behaviour this replaces (each provider
|
||||||
|
already wrapped its ``ut.record`` call in ``try/except: pass``) but logs
|
||||||
|
at debug level instead of discarding the reason entirely, so a broken
|
||||||
|
Dashboard store can at least be diagnosed.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
self._resolve().record(
|
||||||
|
event.provider, event.model,
|
||||||
|
event.input_tokens, event.output_tokens, event.cached_tokens,
|
||||||
|
estimated=event.estimated,
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001 - telemetry must never break a turn
|
||||||
|
logger.debug("usage sink: failed to record %s", event.to_dict(), exc_info=True)
|
||||||
|
|
||||||
|
|
||||||
|
def estimate_tokens(text: str) -> int:
|
||||||
|
"""Approximate token count for ``text`` (~4 characters per token).
|
||||||
|
|
||||||
|
Deliberately identical to ``core.usage_tracker.estimate_tokens`` so that
|
||||||
|
moving estimation into this layer changes no recorded number. Duplicated
|
||||||
|
rather than imported to keep this module free of the legacy dependency;
|
||||||
|
:class:`UsageTrackerSink` is the only bridge back to it.
|
||||||
|
"""
|
||||||
|
return max(0, len(text or "") // _CHARS_PER_TOKEN)
|
||||||
|
|
||||||
|
|
||||||
|
def estimated_event(provider: str, model: str, sent: str, received: str) -> UsageEvent:
|
||||||
|
"""Build an estimated :class:`UsageEvent` from the raw text of a round trip.
|
||||||
|
|
||||||
|
Used when the gateway sends no usage block - most self-hosted OpenAI-compatible
|
||||||
|
servers and Ollama do not.
|
||||||
|
"""
|
||||||
|
return UsageEvent(
|
||||||
|
provider=provider, model=model,
|
||||||
|
input_tokens=estimate_tokens(sent),
|
||||||
|
output_tokens=estimate_tokens(received),
|
||||||
|
cached_tokens=0,
|
||||||
|
estimated=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def openai_usage_event(provider: str, model: str, usage: Dict[str, Any]) -> UsageEvent:
|
||||||
|
"""Build a reported :class:`UsageEvent` from an OpenAI-style usage block."""
|
||||||
|
details = usage.get("prompt_tokens_details") or {}
|
||||||
|
return UsageEvent(
|
||||||
|
provider=provider, model=model,
|
||||||
|
input_tokens=int(usage.get("prompt_tokens", 0) or 0),
|
||||||
|
output_tokens=int(usage.get("completion_tokens", 0) or 0),
|
||||||
|
cached_tokens=int(details.get("cached_tokens", 0) or 0),
|
||||||
|
estimated=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def anthropic_usage_event(provider: str, model: str, usage: Dict[str, Any]) -> UsageEvent:
|
||||||
|
"""Build a reported :class:`UsageEvent` from Anthropic's usage accumulator.
|
||||||
|
|
||||||
|
Anthropic reports input tokens on ``message_start`` and output tokens on
|
||||||
|
``message_delta``, so ``providers/anthropic.py`` accumulates them into a dict
|
||||||
|
keyed ``in``/``out``/``cache`` - this reads that shape.
|
||||||
|
"""
|
||||||
|
return UsageEvent(
|
||||||
|
provider=provider, model=model,
|
||||||
|
input_tokens=int(usage.get("in", 0) or 0),
|
||||||
|
output_tokens=int(usage.get("out", 0) or 0),
|
||||||
|
cached_tokens=int(usage.get("cache", 0) or 0),
|
||||||
|
estimated=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# The sink providers use unless one is injected. A module-level default keeps
|
||||||
|
# the change to the provider classes to a single attribute, and lets a test swap
|
||||||
|
# the destination process-wide with one monkeypatch.
|
||||||
|
default_sink: UsageEventSink = UsageTrackerSink()
|
||||||
|
|
||||||
|
|
||||||
|
def set_default_sink(sink: UsageEventSink) -> UsageEventSink:
|
||||||
|
"""Replace the process-wide default sink; returns the previous one so a
|
||||||
|
caller (or fixture) can restore it."""
|
||||||
|
global default_sink
|
||||||
|
previous = default_sink
|
||||||
|
default_sink = sink
|
||||||
|
return previous
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"UsageEvent",
|
||||||
|
"UsageEventSink",
|
||||||
|
"UsageTrackerSink",
|
||||||
|
"NullUsageSink",
|
||||||
|
"RecordingUsageSink",
|
||||||
|
"estimate_tokens",
|
||||||
|
"estimated_event",
|
||||||
|
"openai_usage_event",
|
||||||
|
"anthropic_usage_event",
|
||||||
|
"default_sink",
|
||||||
|
"set_default_sink",
|
||||||
|
]
|
||||||
+12
-14
@@ -292,21 +292,19 @@ class AnthropicProvider(Provider):
|
|||||||
args = {"_raw": b["json"]}
|
args = {"_raw": b["json"]}
|
||||||
tool_calls.append({"id": b["id"], "name": b["name"], "arguments": args})
|
tool_calls.append({"id": b["id"], "name": b["name"], "arguments": args})
|
||||||
|
|
||||||
# Dashboard usage event — real counts from the stream's usage events,
|
# Dashboard usage event — real counts from the stream's usage events
|
||||||
# else a ~4 chars/token estimate. Never breaks the turn.
|
# (input arrives on message_start, output on message_delta), else a
|
||||||
try:
|
# ~4 chars/token estimate. Delivery is the sink's job (R03-T06), so this
|
||||||
from ..core import usage_tracker as ut
|
# only translates Anthropic's wire shape into a canonical UsageEvent.
|
||||||
|
from ..infrastructure.telemetry import usage_sink as telemetry
|
||||||
|
|
||||||
if usage_seen:
|
if usage_seen:
|
||||||
ut.record(self.name, self.model, usage_seen.get("in", 0),
|
event = telemetry.anthropic_usage_event(self.name, self.model, usage_seen)
|
||||||
usage_seen.get("out", 0), usage_seen.get("cache", 0))
|
else:
|
||||||
else:
|
sent = json.dumps(payload.get("messages", []), ensure_ascii=False)
|
||||||
sent = json.dumps(payload.get("messages", []), ensure_ascii=False)
|
got = "".join(text_parts) + "".join(b["json"] for b in blocks.values())
|
||||||
got = "".join(text_parts) + "".join(b["json"] for b in blocks.values())
|
event = telemetry.estimated_event(self.name, self.model, sent, got)
|
||||||
ut.record(self.name, self.model, ut.estimate_tokens(sent),
|
self._emit_usage(event)
|
||||||
ut.estimate_tokens(got), 0, estimated=True)
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass
|
|
||||||
|
|
||||||
return {"role": "assistant", "content": "".join(text_parts), "tool_calls": tool_calls}
|
return {"role": "assistant", "content": "".join(text_parts), "tool_calls": tool_calls}
|
||||||
|
|
||||||
|
|||||||
@@ -224,6 +224,12 @@ class Provider:
|
|||||||
# silently swallowing the error — Settings' "Test connection" / "Load
|
# silently swallowing the error — Settings' "Test connection" / "Load
|
||||||
# models" surfaces this so "model won't load" has a concrete reason.
|
# models" surfaces this so "model won't load" has a concrete reason.
|
||||||
self.last_error = ""
|
self.last_error = ""
|
||||||
|
# Where this provider's token usage goes (R03-T06). None means "the
|
||||||
|
# process-wide default sink", resolved lazily in _emit_usage so that a
|
||||||
|
# test can swap the destination without rebuilding every provider.
|
||||||
|
# Set it per instance to bill one run somewhere else (a workflow, a
|
||||||
|
# scheduled task) without touching global state.
|
||||||
|
self.usage_sink = None
|
||||||
|
|
||||||
def chat(
|
def chat(
|
||||||
self,
|
self,
|
||||||
@@ -274,6 +280,24 @@ class Provider:
|
|||||||
return True, f"OK — {len(models)} model(s) available."
|
return True, f"OK — {len(models)} model(s) available."
|
||||||
return False, "No models returned. Check base_url/API key and network access."
|
return False, "No models returned. Check base_url/API key and network access."
|
||||||
|
|
||||||
|
# -- telemetry -----------------------------------------------------
|
||||||
|
def _emit_usage(self, event) -> None:
|
||||||
|
"""Hand one ``UsageEvent`` to this provider's usage sink.
|
||||||
|
|
||||||
|
Never raises: recording how many tokens a turn cost must not be able to
|
||||||
|
fail the turn itself. Falls back to the process-wide default sink so
|
||||||
|
existing call sites keep reporting to the Dashboard exactly as before
|
||||||
|
(see infrastructure/telemetry/usage_sink.py)."""
|
||||||
|
try:
|
||||||
|
sink = self.usage_sink
|
||||||
|
if sink is None:
|
||||||
|
from ..infrastructure.telemetry import usage_sink as telemetry
|
||||||
|
|
||||||
|
sink = telemetry.default_sink
|
||||||
|
sink.record(event)
|
||||||
|
except Exception: # noqa: BLE001 — telemetry is never worth a failed turn
|
||||||
|
pass
|
||||||
|
|
||||||
# -- shared helpers ------------------------------------------------
|
# -- shared helpers ------------------------------------------------
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _is_cancelled(cancel) -> bool:
|
def _is_cancelled(cancel) -> bool:
|
||||||
|
|||||||
+18
-15
@@ -268,22 +268,25 @@ class OpenAICompatProvider(Provider):
|
|||||||
def _record_usage(self, messages, text_parts, tool_acc, usage_seen) -> None:
|
def _record_usage(self, messages, text_parts, tool_acc, usage_seen) -> None:
|
||||||
"""One Dashboard usage event per turn: real counts when the server's
|
"""One Dashboard usage event per turn: real counts when the server's
|
||||||
final chunk carried a "usage" block, a ~4 chars/token estimate
|
final chunk carried a "usage" block, a ~4 chars/token estimate
|
||||||
otherwise. Never breaks the turn."""
|
otherwise.
|
||||||
try:
|
|
||||||
from ..core import usage_tracker as ut
|
|
||||||
|
|
||||||
if usage_seen:
|
Building the event and delivering it are now separate concerns (R03-T06):
|
||||||
ut.record(self.name, self.model,
|
this method only translates THIS provider's wire shape into a canonical
|
||||||
usage_seen.get("prompt_tokens", 0),
|
``UsageEvent``; where it ends up is the sink's decision, so a test can
|
||||||
usage_seen.get("completion_tokens", 0),
|
assert on token counts without writing to the real Dashboard store."""
|
||||||
(usage_seen.get("prompt_tokens_details") or {}).get("cached_tokens", 0))
|
from ..infrastructure.telemetry import usage_sink as telemetry
|
||||||
else:
|
|
||||||
sent = json.dumps(self._to_api_messages(messages), ensure_ascii=False)
|
if usage_seen:
|
||||||
got = "".join(text_parts) + "".join(s["args"] for s in tool_acc.values())
|
event = telemetry.openai_usage_event(self.name, self.model, usage_seen)
|
||||||
ut.record(self.name, self.model, ut.estimate_tokens(sent),
|
else:
|
||||||
ut.estimate_tokens(got), 0, estimated=True)
|
# No usage block from the gateway (self-hosted servers and Ollama
|
||||||
except Exception: # noqa: BLE001
|
# never send one) - fall back to estimating from the raw text of
|
||||||
pass
|
# both directions, tool-call arguments included since the model was
|
||||||
|
# billed for generating them.
|
||||||
|
sent = json.dumps(self._to_api_messages(messages), ensure_ascii=False)
|
||||||
|
got = "".join(text_parts) + "".join(s["args"] for s in tool_acc.values())
|
||||||
|
event = telemetry.estimated_event(self.name, self.model, sent, got)
|
||||||
|
self._emit_usage(event)
|
||||||
|
|
||||||
def list_models(self):
|
def list_models(self):
|
||||||
self.last_error = ""
|
self.last_error = ""
|
||||||
|
|||||||
@@ -52,7 +52,16 @@ class AppContext:
|
|||||||
# own event loop), so concurrent model calls never needed serializing.
|
# own event loop), so concurrent model calls never needed serializing.
|
||||||
self._conn_lock = threading.Lock()
|
self._conn_lock = threading.Lock()
|
||||||
self._routing_service = None # lazy RoutingService (Auto Model Routing)
|
self._routing_service = None # lazy RoutingService (Auto Model Routing)
|
||||||
|
# Lazy RoutingApplicationService (R03-T03) — the Qt-free decision layer
|
||||||
|
# every chat surface now routes through. Wraps _routing_service, which
|
||||||
|
# stays the scoring/ranking engine underneath.
|
||||||
|
self._routing_application = None
|
||||||
self._routing_lock = threading.Lock()
|
self._routing_lock = threading.Lock()
|
||||||
|
# A SEPARATE lock for the application service: building it calls
|
||||||
|
# routing(), which takes _routing_lock. threading.Lock is not
|
||||||
|
# reentrant, so sharing one lock across both accessors deadlocks the
|
||||||
|
# first caller instead of just serialising them.
|
||||||
|
self._routing_app_lock = threading.Lock()
|
||||||
# The workspace (project) currently selected in the Workspace screen.
|
# The workspace (project) currently selected in the Workspace screen.
|
||||||
# Per-workspace modes (routing + auto-run) resolve against THIS project
|
# Per-workspace modes (routing + auto-run) resolve against THIS project
|
||||||
# so each workspace keeps its own modes. Updated by WorkspaceTab on
|
# so each workspace keeps its own modes. Updated by WorkspaceTab on
|
||||||
@@ -79,16 +88,28 @@ class AppContext:
|
|||||||
workspace keep its own routing mode."""
|
workspace keep its own routing mode."""
|
||||||
project = self._current_project()
|
project = self._current_project()
|
||||||
if project is not None:
|
if project is not None:
|
||||||
|
# Validated through the single mode vocabulary (R03-T03) rather
|
||||||
|
# than a literal tuple, so a workspace can store any mode the
|
||||||
|
# routing service understands - including "fallback", whose
|
||||||
|
# on-screen toggle arrives in EPIC R08.
|
||||||
|
from .application.model_routing import is_valid_mode, normalize_mode
|
||||||
|
|
||||||
mode = (project.routing_modes or {}).get(surface, "")
|
mode = (project.routing_modes or {}).get(surface, "")
|
||||||
if mode in ("off", "auto", "manual"):
|
# Only a RECOGNISED override wins; an empty or corrupt value falls
|
||||||
return mode
|
# through to the global setting, exactly as before. Validation goes
|
||||||
|
# through the routing vocabulary (R03-T03) instead of a literal
|
||||||
|
# tuple, so a new mode works everywhere the moment it is defined.
|
||||||
|
if is_valid_mode(mode):
|
||||||
|
return normalize_mode(mode)
|
||||||
return self.config.routing_mode_for(surface)
|
return self.config.routing_mode_for(surface)
|
||||||
|
|
||||||
def set_project_routing_mode(self, surface: str, mode: str) -> None:
|
def set_project_routing_mode(self, surface: str, mode: str) -> None:
|
||||||
"""Persist a surface's routing mode for the ACTIVE workspace. With no
|
"""Persist a surface's routing mode for the ACTIVE workspace. With no
|
||||||
workspace selected, falls back to the global setting so behaviour
|
workspace selected, falls back to the global setting so behaviour
|
||||||
outside a project stays global."""
|
outside a project stays global."""
|
||||||
mode = mode if mode in ("off", "auto", "manual") else "off"
|
from .application.model_routing import normalize_mode
|
||||||
|
|
||||||
|
mode = normalize_mode(mode)
|
||||||
project = self._current_project()
|
project = self._current_project()
|
||||||
if project is None:
|
if project is None:
|
||||||
self.config.set_routing_mode_for(surface, mode)
|
self.config.set_routing_mode_for(surface, mode)
|
||||||
@@ -144,6 +165,34 @@ class AppContext:
|
|||||||
self._routing_service = RoutingService(self)
|
self._routing_service = RoutingService(self)
|
||||||
return self._routing_service
|
return self._routing_service
|
||||||
|
|
||||||
|
def routing_application(self):
|
||||||
|
"""The shared :class:`RoutingApplicationService` (R03-T03).
|
||||||
|
|
||||||
|
This is what UI code should call: it owns the Off/Auto/Manual/Fallback
|
||||||
|
policy, the confirm handshake and the never-raise guarantee, while
|
||||||
|
:meth:`routing` remains the scoring engine underneath. Chat, Co4E and
|
||||||
|
AI-Edit all go through this one object, so a change to routing policy is
|
||||||
|
made once instead of three times.
|
||||||
|
|
||||||
|
Built lazily and memoised for the same reason as :meth:`routing`: the
|
||||||
|
pending-switch registry and assessment store must be shared app-wide."""
|
||||||
|
if self._routing_application is None:
|
||||||
|
# Resolve the engine BEFORE taking this lock: routing() takes
|
||||||
|
# _routing_lock, and nesting the two acquisitions is what makes the
|
||||||
|
# ordering fragile in the first place.
|
||||||
|
engine = self.routing()
|
||||||
|
with self._routing_app_lock:
|
||||||
|
if self._routing_application is None:
|
||||||
|
from .application.model_routing import RoutingApplicationService
|
||||||
|
|
||||||
|
self._routing_application = RoutingApplicationService(
|
||||||
|
engine,
|
||||||
|
# Per-workspace mode lookup, so each workspace keeps its
|
||||||
|
# own routing behaviour (see project_routing_mode).
|
||||||
|
mode_reader=self.project_routing_mode,
|
||||||
|
)
|
||||||
|
return self._routing_application
|
||||||
|
|
||||||
def build_active_provider(self):
|
def build_active_provider(self):
|
||||||
"""Construct the currently selected provider (called inside workers)."""
|
"""Construct the currently selected provider (called inside workers)."""
|
||||||
return self.build_provider_for(self.config.active_provider)
|
return self.build_provider_for(self.config.active_provider)
|
||||||
|
|||||||
@@ -0,0 +1,8 @@
|
|||||||
|
"""Contract tests: one shared behaviour suite every implementation must satisfy.
|
||||||
|
|
||||||
|
Unlike unit tests (which test one module in isolation) a contract test is
|
||||||
|
parametrised over EVERY implementation of an interface, so a newly added
|
||||||
|
provider either satisfies the same promises as the existing ones or the suite
|
||||||
|
goes red on the day it is added - not months later, in production, on the one
|
||||||
|
code path that assumed the promise held.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,342 @@
|
|||||||
|
"""Provider contract suite (R03-T01).
|
||||||
|
|
||||||
|
Every provider - the two real adapters and the test double - must honour the
|
||||||
|
same promises declared in ``providers/base.py``:
|
||||||
|
|
||||||
|
1. ``chat()`` returns the canonical assistant message
|
||||||
|
``{"role": "assistant", "content": str, "tool_calls": [...]}``.
|
||||||
|
2. Answer text is streamed through ``on_text`` and equals the returned content.
|
||||||
|
3. Private reasoning goes to ``on_reasoning`` ONLY - it must never leak into the
|
||||||
|
answer, or a reasoning model's chain of thought ends up persisted in history.
|
||||||
|
4. Tool calls come back as ``{"id", "name", "arguments": dict}`` with arguments
|
||||||
|
already parsed - callers must never have to json.loads() them.
|
||||||
|
5. A failure raises ``ProviderError`` and nothing else, so one except clause in
|
||||||
|
the agent loop covers every provider.
|
||||||
|
|
||||||
|
The real adapters are exercised WITHOUT network access by replacing
|
||||||
|
``Provider._request`` with a canned SSE response - which is exactly the seam
|
||||||
|
``providers/base.py`` documents for its TLS retry, so no production code needed
|
||||||
|
changing to make this testable.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cowork_local.domain.models.provider_descriptor import ProviderCapability
|
||||||
|
from cowork_local.infrastructure.providers.provider_registry import (
|
||||||
|
BUILT_IN_PROVIDERS,
|
||||||
|
ProviderRegistry,
|
||||||
|
)
|
||||||
|
from cowork_local.providers.anthropic import AnthropicProvider
|
||||||
|
from cowork_local.providers.base import Provider, ProviderError, ToolSpec
|
||||||
|
from cowork_local.providers.openai_compat import OpenAICompatProvider
|
||||||
|
from tests.fakes import FakeProvider, ScriptedTurn
|
||||||
|
|
||||||
|
|
||||||
|
class _StubResponse:
|
||||||
|
"""Minimal stand-in for a streamed ``requests.Response``.
|
||||||
|
|
||||||
|
Only the members the provider code actually touches are implemented; adding
|
||||||
|
more would invite tests that pass against the stub but not against requests.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, lines: List[str], status_code: int = 200, text: str = "") -> None:
|
||||||
|
self._lines = lines
|
||||||
|
self.status_code = status_code
|
||||||
|
self.text = text
|
||||||
|
self.headers: Dict[str, str] = {}
|
||||||
|
self.encoding = "utf-8"
|
||||||
|
self.closed = False
|
||||||
|
|
||||||
|
def iter_lines(self, decode_unicode: bool = False):
|
||||||
|
yield from self._lines
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self.closed = True
|
||||||
|
|
||||||
|
def json(self) -> Any:
|
||||||
|
return json.loads(self.text or "{}")
|
||||||
|
|
||||||
|
|
||||||
|
def _sse(*payloads: Dict[str, Any]) -> List[str]:
|
||||||
|
"""Render payloads as SSE ``data:`` lines, the wire shape both adapters parse."""
|
||||||
|
return [f"data: {json.dumps(p)}" for p in payloads]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def canned(monkeypatch):
|
||||||
|
"""Return a helper that makes every provider request answer with ``lines``."""
|
||||||
|
|
||||||
|
def _install(lines: List[str], status_code: int = 200, text: str = "") -> Dict[str, Any]:
|
||||||
|
seen: Dict[str, Any] = {}
|
||||||
|
|
||||||
|
def fake_request(self, method, url, **kwargs):
|
||||||
|
# Capture the outgoing payload so tests can assert on how the
|
||||||
|
# canonical message list was translated to the provider's wire format.
|
||||||
|
seen["method"] = method
|
||||||
|
seen["url"] = url
|
||||||
|
seen["json"] = kwargs.get("json")
|
||||||
|
return _StubResponse(lines, status_code=status_code, text=text)
|
||||||
|
|
||||||
|
monkeypatch.setattr(Provider, "_request", fake_request, raising=True)
|
||||||
|
return seen
|
||||||
|
|
||||||
|
return _install
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Shared base-class behaviour every provider inherits
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def _providers_under_test() -> List[Provider]:
|
||||||
|
"""One instance of each implementation, configured but never called."""
|
||||||
|
conf = {"base_url": "https://example.invalid/v1", "api_key": "k", "model": "m"}
|
||||||
|
return [
|
||||||
|
OpenAICompatProvider(dict(conf)),
|
||||||
|
AnthropicProvider(dict(conf)),
|
||||||
|
FakeProvider(),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("provider", _providers_under_test(), ids=lambda p: type(p).__name__)
|
||||||
|
def test_every_provider_exposes_the_base_contract(provider):
|
||||||
|
assert isinstance(provider, Provider)
|
||||||
|
assert callable(provider.chat)
|
||||||
|
assert callable(provider.list_models)
|
||||||
|
assert callable(provider.test_connection)
|
||||||
|
# `name` identifies the provider in usage records and audit entries; an
|
||||||
|
# implementation that forgot to set it would silently report as "base".
|
||||||
|
assert provider.name and provider.name != "base"
|
||||||
|
assert isinstance(provider.supports_vision, bool)
|
||||||
|
assert provider.describe() == f"{provider.name}:{provider.model}"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("provider", _providers_under_test(), ids=lambda p: type(p).__name__)
|
||||||
|
def test_strip_think_removes_inline_reasoning_from_a_final_answer(provider):
|
||||||
|
"""Safety net for gateways that fold reasoning into the content stream: the
|
||||||
|
answer stored in history must never contain a <think> block."""
|
||||||
|
assert provider.strip_think("<think>secret</think>Answer") == "Answer"
|
||||||
|
assert provider.strip_think("Plain answer") == "Plain answer"
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_spec_translates_to_both_wire_formats():
|
||||||
|
"""One ToolSpec must render for both protocols - this is what lets the agent
|
||||||
|
loop build its tool catalogue once and reuse it across providers."""
|
||||||
|
spec = ToolSpec(name="save_file", description="Write a file",
|
||||||
|
parameters={"type": "object", "properties": {}})
|
||||||
|
|
||||||
|
openai_shape = spec.to_openai()
|
||||||
|
anthropic_shape = spec.to_anthropic()
|
||||||
|
|
||||||
|
assert openai_shape["type"] == "function"
|
||||||
|
assert openai_shape["function"]["name"] == "save_file"
|
||||||
|
assert openai_shape["function"]["parameters"] == spec.parameters
|
||||||
|
# Anthropic names the same field `input_schema`; the values must stay equal,
|
||||||
|
# otherwise the same tool would validate differently per provider.
|
||||||
|
assert anthropic_shape["name"] == "save_file"
|
||||||
|
assert anthropic_shape["input_schema"] == spec.parameters
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Streaming contract - real adapters, canned transport
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_openai_compat_streams_text_and_returns_canonical_message(canned):
|
||||||
|
canned(_sse(
|
||||||
|
{"choices": [{"delta": {"content": "Hel"}}]},
|
||||||
|
{"choices": [{"delta": {"content": "lo"}}]},
|
||||||
|
) + ["data: [DONE]"])
|
||||||
|
provider = OpenAICompatProvider({"base_url": "https://x.invalid/v1",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
chunks: List[str] = []
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "hi"}], on_text=chunks.append)
|
||||||
|
|
||||||
|
assert "".join(chunks) == "Hello"
|
||||||
|
assert result["role"] == "assistant"
|
||||||
|
assert result["content"] == "Hello"
|
||||||
|
assert result["tool_calls"] == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_openai_compat_keeps_reasoning_out_of_the_answer(canned):
|
||||||
|
canned(_sse(
|
||||||
|
{"choices": [{"delta": {"reasoning_content": "hmm..."}}]},
|
||||||
|
{"choices": [{"delta": {"content": "42"}}]},
|
||||||
|
) + ["data: [DONE]"])
|
||||||
|
provider = OpenAICompatProvider({"base_url": "https://x.invalid/v1",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
text: List[str] = []
|
||||||
|
reasoning: List[str] = []
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "q"}],
|
||||||
|
on_text=text.append, on_reasoning=reasoning.append)
|
||||||
|
|
||||||
|
assert reasoning == ["hmm..."]
|
||||||
|
assert result["content"] == "42"
|
||||||
|
assert "hmm" not in result["content"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_openai_compat_returns_tool_calls_with_parsed_arguments(canned):
|
||||||
|
"""Arguments arrive as a JSON string split across chunks; the contract says
|
||||||
|
the caller receives a ready-to-use dict."""
|
||||||
|
canned(_sse(
|
||||||
|
{"choices": [{"delta": {"tool_calls": [
|
||||||
|
{"index": 0, "id": "call_a", "function": {"name": "save_file",
|
||||||
|
"arguments": '{"filename":'}}]}}]},
|
||||||
|
{"choices": [{"delta": {"tool_calls": [
|
||||||
|
{"index": 0, "function": {"arguments": '"a.md"}'}}]}}]},
|
||||||
|
) + ["data: [DONE]"])
|
||||||
|
provider = OpenAICompatProvider({"base_url": "https://x.invalid/v1",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "save it"}])
|
||||||
|
|
||||||
|
assert len(result["tool_calls"]) == 1
|
||||||
|
call = result["tool_calls"][0]
|
||||||
|
assert call["id"] == "call_a"
|
||||||
|
assert call["name"] == "save_file"
|
||||||
|
assert call["arguments"] == {"filename": "a.md"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_anthropic_streams_text_and_returns_canonical_message(canned):
|
||||||
|
canned(_sse(
|
||||||
|
{"type": "content_block_delta", "index": 0,
|
||||||
|
"delta": {"type": "text_delta", "text": "Hel"}},
|
||||||
|
{"type": "content_block_delta", "index": 0,
|
||||||
|
"delta": {"type": "text_delta", "text": "lo"}},
|
||||||
|
{"type": "message_stop"},
|
||||||
|
))
|
||||||
|
provider = AnthropicProvider({"base_url": "https://x.invalid",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
chunks: List[str] = []
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "hi"}], on_text=chunks.append)
|
||||||
|
|
||||||
|
assert "".join(chunks) == "Hello"
|
||||||
|
assert result["content"] == "Hello"
|
||||||
|
assert result["role"] == "assistant"
|
||||||
|
|
||||||
|
|
||||||
|
def test_anthropic_keeps_extended_thinking_out_of_the_answer(canned):
|
||||||
|
canned(_sse(
|
||||||
|
{"type": "content_block_delta", "index": 0,
|
||||||
|
"delta": {"type": "thinking_delta", "thinking": "reasoning..."}},
|
||||||
|
{"type": "content_block_delta", "index": 0,
|
||||||
|
"delta": {"type": "text_delta", "text": "42"}},
|
||||||
|
{"type": "message_stop"},
|
||||||
|
))
|
||||||
|
provider = AnthropicProvider({"base_url": "https://x.invalid",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
reasoning: List[str] = []
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "q"}], on_reasoning=reasoning.append)
|
||||||
|
|
||||||
|
assert reasoning == ["reasoning..."]
|
||||||
|
assert result["content"] == "42"
|
||||||
|
|
||||||
|
|
||||||
|
def test_anthropic_returns_tool_calls_with_parsed_arguments(canned):
|
||||||
|
canned(_sse(
|
||||||
|
{"type": "content_block_start", "index": 0,
|
||||||
|
"content_block": {"type": "tool_use", "id": "toolu_1", "name": "save_file"}},
|
||||||
|
{"type": "content_block_delta", "index": 0,
|
||||||
|
"delta": {"type": "input_json_delta", "partial_json": '{"filename":"a.md"}'}},
|
||||||
|
{"type": "message_stop"},
|
||||||
|
))
|
||||||
|
provider = AnthropicProvider({"base_url": "https://x.invalid",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "save"}])
|
||||||
|
|
||||||
|
assert result["tool_calls"] == [
|
||||||
|
{"id": "toolu_1", "name": "save_file", "arguments": {"filename": "a.md"}}
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("factory", [
|
||||||
|
lambda: OpenAICompatProvider({"base_url": "https://x.invalid/v1", "api_key": "k", "model": "m"}),
|
||||||
|
lambda: AnthropicProvider({"base_url": "https://x.invalid", "api_key": "k", "model": "m"}),
|
||||||
|
], ids=["openai_compat", "anthropic"])
|
||||||
|
def test_transport_failure_surfaces_as_provider_error(canned, factory):
|
||||||
|
"""Every failure mode must arrive as ProviderError so the agent loop needs
|
||||||
|
exactly one except clause, whichever provider is active."""
|
||||||
|
canned([], status_code=500, text="boom")
|
||||||
|
|
||||||
|
with pytest.raises(ProviderError):
|
||||||
|
factory().chat([{"role": "user", "content": "hi"}])
|
||||||
|
|
||||||
|
|
||||||
|
def test_fake_provider_satisfies_the_same_streaming_contract():
|
||||||
|
"""The double is only useful as a stand-in if it keeps the same promises the
|
||||||
|
real adapters are held to above."""
|
||||||
|
provider = FakeProvider([ScriptedTurn(text="Hello", reasoning="hmm")])
|
||||||
|
text: List[str] = []
|
||||||
|
reasoning: List[str] = []
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "hi"}],
|
||||||
|
on_text=text.append, on_reasoning=reasoning.append)
|
||||||
|
|
||||||
|
assert "".join(text) == result["content"] == "Hello"
|
||||||
|
assert reasoning == ["hmm"]
|
||||||
|
assert result["role"] == "assistant"
|
||||||
|
assert result["tool_calls"] == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_fake_provider_raises_provider_error_like_the_real_ones():
|
||||||
|
provider = FakeProvider([ScriptedTurn(error="gateway exploded")])
|
||||||
|
|
||||||
|
with pytest.raises(ProviderError):
|
||||||
|
provider.chat([{"role": "user", "content": "hi"}])
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Registry <-> implementation agreement
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
@pytest.mark.parametrize("descriptor", BUILT_IN_PROVIDERS, ids=lambda d: d.id)
|
||||||
|
def test_every_descriptor_builds_a_working_provider(descriptor):
|
||||||
|
"""A descriptor that cannot be built is a catalogue lying to the UI: Settings
|
||||||
|
would list the provider and selecting it would fail at the first message."""
|
||||||
|
registry = ProviderRegistry()
|
||||||
|
conf = {"base_url": "https://x.invalid/v1", "api_key": "k"}
|
||||||
|
|
||||||
|
provider = registry.build(descriptor.id, conf)
|
||||||
|
|
||||||
|
assert isinstance(provider, Provider)
|
||||||
|
# The id, not the shared adapter class name: three descriptors map onto
|
||||||
|
# OpenAICompatProvider, and usage/audit records must still tell them apart.
|
||||||
|
assert provider.name == descriptor.id
|
||||||
|
assert provider.model == descriptor.default_model
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("descriptor", BUILT_IN_PROVIDERS, ids=lambda d: d.id)
|
||||||
|
def test_declared_vision_capability_matches_the_implementation(descriptor):
|
||||||
|
"""``supports_vision`` decides whether an image block may be sent. A
|
||||||
|
descriptor claiming vision for an adapter that cannot translate the block
|
||||||
|
would route image turns into a guaranteed failure."""
|
||||||
|
provider = ProviderRegistry().build(descriptor.id, {"base_url": "u", "api_key": "k"})
|
||||||
|
|
||||||
|
if descriptor.supports(ProviderCapability.VISION):
|
||||||
|
assert provider.supports_vision is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_registry_build_never_mutates_the_caller_config():
|
||||||
|
"""The routing layer runs one turn on a different model; if build() wrote
|
||||||
|
that model back into the config dict it was handed, the override would
|
||||||
|
silently become the user's saved default."""
|
||||||
|
registry = ProviderRegistry()
|
||||||
|
conf = {"base_url": "u", "api_key": "k", "model": "configured-model"}
|
||||||
|
|
||||||
|
provider = registry.build("openai_compat", conf, model="routed-model")
|
||||||
|
|
||||||
|
assert provider.model == "routed-model"
|
||||||
|
assert conf["model"] == "configured-model"
|
||||||
|
|
||||||
|
|
||||||
|
def test_registry_rejects_an_unknown_provider_with_provider_error():
|
||||||
|
with pytest.raises(ProviderError) as excinfo:
|
||||||
|
ProviderRegistry().build("does_not_exist", {})
|
||||||
|
|
||||||
|
# The message lists what IS known, so a typo in config is fixable from the
|
||||||
|
# error alone without opening the source.
|
||||||
|
assert "openai_compat" in str(excinfo.value)
|
||||||
@@ -0,0 +1,346 @@
|
|||||||
|
"""Unit tests for :mod:`application.model_routing` (R03-T03).
|
||||||
|
|
||||||
|
These run against a hand-written fake router rather than ``core.routing``: the
|
||||||
|
point of the service is the DECISION policy around the engine (mode handling,
|
||||||
|
the manual confirm, never-raise behaviour, failure fallback), and mixing in the
|
||||||
|
real scorer would test the wrong thing and drag the suite over its time budget.
|
||||||
|
|
||||||
|
No Qt, no config, no network - the whole file runs in milliseconds, which is the
|
||||||
|
concrete payoff of moving this logic out of ``ui/chat_panel.py``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, List, Optional, Tuple
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cowork_local.application.model_routing import (
|
||||||
|
RoutingApplicationService,
|
||||||
|
RoutingDecision,
|
||||||
|
RoutingMode,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Test doubles shaped like core.routing's RouteResult / SwitchDecision
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
@dataclass
|
||||||
|
class _TaskType:
|
||||||
|
value: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _Decision:
|
||||||
|
score_gain: float = 0.0
|
||||||
|
reason: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _RouteResult:
|
||||||
|
should_switch: bool
|
||||||
|
to: Optional[Tuple[str, str]] = None
|
||||||
|
task_type: Any = None
|
||||||
|
decision: Any = None
|
||||||
|
|
||||||
|
def target(self) -> Optional[Tuple[str, str]]:
|
||||||
|
return self.to
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeRouter:
|
||||||
|
"""Records every route() call and replays a canned result."""
|
||||||
|
|
||||||
|
def __init__(self, result: Any = None, raises: bool = False) -> None:
|
||||||
|
self._result = result or _RouteResult(should_switch=False, decision=_Decision())
|
||||||
|
self._raises = raises
|
||||||
|
self.calls: List[dict] = []
|
||||||
|
|
||||||
|
def route(self, surface, prompt, current_provider, current_model, **kwargs):
|
||||||
|
self.calls.append({"surface": surface, "prompt": prompt,
|
||||||
|
"provider": current_provider, "model": current_model, **kwargs})
|
||||||
|
if self._raises:
|
||||||
|
raise RuntimeError("assessment store is corrupt")
|
||||||
|
return self._result
|
||||||
|
|
||||||
|
|
||||||
|
def _switch_to(provider: str, model: str, gain: float = 0.2, task: str = "coding") -> _RouteResult:
|
||||||
|
return _RouteResult(
|
||||||
|
should_switch=True, to=(provider, model), task_type=_TaskType(task),
|
||||||
|
decision=_Decision(score_gain=gain, reason=f"{task} fit beats current by {gain}"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Mode parsing
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
@pytest.mark.parametrize("raw,expected", [
|
||||||
|
("off", RoutingMode.OFF),
|
||||||
|
("AUTO", RoutingMode.AUTO),
|
||||||
|
(" manual ", RoutingMode.MANUAL),
|
||||||
|
("fallback", RoutingMode.FALLBACK),
|
||||||
|
])
|
||||||
|
def test_parse_accepts_the_config_spellings(raw, expected):
|
||||||
|
assert RoutingMode.parse(raw) is expected
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("raw", ["", None, "nonsense", 0])
|
||||||
|
def test_parse_degrades_unknown_values_to_off(raw):
|
||||||
|
"""A corrupt setting must leave the user's own model alone rather than
|
||||||
|
silently moving their work onto another model."""
|
||||||
|
assert RoutingMode.parse(raw) is RoutingMode.OFF
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# OFF
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_off_never_consults_the_engine():
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "hi", "openai_compat", "gpt-4o-mini",
|
||||||
|
mode="off")
|
||||||
|
|
||||||
|
assert router.calls == [] # not even scored: OFF costs nothing
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.target() == ("openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
|
||||||
|
def test_blank_prompt_is_never_routed():
|
||||||
|
"""An empty message carries no signal to classify; all three legacy copies
|
||||||
|
guarded this and the guard has to survive the move."""
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", " ", "openai_compat", "m", mode="auto")
|
||||||
|
|
||||||
|
assert router.calls == []
|
||||||
|
assert decision.switched is False
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# AUTO
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_auto_switches_silently_and_reports_the_target():
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude-sonnet-4-6", gain=0.31))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "write a function", "openai_compat", "gpt-4o-mini",
|
||||||
|
mode="auto")
|
||||||
|
|
||||||
|
assert decision.switched is True
|
||||||
|
assert decision.target() == ("anthropic", "claude-sonnet-4-6")
|
||||||
|
assert decision.task_type == "coding"
|
||||||
|
assert decision.score_gain == pytest.approx(0.31)
|
||||||
|
assert decision.should_notify is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_auto_keeps_the_current_model_when_no_candidate_wins():
|
||||||
|
router = _FakeRouter(_RouteResult(should_switch=False, decision=_Decision(reason="no gain")))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "hello", "openai_compat", "gpt-4o-mini", mode="auto")
|
||||||
|
|
||||||
|
assert decision.switched is False
|
||||||
|
# The decision still names a model to run on, so the call site never has to
|
||||||
|
# re-derive the fallback itself - the exact drift the three copies suffered.
|
||||||
|
assert decision.target() == ("openai_compat", "gpt-4o-mini")
|
||||||
|
assert decision.should_notify is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_auto_never_asks_for_confirmation():
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
asked: List[RoutingDecision] = []
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
service.route_turn("cowork", "q", "openai_compat", "m", mode="auto",
|
||||||
|
confirm=lambda d: asked.append(d) or True)
|
||||||
|
|
||||||
|
assert asked == []
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# MANUAL
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_manual_switches_only_after_the_user_approves():
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
seen: List[RoutingDecision] = []
|
||||||
|
|
||||||
|
def confirm(proposal: RoutingDecision) -> bool:
|
||||||
|
seen.append(proposal)
|
||||||
|
return True
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "q", "openai_compat", "m",
|
||||||
|
mode="manual", confirm=confirm)
|
||||||
|
|
||||||
|
assert decision.switched is True
|
||||||
|
assert decision.target() == ("anthropic", "claude")
|
||||||
|
# The dialog is handed the full proposal so it can explain the trade-off.
|
||||||
|
assert seen[0].model == "claude"
|
||||||
|
assert seen[0].score_gain > 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_manual_keeps_the_current_model_when_declined():
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "q", "openai_compat", "gpt-4o-mini",
|
||||||
|
mode="manual", confirm=lambda d: False)
|
||||||
|
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.declined is True
|
||||||
|
assert decision.target() == ("openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
|
||||||
|
def test_manual_without_a_confirm_callback_does_not_switch():
|
||||||
|
"""A headless caller (scheduler) has nobody to ask, so Manual must behave as
|
||||||
|
"not approved" rather than as "approved by default"."""
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "q", "openai_compat", "m", mode="manual")
|
||||||
|
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.declined is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_confirm_dialog_that_raises_counts_as_declined():
|
||||||
|
"""If the modal blows up (window closing mid-turn) the safe reading is that
|
||||||
|
the user did NOT consent to running on another model."""
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
def confirm(_proposal):
|
||||||
|
raise RuntimeError("dialog destroyed")
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "q", "openai_compat", "m",
|
||||||
|
mode="manual", confirm=confirm)
|
||||||
|
|
||||||
|
assert decision.switched is False
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# FALLBACK
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_fallback_does_not_switch_up_front():
|
||||||
|
"""The whole point of the mode: honour the user's model choice until it
|
||||||
|
actually fails."""
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "q", "openai_compat", "gpt-4o-mini",
|
||||||
|
mode="fallback")
|
||||||
|
|
||||||
|
assert router.calls == []
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.target() == ("openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
|
||||||
|
def test_fallback_switches_after_a_failure():
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude", gain=0.4))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
decision = service.fallback_after_failure("cowork", "q", "openai_compat", "gpt-4o-mini",
|
||||||
|
mode="fallback")
|
||||||
|
|
||||||
|
assert decision is not None
|
||||||
|
assert decision.switched is True
|
||||||
|
assert decision.target() == ("anthropic", "claude")
|
||||||
|
assert "failed" in decision.reason
|
||||||
|
|
||||||
|
|
||||||
|
def test_fallback_never_returns_the_model_that_just_failed():
|
||||||
|
"""Retrying the model that just went down would spin on the outage."""
|
||||||
|
router = _FakeRouter(_switch_to("openai_compat", "gpt-4o-mini"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
assert service.fallback_after_failure(
|
||||||
|
"cowork", "q", "openai_compat", "gpt-4o-mini", mode="fallback") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_fallback_returns_none_when_there_is_no_alternative():
|
||||||
|
router = _FakeRouter(_RouteResult(should_switch=False, decision=_Decision()))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
assert service.fallback_after_failure("cowork", "q", "openai_compat", "m",
|
||||||
|
mode="auto") is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mode", ["off", "manual"])
|
||||||
|
def test_off_and_manual_do_not_auto_recover_from_a_failure(mode):
|
||||||
|
"""Both modes exist to keep the user in control of which model runs their
|
||||||
|
work; moving it on failure would break that promise silently."""
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
assert service.fallback_after_failure("cowork", "q", "openai_compat", "m",
|
||||||
|
mode=mode) is None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Robustness - routing must never break a chat turn
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_engine_failure_degrades_to_keeping_the_current_model():
|
||||||
|
service = RoutingApplicationService(_FakeRouter(raises=True))
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "q", "openai_compat", "gpt-4o-mini", mode="auto")
|
||||||
|
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.target() == ("openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
|
||||||
|
def test_engine_failure_during_fallback_returns_none():
|
||||||
|
"""A broken router must not mask the original provider error with its own."""
|
||||||
|
service = RoutingApplicationService(_FakeRouter(raises=True))
|
||||||
|
|
||||||
|
assert service.fallback_after_failure("cowork", "q", "p", "m", mode="auto") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_malformed_route_result_is_treated_as_no_switch():
|
||||||
|
"""The engine is a legacy module still under refactor; a missing attribute
|
||||||
|
must degrade, not raise into the middle of a turn."""
|
||||||
|
class _Garbage:
|
||||||
|
should_switch = True # claims a switch but exposes no target()
|
||||||
|
|
||||||
|
service = RoutingApplicationService(_FakeRouter(_Garbage()))
|
||||||
|
|
||||||
|
decision = service.route_turn("cowork", "q", "openai_compat", "m", mode="auto")
|
||||||
|
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.target() == ("openai_compat", "m")
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Per-surface mode lookup
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_mode_is_read_per_surface_when_not_passed_explicitly():
|
||||||
|
"""Each screen has its own Off/Auto/Manual toggle, and workspaces override
|
||||||
|
it - so the surface, not a global setting, decides."""
|
||||||
|
router = _FakeRouter(_switch_to("anthropic", "claude"))
|
||||||
|
modes = {"cowork": "auto", "ai_edit": "off"}
|
||||||
|
service = RoutingApplicationService(router, mode_reader=modes.get)
|
||||||
|
|
||||||
|
assert service.route_turn("cowork", "q", "p", "m").switched is True
|
||||||
|
assert service.route_turn("ai_edit", "q", "p", "m").switched is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_failing_mode_reader_falls_back_to_off():
|
||||||
|
def broken(_surface):
|
||||||
|
raise KeyError("config not loaded yet")
|
||||||
|
|
||||||
|
service = RoutingApplicationService(_FakeRouter(_switch_to("a", "b")),
|
||||||
|
mode_reader=broken)
|
||||||
|
|
||||||
|
assert service.route_turn("cowork", "q", "p", "m").switched is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_required_capabilities_are_passed_through_to_the_engine():
|
||||||
|
"""An image turn must only be routed to a vision-capable model; the filter
|
||||||
|
has to reach the scorer or the constraint is silently dropped."""
|
||||||
|
router = _FakeRouter()
|
||||||
|
service = RoutingApplicationService(router)
|
||||||
|
|
||||||
|
service.route_turn("cowork", "describe this", "p", "m", mode="auto",
|
||||||
|
required_capabilities=["vision"])
|
||||||
|
|
||||||
|
assert router.calls[0]["required_capabilities"] == ["vision"]
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
"""Integration test for the AppContext routing wiring (R03-T04 / R03-T05).
|
||||||
|
|
||||||
|
The three chat surfaces now call ``ctx.routing_application()`` instead of each
|
||||||
|
carrying their own copy of the routing algorithm. The unit tests cover the
|
||||||
|
policy; this file covers the WIRING, which unit tests with a fake router cannot
|
||||||
|
see:
|
||||||
|
|
||||||
|
* the service is built and memoised on the context
|
||||||
|
* it reads the per-workspace mode through ``project_routing_mode``
|
||||||
|
* the legacy ``core.routing.RoutingService`` is what sits underneath it
|
||||||
|
* ``fallback`` survives a round trip through the per-workspace mode store
|
||||||
|
|
||||||
|
Still Qt-free: ``AppContext`` itself imports no widgets, and the config is
|
||||||
|
written into a tmp dir so nothing touches ``~/.cowork_local``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cowork_local.application.model_routing import (
|
||||||
|
RoutingApplicationService,
|
||||||
|
RoutingMode,
|
||||||
|
)
|
||||||
|
from cowork_local.config import AppConfig
|
||||||
|
from cowork_local.state import AppContext
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def ctx(tmp_path: Path) -> AppContext:
|
||||||
|
"""An AppContext backed by a throwaway config file."""
|
||||||
|
return AppContext(AppConfig.load(tmp_path / "config.json"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_routing_application_is_built_and_memoised(ctx):
|
||||||
|
"""One instance per app: the pending-switch registry underneath it must be
|
||||||
|
shared by every surface, so a second call has to return the same object."""
|
||||||
|
first = ctx.routing_application()
|
||||||
|
|
||||||
|
assert isinstance(first, RoutingApplicationService)
|
||||||
|
assert ctx.routing_application() is first
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_legacy_engine_sits_underneath_the_new_service():
|
||||||
|
"""Strangler-fig check (ADR-001 section 4): the scoring engine is reused, not
|
||||||
|
reimplemented. If this ever stops holding, the assessment scores the
|
||||||
|
scheduler probes would no longer be the ones routing decisions use."""
|
||||||
|
from cowork_local.core.routing.service import RoutingService
|
||||||
|
|
||||||
|
config = AppConfig.load(Path("does-not-exist.json"))
|
||||||
|
context = AppContext(config)
|
||||||
|
|
||||||
|
service = context.routing_application()
|
||||||
|
|
||||||
|
assert isinstance(service._router, RoutingService)
|
||||||
|
assert service._router is context.routing()
|
||||||
|
|
||||||
|
|
||||||
|
def test_mode_is_read_through_the_per_workspace_lookup(ctx, monkeypatch):
|
||||||
|
seen = []
|
||||||
|
|
||||||
|
def fake_mode(surface: str) -> str:
|
||||||
|
seen.append(surface)
|
||||||
|
return "off"
|
||||||
|
|
||||||
|
monkeypatch.setattr(ctx, "project_routing_mode", fake_mode)
|
||||||
|
# Built after the patch so the service captures the patched reader.
|
||||||
|
service = RoutingApplicationService(ctx.routing(), mode_reader=ctx.project_routing_mode)
|
||||||
|
|
||||||
|
decision = service.route_turn("co4e", "hello", "openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
assert seen == ["co4e"]
|
||||||
|
assert decision.switched is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_routing_off_by_default_leaves_the_selected_model_alone(ctx):
|
||||||
|
"""Default config has routing off on every surface, so a fresh install must
|
||||||
|
never move a turn to another model."""
|
||||||
|
decision = ctx.routing_application().route_turn(
|
||||||
|
"cowork", "write a function", "openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
assert decision.mode is RoutingMode.OFF
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.target() == ("openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mode", ["off", "auto", "manual", "fallback"])
|
||||||
|
def test_every_mode_survives_a_round_trip_through_the_config(ctx, mode):
|
||||||
|
"""``fallback`` is new (R03-T03); the per-surface store used to whitelist
|
||||||
|
only three values and would have silently downgraded it to "off"."""
|
||||||
|
ctx.set_project_routing_mode("cowork", mode)
|
||||||
|
|
||||||
|
assert ctx.project_routing_mode("cowork") == mode
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unknown_mode_still_falls_back_to_off(ctx):
|
||||||
|
ctx.set_project_routing_mode("cowork", "turbo")
|
||||||
|
|
||||||
|
assert ctx.project_routing_mode("cowork") == "off"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_real_route_call_never_raises_without_any_assessments(ctx):
|
||||||
|
"""The store is empty on a fresh install. Routing must degrade to "keep the
|
||||||
|
current model" rather than raise into the middle of the first message."""
|
||||||
|
ctx.set_project_routing_mode("cowork", "auto")
|
||||||
|
|
||||||
|
decision = ctx.routing_application().route_turn(
|
||||||
|
"cowork", "hello there", "openai_compat", "gpt-4o-mini")
|
||||||
|
|
||||||
|
assert decision.switched is False
|
||||||
|
assert decision.target() == ("openai_compat", "gpt-4o-mini")
|
||||||
@@ -0,0 +1,249 @@
|
|||||||
|
"""Unit tests for :mod:`infrastructure.telemetry.usage_sink` (R03-T06).
|
||||||
|
|
||||||
|
Two things are being protected here:
|
||||||
|
|
||||||
|
1. The **numbers do not change**. Extracting usage recording out of the two
|
||||||
|
providers is only safe if the events built from each wire format carry
|
||||||
|
exactly what ``core.usage_tracker.record`` used to receive - a silent change
|
||||||
|
would corrupt the Dashboard's cost history.
|
||||||
|
2. The **sink can never break a turn**. Telemetry is observability; a broken
|
||||||
|
store must be swallowed (and logged), never raised into a chat turn.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from cowork_local.infrastructure.telemetry import usage_sink as telemetry
|
||||||
|
from cowork_local.providers.anthropic import AnthropicProvider
|
||||||
|
from cowork_local.providers.base import Provider
|
||||||
|
from cowork_local.providers.openai_compat import OpenAICompatProvider
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Event construction - one per wire format
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_openai_usage_block_maps_onto_the_canonical_event():
|
||||||
|
event = telemetry.openai_usage_event("openai_compat", "gpt-4o-mini", {
|
||||||
|
"prompt_tokens": 120,
|
||||||
|
"completion_tokens": 45,
|
||||||
|
"prompt_tokens_details": {"cached_tokens": 100},
|
||||||
|
})
|
||||||
|
|
||||||
|
assert (event.input_tokens, event.output_tokens, event.cached_tokens) == (120, 45, 100)
|
||||||
|
assert event.estimated is False
|
||||||
|
# Cached tokens are a SUBSET of input, so adding them would double-count.
|
||||||
|
assert event.total_tokens == 165
|
||||||
|
|
||||||
|
|
||||||
|
def test_anthropic_usage_accumulator_maps_onto_the_canonical_event():
|
||||||
|
"""Anthropic reports input on message_start and output on message_delta, so
|
||||||
|
providers/anthropic.py accumulates them into in/out/cache keys."""
|
||||||
|
event = telemetry.anthropic_usage_event("anthropic", "claude", {
|
||||||
|
"in": 200, "out": 80, "cache": 150,
|
||||||
|
})
|
||||||
|
|
||||||
|
assert (event.input_tokens, event.output_tokens, event.cached_tokens) == (200, 80, 150)
|
||||||
|
assert event.estimated is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_missing_usage_block_produces_an_estimated_event():
|
||||||
|
event = telemetry.estimated_event("ollama", "llama3.1", "x" * 400, "y" * 40)
|
||||||
|
|
||||||
|
assert event.estimated is True
|
||||||
|
assert event.input_tokens == 100 # ~4 characters per token
|
||||||
|
assert event.output_tokens == 10
|
||||||
|
assert event.cached_tokens == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_estimation_matches_the_legacy_tracker_formula():
|
||||||
|
"""The extraction must not shift a single recorded number, so the estimator
|
||||||
|
is pinned against the one it replaced."""
|
||||||
|
from cowork_local.core import usage_tracker
|
||||||
|
|
||||||
|
for text in ("", "short", "x" * 4001, "unicode - tiếng Việt"):
|
||||||
|
assert telemetry.estimate_tokens(text) == usage_tracker.estimate_tokens(text)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Sinks
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def test_recording_sink_collects_events_for_assertions():
|
||||||
|
sink = telemetry.RecordingUsageSink()
|
||||||
|
|
||||||
|
sink.record(telemetry.UsageEvent("p", "m", input_tokens=10, output_tokens=5))
|
||||||
|
sink.record(telemetry.UsageEvent("p", "m", input_tokens=1, output_tokens=1))
|
||||||
|
|
||||||
|
assert len(sink.events) == 2
|
||||||
|
assert sink.total_tokens == 17
|
||||||
|
|
||||||
|
|
||||||
|
def test_null_sink_discards_without_error():
|
||||||
|
telemetry.NullUsageSink().record(telemetry.UsageEvent("p", "m"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_tracker_sink_forwards_every_field_positionally():
|
||||||
|
"""``core.usage_tracker.record`` takes positional counts plus an ``estimated``
|
||||||
|
keyword; the adapter has to preserve that exact call shape."""
|
||||||
|
seen: Dict[str, Any] = {}
|
||||||
|
|
||||||
|
class _Tracker:
|
||||||
|
@staticmethod
|
||||||
|
def record(provider, model, input_tokens, output_tokens, cached_tokens,
|
||||||
|
estimated=False):
|
||||||
|
# Fields captured explicitly rather than via locals(), which would
|
||||||
|
# also drag in the closed-over `seen` binding itself.
|
||||||
|
seen.update({"provider": provider, "model": model,
|
||||||
|
"input_tokens": input_tokens, "output_tokens": output_tokens,
|
||||||
|
"cached_tokens": cached_tokens, "estimated": estimated})
|
||||||
|
|
||||||
|
telemetry.UsageTrackerSink(tracker=_Tracker()).record(
|
||||||
|
telemetry.UsageEvent("anthropic", "claude", 7, 3, 2, estimated=True))
|
||||||
|
|
||||||
|
assert seen == {"provider": "anthropic", "model": "claude", "input_tokens": 7,
|
||||||
|
"output_tokens": 3, "cached_tokens": 2, "estimated": True}
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_failing_tracker_never_raises_into_the_turn():
|
||||||
|
class _Broken:
|
||||||
|
@staticmethod
|
||||||
|
def record(*_args, **_kwargs):
|
||||||
|
raise OSError("usage store is read-only")
|
||||||
|
|
||||||
|
# Must not raise - the turn that produced this event has already succeeded.
|
||||||
|
telemetry.UsageTrackerSink(tracker=_Broken()).record(telemetry.UsageEvent("p", "m"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_set_default_sink_returns_the_previous_one_for_restoration():
|
||||||
|
replacement = telemetry.RecordingUsageSink()
|
||||||
|
|
||||||
|
previous = telemetry.set_default_sink(replacement)
|
||||||
|
try:
|
||||||
|
assert telemetry.default_sink is replacement
|
||||||
|
finally:
|
||||||
|
telemetry.set_default_sink(previous)
|
||||||
|
assert telemetry.default_sink is previous
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# Provider integration - the seam actually being used
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
class _StubResponse:
|
||||||
|
"""The few members the provider streaming loop touches."""
|
||||||
|
|
||||||
|
def __init__(self, lines: List[str]) -> None:
|
||||||
|
self._lines = lines
|
||||||
|
self.status_code = 200
|
||||||
|
self.headers: Dict[str, str] = {}
|
||||||
|
self.encoding = "utf-8"
|
||||||
|
self.text = ""
|
||||||
|
|
||||||
|
def iter_lines(self, decode_unicode: bool = False):
|
||||||
|
yield from self._lines
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def sink(monkeypatch):
|
||||||
|
"""A per-instance recording sink, so nothing touches the real usage store."""
|
||||||
|
return telemetry.RecordingUsageSink()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def canned(monkeypatch):
|
||||||
|
def _install(lines: List[str]):
|
||||||
|
monkeypatch.setattr(Provider, "_request",
|
||||||
|
lambda self, method, url, **kw: _StubResponse(lines))
|
||||||
|
return _install
|
||||||
|
|
||||||
|
|
||||||
|
def test_openai_provider_reports_server_counts_to_its_sink(canned, sink):
|
||||||
|
canned([
|
||||||
|
'data: ' + json.dumps({"choices": [{"delta": {"content": "hi"}}],
|
||||||
|
"usage": {"prompt_tokens": 11, "completion_tokens": 2,
|
||||||
|
"prompt_tokens_details": {"cached_tokens": 4}}}),
|
||||||
|
"data: [DONE]",
|
||||||
|
])
|
||||||
|
provider = OpenAICompatProvider({"base_url": "https://x.invalid/v1",
|
||||||
|
"api_key": "k", "model": "gpt-4o-mini"})
|
||||||
|
provider.usage_sink = sink
|
||||||
|
|
||||||
|
provider.chat([{"role": "user", "content": "hi"}])
|
||||||
|
|
||||||
|
assert len(sink.events) == 1
|
||||||
|
event = sink.events[0]
|
||||||
|
assert (event.input_tokens, event.output_tokens, event.cached_tokens) == (11, 2, 4)
|
||||||
|
assert event.estimated is False
|
||||||
|
assert event.model == "gpt-4o-mini"
|
||||||
|
|
||||||
|
|
||||||
|
def test_openai_provider_estimates_when_the_gateway_sends_no_usage(canned, sink):
|
||||||
|
canned(['data: ' + json.dumps({"choices": [{"delta": {"content": "hello"}}]}),
|
||||||
|
"data: [DONE]"])
|
||||||
|
provider = OpenAICompatProvider({"base_url": "https://x.invalid/v1",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
provider.usage_sink = sink
|
||||||
|
|
||||||
|
provider.chat([{"role": "user", "content": "hi"}])
|
||||||
|
|
||||||
|
assert sink.events[0].estimated is True
|
||||||
|
assert sink.events[0].output_tokens >= 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_anthropic_provider_reports_stream_counts_to_its_sink(canned, sink):
|
||||||
|
canned(['data: ' + json.dumps(p) for p in (
|
||||||
|
{"type": "message_start", "message": {"usage": {"input_tokens": 30,
|
||||||
|
"cache_read_input_tokens": 10}}},
|
||||||
|
{"type": "content_block_delta", "index": 0,
|
||||||
|
"delta": {"type": "text_delta", "text": "ok"}},
|
||||||
|
{"type": "message_delta", "usage": {"output_tokens": 5}},
|
||||||
|
{"type": "message_stop"},
|
||||||
|
)])
|
||||||
|
provider = AnthropicProvider({"base_url": "https://x.invalid", "api_key": "k",
|
||||||
|
"model": "claude"})
|
||||||
|
provider.usage_sink = sink
|
||||||
|
|
||||||
|
provider.chat([{"role": "user", "content": "hi"}])
|
||||||
|
|
||||||
|
event = sink.events[0]
|
||||||
|
assert (event.input_tokens, event.output_tokens, event.cached_tokens) == (30, 5, 10)
|
||||||
|
assert event.estimated is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_provider_without_an_explicit_sink_uses_the_process_default(canned):
|
||||||
|
"""Existing call sites set no sink, so the default has to keep working -
|
||||||
|
that is what makes this extraction a no-op for production behaviour."""
|
||||||
|
canned(['data: ' + json.dumps({"choices": [{"delta": {"content": "x"}}]}),
|
||||||
|
"data: [DONE]"])
|
||||||
|
recorder = telemetry.RecordingUsageSink()
|
||||||
|
previous = telemetry.set_default_sink(recorder)
|
||||||
|
try:
|
||||||
|
OpenAICompatProvider({"base_url": "https://x.invalid/v1", "api_key": "k",
|
||||||
|
"model": "m"}).chat([{"role": "user", "content": "hi"}])
|
||||||
|
finally:
|
||||||
|
telemetry.set_default_sink(previous)
|
||||||
|
|
||||||
|
assert len(recorder.events) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_sink_that_raises_does_not_fail_the_turn(canned):
|
||||||
|
"""The answer has already been produced by the time usage is recorded;
|
||||||
|
losing the telemetry is strictly better than losing the answer."""
|
||||||
|
canned(['data: ' + json.dumps({"choices": [{"delta": {"content": "x"}}]}),
|
||||||
|
"data: [DONE]"])
|
||||||
|
|
||||||
|
class _Exploding:
|
||||||
|
def record(self, _event):
|
||||||
|
raise RuntimeError("sink is down")
|
||||||
|
|
||||||
|
provider = OpenAICompatProvider({"base_url": "https://x.invalid/v1",
|
||||||
|
"api_key": "k", "model": "m"})
|
||||||
|
provider.usage_sink = _Exploding()
|
||||||
|
|
||||||
|
result = provider.chat([{"role": "user", "content": "hi"}])
|
||||||
|
|
||||||
|
assert result["content"] == "x"
|
||||||
+32
-35
@@ -638,12 +638,16 @@ class ChatPanel(QWidget):
|
|||||||
def _apply_routing(self, text: str, turn: Dict[str, Any]) -> None:
|
def _apply_routing(self, text: str, turn: Dict[str, Any]) -> None:
|
||||||
"""Auto Model Routing hook — run once per outgoing message.
|
"""Auto Model Routing hook — run once per outgoing message.
|
||||||
|
|
||||||
Off → no-op. Auto → silently switch to the best-fit model. Manual → ask
|
The decision itself lives in ``application/model_routing`` (R03-T04):
|
||||||
the user (modal, with the configured confirm timeout) before switching.
|
this method is now only the presentation half — supply the current
|
||||||
|
model, open the confirm dialog when the service asks for one, and render
|
||||||
|
the notice. Off/Auto/Manual/Fallback semantics, the never-raise
|
||||||
|
guarantee and the "which model do we end up on" fallback are the
|
||||||
|
service's job, and are shared with Co4E and AI-Edit instead of being
|
||||||
|
re-implemented here.
|
||||||
|
|
||||||
Sets ``self._routed_provider``/``self._routed_model`` for THIS turn;
|
Sets ``self._routed_provider``/``self._routed_model`` for THIS turn;
|
||||||
:meth:`build_provider` honours them. Never raises — a routing failure
|
:meth:`build_provider` honours them.
|
||||||
must never block sending a message; it just falls back to the tab's
|
|
||||||
own model.
|
|
||||||
"""
|
"""
|
||||||
# Recompute fresh each message; clear any previous turn's override.
|
# Recompute fresh each message; clear any previous turn's override.
|
||||||
self._routed_provider = None
|
self._routed_provider = None
|
||||||
@@ -651,37 +655,30 @@ class ChatPanel(QWidget):
|
|||||||
# An explicitly-pinned Admin agent takes precedence over routing.
|
# An explicitly-pinned Admin agent takes precedence over routing.
|
||||||
if getattr(self, "_admin_agent", None) is not None:
|
if getattr(self, "_admin_agent", None) is not None:
|
||||||
return
|
return
|
||||||
if not (text or "").strip():
|
cur_provider = self.ctx.config.active_provider
|
||||||
|
cur_model = self._model or self.ctx.config.provider_conf(cur_provider).get("model", "")
|
||||||
|
decision = self.ctx.routing_application().route_turn(
|
||||||
|
self.kind, text, cur_provider, cur_model, confirm=self._confirm_routing_switch,
|
||||||
|
)
|
||||||
|
if not decision.switched:
|
||||||
return
|
return
|
||||||
try:
|
self._routed_provider, self._routed_model = decision.target()
|
||||||
mode = self.ctx.project_routing_mode(self.kind) # per-workspace mode
|
notice = self.chat_view.add_status(tr(
|
||||||
if mode == "off":
|
"routing.switched_notice",
|
||||||
return
|
model=decision.model, task=decision.task_type,
|
||||||
service = self.ctx.routing()
|
gain=f"{decision.score_gain:.2f}"))
|
||||||
cur_provider = self.ctx.config.active_provider
|
turn["bubbles"].append(notice)
|
||||||
cur_model = self._model or self.ctx.config.provider_conf(cur_provider).get("model", "")
|
|
||||||
result = service.route(self.kind, text, cur_provider, cur_model, mode_override=mode)
|
def _confirm_routing_switch(self, decision) -> bool:
|
||||||
if not result.should_switch:
|
"""Manual mode: ask the user before moving this turn to another model.
|
||||||
return
|
|
||||||
target = result.target()
|
Passed to the routing service as a callback so the pure-Python decision
|
||||||
if target is None:
|
layer never has to know a modal dialog exists. Returning False (declined
|
||||||
return
|
or timed out) keeps the tab's own model."""
|
||||||
to_provider, to_model = target
|
from .routing_toggle import confirm_switch
|
||||||
if mode == "manual":
|
|
||||||
from .routing_toggle import confirm_switch
|
timeout = float(self.ctx.config.routing.get("confirm_timeout_sec", 60) or 60)
|
||||||
timeout = float(self.ctx.config.routing.get("confirm_timeout_sec", 60) or 60)
|
return bool(confirm_switch(self, decision, timeout))
|
||||||
if not confirm_switch(self, result.decision, timeout):
|
|
||||||
return # declined / timed out → keep current model
|
|
||||||
self._routed_provider = to_provider
|
|
||||||
self._routed_model = to_model
|
|
||||||
notice = self.chat_view.add_status(tr(
|
|
||||||
"routing.switched_notice",
|
|
||||||
model=to_model, task=result.task_type.value,
|
|
||||||
gain=f"{result.decision.score_gain:.2f}"))
|
|
||||||
turn["bubbles"].append(notice)
|
|
||||||
except Exception: # noqa: BLE001 — routing must never block a chat turn
|
|
||||||
self._routed_provider = None
|
|
||||||
self._routed_model = None
|
|
||||||
|
|
||||||
def _compress_messages(self) -> None:
|
def _compress_messages(self) -> None:
|
||||||
"""Manual compress: keep the system prompt + the last 2 turns verbatim and
|
"""Manual compress: keep the system prompt + the last 2 turns verbatim and
|
||||||
|
|||||||
+33
-33
@@ -1847,41 +1847,41 @@ class Co4ETab(QWidget):
|
|||||||
self._run_chat_turn(system_parts, request, model)
|
self._run_chat_turn(system_parts, request, model)
|
||||||
|
|
||||||
def _apply_co4e_routing(self, request: str) -> str:
|
def _apply_co4e_routing(self, request: str) -> str:
|
||||||
"""Route this Co4E turn to the best-fit model. Returns the model id to
|
"""Route this Co4E turn to the best-fit model.
|
||||||
use ('' → provider default) and sets ``self._co4e_routed_provider`` when
|
|
||||||
a cross-provider switch is chosen. Off → no-op. Manual → confirm first.
|
Returns the model id to use ('' -> provider default) and sets
|
||||||
Never raises — falls back to the default model on any error."""
|
``self._co4e_routed_provider`` when a cross-provider switch is chosen.
|
||||||
|
|
||||||
|
The decision comes from the shared ``RoutingApplicationService``
|
||||||
|
(R03-T05) - Off/Auto/Manual/Fallback handling, the confirm handshake and
|
||||||
|
the never-raise guarantee are no longer duplicated here. What stays is
|
||||||
|
only the Co4E-specific presentation: the surface key, and where the
|
||||||
|
notice is rendered.
|
||||||
|
"""
|
||||||
self._co4e_routed_provider = None
|
self._co4e_routed_provider = None
|
||||||
if not (request or "").strip():
|
cur_provider = self.ctx.config.active_provider
|
||||||
return ""
|
cur_model = self.ctx.config.provider_conf(cur_provider).get("model", "")
|
||||||
try:
|
decision = self.ctx.routing_application().route_turn(
|
||||||
mode = self.ctx.project_routing_mode("co4e") # per-workspace mode
|
"co4e", request, cur_provider, cur_model, confirm=self._confirm_routing_switch,
|
||||||
if mode == "off":
|
)
|
||||||
return ""
|
if not decision.switched:
|
||||||
service = self.ctx.routing()
|
|
||||||
cur_provider = self.ctx.config.active_provider
|
|
||||||
cur_model = self.ctx.config.provider_conf(cur_provider).get("model", "")
|
|
||||||
result = service.route("co4e", request, cur_provider, cur_model, mode_override=mode)
|
|
||||||
if not result.should_switch:
|
|
||||||
return ""
|
|
||||||
target = result.target()
|
|
||||||
if target is None:
|
|
||||||
return ""
|
|
||||||
to_provider, to_model = target
|
|
||||||
if mode == "manual":
|
|
||||||
from .routing_toggle import confirm_switch
|
|
||||||
timeout = float(self.ctx.config.routing.get("confirm_timeout_sec", 60) or 60)
|
|
||||||
if not confirm_switch(self, result.decision, timeout):
|
|
||||||
return ""
|
|
||||||
self._co4e_routed_provider = to_provider
|
|
||||||
self._append_chat("system", tr(
|
|
||||||
"routing.switched_notice",
|
|
||||||
model=to_model, task=result.task_type.value,
|
|
||||||
gain=f"{result.decision.score_gain:.2f}"))
|
|
||||||
return to_model
|
|
||||||
except Exception: # noqa: BLE001 — routing must never block a Co4E turn
|
|
||||||
self._co4e_routed_provider = None
|
|
||||||
return ""
|
return ""
|
||||||
|
self._co4e_routed_provider = decision.provider
|
||||||
|
self._append_chat("system", tr(
|
||||||
|
"routing.switched_notice",
|
||||||
|
model=decision.model, task=decision.task_type,
|
||||||
|
gain=f"{decision.score_gain:.2f}"))
|
||||||
|
return decision.model
|
||||||
|
|
||||||
|
def _confirm_routing_switch(self, decision) -> bool:
|
||||||
|
"""Manual mode: ask before moving this Co4E turn to another model.
|
||||||
|
|
||||||
|
Handed to the routing service as a callback so the pure-Python decision
|
||||||
|
layer never needs to know a modal dialog exists."""
|
||||||
|
from .routing_toggle import confirm_switch
|
||||||
|
|
||||||
|
timeout = float(self.ctx.config.routing.get("confirm_timeout_sec", 60) or 60)
|
||||||
|
return bool(confirm_switch(self, decision, timeout))
|
||||||
|
|
||||||
def _extract_agent_directive(self, text: str):
|
def _extract_agent_directive(self, text: str):
|
||||||
m = re.search(r"(?<!\S)/agent:([\w\-.]+)", text)
|
m = re.search(r"(?<!\S)/agent:([\w\-.]+)", text)
|
||||||
|
|||||||
+33
-37
@@ -923,46 +923,42 @@ class FolderTab(QWidget):
|
|||||||
def _ai_apply_routing(self, instruction: str) -> None:
|
def _ai_apply_routing(self, instruction: str) -> None:
|
||||||
"""Auto Model Routing for the AI-Edit surface (always a CODING task).
|
"""Auto Model Routing for the AI-Edit surface (always a CODING task).
|
||||||
|
|
||||||
Off → no-op. Auto → silently pick the best coding model. Manual → ask
|
Sets ``self._ai_routed_provider``/``_ai_routed_model`` for this run;
|
||||||
first. Sets ``self._ai_routed_provider``/``_ai_routed_model`` for this
|
:meth:`_ai_provider` honours them.
|
||||||
run; :meth:`_ai_provider` honours them. Never raises."""
|
|
||||||
|
The policy itself lives in the shared ``RoutingApplicationService``
|
||||||
|
(R03-T05). What stays here is genuinely AI-Edit-specific: the task type
|
||||||
|
is pinned to CODING (an edit instruction is never a QA question, so
|
||||||
|
classifying it would only add noise), and the current model comes from
|
||||||
|
this screen's own picker rather than the global active model."""
|
||||||
|
from ..core.routing.models import TaskType
|
||||||
|
|
||||||
self._ai_routed_provider = None
|
self._ai_routed_provider = None
|
||||||
self._ai_routed_model = None
|
self._ai_routed_model = None
|
||||||
if not (instruction or "").strip():
|
cur_provider = self.ctx.config.active_provider
|
||||||
|
picked = self.ai_model_combo.currentData() if hasattr(self, "ai_model_combo") else None
|
||||||
|
cur_model = picked or self.ctx.config.provider_conf(cur_provider).get("model", "")
|
||||||
|
decision = self.ctx.routing_application().route_turn(
|
||||||
|
"ai_edit", instruction, cur_provider, cur_model,
|
||||||
|
task_type=TaskType.CODING, confirm=self._confirm_routing_switch,
|
||||||
|
)
|
||||||
|
if not decision.switched:
|
||||||
return
|
return
|
||||||
try:
|
self._ai_routed_provider, self._ai_routed_model = decision.target()
|
||||||
from ..core.routing.models import TaskType
|
self.ai_chat.add_status(tr(
|
||||||
mode = self.ctx.project_routing_mode("ai_edit") # per-workspace mode
|
"routing.switched_notice",
|
||||||
if mode == "off":
|
model=decision.model, task=decision.task_type,
|
||||||
return
|
gain=f"{decision.score_gain:.2f}"))
|
||||||
service = self.ctx.routing()
|
|
||||||
cur_provider = self.ctx.config.active_provider
|
def _confirm_routing_switch(self, decision) -> bool:
|
||||||
picked = self.ai_model_combo.currentData() if hasattr(self, "ai_model_combo") else None
|
"""Manual mode: ask before moving this AI-Edit run to another model.
|
||||||
cur_model = picked or self.ctx.config.provider_conf(cur_provider).get("model", "")
|
|
||||||
result = service.route(
|
Passed to the routing service as a callback, keeping the pure-Python
|
||||||
"ai_edit", instruction, cur_provider, cur_model,
|
decision layer free of any Qt dialog knowledge."""
|
||||||
mode_override=mode, task_type=TaskType.CODING,
|
from .routing_toggle import confirm_switch
|
||||||
)
|
|
||||||
if not result.should_switch:
|
timeout = float(self.ctx.config.routing.get("confirm_timeout_sec", 60) or 60)
|
||||||
return
|
return bool(confirm_switch(self, decision, timeout))
|
||||||
target = result.target()
|
|
||||||
if target is None:
|
|
||||||
return
|
|
||||||
to_provider, to_model = target
|
|
||||||
if mode == "manual":
|
|
||||||
from .routing_toggle import confirm_switch
|
|
||||||
timeout = float(self.ctx.config.routing.get("confirm_timeout_sec", 60) or 60)
|
|
||||||
if not confirm_switch(self, result.decision, timeout):
|
|
||||||
return
|
|
||||||
self._ai_routed_provider = to_provider
|
|
||||||
self._ai_routed_model = to_model
|
|
||||||
self.ai_chat.add_status(tr(
|
|
||||||
"routing.switched_notice",
|
|
||||||
model=to_model, task=result.task_type.value,
|
|
||||||
gain=f"{result.decision.score_gain:.2f}"))
|
|
||||||
except Exception: # noqa: BLE001 — routing must never block an edit
|
|
||||||
self._ai_routed_provider = None
|
|
||||||
self._ai_routed_model = None
|
|
||||||
|
|
||||||
def _ai_image_model(self):
|
def _ai_image_model(self):
|
||||||
"""Resolve the model+endpoint for image generation, searching ALL
|
"""Resolve the model+endpoint for image generation, searching ALL
|
||||||
|
|||||||
Reference in New Issue
Block a user