diff --git a/docs/explore-notes/usage-monitoring.md b/docs/explore-notes/usage-monitoring.md index 3d913ee4..4ba1d7b1 100644 --- a/docs/explore-notes/usage-monitoring.md +++ b/docs/explore-notes/usage-monitoring.md @@ -1,15 +1,18 @@ # Usage monitoring, gate & throttle > **Explore-note — verify before trusting.** Distilled map of a subsystem, not authoritative. -> Last verified against commit `f6cb825` (2026-08-05), plus the uncommitted per-bucket-throttle / -> draggable-gauge change of 2026-08-06 (this note already describes that newer state). -> Drift check: `git log --oneline f6cb825..HEAD -- src/ClaudeDo.Worker/Usage src/ClaudeDo.Worker/Queue src/ClaudeDo.Ui/ViewModels/UsagePillViewModel.cs` +> Last verified against commit `bf56cd8c` (2026-08-24). +> Drift check: `git log --oneline bf56cd8c..HEAD -- src/ClaudeDo.Worker/Usage src/ClaudeDo.Worker/Queue src/ClaudeDo.Ui/ViewModels/UsagePillViewModel.cs` > Stable structure only (no line numbers). See docs/explore-notes/README.md. Covers `src/ClaudeDo.Worker/Usage/`, the queue's throttle/gate integration, per-run token accounting, and the UI surfaces (usage pill + usage monitor modal). -## Data source +**Two independent data sources, on purpose.** Limits/gate/throttle come from the OAuth poll +below; the *analytics* (per-model and per-task breakdown, cost) come from the external +TokenTracker CLI. Neither one can break the other. + +## Data source (limits, gate, throttle) `GET https://api.anthropic.com/api/oauth/usage` — an **undocumented** Anthropic endpoint, authenticated with the Bearer access token Claude Code itself keeps fresh at @@ -29,9 +32,11 @@ fails open**. That is the single most important invariant here. | `UsageSnapshotBuilder` | Builds the Hub-facing `UsageSnapshotDto` from `UsageState` + `IUsageGate` + `AppSettings`. The one shared place for stale/threshold/gate logic — `WorkerHub.GetUsageSnapshot` and `UsageMonitorService` must not diverge. | | `UsageGate` | Hard pause decision → `UsageGateDecision(IsBlocked, Reason)`. | | `UsageThrottle` | Pure static staging of parallelism ahead of the gate. | -| `TranscriptUsageReader` | Aggregates token usage from Claude Code transcripts. | +| `TranscriptUsageReader` | Reads **one** session's cumulative totals for per-run accounting. **No** aggregation — the date/model/scope roll-up moved to `TokenTracker/`. | +| `TokenTracker/` | The external analytics backend: cost + per-model/per-task breakdown. Own section below. | -Interfaces in `Usage/Interfaces/`: `IUsageClient`, `ITranscriptUsageReader`, `IUsageGate`. +Interfaces in `Usage/Interfaces/`: `IUsageClient`, `ITranscriptUsageReader`, `IUsageGate`; +plus `Usage/TokenTracker/Interfaces/ITokenTrackerClient`. ## The gate (hard pause) @@ -109,18 +114,56 @@ stores the **delta** against prior `task_runs` rows sharing the same `session_id A missing/unreadable transcript leaves all four fields `null`; it never fails the run. +`ReadSessionTotalsAsync` **stays** — it is the per-run source of truth and is deliberately *not* +routed through TokenTracker, whose export has no per-run granularity (it aggregates a whole +session, which a `--resume`'d task spreads over several runs). + ### `TranscriptUsageReader` details -Reads `~/.claude/projects/**/*.jsonl`, aggregating by date / model / scope (ClaudeDo vs -Other), deduped by `requestId`, with a per-file length+mtime cache. +Locates a single `{sessionId}.jsonl` under `~/.claude/projects/**`, sums the raw `usage` fields of +every assistant message, deduped by `requestId` (falling back to `message.id`), with a per-file +length+mtime cache. ``-model lines are skipped — they are not real API calls. -`ReadAsync` **skips any file whose mtime predates the window start minus one day** — it cannot hold -a record inside the range, and the full history is large (measured 2026-08-06: 501 files / 230 MB / -77k lines ≈ 1.7 s to parse cold; a 7-day range touches ~190 files / ~106 MB). The one-day slack -absorbs local-vs-UTC skew between mtime and record timestamps. `ReadSessionTotalsAsync` is -unaffected — it looks up a single `{sessionId}.jsonl`. +There is no history-wide `ReadAsync` any more; `WorkerConfig` is still a constructor parameter +purely so the DI registration and call sites stayed untouched. -``-model lines are skipped **everywhere** — they are not real API calls. +## Analytics via TokenTracker + +The per-model / per-task / cost view comes from **TokenTracker**, an external MIT-licensed CLI +(`tokentracker-cli`, npm) that reads the same local Claude transcripts. Everything lives in +`Usage/TokenTracker/`; the pure parts (`SessionHash`, `TokenTrackerArgs`, +`TokenTrackerExportParser`, `TokenTrackerAggregator`) have no I/O and are unit-tested, only +`TokenTrackerClient` (processes) and `TokenTrackerService` (cache coordination) touch the world. + +- **Invocation:** `tokentracker sessions --from --to --no-git --format json` + (`TokenTrackerArgs`). `--no-git` matters — without it TokenTracker runs `git log` inside every + session's working directory. We **never** call `tokentracker init`: that would write hooks into + the user's global `~/.claude/settings.json` and switch on cloud sync. Install is an explicit + `npm i -g tokentracker-cli`, never `npx`. Resolution goes through the shared + `ExecutableResolver` because an npm CLI is a `.cmd` shim. +- **The join:** the export never carries a raw session id, only `session_hash` = + `sha256("claude\0" + sessionId)` hex, first 24 chars (`SessionHash.ForClaudeSession`, + mirroring TokenTracker's `lib/session-analytics.js`). That hash set — built from + `task_runs.session_id` — is also what decides the **ClaudeDo vs. other** scope split. Note this + is *narrower* than the old cwd-based split: an interactive or planning session has no + `task_runs` row and therefore counts as "other". +- **One wide fetch, local filtering:** `TokenTrackerService.WindowDays` = 90. TokenTracker's + `buildSessionAnalytics` walks the whole history regardless of `--from`/`--to` (v0.88.4 returns + the same rows either way), so narrowing the request buys nothing while making every range switch + pay again. Range switching is then pure in-memory filtering in `TokenTrackerAggregator`. +- **Freshness:** `EnsureFreshAsync(maxAge)`. No export at all → the caller awaits the fetch; + an export older than **15 minutes** → served immediately while a refresh runs behind the + caller's back. `RefreshAsync` is single-flight (a second caller waits for the in-flight run + instead of spawning another process). The probe (CLI present? Node ≥ 20?) is cached until an + install forces it. +- **Version check:** the export's `version` field is compared against + `TokenTrackerExportParser.SupportedFormatVersion` (11). A mismatch is treated as a **failure** + — the state keeps the last good export and reports the version in `LastError`. +- **Fail-open everywhere:** `TokenTrackerState` mirrors `UsageState` — a failed or unparseable + fetch never overwrites a good export, it only sets `LastError`. CLI missing → empty analytics + plus a hint card in the UI. Nothing here can throw into the queue. +- **Limits, gate and throttle do *not* come from TokenTracker.** They are the OAuth poll above. + A broken or absent TokenTracker costs you the cost columns, nothing else. ## UI surfaces @@ -153,17 +196,36 @@ unaffected — it looks up a single `{sessionId}.jsonl`. `seven_day_sonnet`-style buckets the raw API returns are plan-dependent and come back `null` on plans that don't have them; a fixed gauge layout would break. Also shows model usage (`GetModelUsageAsync`, ClaudeDo-vs-Other split per model) and top-task usage - (`GetTaskUsageAsync`) over a 7d/30d preset or custom range. + (`GetTaskUsageAsync`) over a 7d/30d preset or custom range — both with a **cost** column, the + task table additionally with retries. +- **Analytics header vs. hint card** — the modal shows exactly one of the two. With TokenTracker + present: total tokens + total cost, a freshness stamp, and a manual `RefreshAnalytics` button. + Without it: a hint card explaining what TokenTracker is (local only, no `init`, no cloud sync) + plus an `Install TokenTracker` button — which is hidden and replaced by a Node-version hint when + Node < 20 or Node is missing, because the install would just fail. Install progress and every + failure go through `ErrorReported` into the shell's footer log strip, not a private banner. ## Hub surface - `GetUsageSnapshot() -> UsageSnapshotDto` — percentages/limits/`FetchedAtUtc` are null and `IsStale=true` when no snapshot has landed yet. `IsStale` also trips on a failed last poll or a snapshot older than 3× `usage_poll_interval_seconds`. -- `GetModelUsage(from, to)` — thin wrapper over `ITranscriptUsageReader.ReadAsync`. -- `GetTaskUsage(from, to)` — top consumers from `task_runs` joined to task/list, grouped per - task. Null token columns count as **0**, never drop the row. `Model` comes from that task's - most recent run. Sorted by total tokens descending, capped at 100. +- `GetModelUsage(from, to)` — filters + aggregates the **cached TokenTracker export** + (`TokenTrackerAggregator.ByModel`). Returns an empty list rather than throwing when TokenTracker + is unconfigured or no export landed. `Sessions` replaced the old assistant-message count — the + export has no message granularity. +- `GetTaskUsage(from, to)` — unchanged source: top consumers from `task_runs` joined to task/list, + grouped per task. Null token columns count as **0**, never drop the row. `Model` comes from that + task's most recent run. Sorted by total tokens descending, capped at 100. Cost/retries/ + productive/one-shot are an **enrichment** via the `session_hash` join — a missing export just + leaves those columns empty, it never triggers a fetch (`GetModelUsage` is what does). +- `GetTokenTrackerStatus()` — probe + state, as `TokenTrackerStatusDto(Installed, Version, NodeOk, + NodeVersion, LastFetchedUtc, LastError, FormatVersion, SessionCount)`. Never throws; an + unconfigured worker comes back as not-installed with a reason. +- `RefreshTokenTracker()` — forces a fetch, then re-probes and returns the status. +- `InstallTokenTracker()` — `npm i -g tokentracker-cli`, streaming npm's output line by line into + `WorkerLog` (npm writes progress to stderr; forwarding it is what keeps a multi-minute global + install from looking hung), then force-re-probes. - `UsageUpdated` event carries the same `UsageSnapshotDto`. ## Settings columns diff --git a/src/ClaudeDo.Ui/CLAUDE.md b/src/ClaudeDo.Ui/CLAUDE.md index 3513d04c..1b7f5a78 100644 --- a/src/ClaudeDo.Ui/CLAUDE.md +++ b/src/ClaudeDo.Ui/CLAUDE.md @@ -58,7 +58,7 @@ Design/ — Tokens.axaml (design tokens; merged before styles) | `ListSettingsModalViewModel` | Name, working dir, commit type, "manual list" flag, `VerifyCommand`, delete. Hosts the shared `AgentConfigEditorViewModel` as `Agent` (scope=List) — ⚠️ save delegates to `Agent.SaveAsync(verifyCommand)` because both land in the same `list_config` row via one `UpdateListConfig` call and would otherwise clobber each other. | | `WeeklyReportModalViewModel` | Range pickers default "since last standup weekday → today", cached per range. | | `MergeHelperSelectionModalViewModel` | "Let Claude handle it" picker → [conpty-sessions](../../docs/explore-notes/conpty-sessions.md). | -| `UsageMonitorModalViewModel` | Opened from the usage pill (shown **before** the data loads via `BeginLoad`); gauges are **dynamic** per `UsageSnapshotDto.Limits` row, and the 5h/7d ones carry three draggable stage markers (soft/hard/gate) via `UsageGaugeBar` + the pure `UsageThresholdDrag`, plus a colour-matched legend with a `NumericUpDown` per stage → [usage-monitoring](../../docs/explore-notes/usage-monitoring.md). | +| `UsageMonitorModalViewModel` | Opened from the usage pill (shown **before** the data loads via `BeginLoad`); gauges are **dynamic** per `UsageSnapshotDto.Limits` row, and the 5h/7d ones carry three draggable stage markers (soft/hard/gate) via `UsageGaugeBar` + the pure `UsageThresholdDrag`, plus a colour-matched legend with a `NumericUpDown` per stage → [usage-monitoring](../../docs/explore-notes/usage-monitoring.md). The **analytics** part (models/tasks/cost) comes from the external TokenTracker export, not from us: without the CLI the modal shows a hint card with an install button (gated on Node ≥ 20) instead of the totals header. Gauges, gate and throttle are unaffected by it. | Self-explanatory: `RepoImportModalViewModel` (bulk-create lists from git repos; already-wired repos disabled), `MergeModalViewModel`, `WorktreesOverviewModalViewModel`, diff --git a/src/ClaudeDo.Worker/CLAUDE.md b/src/ClaudeDo.Worker/CLAUDE.md index b224009f..8e6f8149 100644 --- a/src/ClaudeDo.Worker/CLAUDE.md +++ b/src/ClaudeDo.Worker/CLAUDE.md @@ -36,7 +36,8 @@ Worker/ Prime/ — daily prep ("Prime Claude"): PrimeScheduler, PrimeRunner, DailyPrepPrompt, NextDueCalculator, PrimeScheduleSignal Online/ — optional Online Inbox sync (off by default; zero network when disabled) - Usage/ — OAuth usage monitor, gate, throttle, transcript token reader + Usage/ — OAuth usage monitor, gate, throttle, per-session token reader; + TokenTracker/ = the external analytics backend (cost + per-model/per-task breakdown) ``` Interfaces (`IQueueWaker`, `IPrimeClock`, `ITaskStateService`, …) live in an `Interfaces/`