Merge branch 'claudedo/f359858ac98a439593e459df9c5d0a5d'
This commit is contained in:
mika kuns
@@ -0,0 +1,142 @@
|
||||
# Usage monitoring, gate & throttle
|
||||
|
||||
> **Explore-note — verify before trusting.** Distilled map of a subsystem, not authoritative.
|
||||
> Last verified against commit `f6cb825` (2026-08-05).
|
||||
> Drift check: `git log --oneline f6cb825..HEAD -- src/ClaudeDo.Worker/Usage src/ClaudeDo.Worker/Queue src/ClaudeDo.Ui/ViewModels/UsagePillViewModel.cs`
|
||||
> Stable structure only (no line numbers). See docs/explore-notes/README.md.
|
||||
|
||||
Covers `src/ClaudeDo.Worker/Usage/`, the queue's throttle/gate integration, per-run token
|
||||
accounting, and the UI surfaces (usage pill + usage monitor modal).
|
||||
|
||||
## Data source
|
||||
|
||||
`GET https://api.anthropic.com/api/oauth/usage` — an **undocumented** Anthropic endpoint,
|
||||
authenticated with the Bearer access token Claude Code itself keeps fresh at
|
||||
`~/.claude/.credentials.json`. ClaudeDo reads that token, never refreshes it, never logs it.
|
||||
|
||||
Because the endpoint is undocumented and can change without notice, **every consumer
|
||||
fails open**. That is the single most important invariant here.
|
||||
|
||||
## Components (`Usage/`)
|
||||
|
||||
| Type | Role |
|
||||
|---|---|
|
||||
| `UsageModels` | `UsageBucket` / `UsageLimitRow` / `UsageSnapshot`. `UsageBucket.Utilization` is already a 0–100 percent — compare directly against thresholds, don't rescale. |
|
||||
| `ClaudeOAuthUsageClient` | Reads the token, calls the endpoint. Defensive parsing: missing/null buckets → null, missing `limits` → empty list. |
|
||||
| `UsageState` | Threadsafe singleton. A failed poll **never** overwrites the last good snapshot — it only sets `LastError`. |
|
||||
| `UsageMonitorService` | `BackgroundService`; polls on `usage_poll_interval_seconds` (default 60, clamped to min 15 on config load), one poll at startup. Logs a failure at most once per distinct error message. Broadcasts `HubBroadcaster.UsageUpdated` after **every** tick, success or failure. |
|
||||
| `UsageSnapshotBuilder` | Builds the Hub-facing `UsageSnapshotDto` from `UsageState` + `IUsageGate` + `AppSettings`. The one shared place for stale/threshold/gate logic — `WorkerHub.GetUsageSnapshot` and `UsageMonitorService` must not diverge. |
|
||||
| `UsageGate` | Hard pause decision → `UsageGateDecision(IsBlocked, Reason)`. |
|
||||
| `UsageThrottle` | Pure static staging of parallelism ahead of the gate. |
|
||||
| `TranscriptUsageReader` | Aggregates token usage from Claude Code transcripts. |
|
||||
|
||||
Interfaces in `Usage/Interfaces/`: `IUsageClient`, `ITranscriptUsageReader`, `IUsageGate`.
|
||||
|
||||
## The gate (hard pause)
|
||||
|
||||
Thresholds: `AppSettings.UsageGateFiveHourPct` / `UsageGateSevenDayPct` (defaults 80/90).
|
||||
Blocked once `five_hour >= UsageGateFiveHourPct` **or** `seven_day >= UsageGateSevenDayPct`
|
||||
(`>=`, not `>`). Threshold `0` = that bucket never gates.
|
||||
|
||||
What it pauses: **only the queue's slot-fill loop** — new queued tasks don't start.
|
||||
Unaffected: already-running runs, `RunNow`, `ContinueTask`, interactive ConPTY sessions,
|
||||
planning sessions, daily prep (all bypass the queue).
|
||||
|
||||
**Fail-open**: no snapshot yet, a failed last poll, or an app-settings read error all
|
||||
resolve to not-blocked.
|
||||
|
||||
There is **no persistent pause state**. Recovery is just the queue's 30 s backstop timer
|
||||
(`queue_backstop_interval_ms`) re-evaluating the gate on its own once usage drops back
|
||||
under the threshold. A blocked↔free transition is logged and broadcast (`WorkerLog`, Warn
|
||||
on block / Info on resume) exactly **once per change**, not every tick.
|
||||
|
||||
## The throttle (staged parallelism)
|
||||
|
||||
`UsageThrottle.EffectiveSlots(configuredSlots, fiveHourPct, sevenDayPct, softPct, hardPct,
|
||||
gateFiveHourPct, gateSevenDayPct)` — pure static, no state.
|
||||
|
||||
Thresholds `usage_throttle_soft_pct` / `usage_throttle_hard_pct` (defaults 50/65).
|
||||
Whichever of 5h/7d is **more utilized** decides the stage:
|
||||
|
||||
| Utilization | Effective slots |
|
||||
|---|---|
|
||||
| below soft | full configured `max_parallel_executions` |
|
||||
| `>= softPct` | capped at 2 |
|
||||
| `>= hardPct` | capped at 1 |
|
||||
| `>=` either gate threshold | 0 — same hard block as `UsageGate` |
|
||||
|
||||
A threshold of `0` disables that stage. The `0` return is deliberately kept in sync with
|
||||
`UsageGate`'s hard block because both read the same gate thresholds — change one, change both.
|
||||
|
||||
Only **new** slot fills are affected; a run already occupying a slot when the stage tightens
|
||||
runs to completion. Same fail-open policy: no snapshot means no throttling.
|
||||
|
||||
The effective stage (configured vs. effective slots + the decisive bucket, `ThrottleBucket`
|
||||
= `"five_hour"` / `"seven_day"`) rides along on `UsageSnapshotDto` purely for UI display.
|
||||
It does **not** change what the gate gates on.
|
||||
|
||||
## Queue integration (`Queue/QueueService`)
|
||||
|
||||
Per loop tick:
|
||||
|
||||
1. `GetEffectiveMaxParallelAsync` reads `AppSettings.MaxParallelExecutions` and steps it
|
||||
down via `UsageThrottle.EffectiveSlots` against the current `UsageState` snapshot.
|
||||
A missing/failed snapshot fails open to the configured value. A **stage change** (not
|
||||
every tick) logs once.
|
||||
2. Separately, `IUsageGate.EvaluateAsync` — if blocked, the slot-fill loop is skipped
|
||||
entirely for that tick.
|
||||
|
||||
## Per-run token accounting
|
||||
|
||||
`task_runs` stores four raw token fields: `tokens_in` / `tokens_out` /
|
||||
`cache_read_tokens` / `cache_write_tokens`.
|
||||
|
||||
These are **not** read from the stream-json `result` event's `usage.input_tokens` — that is
|
||||
only the uncached remainder of a single API call and undercounts the real prompt size by
|
||||
orders of magnitude once caching kicks in.
|
||||
|
||||
Instead `TaskRunner.ApplyUsageAsync` calls
|
||||
`ITranscriptUsageReader.ReadSessionTotalsAsync(sessionId)` — the session transcript's
|
||||
cumulative raw totals across every assistant message, located by `{sessionId}.jsonl` — and
|
||||
stores the **delta** against prior `task_runs` rows sharing the same `session_id`, so a
|
||||
`--resume`'d run doesn't double-count turns already billed to an earlier run.
|
||||
|
||||
A missing/unreadable transcript leaves all four fields `null`; it never fails the run.
|
||||
|
||||
### `TranscriptUsageReader` details
|
||||
|
||||
Reads `~/.claude/projects/**/*.jsonl`, aggregating by date / model / scope (ClaudeDo vs
|
||||
Other), deduped by `requestId`, with a per-file length+mtime cache.
|
||||
|
||||
`<synthetic>`-model lines are skipped **everywhere** — they are not real API calls.
|
||||
|
||||
## UI surfaces
|
||||
|
||||
- **`UsagePillViewModel`** — one shared instance backs the `UsagePill` control in both the
|
||||
footer and the Mission Control header. Loads via `GetUsageSnapshotAsync`, updates live off
|
||||
`IWorkerClient.UsageUpdatedEvent`. Dot state priority is mutually exclusive:
|
||||
**blocked > stale > warn > normal**. `IsThrottled` (effective slots below configured, and
|
||||
not gate-blocked) adds a tooltip line naming effective/configured slots + decisive bucket.
|
||||
- **`UsageMonitorModalViewModel`** — opened from the pill. Renders one gauge **per row** in
|
||||
`UsageSnapshotDto.Limits` — deliberately **dynamic**, because the `seven_day_opus` /
|
||||
`seven_day_sonnet`-style buckets the raw API returns are plan-dependent and come back
|
||||
`null` on plans that don't have them; a fixed gauge layout would break. Also shows model
|
||||
usage (`GetModelUsageAsync`, ClaudeDo-vs-Other split per model) and top-task usage
|
||||
(`GetTaskUsageAsync`) over a 7d/30d preset or custom range.
|
||||
|
||||
## Hub surface
|
||||
|
||||
- `GetUsageSnapshot() -> UsageSnapshotDto` — percentages/limits/`FetchedAtUtc` are null and
|
||||
`IsStale=true` when no snapshot has landed yet. `IsStale` also trips on a failed last poll
|
||||
or a snapshot older than 3× `usage_poll_interval_seconds`.
|
||||
- `GetModelUsage(from, to)` — thin wrapper over `ITranscriptUsageReader.ReadAsync`.
|
||||
- `GetTaskUsage(from, to)` — top consumers from `task_runs` joined to task/list, grouped per
|
||||
task. Null token columns count as **0**, never drop the row. `Model` comes from that task's
|
||||
most recent run. Sorted by total tokens descending, capped at 100.
|
||||
- `UsageUpdated` event carries the same `UsageSnapshotDto`.
|
||||
|
||||
## Settings columns
|
||||
|
||||
`app_settings`: `usage_gate_five_hour_pct` / `usage_gate_seven_day_pct` (80/90),
|
||||
`usage_throttle_soft_pct` / `usage_throttle_hard_pct` (50/65). All four clamped 0..100 by
|
||||
`AppSettingsRepository.UpdateAsync`. Worker config: `usage_poll_interval_seconds`.
|
||||
Reference in New Issue
Block a user