diff --git a/CLAUDE.md b/CLAUDE.md index c257ef15..44372b20 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -84,3 +84,4 @@ dotnet test tests/ClaudeDo.Worker.Tests/ClaudeDo.Worker.Tests.csproj -c Release - `docs/improvement-plan.md` — improvement snapshot from 2026-04-13 (historical) - `docs/prompts-inventory.md`, `docs/mailbox-proposal.md` — reference material (mailbox integration is parked) - `CHANGELOG.md` — Keep a Changelog format, maintained on release +- `docs/explore-notes/` — distilled maps of complex subsystems from deep exploration (detail too fine for a CLAUDE.md). **Before** deep-exploring a subsystem, check for a matching note first; **after** a deep explore, distill durable findings back and bump its "verified against" commit. Always verify against current code before trusting. See `docs/explore-notes/README.md`. diff --git a/docs/explore-notes/README.md b/docs/explore-notes/README.md new file mode 100644 index 00000000..e68e0e79 --- /dev/null +++ b/docs/explore-notes/README.md @@ -0,0 +1,38 @@ +# Explore-notes + +Distilled, reusable maps of complex subsystems, produced by deep code exploration. +The goal: stop re-exploring the same subsystem from scratch in every new session. + +These sit **between** the CLAUDE.md files and the code: + +- **CLAUDE.md** — high-level orientation, hand-maintained, always-loaded. +- **explore-notes** — deeper subsystem detail (flows, who-calls-whom, invariants) that is + too fine-grained for a CLAUDE.md but stable enough to be worth caching. Read on demand. +- **code** — the only source of truth. + +## Rules + +- **Only stable structure.** Flows, responsibilities, entry points, invariants, relative + file paths. **No line numbers**, no exhaustive symbol dumps — those rot fastest. +- **Verify before trusting.** A note is a starting map, not authority. Always confirm + against current code before acting on it. Each note records the commit it was verified + against so you can diff for drift. +- **Not a substitute for CLAUDE.md.** If a fact belongs in orientation, put it there. + +## Header every note must carry + +``` +> **Explore-note — verify before trusting.** Distilled map of a subsystem, not authoritative. +> Last verified against commit `` (). +> Drift check: `git log --oneline ..HEAD -- ` +> Stable structure only (no line numbers). See docs/explore-notes/README.md. +``` + +## Workflow + +1. **Before** deep-exploring a subsystem, check for a matching note here and read it first; + explore only to fill gaps or confirm. +2. **After** a deep explore, distill the durable findings into a new/updated note and bump + its "verified against" commit line. +3. If the drift check shows the covered paths changed a lot since the verified commit, treat + the note as suspect and re-verify the parts you rely on. diff --git a/docs/explore-notes/worker-task-pipeline.md b/docs/explore-notes/worker-task-pipeline.md new file mode 100644 index 00000000..e5eadd63 --- /dev/null +++ b/docs/explore-notes/worker-task-pipeline.md @@ -0,0 +1,124 @@ +> **Explore-note — verify before trusting.** Distilled map of a subsystem, not authoritative. +> Last verified against commit `896d4b5` (2026-07-23). +> Drift check: `git log --oneline 896d4b5..HEAD -- src/ClaudeDo.Worker` +> Stable structure only (no line numbers). See docs/explore-notes/README.md. + +# Worker: Task Execution Pipeline + +How a task moves Queued → Running → terminal, across `src/ClaudeDo.Worker` +(Queue, Runner, Lifecycle, State, Agents, Worktrees, Hub). + +## End-to-End Flow (Queued → Terminal) + +1. **Enqueue** — `ITaskStateService.EnqueueAsync()` (State/TaskStateService.cs) + - Idle → Queued, then wakes the dispatcher via `IQueueWaker.Wake()`. + +2. **Dispatch** — `QueueService` loop (Queue/QueueService.cs) + - `BackgroundService`; waits for a wake signal or a backstop timer. + - Reads the max-parallel limit from settings; claims a free slot if under limit. + +3. **Atomic Claim** — `IQueuePicker.ClaimNextAsync()` (Queue/QueuePicker.cs) + - Raw SQL `UPDATE ... RETURNING` in one transaction: picks an eligible Queued task + (unblocked, due or unscheduled; sorted by sort_order/created_at), sets status→Running + + started_at, returns the row. Prevents two workers claiming the same task (TOCTOU). + +4. **Slot Execution** — `QueueService.RunInSlotAsync()` (Queue/QueueService.cs) + - For review feedback: resume the prior session if one exists, else fold feedback into + the prompt. Calls `TaskRunner.RunAsync()` / `ContinueAsync()` with `alreadyClaimed=true`. + +5. **Run Preparation** — `TaskRunner.RunAsync()` (Runner/TaskRunner.cs) + - Loads task, list config, subtasks, attachments from the DB. + - `PrepareRunDirectoryAsync()`: worktree (via WorktreeManager) if the list has a WorkingDir, + else sandbox. Generates a per-run MCP token, writes MCP config to disk. + - `StartRunningAsync()` (only if not pre-claimed): atomic Queued → Running. Broadcasts TaskStarted. + +6. **Claude Execution** — `TaskRunner.RunOnceAsync()` (Runner/TaskRunner.cs) + - Creates a TaskRunEntity, points the task at the run's log path. + - Builds claude CLI args (ClaudeArgsBuilder), spawns the process via `IClaudeProcess.RunAsync()` + with prompt + working dir + streaming callback. + - Stream lines → NDJSON log + broadcast via TaskMessage. MCP tools (AskUser, SuggestImprovement) + are scoped by the per-run token. + +7. **Result Handling** — `TaskRunner.HandleSuccess()` / `MarkFailed()` (Runner/TaskRunner.cs) + - Success (exit 0 + result markdown): if worktree, commit + broadcast WorktreeUpdated; then + transition to Done / WaitingForReview / WaitingForChildren (CompleteAsync / SubmitForReviewAsync + / SubmitForChildrenAsync). + - Failure: if a session exists, auto-retry once via ContinueAsync; else MarkFailed → FailAsync. + - All terminal writes use `CancellationToken.None` so a task is never left Running. + +8. **Terminal States** — `ITaskStateService` transitions (State/TaskStateService.cs) + - **Done** CompleteAsync (Running → Done) — top-level success. + - **WaitingForReview** SubmitForReviewAsync (Running → WaitingForReview) — review gate. + - **WaitingForChildren** SubmitForChildrenAsync (Running → WaitingForChildren) — blocks on children. + - **Failed** FailAsync (Running/Queued → Failed). + - **Cancelled** CancelAsync (Running/Queued/WaitingForReview/WaitingForChildren → Cancelled). + +## Component Responsibilities + +**Queue/** +- `QueueService` — main dispatch loop; slot limit; decides when to start tasks. +- `QueuePicker` — atomic Queued→Running claim via raw SQL. +- `QueueWaker` — semaphore for non-blocking, idempotent wake signals. +- `OverrideSlotService` — owns the RunNow / ContinueTask slot (bypasses the queue). + +**Runner/** +- `TaskRunner` — orchestrates the run (prepare, execute, handle result). +- `WorktreeManager` — creates/manages git worktrees; self-heals stale branches. +- `ClaudeProcess` — spawns the claude CLI subprocess; manages streams/logs. +- `TaskRunMcpService` — runtime MCP tools (AskUser, SuggestImprovement). +- `TaskRunTokenRegistry` — per-run MCP identity for tool-access control. +- `InteractiveLaunchSpecService` — config for the task's claude run. + +**State/** +- `TaskStateService` — all task status transitions; guards preconditions; signals queue/hub. + +**Lifecycle/** (startup recovery) +- `StaleTaskRecovery` — tasks stuck Running after a crash/restart → Failed. +- `OrphanRecovery` — dequeues children whose parent is no longer planning (stays attached). +- `AttachmentOrphanRecovery` — cleans orphaned attachment files. +- `TaskResetService` — manual reset to Idle. +- `TaskMergeService` — conflict resolution for worktree merges. + +**Hub/** +- `HubBroadcaster` — single SignalR broadcast point (TaskStarted/TaskUpdated/TaskMessage/RunCreated…). +- `WorkerHub` — SignalR hub + client methods. + +**Agents/** +- `AgentFileService` — file I/O for custom agents. +- `DefaultAgentSeeder` — seeds built-in agents on startup. + +**Worktrees/** +- `WorktreeMaintenanceService` — cleanup, state tracking, overview reporting. + +## Entry Points & Call Chain + +``` +Program.cs (DI setup) + ├─ QueueService (BackgroundService) → ExecuteAsync loop + │ ├─ waits: IQueueWaker.WaitAsync() or timer + │ ├─ claims: IQueuePicker.ClaimNextAsync() + │ └─ runs: TaskRunner.RunAsync() / ContinueAsync() + ├─ Hub clients → WorkerHub methods + │ ├─ Enqueue → ITaskStateService.EnqueueAsync() → Wake() + │ ├─ RunNow → OverrideSlotService.RunNow() → TaskRunner.RunAsync() + │ ├─ ContinueTask→ OverrideSlotService.ContinueTask()→ TaskRunner.ContinueAsync() + │ └─ CancelTask → QueueService.CancelTask() + ├─ Lifecycle recovery (startup): StaleTaskRecovery / OrphanRecovery / AttachmentOrphanRecovery + └─ State transitions → HubBroadcaster.TaskUpdated() +``` + +## Invariants & Conventions + +- **Atomic claiming** — QueuePicker's `UPDATE ... RETURNING` makes Queued→Running atomic. +- **Slot limit** — respects MaxParallelExecutions; a backstop timer wakes even if a Wake() is missed. +- **Pre-claimed tasks** — the dispatcher pre-claims via the picker; the override slot + (RunNow/ContinueTask) must call StartRunningAsync if a task is not pre-claimed. +- **Terminal writes** — use `CancellationToken.None`; a task is never left Running after crash/cancel. +- **Per-run MCP tokens** — each run gets a unique token scoping tool access; unregistered on end. +- **Auto-retry** — one automatic retry if a session exists and the first run failed. +- **Worktree self-heal** — on branch collision, remove phantom worktrees, prune, delete branch, retry add. +- **Review feedback** — stored on the task; consumed once a run reaches a terminal state; a re-queued + task resumes the session or folds feedback into the prompt. +- **Child tasks** — planning creates draft children; finalization requires no Queued children remain; + OrphanRecovery dequeues children if the parent is not planning. +- **Lifecycle recovery** runs at startup: stale-Running → Failed; orphaned children → dequeued but attached. diff --git a/docs/verification-handoff.md b/docs/verification-handoff.md new file mode 100644 index 00000000..77c8e136 --- /dev/null +++ b/docs/verification-handoff.md @@ -0,0 +1,89 @@ +# Verifikations-Handoff (2026-07-23) + +Konsolidierte manuelle Verifikationen aus `docs/open.md` + Memory-Ständen — gedacht für eine frische Session, die diese Punkte am laufenden System durchspielt. Mika bedient die UI, die Session protokolliert Pass/Fail. + +**Vorbedingungen:** Worker + App laufen (SignalR-Port lt. `~/.todo-app/worker.config.json`, aktuell 37821; External MCP 47822). Testliste `ClaudeDoTests` (`C:\TestRepos\ClaudeDoTests`) für zerstörungsfreie Runs nutzen. + +**Achtung:** 7 Bugfix-/Chore-Tasks vom 2026-07-23 (createdBy `claude-code-gap-analysis`) liegen in der Queue der Liste „Claude do". Die Merge-bezogenen Checks (Abschnitte 2 und 4) erst NACH Review/Merge dieser Tasks final abhaken — sie ändern ggf. genau dieses Verhalten. + +**Nacharbeit:** Erledigtes aus `docs/open.md` austragen, dieses File danach löschen. + +--- + +## 1. Detail-Insel & Diff-Viewer (reines Durchklicken) + +- [ ] Detail-Insel komplett: Output/Git/Session-Tabs, Merge-Sektion, Agent-Settings-Overrides (InheritedBadge korrekt), Prep-Panel — nach dem VM-Split (`DetailsIslandViewModel` → Sektions-VMs) alles gebunden, keine leeren Panels. +- [ ] Diff-Viewer: Dateiliste, Added/Deleted/Renamed/Binary-Erkennung, Commit-Range-Diff nach einem Merge. +- [ ] DiffModal-Fehler-State: Commit-Range ohne aufgezeichnete Commits → „Diff nicht mehr verfügbar" statt Crash/leer. +- [ ] „children need attention"-Band auf dem Session-Tab eines Parents mit failed/blocked Kind. + +## 2. Worktree-Pipeline (3 falsifizierbare Fälle) + +- [ ] Happy-Path: Task mit WorkingDir → `worktrees.state='active'`, `head_commit` gesetzt, `diff_stat` non-empty, Branch `claudedo/` existiert auf Disk. +- [ ] No-Changes-Run: → `status='Done'`, `head_commit IS NULL`, `diff_stat IS NULL`. +- [ ] Kein Git-Repo (WorkingDir = `C:\Temp`): → `status='Failed'`, KEINE `worktrees`-Row, Git-Fehler im Log. + +## 3. Planning-Flow-Walkthrough + +- [ ] Draft → Finalize → Kette: Finalize queued NICHT automatisch (Kinder bleiben Idle); „Queue plan" setzt alle nicht-terminalen Kinder Queued, Kette läuft sequenziell durch (blocked-by löst sich je Vorgänger). +- [ ] Parent landet nach letztem terminalen Kind in WaitingForReview; Approve merged die ganze Unit (Parent-Worktree falls Active + jedes Done-Kind in Reihenfolge). +- [ ] UnfinishedPlanning-Modal: Resume / FinalizeNow / Discard. + +## 4. Merge-Editor (Rider-Style 3-Pane) mit echtem Konflikt + +Konflikt provozieren: gleiche Datei auf main ändern, während der Task-Branch sie ändert. Beide Wege testen: **(a)** Single-Task-Approve mit Konflikt, **(b)** Planning-Unit-Merge mit Konflikt in einem Subtask (`PlanningMergeConflict` → Editor öffnet pro Subtask). + +- [ ] Drei Panes: MAIN read-only | Result editierbar | INCOMING read-only; Konfliktblöcke rot, aufgelöst grün, in allen Panes. +- [ ] Gutter-Toggle `›`/`‹`: Seite rein/raus, Klickreihenfolge = Reihenfolge im Result; main/incoming/beide/keine möglich. +- [ ] Nur Konfliktregionen im Result editierbar (Stable read-only); Edits fließen in den Block zurück. +- [ ] Synchrones vertikales Scrollen; File-Switcher bei mehreren Dateien; `M conflicts · K resolved`-Readout; Conflict-Ruler (Klick springt). +- [ ] Continue erst aktiv, wenn ALLE Konflikte in ALLEN Dateien gelöst; Binär-Guard greift. +- [ ] Abort: Tree sauber, Task bleibt WaitingForReview. +- Bekannte Kanten (nur gegenprüfen, nicht als Fail werten): leere Ours-Seite → null-lange Result-Region (Accept geht, Handtippen fummelig); Gutter-Y-Ausrichtung bei sehr hohen Fenstern/großem Scroll; vertikaler Drift nachfolgender Blöcke nach Konflikt mit ungleicher Zeilenzahl. + +## 5. Embedded ConPTY / Mission Control + +- [ ] **Kritisch:** Task-basiert (Kontextmenü „Open ConPTY session"): frischer Task → Worktree wird on-demand angelegt, `claude` startet — verifizieren, dass der Task-Prompt wirklich GESENDET wird (Antwort beginnt), nicht nur im Eingabefeld vorbefüllt ist. +- [ ] Ad-hoc: „New session"-Button → Ordnerwahl → freie Session im gewählten Verzeichnis. +- [ ] Grid↔Tabs-Toggle; Close killt den Prozess + entfernt die Kachel; mehrere Sessions parallel; Pane-Resize reflowt das TUI. +- [ ] Resume: im Worktree-Dir per `claude --continue` möglich (ClaudeDo persistiert die ConPTY-Session-Id bewusst nicht). + +## 6. Pick up in terminal + +- [ ] Sichtbarkeit: Kontextmenü-Eintrag + Terminal-Button (ArrowOut) NUR bei WaitingForReview und Failed; bei Idle/Running/Queued/Done nicht. +- [ ] Klick → neues Windows-Terminal im Worktree-Verzeichnis, `claude --resume ` nimmt die Session mit Kontext wieder auf. +- [ ] Fehlerfälle surfacen sauber (Footer-Strip bzw. Fehlerdialog): laufende/gequeuete Task, keine persistierte Session-Id, kein aktiver Worktree. +- Bekannte Kante: parked-Idle (reject-park) hat oft Session+Worktree, zeigt die Aktion aber bewusst NICHT (Idle nicht unterscheidbar). Nervt das in der Praxis → `CanPickUpInTerminal` erweitern. + +## 7. AskUser (Frage aus laufendem Task) + +- [ ] Task, dessen Prompt eine Rückfrage erzwingt → Frage erscheint im Task-Monitor (Mission Control), Prozess wartet. +- [ ] Antwort inline absenden → Run läuft mit der Antwort weiter. +- [ ] Timeout-/Cleanup-Verhalten der PendingQuestionRegistry (Frage unbeantwortet lassen): Task schlägt kontrolliert fehl, UI räumt die Frage auf (MCP_TOOL_TIMEOUT-Gotcha). + +## 8. Session Skills (E2E + UI) + +- [ ] Settings → Skills: `https://github.com/DietrichGebert/ponytail` installieren → 6 Skills erscheinen (ponytail, -help, -review, -audit, -debt, -gain), auf Commit gepinnt, Dateien unter `~/.todo-app/session-skills//`. +- [ ] Skill per-Task (Agent-Settings-Flyout) oder global aktivieren → Task laufen lassen → im Worktree liegt `.claude/skills//`, Agent kann ihn nutzen, `git status` im Worktree bleibt sauber (info/exclude greift). +- [ ] Gegenprobe: nicht-aktivierte/andere interaktive Session sieht den Skill NICHT (kein Leak nach `~/.claude`). +- [ ] UI: Skills-Tab (Install-Zeile, Karten mit Update/Remove), Checkbox-Listen im General-Tab + AgentConfigEditor (Flyout-Höhe!), leerer Zustand (0 Skills — Empty-State fehlt evtl., dann entscheiden), lange Namen/URLs (Trimming). + +## 9. Attachments (Drag & Drop + MCP) + +- [ ] Drop aufs Detail-Pane: „Drop to attach"-Overlay, Datei erscheint in der Liste, landet unter `~/.todo-app/attachments//`; „Add file…"-Picker; Remove-Button. +- [ ] `ComposedPreview` enthält die Attachment-Pfade („## Reference files"). +- [ ] MCP: `add_task_attachment` / `list_task_attachments` / `remove_task_attachment`; Running-Task verweigert add/remove. + +## 10. Daily Prep (Prime) & Weekly Report + +- [ ] Prime-Trigger: Schedule feuert bzw. „Plan day" manuell → Prep-Log streamt live, `daily-prep.log` enthält letzten Run, MyDay-Auswahl respektiert `DailyPrepMaxTasks`. +- [ ] Weekly Report: Range-Default „seit letztem Standup-Wochentag → heute", Markdown rendert, Cache pro Range. + +## 11. Self-Update & Autostart (am Gerät) + +- [ ] Update-Banner → Update durchführen → danach „up to date". +- [ ] Autostart: Logoff/Logon startet den Worker (Startup-`.lnk`); Update-Pfad erhält den Autostart; Uninstall entfernt die `.lnk`. + +## 12. Status-Bar / RunNow (Mini-Codecheck, erst messen) + +- [ ] Worker trennen/verbinden → prüfen, ob RunNow-Enable pro Task-Row sauber re-evaluiert (Connection-State lebt in `IslandsShellViewModel`). Nur fixen, wenn tatsächlich kaputt.