docs: explore-notes convention + verification handoff for manual checks
This commit is contained in:
mika kuns
@@ -0,0 +1,124 @@
|
||||
> **Explore-note — verify before trusting.** Distilled map of a subsystem, not authoritative.
|
||||
> Last verified against commit `896d4b5` (2026-07-23).
|
||||
> Drift check: `git log --oneline 896d4b5..HEAD -- src/ClaudeDo.Worker`
|
||||
> Stable structure only (no line numbers). See docs/explore-notes/README.md.
|
||||
|
||||
# Worker: Task Execution Pipeline
|
||||
|
||||
How a task moves Queued → Running → terminal, across `src/ClaudeDo.Worker`
|
||||
(Queue, Runner, Lifecycle, State, Agents, Worktrees, Hub).
|
||||
|
||||
## End-to-End Flow (Queued → Terminal)
|
||||
|
||||
1. **Enqueue** — `ITaskStateService.EnqueueAsync()` (State/TaskStateService.cs)
|
||||
- Idle → Queued, then wakes the dispatcher via `IQueueWaker.Wake()`.
|
||||
|
||||
2. **Dispatch** — `QueueService` loop (Queue/QueueService.cs)
|
||||
- `BackgroundService`; waits for a wake signal or a backstop timer.
|
||||
- Reads the max-parallel limit from settings; claims a free slot if under limit.
|
||||
|
||||
3. **Atomic Claim** — `IQueuePicker.ClaimNextAsync()` (Queue/QueuePicker.cs)
|
||||
- Raw SQL `UPDATE ... RETURNING` in one transaction: picks an eligible Queued task
|
||||
(unblocked, due or unscheduled; sorted by sort_order/created_at), sets status→Running
|
||||
+ started_at, returns the row. Prevents two workers claiming the same task (TOCTOU).
|
||||
|
||||
4. **Slot Execution** — `QueueService.RunInSlotAsync()` (Queue/QueueService.cs)
|
||||
- For review feedback: resume the prior session if one exists, else fold feedback into
|
||||
the prompt. Calls `TaskRunner.RunAsync()` / `ContinueAsync()` with `alreadyClaimed=true`.
|
||||
|
||||
5. **Run Preparation** — `TaskRunner.RunAsync()` (Runner/TaskRunner.cs)
|
||||
- Loads task, list config, subtasks, attachments from the DB.
|
||||
- `PrepareRunDirectoryAsync()`: worktree (via WorktreeManager) if the list has a WorkingDir,
|
||||
else sandbox. Generates a per-run MCP token, writes MCP config to disk.
|
||||
- `StartRunningAsync()` (only if not pre-claimed): atomic Queued → Running. Broadcasts TaskStarted.
|
||||
|
||||
6. **Claude Execution** — `TaskRunner.RunOnceAsync()` (Runner/TaskRunner.cs)
|
||||
- Creates a TaskRunEntity, points the task at the run's log path.
|
||||
- Builds claude CLI args (ClaudeArgsBuilder), spawns the process via `IClaudeProcess.RunAsync()`
|
||||
with prompt + working dir + streaming callback.
|
||||
- Stream lines → NDJSON log + broadcast via TaskMessage. MCP tools (AskUser, SuggestImprovement)
|
||||
are scoped by the per-run token.
|
||||
|
||||
7. **Result Handling** — `TaskRunner.HandleSuccess()` / `MarkFailed()` (Runner/TaskRunner.cs)
|
||||
- Success (exit 0 + result markdown): if worktree, commit + broadcast WorktreeUpdated; then
|
||||
transition to Done / WaitingForReview / WaitingForChildren (CompleteAsync / SubmitForReviewAsync
|
||||
/ SubmitForChildrenAsync).
|
||||
- Failure: if a session exists, auto-retry once via ContinueAsync; else MarkFailed → FailAsync.
|
||||
- All terminal writes use `CancellationToken.None` so a task is never left Running.
|
||||
|
||||
8. **Terminal States** — `ITaskStateService` transitions (State/TaskStateService.cs)
|
||||
- **Done** CompleteAsync (Running → Done) — top-level success.
|
||||
- **WaitingForReview** SubmitForReviewAsync (Running → WaitingForReview) — review gate.
|
||||
- **WaitingForChildren** SubmitForChildrenAsync (Running → WaitingForChildren) — blocks on children.
|
||||
- **Failed** FailAsync (Running/Queued → Failed).
|
||||
- **Cancelled** CancelAsync (Running/Queued/WaitingForReview/WaitingForChildren → Cancelled).
|
||||
|
||||
## Component Responsibilities
|
||||
|
||||
**Queue/**
|
||||
- `QueueService` — main dispatch loop; slot limit; decides when to start tasks.
|
||||
- `QueuePicker` — atomic Queued→Running claim via raw SQL.
|
||||
- `QueueWaker` — semaphore for non-blocking, idempotent wake signals.
|
||||
- `OverrideSlotService` — owns the RunNow / ContinueTask slot (bypasses the queue).
|
||||
|
||||
**Runner/**
|
||||
- `TaskRunner` — orchestrates the run (prepare, execute, handle result).
|
||||
- `WorktreeManager` — creates/manages git worktrees; self-heals stale branches.
|
||||
- `ClaudeProcess` — spawns the claude CLI subprocess; manages streams/logs.
|
||||
- `TaskRunMcpService` — runtime MCP tools (AskUser, SuggestImprovement).
|
||||
- `TaskRunTokenRegistry` — per-run MCP identity for tool-access control.
|
||||
- `InteractiveLaunchSpecService` — config for the task's claude run.
|
||||
|
||||
**State/**
|
||||
- `TaskStateService` — all task status transitions; guards preconditions; signals queue/hub.
|
||||
|
||||
**Lifecycle/** (startup recovery)
|
||||
- `StaleTaskRecovery` — tasks stuck Running after a crash/restart → Failed.
|
||||
- `OrphanRecovery` — dequeues children whose parent is no longer planning (stays attached).
|
||||
- `AttachmentOrphanRecovery` — cleans orphaned attachment files.
|
||||
- `TaskResetService` — manual reset to Idle.
|
||||
- `TaskMergeService` — conflict resolution for worktree merges.
|
||||
|
||||
**Hub/**
|
||||
- `HubBroadcaster` — single SignalR broadcast point (TaskStarted/TaskUpdated/TaskMessage/RunCreated…).
|
||||
- `WorkerHub` — SignalR hub + client methods.
|
||||
|
||||
**Agents/**
|
||||
- `AgentFileService` — file I/O for custom agents.
|
||||
- `DefaultAgentSeeder` — seeds built-in agents on startup.
|
||||
|
||||
**Worktrees/**
|
||||
- `WorktreeMaintenanceService` — cleanup, state tracking, overview reporting.
|
||||
|
||||
## Entry Points & Call Chain
|
||||
|
||||
```
|
||||
Program.cs (DI setup)
|
||||
├─ QueueService (BackgroundService) → ExecuteAsync loop
|
||||
│ ├─ waits: IQueueWaker.WaitAsync() or timer
|
||||
│ ├─ claims: IQueuePicker.ClaimNextAsync()
|
||||
│ └─ runs: TaskRunner.RunAsync() / ContinueAsync()
|
||||
├─ Hub clients → WorkerHub methods
|
||||
│ ├─ Enqueue → ITaskStateService.EnqueueAsync() → Wake()
|
||||
│ ├─ RunNow → OverrideSlotService.RunNow() → TaskRunner.RunAsync()
|
||||
│ ├─ ContinueTask→ OverrideSlotService.ContinueTask()→ TaskRunner.ContinueAsync()
|
||||
│ └─ CancelTask → QueueService.CancelTask()
|
||||
├─ Lifecycle recovery (startup): StaleTaskRecovery / OrphanRecovery / AttachmentOrphanRecovery
|
||||
└─ State transitions → HubBroadcaster.TaskUpdated()
|
||||
```
|
||||
|
||||
## Invariants & Conventions
|
||||
|
||||
- **Atomic claiming** — QueuePicker's `UPDATE ... RETURNING` makes Queued→Running atomic.
|
||||
- **Slot limit** — respects MaxParallelExecutions; a backstop timer wakes even if a Wake() is missed.
|
||||
- **Pre-claimed tasks** — the dispatcher pre-claims via the picker; the override slot
|
||||
(RunNow/ContinueTask) must call StartRunningAsync if a task is not pre-claimed.
|
||||
- **Terminal writes** — use `CancellationToken.None`; a task is never left Running after crash/cancel.
|
||||
- **Per-run MCP tokens** — each run gets a unique token scoping tool access; unregistered on end.
|
||||
- **Auto-retry** — one automatic retry if a session exists and the first run failed.
|
||||
- **Worktree self-heal** — on branch collision, remove phantom worktrees, prune, delete branch, retry add.
|
||||
- **Review feedback** — stored on the task; consumed once a run reaches a terminal state; a re-queued
|
||||
task resumes the session or folds feedback into the prompt.
|
||||
- **Child tasks** — planning creates draft children; finalization requires no Queued children remain;
|
||||
OrphanRecovery dequeues children if the parent is not planning.
|
||||
- **Lifecycle recovery** runs at startup: stale-Running → Failed; orphaned children → dequeued but attached.
|
||||
Reference in New Issue
Block a user