Files
ClaudeDo/src/ClaudeDo.Worker/CLAUDE.md
Mika Kuns 1c94fbdb14 feat(worker): batch MCP tools for the external endpoint
Add seven best-effort batch variants of the single-entity external MCP
tools: batch_get_tasks, batch_add_tasks, batch_update_task_status,
batch_cancel_tasks, batch_delete_tasks, batch_set_my_day, and
batch_cleanup_task_worktrees. Each loops the existing ExternalMcpService
methods sequentially (scoped DbContext is not thread-safe), returns a
per-item result array so a failing item never aborts the rest, and
rejects empty or over-100-item batches. Merge/review stay single-task.
2026-06-26 16:11:51 +02:00

18 KiB

ClaudeDo.Worker

ASP.NET Core hosted service that executes tasks via Claude CLI in isolated environments.

Folder Layout

Worker/
  State/        — TaskStateService + TransitionResult (sole owner of Status/PlanningPhase/BlockedBy writes)
  Queue/        — IQueueWaker, IQueuePicker, QueueService (BackgroundService), OverrideSlotService
  Lifecycle/    — StaleTaskRecovery, TaskResetService, TaskMergeService, ClaudeCliPreflight, OrphanRecovery, PlanningLineageRecovery, AttachmentOrphanRecovery (startup sweep: deletes any `attachments/<taskId>/` dirs whose task no longer exists)
  Worktrees/    — WorktreeMaintenanceService
  Agents/       — AgentFileService, DefaultAgentSeeder
  Runner/       — TaskRunner + Claude CLI integration; TaskRunMcpService/TaskRunMcpContext/TaskRunTokenRegistry (in-task MCP wired during execution)
  Planning/     — PlanningSessionManager, PlanningChainCoordinator, PlanningMcpService, PlanningMergeOrchestrator, PlanningAggregator, PlanningSessionContext/PlanningTokenAuth/PlanningMcpContextAccessor, WindowsTerminalLauncher (ITerminalLauncher) — wt launcher for planning + interactive sessions
  Refine/       — RefineRunner + RefinePrompt (hub `RefineTask`; broadcasts RefineStarted/RefineFinished)
  External/     — ExternalMcpService + sibling tool classes
  Config/       — WorkerConfig
  Hub/          — WorkerHub, HubBroadcaster
  Logging/      — LogRingBuffer (30-min in-memory log window) + BroadcastLogSink (Serilog sink → footer + overlay)
  Report/       — ClaudeHistoryReader, WeekReportPromptBuilder, WeekReportService; interfaces in Report/Interfaces/
  Prime/        — daily-prep ("Prime Claude"): PrimeScheduler (BackgroundService), PrimeRunner (runs the daily prep), DailyPrepPrompt (fixed prompt + CLI args + LogPath() helper), NextDueCalculator, PrimeScheduleSignal; interfaces in Prime/Interfaces/ (IPrimeRunner, IPrimeClock, IPrimeScheduleSignal, IPrimeBroadcaster)
  Online/       — optional Online Inbox sync: OnlineInboxConfig (config record), Dtos (RemoteList/RemoteTask/MirrorTask), IOnlineInboxApi, OnlineInboxApiClient (typed HttpClient, bearer auth, HTTPS guard), OnlineTokenStore (DPAPI refresh-token store, Windows-only), StaticTokenAuthProvider (default/test IOnlineAuthProvider), ZitadelAuthProvider (stub — TODO(online-inbox) Phase 2), OnlineSyncService (BackgroundService: reconcile loop), OnlineBacklog (Idle-backlog filter/query); interface in Online/Interfaces/ (IOnlineAuthProvider)

Interfaces (e.g. IQueueWaker, IPrimeClock, ITaskStateService) live in an Interfaces/ subfolder within their area; the namespace stays the area namespace.

Architecture

  • Program.cs — loads config, inits schema, registers DI, configures SignalR on /hub, binds to 127.0.0.1:47821
  • TaskStateService — only component that writes Status, PlanningPhase, BlockedByTaskId. All transitions return a TransitionResult (no exceptions on invalid moves). Wakes the queue and broadcasts TaskUpdated automatically; advances the planning chain on child terminal transitions.
  • IQueueWaker / IQueuePicker / QueueService — waker is a singleton SemaphoreSlim; picker performs the atomic Queued → Running claim filtered by BlockedByTaskId IS NULL and schedule; QueueService is a thin BackgroundService that loops on the waker and dispatches via TaskRunner.
  • OverrideSlotService — owns RunNow / ContinueTask; goes through TaskStateService.StartRunningAsync (caller-driven, serialized by slot lock).
  • StaleTaskRecovery — startup-only service; calls TaskStateService.RecoverStaleRunningAsync to flip orphaned Running rows to Failed.
  • External/* — always-on MCP tools for general Claude sessions, scoped to starting and observing sessions (no worktree/merge, multi-turn, planning, or app-settings writes). Auth via optional X-ClaudeDo-Key header. Registered explicitly in Program.cs's external app via .WithTools<T>(). Organized by concern:
    • ExternalMcpService — task CRUD + execution: ListTaskLists, ListTasks, GetTask, AddTask, AddSubtask, UpdateTask, UpdateTaskStatus (Idle / Queued), GetTaskStatusValues, ReviewTask (approve / reject_rerun / reject_park / cancel for a WaitingForReview task), RunTaskNow, ContinueTask, CancelTask, DeleteTask; worktree/git: GetTaskWorktree, GetTaskDiff, MergeTask, ListWorktrees, CleanupTaskWorktree
    • BatchMcpTools — best-effort batch variants that loop the ExternalMcpService single-entity methods (sequential — the scoped DbContext is not thread-safe; merge/review stay single-task): BatchGetTasks, BatchAddTasks, BatchUpdateTaskStatus, BatchCancelTasks, BatchDeleteTasks, BatchSetMyDay, BatchCleanupTaskWorktrees. Every tool returns a per-item result array ({ id/index, ok, error?, … }) — a failing item never aborts the rest — and rejects batches over 100 items.
    • ListMcpToolsCreateList, UpdateList, DeleteList
    • ConfigMcpToolsGetListConfig, SetListConfig, GetTaskConfig, SetTaskConfig
    • RunHistoryMcpToolsListRuns, GetRun, GetTaskLog (latest run's log, tail-capped at 256 KB)
    • AgentMcpToolsListAgents (class lives in LifecycleMcpTools.cs)
    • LifecycleMcpToolsResetFailedTask
    • AppSettingsMcpToolsGetAppSettings (read-only)
    • AttachmentMcpToolsAddTaskAttachment(taskId, fileName, textContent?|base64Content?), ListTaskAttachments, RemoveTaskAttachment. Re-attaching the same fileName overwrites; add/remove refuse on a Running task.
    • ExternalMcpService also exposes two daily-prep tools:
      • GetDailyPrepCandidates — returns Idle, non-blocked tasks in a git repo NOT excluded by AppSettings.ReportExcludedPaths and not already IsMyDay, plus the current Idle MyDay tasks and maxTasks (= DailyPrepMaxTasks). Repo-exclusion logic lives in the DailyPrepFilter helper (same file).
      • SetMyDay — sets a task's IsMyDay (+ optional SortOrder); server-side cap-guard rejects turning on MyDay beyond DailyPrepMaxTasks open (Idle) MyDay tasks.

Daily Prep (Prime Claude)

  • PrimeScheduler (hosted BackgroundService) computes the next due time from the prime_schedules table and at that time calls IPrimeRunner.FireAsync. A manual run arrives via WorkerHub.RunDailyPrepNow. A SemaphoreSlim single-flight gate in PrimeRunner prevents overlapping runs (returns "already running"); both scheduled and manual runs go through it.
  • PrimeRunner builds a fixed prompt via DailyPrepPrompt.BuildPrompt, parameterized by AppSettings.DailyPrepMaxTasks and today's date, then invokes:
    claude -p --output-format stream-json --verbose --permission-mode acceptEdits --max-turns 30
           --allowedTools mcp__claudedo__get_daily_prep_candidates mcp__claudedo__set_my_day
    
    It relies on the globally-registered claudedo MCP (installer's RegisterMcpStep) — no separate --mcp-config. This replaced the old warm-up "ping".
  • Each stdout line is streamed to the UI via IPrimeBroadcaster.PrepLineAsync AND written to DailyPrepPrompt.LogPath() = <appdata>/logs/daily-prep.log (truncated at the start of each run → last run only). PrepStarted/PrepFinished events bracket the run.
  • Agentic behaviour: Claude calls get_daily_prep_candidates, picks an effort-aware subset capped at DailyPrepMaxTasks, and marks them via set_my_day (which broadcasts TaskUpdated so the UI updates live).

Status Model

TaskEntity carries three orthogonal fields. Lifecycle, planning hierarchy, and chain blocking are no longer conflated.

Field Values Meaning
Status Idle, Queued, Running, WaitingForChildren, WaitingForReview, Done, Failed, Cancelled Lifecycle only. WaitingForChildren = parent's own work is done, waiting on its children.
PlanningPhase None, Active, Finalized Parent-only marker. Active ≈ legacy Planning; Finalized ≈ legacy Planned.
BlockedByTaskId nullable FK Replaces legacy Waiting. A queued row with BlockedByTaskId != NULL is skipped by the picker.
ReviewFeedback nullable string Reviewer's rejection comment. Set by RejectToQueueAsync; consumed and cleared by QueueService on the next re-run (resumes the Claude session with it as the next-turn prompt).

Allowed transitions (enforced by TaskStateService):

Idle               → Queued | Running (RunNow)
Queued             → Running | Cancelled | Idle | Failed (OverrideSlotService preflight gap: RunAsync can fail before StartRunningAsync is called)
Running            → WaitingForReview (standalone success, no children)
                   | WaitingForChildren (parent with pending children)
                   | Done (planning/improvement child success) | Failed | Cancelled
WaitingForChildren → WaitingForReview (all children terminal) | Cancelled
WaitingForReview   → Done (approve) | Queued (reject-rerun, +feedback) | Idle (reject-park) | Cancelled
Done               → Idle (re-run)
Failed             → Idle | Queued
Cancelled          → Idle | Queued

Unified parent model. Every parent — planning or improvement — flows … → WaitingForChildren → WaitingForReview → Done, advanced by the single TaskStateService.TryAdvanceParentAsync (surfaces any WaitingForChildren parent for review once all children are terminal; failed/cancelled children are annotated on the result, not wedged). A planning parent enters WaitingForChildren at FinalizePlanningAsync (or WaitingForReview directly if it has no children); an improvement parent enters it from TaskRunner.HandleSuccess when its run spawned children. Planning/improvement children still go straight to Done (no individual review) — only the parent is reviewed.

Approve = merge the whole unit. ApproveReview/review_task approve, for a task that has children, drives PlanningMergeOrchestrator (merges the parent worktree if Active + each Done child in order, sets the parent Done, and on a mid-merge conflict pauses for ContinuePlanningMerge/AbortPlanningMerge). Childless tasks use TaskMergeService.ApproveAndMergeAsync. There is no separate "Merge all" entry — approve is the single review+merge action. Review transitions live in TaskStateService (SubmitForReviewAsync, SubmitForChildrenAsync, ApproveReviewAsync, RejectToQueueAsync, RejectToIdleAsync, ClearReviewFeedbackAsync).

Planning Flow

PlanningSessionManager.FinalizeAsync is the single path:

  1. _state.FinalizePlanningAsync(parent) flips parent PlanningPhase to Finalized and sets Status to WaitingForChildren (or WaitingForReview if the parent has no children).
  2. PlanningChainCoordinator.SetupChainAsync(parent, enqueue: false) establishes the blocked-by chain (BlockOns child[i] → child[i-1]) but leaves children Idle — finalize never auto-queues. Queueing is a deliberate user action: QueuePlanAsync (hub QueuePlanningSubtasksAsync, the "Queue plan" button) calls SetupChainAsync(parent, enqueue: true), which sets every non-terminal child Queued and re-applies the chain.
  3. Once queued, the first child is woken automatically; successors unblock as their predecessor reaches a terminal state via OnChildFinishedAsync.

A child that hits a roadblock (fails, or reports CLAUDEDO_BLOCKED roadblocks) does not advance the parent — the parent stays in WaitingForChildren until every child is terminal. The UI surfaces blocked children on the parent's Session tab (ChildOutcomes + a "children need attention" band) so the roadblock is visible without forcing a transition.

TaskRepository.FinalizePlanningAsync no longer exists. The Mark*Async repository helpers are internal — only TaskStateService calls them.

Task Execution Pipeline

TaskRunner orchestrates:

  1. Load task + list metadata from DB; resolve config from list_config + task-level overrides (model, system_prompt, agent_path)
  2. Create worktree (if WorkingDir set) or sandbox directory
  3. Mark task "running", broadcast TaskStarted
  4. Build CLI args via ClaudeArgsBuilder; inject attachment absolute paths via TaskPromptComposer.Compose (appends a read-only "## Reference files" section); invoke ClaudeProcess with task prompt
  5. Stream NDJSON output through StreamAnalyzer; lines forwarded to log file and SignalR (TaskMessage)
  6. On success: auto-commit changes (worktree only), store run record, mark "done"
  7. On failure: retry once if session ID available (--resume), then mark "failed"

Key Components

  • ClaudeProcess — spawns claude -p --output-format stream-json --verbose --permission-mode auto (or whatever permission mode the app settings specify). Writes prompt to stdin, reads NDJSON from stdout. Supports CancellationToken (kills process tree).
  • ClaudeArgsBuilder — dynamically constructs CLI args; supports --model, --append-system-prompt, --agents, --json-schema, --resume
  • StreamAnalyzer — parses rich NDJSON output; extracts session_id, token counts, turn counts, result text, structured output. Replaces MessageParser.
  • TaskResetService — discards a failed task's worktree and resets the task row to Idle; preserves run history.
  • WorktreeManager — creates worktrees at claudedo/{taskId[:8]} branches, commits changes with semantic messages, updates DB with head commit and diff stats
  • CommitMessageBuilder — formats {commitType}(slug): title\n\ndescription\n\nClaudeDo-Task: taskId
  • AgentFileService — manages ~/.todo-app/agents/*.md agent definition files; exposes list/refresh via SignalR
  • LogWriter — async StreamWriter wrapper, auto-creates parent dirs

Execution History

Each CLI invocation is recorded in the task_runs table via TaskRunRepository:

  • Fields: session_id, input/output/cache token counts, turn count, result text, structured output JSON
  • Enables auto-retry on failure (resume last session) and multi-turn follow-up via ContinueAsync

Multi-Turn / Continue

TaskRunner.ContinueAsync sends a follow-up prompt to an existing Claude session using --resume <session_id> with the stored session ID from the last run.

SignalR Hub

WorkerHub methods, grouped:

  • Execution: Ping, GetActive, RunNow, CancelTask, WakeQueue, ContinueTask, ResetTask, SetTaskStatus, RefineTask
  • Review/merge: ApproveReview(taskId, targetBranch) -> MergeResultDto (childless task: merges its worktree then Done, conflict stays WaitingForReview; task with children: drives PlanningMergeOrchestrator to merge the whole unit), ContinuePlanningMerge / AbortPlanningMerge (resolve a unit-merge conflict), PreviewMerge(taskId, targetBranch) -> MergePreviewDto (non-destructive mergeability check), RejectReviewToQueue, RejectReviewToIdle, CancelReview, MergeTask, GetMergeTargets
  • Single-task conflict resolver (Layer C): StartConflictMerge, GetMergeConflictDocuments (segments), WriteConflictResolution, ContinueConflictMerge, AbortConflictMerge (service-level TaskMergeService.ContinueMergeAsync/AbortMergeAsync keep their names)
  • Planning sessions: StartPlanningSession, ResumePlanningSession, DiscardPlanningSession, FinalizePlanningSession, QueuePlanningSubtasks, GetPendingDraftCount, OpenInteractiveTerminal, GetPlanningAggregate (per-subtask diffs), BuildPlanningIntegrationBranch (combined diff)
  • Worktrees: CleanupFinishedWorktrees, ResetAllWorktrees, GetWorktreesOverview, SetWorktreeState, ForceRemoveWorktree
  • Agents/settings/lists: GetAgents, RefreshAgents, RestoreDefaultAgents, GetAppSettings, UpdateAppSettings, UpdateList, UpdateListConfig, GetListConfig, UpdateTaskAgentSettings
  • Reports/notes/prep: GetWeekReport, GenerateWeekReport, GetDailyNotes, AddDailyNote, UpdateDailyNote, DeleteDailyNote, RunDailyPrepNow, ClearMyDay, GetLastPrepLog, ListPrimeSchedules, UpsertPrimeSchedule, DeletePrimeSchedule
  • Diagnostics: GetRecentLogs (last 30 min of buffered log records, all levels, for the Log Visualizer overlay)

HubBroadcaster events: TaskStarted, TaskFinished, TaskMessage, WorktreeUpdated, TaskUpdated, RunCreated, ListUpdated, WorkerLog, PrimeFired, PrepStarted, PrepLine, PrepFinished, PlanningMergeStarted, PlanningSubtaskMerged, PlanningMergeConflict, PlanningMergeAborted, PlanningCompleted, RefineStarted, RefineFinished

WorkerLog carries two sources: the hand-curated business events (_broadcaster.WorkerLog(...) in TaskRunner/TaskMergeService/TaskResetService) and every Serilog Warn/Error event, re-broadcast by BroadcastLogSink (deduped within a 120 s per-message window; SignalR plumbing source-contexts filtered to avoid feedback loops). The sink also buffers all levels into LogRingBuffer for GetRecentLogs.

Config

Loaded from ~/.todo-app/worker.config.json:

  • db_path, sandbox_root, log_root
  • worktree_root_strategy ("sibling" | "central"), central_worktree_root
  • queue_backstop_interval_ms (default 30000)
  • signalr_port (default 47821)
  • claude_bin (path to claude CLI)
  • online_inbox — Online Inbox config (default: enabled=false, zero network when disabled):
    • enabled (bool, default false) — when false the entire Online/ stack is not registered
    • api_base_url (string) — must be HTTPS or loopback; validated at startup when enabled
    • poll_interval_seconds (int, default 60)
    • zitadel.authority, zitadel.client_id, zitadel.scopes (Phase 2; not used until ZitadelAuthProvider is wired)
    • The refresh token is NOT in this file — stored encrypted via DPAPI at ~/.todo-app/online-inbox.token

Per-list config (list_config in DB) provides defaults for model, system_prompt, agent_path; tasks can override each individually. Task-generating MCP tools (AddTask, planning CreateChildTask, SuggestImprovement) accept an optional model (alias-validated via ModelRegistry.NormalizeAliashaiku/sonnet/opus, blank = inherit) so Claude assigns the cheapest capable model at creation time; the planning/system/improvement prompts instruct it to do so (ModelRegistry.ByCostAscending = the cost order).

Notes

  • The worker runs standalone — start it separately from the UI
  • Only listens on loopback (127.0.0.1)
  • ClaudeProcess uses --permission-mode auto by default; legacy "bypassPermissions" settings are mapped to auto at dispatch time. acceptEdits, plan, and default pass through unchanged.
  • Worktree branches follow claudedo/{id} naming convention