Feat/temporal (#46)

* refactor: modularize claude-executor and extract shared utilities

- Extract message handling into src/ai/message-handlers.ts with pure functions
- Extract output formatting into src/ai/output-formatters.ts
- Extract progress management into src/ai/progress-manager.ts
- Add audit-logger.ts with Null Object pattern for optional logging
- Add shared utilities: formatting.ts, file-io.ts, functional.ts
- Consolidate getPromptNameForAgent into src/types/agents.ts

* feat: add Claude Code custom commands for debug and review

* feat: add Temporal integration foundation (phase 1-2)

- Add Temporal SDK dependencies (@temporalio/client, worker, workflow, activity)
- Add shared types for pipeline state, metrics, and progress queries
- Add classifyErrorForTemporal() for retry behavior classification
- Add docker-compose for Temporal server with SQLite persistence

* feat: add Temporal activities for agent execution (phase 3)

- Add activities.ts with heartbeat loop, git checkpoint/rollback, and error classification
- Export runClaudePrompt, validateAgentOutput, ClaudePromptResult for Temporal use
- Track attempt number via Temporal Context for accurate audit logging
- Rollback git workspace before retry to ensure clean state

* feat: add Temporal workflow for 5-phase pipeline orchestration (phase 4)

* feat: add Temporal worker, client, and query tools (phase 5)

- Add worker.ts with workflow bundling and graceful shutdown
- Add client.ts CLI to start pipelines with progress polling
- Add query.ts CLI to inspect running workflow state
- Fix buffer overflow by truncating error messages and stack traces
- Skip git operations gracefully on non-git repositories
- Add kill.sh/start.sh dev scripts and Dockerfile.worker

* feat: fix Docker worker container setup

- Install uv instead of deprecated uvx package
- Add mcp-server and configs directories to container
- Mount target repo dynamically via TARGET_REPO env variable

* fix: add report assembly step to Temporal workflow

- Add assembleReportActivity to concatenate exploitation evidence files before report agent runs
- Call assembleFinalReport in workflow Phase 5 before runReportAgent
- Ensure deliverables directory exists before writing final report
- Simplify pipeline-testing report prompt to just prepend header

* refactor: consolidate Docker setup to root docker-compose.yml

* feat: improve Temporal client UX and env handling

- Change default to fire-and-forget (--wait flag to opt-in)
- Add splash screen and improve console output formatting
- Add .env to gitignore, remove from dockerignore for container access
- Add Taskfile for common development commands

* refactor: simplify session ID handling and improve Taskfile options

- Include hostname in workflow ID for better audit log organization
- Extract sanitizeHostname utility to audit/utils.ts for reuse
- Remove unused generateSessionLogPath and buildLogFilePath functions
- Simplify Taskfile with CONFIG/OUTPUT/CLEAN named parameters

* chore: add .env.example and simplify .gitignore

* docs: update README and CLAUDE.md for Temporal workflow usage

- Replace Docker CLI instructions with Task-based commands
- Add monitoring/stopping sections and workflow examples
- Document Temporal orchestration layer and troubleshooting
- Simplify file structure to key files overview

* refactor: replace Taskfile with bash CLI script

- Add shannon bash script with start/logs/query/stop/help commands
- Remove Taskfile.yml dependency (no longer requires Task installation)
- Update README.md and CLAUDE.md to use ./shannon commands
- Update client.ts output to show ./shannon commands

* docs: fix deliverable filename in README

* refactor: remove direct CLI and .shannon-store.json in favor of Temporal

- Delete src/shannon.ts direct CLI entry point (Temporal is now the only mode)
- Remove .shannon-store.json session lock (Temporal handles workflow deduplication)
- Remove broken scripts/export-metrics.js (imported non-existent function)
- Update package.json to remove main, start script, and bin entry
- Clean up CLAUDE.md and debug.md to remove obsolete references

* chore: remove licensing comments from prompt files to prevent leaking into actual prompts

* fix: resolve parallel workflow race conditions and retry logic bugs

- Fix save_deliverable race condition using closure pattern instead of global variable
- Fix error classification order so OutputValidationError matches before generic validation
- Fix ApplicationFailure re-classification bug by checking instanceof before re-throwing
- Add per-error-type retry limits (3 for output validation, 50 for billing)
- Add fast retry intervals for pipeline testing mode (10s vs 5min)
- Increase worker concurrent activities to 25 for parallel workflows

* refactor: pipeline vuln→exploit workflow for parallel execution

- Replace sync barrier between vuln/exploit phases with independent pipelines
- Each vuln type runs: vuln agent → queue check → conditional exploit
- Add checkExploitationQueue activity to skip exploits when no vulns found
- Use Promise.allSettled for graceful failure handling across pipelines
- Add PipelineSummary type for aggregated cost/duration/turns metrics

* fix: re-throw retryable errors in checkExploitationQueue

* fix: detect and retry on Claude Code spending cap errors

- Add spending cap pattern detection in detectApiError() with retryable error
- Add matching patterns to classifyErrorForTemporal() for proper Temporal retry
- Add defense-in-depth safeguard in runClaudePrompt() for $0 cost / low turn detection
- Add final sanity check in activities before declaring success

* fix: increase heartbeat timeout to prevent false worker-dead detection

Original 30s timeout was from POC spec assuming <5min activities. With
hour-long activities and multiple concurrent workflows sharing one worker,
resource contention causes event loop stalls exceeding 30s, triggering
false heartbeat timeouts. Increased to 10min (prod) and 5min (testing).

* fix: temporal db init

* fix: persist home dir

* feat: add per-workflow unified logging with ./shannon logs ID=<workflow-id>

- Add WorkflowLogger class for human-readable, per-workflow log files
- Create workflow.log in audit-logs/{workflowId}/ with phase, agent, tool, and LLM events
- Update ./shannon logs to require ID param and tail specific workflow log
- Add phase transition logging at workflow boundaries
- Include workflow completion summary with agent breakdown (duration, cost)
- Mount audit-logs volume in docker-compose for host access

---------

Co-authored-by: ezl-keygraph <ezhil@keygraph.io>

src/audit/audit-session.ts

@@ -12,8 +12,10 @@
  */
 
 import { AgentLogger } from './logger.js';
+import { WorkflowLogger, type AgentLogDetails, type WorkflowSummary } from './workflow-logger.js';
 import { MetricsTracker } from './metrics-tracker.js';
-import { initializeAuditStructure, formatTimestamp, type SessionMetadata } from './utils.js';
+import { initializeAuditStructure, type SessionMetadata } from './utils.js';
+import { formatTimestamp } from '../utils/formatting.js';
 import { SessionMutex } from '../utils/concurrency.js';
 
 // Global mutex instance
@@ -36,7 +38,9 @@ export class AuditSession {
   private sessionMetadata: SessionMetadata;
   private sessionId: string;
   private metricsTracker: MetricsTracker;
+  private workflowLogger: WorkflowLogger;
   private currentLogger: AgentLogger | null = null;
+  private currentAgentName: string | null = null;
   private initialized: boolean = false;
 
   constructor(sessionMetadata: SessionMetadata) {
@@ -53,6 +57,7 @@ export class AuditSession {
 
     // Components
     this.metricsTracker = new MetricsTracker(sessionMetadata);
+    this.workflowLogger = new WorkflowLogger(sessionMetadata);
   }
 
   /**
@@ -70,6 +75,9 @@ export class AuditSession {
     // Initialize metrics tracker (loads or creates session.json)
     await this.metricsTracker.initialize();
 
+    // Initialize workflow logger
+    await this.workflowLogger.initialize();
+
     this.initialized = true;
   }
 
@@ -97,6 +105,9 @@ export class AuditSession {
       await AgentLogger.savePrompt(this.sessionMetadata, agentName, promptContent);
     }
 
+    // Track current agent name for workflow logging
+    this.currentAgentName = agentName;
+
     // Create and initialize logger for this attempt
     this.currentLogger = new AgentLogger(this.sessionMetadata, agentName, attemptNumber);
     await this.currentLogger.initialize();
@@ -110,6 +121,9 @@ export class AuditSession {
       attemptNumber,
       timestamp: formatTimestamp(),
     });
+
+    // Log to unified workflow log
+    await this.workflowLogger.logAgent(agentName, 'start', { attemptNumber });
   }
 
   /**
@@ -120,7 +134,30 @@ export class AuditSession {
       throw new Error('No active logger. Call startAgent() first.');
     }
 
+    // Log to agent-specific log file (JSON format)
     await this.currentLogger.logEvent(eventType, eventData);
+
+    // Also log to unified workflow log (human-readable format)
+    const data = eventData as Record<string, unknown>;
+    const agentName = this.currentAgentName || 'unknown';
+    switch (eventType) {
+      case 'tool_start':
+        await this.workflowLogger.logToolStart(
+          agentName,
+          String(data.toolName || ''),
+          data.parameters
+        );
+        break;
+      case 'llm_response':
+        await this.workflowLogger.logLlmResponse(
+          agentName,
+          Number(data.turn || 0),
+          String(data.content || '')
+        );
+        break;
+      // tool_end and error events are intentionally not logged to workflow log
+      // to reduce noise - the agent completion message captures the outcome
+    }
   }
 
   /**
@@ -142,10 +179,23 @@ export class AuditSession {
       this.currentLogger = null;
     }
 
+    // Reset current agent name
+    this.currentAgentName = null;
+
+    // Log to unified workflow log
+    const agentLogDetails: AgentLogDetails = {
+      attemptNumber: result.attemptNumber,
+      duration_ms: result.duration_ms,
+      cost_usd: result.cost_usd,
+      success: result.success,
+      ...(result.error !== undefined && { error: result.error }),
+    };
+    await this.workflowLogger.logAgent(agentName, 'end', agentLogDetails);
+
     // Mutex-protected update to session.json
     const unlock = await sessionMutex.lock(this.sessionId);
     try {
-      // Reload metrics (in case of parallel updates)
+      // Reload inside mutex to prevent lost updates during parallel exploitation phase
       await this.metricsTracker.reload();
 
       // Update metrics
@@ -177,4 +227,28 @@ export class AuditSession {
     await this.ensureInitialized();
     return this.metricsTracker.getMetrics();
   }
+
+  /**
+   * Log phase start to unified workflow log
+   */
+  async logPhaseStart(phase: string): Promise<void> {
+    await this.ensureInitialized();
+    await this.workflowLogger.logPhase(phase, 'start');
+  }
+
+  /**
+   * Log phase completion to unified workflow log
+   */
+  async logPhaseComplete(phase: string): Promise<void> {
+    await this.ensureInitialized();
+    await this.workflowLogger.logPhase(phase, 'complete');
+  }
+
+  /**
+   * Log workflow completion to unified workflow log
+   */
+  async logWorkflowComplete(summary: WorkflowSummary): Promise<void> {
+    await this.ensureInitialized();
+    await this.workflowLogger.logWorkflowComplete(summary);
+  }
 }