Feat/temporal (#46)

* refactor: modularize claude-executor and extract shared utilities - Extract message handling into src/ai/message-handlers.ts with pure functions - Extract output formatting into src/ai/output-formatters.ts - Extract progress management into src/ai/progress-manager.ts - Add audit-logger.ts with Null Object pattern for optional logging - Add shared utilities: formatting.ts, file-io.ts, functional.ts - Consolidate getPromptNameForAgent into src/types/agents.ts * feat: add Claude Code custom commands for debug and review * feat: add Temporal integration foundation (phase 1-2) - Add Temporal SDK dependencies (@temporalio/client, worker, workflow, activity) - Add shared types for pipeline state, metrics, and progress queries - Add classifyErrorForTemporal() for retry behavior classification - Add docker-compose for Temporal server with SQLite persistence * feat: add Temporal activities for agent execution (phase 3) - Add activities.ts with heartbeat loop, git checkpoint/rollback, and error classification - Export runClaudePrompt, validateAgentOutput, ClaudePromptResult for Temporal use - Track attempt number via Temporal Context for accurate audit logging - Rollback git workspace before retry to ensure clean state * feat: add Temporal workflow for 5-phase pipeline orchestration (phase 4) * feat: add Temporal worker, client, and query tools (phase 5) - Add worker.ts with workflow bundling and graceful shutdown - Add client.ts CLI to start pipelines with progress polling - Add query.ts CLI to inspect running workflow state - Fix buffer overflow by truncating error messages and stack traces - Skip git operations gracefully on non-git repositories - Add kill.sh/start.sh dev scripts and Dockerfile.worker * feat: fix Docker worker container setup - Install uv instead of deprecated uvx package - Add mcp-server and configs directories to container - Mount target repo dynamically via TARGET_REPO env variable * fix: add report assembly step to Temporal workflow - Add assembleReportActivity to concatenate exploitation evidence files before report agent runs - Call assembleFinalReport in workflow Phase 5 before runReportAgent - Ensure deliverables directory exists before writing final report - Simplify pipeline-testing report prompt to just prepend header * refactor: consolidate Docker setup to root docker-compose.yml * feat: improve Temporal client UX and env handling - Change default to fire-and-forget (--wait flag to opt-in) - Add splash screen and improve console output formatting - Add .env to gitignore, remove from dockerignore for container access - Add Taskfile for common development commands * refactor: simplify session ID handling and improve Taskfile options - Include hostname in workflow ID for better audit log organization - Extract sanitizeHostname utility to audit/utils.ts for reuse - Remove unused generateSessionLogPath and buildLogFilePath functions - Simplify Taskfile with CONFIG/OUTPUT/CLEAN named parameters * chore: add .env.example and simplify .gitignore * docs: update README and CLAUDE.md for Temporal workflow usage - Replace Docker CLI instructions with Task-based commands - Add monitoring/stopping sections and workflow examples - Document Temporal orchestration layer and troubleshooting - Simplify file structure to key files overview * refactor: replace Taskfile with bash CLI script - Add shannon bash script with start/logs/query/stop/help commands - Remove Taskfile.yml dependency (no longer requires Task installation) - Update README.md and CLAUDE.md to use ./shannon commands - Update client.ts output to show ./shannon commands * docs: fix deliverable filename in README * refactor: remove direct CLI and .shannon-store.json in favor of Temporal - Delete src/shannon.ts direct CLI entry point (Temporal is now the only mode) - Remove .shannon-store.json session lock (Temporal handles workflow deduplication) - Remove broken scripts/export-metrics.js (imported non-existent function) - Update package.json to remove main, start script, and bin entry - Clean up CLAUDE.md and debug.md to remove obsolete references * chore: remove licensing comments from prompt files to prevent leaking into actual prompts * fix: resolve parallel workflow race conditions and retry logic bugs - Fix save_deliverable race condition using closure pattern instead of global variable - Fix error classification order so OutputValidationError matches before generic validation - Fix ApplicationFailure re-classification bug by checking instanceof before re-throwing - Add per-error-type retry limits (3 for output validation, 50 for billing) - Add fast retry intervals for pipeline testing mode (10s vs 5min) - Increase worker concurrent activities to 25 for parallel workflows * refactor: pipeline vuln→exploit workflow for parallel execution - Replace sync barrier between vuln/exploit phases with independent pipelines - Each vuln type runs: vuln agent → queue check → conditional exploit - Add checkExploitationQueue activity to skip exploits when no vulns found - Use Promise.allSettled for graceful failure handling across pipelines - Add PipelineSummary type for aggregated cost/duration/turns metrics * fix: re-throw retryable errors in checkExploitationQueue * fix: detect and retry on Claude Code spending cap errors - Add spending cap pattern detection in detectApiError() with retryable error - Add matching patterns to classifyErrorForTemporal() for proper Temporal retry - Add defense-in-depth safeguard in runClaudePrompt() for $0 cost / low turn detection - Add final sanity check in activities before declaring success * fix: increase heartbeat timeout to prevent false worker-dead detection Original 30s timeout was from POC spec assuming <5min activities. With hour-long activities and multiple concurrent workflows sharing one worker, resource contention causes event loop stalls exceeding 30s, triggering false heartbeat timeouts. Increased to 10min (prod) and 5min (testing). * fix: temporal db init * fix: persist home dir * feat: add per-workflow unified logging with ./shannon logs ID=<workflow-id> - Add WorkflowLogger class for human-readable, per-workflow log files - Create workflow.log in audit-logs/{workflowId}/ with phase, agent, tool, and LLM events - Update ./shannon logs to require ID param and tail specific workflow log - Add phase transition logging at workflow boundaries - Include workflow completion summary with agent breakdown (duration, cost) - Mount audit-logs volume in docker-compose for host access --------- Co-authored-by: ezl-keygraph <ezhil@keygraph.io>
2026-01-15 10:36:11 -08:00
parent 45acb16711
commit 51e621d0d5
77 changed files with 6117 additions and 2417 deletions
@@ -0,0 +1,382 @@
+// Copyright (C) 2025 Keygraph, Inc.
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License version 3
+// as published by the Free Software Foundation.
+
+/**
+ * Workflow Logger
+ *
+ * Provides a unified, human-readable log file per workflow.
+ * Optimized for `tail -f` viewing during concurrent workflow execution.
+ */
+
+import fs from 'fs';
+import path from 'path';
+import { generateWorkflowLogPath, ensureDirectory, type SessionMetadata } from './utils.js';
+import { formatDuration, formatTimestamp } from '../utils/formatting.js';
+
+export interface AgentLogDetails {
+  attemptNumber?: number;
+  duration_ms?: number;
+  cost_usd?: number;
+  success?: boolean;
+  error?: string;
+}
+
+export interface AgentMetricsSummary {
+  durationMs: number;
+  costUsd: number | null;
+}
+
+export interface WorkflowSummary {
+  status: 'completed' | 'failed';
+  totalDurationMs: number;
+  totalCostUsd: number;
+  completedAgents: string[];
+  agentMetrics: Record<string, AgentMetricsSummary>;
+  error?: string;
+}
+
+/**
+ * WorkflowLogger - Manages the unified workflow log file
+ */
+export class WorkflowLogger {
+  private sessionMetadata: SessionMetadata;
+  private logPath: string;
+  private stream: fs.WriteStream | null = null;
+  private initialized: boolean = false;
+
+  constructor(sessionMetadata: SessionMetadata) {
+    this.sessionMetadata = sessionMetadata;
+    this.logPath = generateWorkflowLogPath(sessionMetadata);
+  }
+
+  /**
+   * Initialize the log stream (creates file and writes header)
+   */
+  async initialize(): Promise<void> {
+    if (this.initialized) {
+      return;
+    }
+
+    // Ensure directory exists
+    await ensureDirectory(path.dirname(this.logPath));
+
+    // Create write stream with append mode
+    this.stream = fs.createWriteStream(this.logPath, {
+      flags: 'a',
+      encoding: 'utf8',
+      autoClose: true,
+    });
+
+    this.initialized = true;
+
+    // Write header only if file is new (empty)
+    const stats = await fs.promises.stat(this.logPath).catch(() => null);
+    if (!stats || stats.size === 0) {
+      await this.writeHeader();
+    }
+  }
+
+  /**
+   * Write header to log file
+   */
+  private async writeHeader(): Promise<void> {
+    const header = [
+      `================================================================================`,
+      `Shannon Pentest - Workflow Log`,
+      `================================================================================`,
+      `Workflow ID: ${this.sessionMetadata.id}`,
+      `Target URL:  ${this.sessionMetadata.webUrl}`,
+      `Started:     ${formatTimestamp()}`,
+      `================================================================================`,
+      ``,
+    ].join('\n');
+
+    return this.writeRaw(header);
+  }
+
+  /**
+   * Write raw text to log file with immediate flush
+   */
+  private writeRaw(text: string): Promise<void> {
+    return new Promise((resolve, reject) => {
+      if (!this.initialized || !this.stream) {
+        reject(new Error('WorkflowLogger not initialized'));
+        return;
+      }
+
+      const needsDrain = !this.stream.write(text, 'utf8', (error) => {
+        if (error) reject(error);
+      });
+
+      if (needsDrain) {
+        this.stream.once('drain', resolve);
+      } else {
+        resolve();
+      }
+    });
+  }
+
+  /**
+   * Format timestamp for log line (local time, human readable)
+   */
+  private formatLogTime(): string {
+    const now = new Date();
+    return now.toISOString().replace('T', ' ').slice(0, 19);
+  }
+
+  /**
+   * Log a phase transition event
+   */
+  async logPhase(phase: string, event: 'start' | 'complete'): Promise<void> {
+    await this.ensureInitialized();
+
+    const action = event === 'start' ? 'Starting' : 'Completed';
+    const line = `[${this.formatLogTime()}] [PHASE] ${action}: ${phase}\n`;
+
+    // Add blank line before phase start for readability
+    if (event === 'start') {
+      await this.writeRaw('\n');
+    }
+
+    await this.writeRaw(line);
+  }
+
+  /**
+   * Log an agent event
+   */
+  async logAgent(
+    agentName: string,
+    event: 'start' | 'end',
+    details?: AgentLogDetails
+  ): Promise<void> {
+    await this.ensureInitialized();
+
+    let message: string;
+
+    if (event === 'start') {
+      const attempt = details?.attemptNumber ?? 1;
+      message = `${agentName}: Starting (attempt ${attempt})`;
+    } else {
+      const parts: string[] = [agentName + ':'];
+
+      if (details?.success === false) {
+        parts.push('Failed');
+        if (details?.error) {
+          parts.push(`- ${details.error}`);
+        }
+      } else {
+        parts.push('Completed');
+      }
+
+      if (details?.duration_ms !== undefined) {
+        parts.push(`(${formatDuration(details.duration_ms)}`);
+        if (details?.cost_usd !== undefined) {
+          parts.push(`$${details.cost_usd.toFixed(2)})`);
+        } else {
+          parts.push(')');
+        }
+      }
+
+      message = parts.join(' ');
+    }
+
+    const line = `[${this.formatLogTime()}] [AGENT] ${message}\n`;
+    await this.writeRaw(line);
+  }
+
+  /**
+   * Log a general event
+   */
+  async logEvent(eventType: string, message: string): Promise<void> {
+    await this.ensureInitialized();
+
+    const line = `[${this.formatLogTime()}] [${eventType.toUpperCase()}] ${message}\n`;
+    await this.writeRaw(line);
+  }
+
+  /**
+   * Log an error
+   */
+  async logError(error: Error, context?: string): Promise<void> {
+    await this.ensureInitialized();
+
+    const contextStr = context ? ` (${context})` : '';
+    const line = `[${this.formatLogTime()}] [ERROR] ${error.message}${contextStr}\n`;
+    await this.writeRaw(line);
+  }
+
+  /**
+   * Truncate string to max length with ellipsis
+   */
+  private truncate(str: string, maxLen: number): string {
+    if (str.length <= maxLen) return str;
+    return str.slice(0, maxLen - 3) + '...';
+  }
+
+  /**
+   * Format tool parameters for human-readable display
+   */
+  private formatToolParams(toolName: string, params: unknown): string {
+    if (!params || typeof params !== 'object') {
+      return '';
+    }
+
+    const p = params as Record<string, unknown>;
+
+    // Tool-specific formatting for common tools
+    switch (toolName) {
+      case 'Bash':
+        if (p.command) {
+          return this.truncate(String(p.command).replace(/\n/g, ' '), 100);
+        }
+        break;
+      case 'Read':
+        if (p.file_path) {
+          return String(p.file_path);
+        }
+        break;
+      case 'Write':
+        if (p.file_path) {
+          return String(p.file_path);
+        }
+        break;
+      case 'Edit':
+        if (p.file_path) {
+          return String(p.file_path);
+        }
+        break;
+      case 'Glob':
+        if (p.pattern) {
+          return String(p.pattern);
+        }
+        break;
+      case 'Grep':
+        if (p.pattern) {
+          const path = p.path ? ` in ${p.path}` : '';
+          return `"${this.truncate(String(p.pattern), 50)}"${path}`;
+        }
+        break;
+      case 'WebFetch':
+        if (p.url) {
+          return String(p.url);
+        }
+        break;
+      case 'mcp__playwright__browser_navigate':
+        if (p.url) {
+          return String(p.url);
+        }
+        break;
+      case 'mcp__playwright__browser_click':
+        if (p.selector) {
+          return this.truncate(String(p.selector), 60);
+        }
+        break;
+      case 'mcp__playwright__browser_type':
+        if (p.selector) {
+          const text = p.text ? `: "${this.truncate(String(p.text), 30)}"` : '';
+          return `${this.truncate(String(p.selector), 40)}${text}`;
+        }
+        break;
+    }
+
+    // Default: show first string-valued param truncated
+    for (const [key, val] of Object.entries(p)) {
+      if (typeof val === 'string' && val.length > 0) {
+        return `${key}=${this.truncate(val, 60)}`;
+      }
+    }
+
+    return '';
+  }
+
+  /**
+   * Log tool start event
+   */
+  async logToolStart(agentName: string, toolName: string, parameters: unknown): Promise<void> {
+    await this.ensureInitialized();
+
+    const params = this.formatToolParams(toolName, parameters);
+    const paramStr = params ? `: ${params}` : '';
+    const line = `[${this.formatLogTime()}] [${agentName}] [TOOL] ${toolName}${paramStr}\n`;
+    await this.writeRaw(line);
+  }
+
+  /**
+   * Log LLM response
+   */
+  async logLlmResponse(agentName: string, turn: number, content: string): Promise<void> {
+    await this.ensureInitialized();
+
+    // Show full content, replacing newlines with escaped version for single-line output
+    const escaped = content.replace(/\n/g, '\\n');
+    const line = `[${this.formatLogTime()}] [${agentName}] [LLM] Turn ${turn}: ${escaped}\n`;
+    await this.writeRaw(line);
+  }
+
+  /**
+   * Log workflow completion with full summary
+   */
+  async logWorkflowComplete(summary: WorkflowSummary): Promise<void> {
+    await this.ensureInitialized();
+
+    const status = summary.status === 'completed' ? 'COMPLETED' : 'FAILED';
+
+    await this.writeRaw('\n');
+    await this.writeRaw(`================================================================================\n`);
+    await this.writeRaw(`Workflow ${status}\n`);
+    await this.writeRaw(`────────────────────────────────────────\n`);
+    await this.writeRaw(`Workflow ID: ${this.sessionMetadata.id}\n`);
+    await this.writeRaw(`Status:      ${summary.status}\n`);
+    await this.writeRaw(`Duration:    ${formatDuration(summary.totalDurationMs)}\n`);
+    await this.writeRaw(`Total Cost:  $${summary.totalCostUsd.toFixed(4)}\n`);
+    await this.writeRaw(`Agents:      ${summary.completedAgents.length} completed\n`);
+
+    if (summary.error) {
+      await this.writeRaw(`Error:       ${summary.error}\n`);
+    }
+
+    await this.writeRaw(`\n`);
+    await this.writeRaw(`Agent Breakdown:\n`);
+
+    for (const agentName of summary.completedAgents) {
+      const metrics = summary.agentMetrics[agentName];
+      if (metrics) {
+        const duration = formatDuration(metrics.durationMs);
+        const cost = metrics.costUsd !== null ? `$${metrics.costUsd.toFixed(4)}` : 'N/A';
+        await this.writeRaw(`  - ${agentName} (${duration}, ${cost})\n`);
+      } else {
+        await this.writeRaw(`  - ${agentName}\n`);
+      }
+    }
+
+    await this.writeRaw(`================================================================================\n`);
+  }
+
+  /**
+   * Ensure initialized (helper for lazy initialization)
+   */
+  private async ensureInitialized(): Promise<void> {
+    if (!this.initialized) {
+      await this.initialize();
+    }
+  }
+
+  /**
+   * Close the log stream
+   */
+  async close(): Promise<void> {
+    if (!this.initialized || !this.stream) {
+      return;
+    }
+
+    return new Promise((resolve) => {
+      this.stream!.end(() => {
+        this.initialized = false;
+        resolve();
+      });
+    });
+  }
+}