feat: add npx CLI with monorepo, CI/CD, and ephemeral worker architecture (#256)
* feat: integrate npx CLI, CI/CD, and ephemeral worker architecture
Bring in changes from shannon-npx: npx-distributable CLI package (cli/),
semantic-release CI/CD workflows, ephemeral per-scan worker containers,
TOML config support, setup wizard, and workspace management.
Preserves all shannon-only changes: security hardening (localhost-bound
ports, MCP env allowlist, path traversal guard), updated benchmarks
(XBEN 19/31/35/44), README assets, and prompt injection disclaimer.
Applies security hardening to cli/infra/compose.yml as well.
* refactor: migrate to Turborepo + pnpm + Biome monorepo
Restructure into apps/worker, apps/cli, packages/mcp-server with
Turborepo task orchestration, pnpm workspaces, Biome linting/formatting,
and tsdown CLI bundling.
Key changes:
- src/ -> apps/worker/src/, cli/ -> apps/cli/, mcp-server/ -> packages/mcp-server/
- prompts/ and configs/ moved into apps/worker/
- npm replaced with pnpm, package-lock.json replaced with pnpm-lock.yaml
- Dockerfile updated for pnpm-based builds
- CLI logs command rewritten with chokidar for cross-platform reliability
- Router health checking added for auto-detected router mode
- Centralized path resolution via apps/worker/src/paths.ts
* fix: resolve all biome warnings and formatting issues
- Remove unnecessary non-null assertions where values are guaranteed
- Replace array index access with .at() for safer element retrieval
- Use local variables to avoid repeated process.env lookups
- Replace any types with unknown in functional utilities
- Use nullish coalescing for TOTP hash byte access
- Auto-format security patches to match biome config
* fix: pin pnpm to 10.12.1 in Dockerfile for catalog support
* fix: handle Esc cancellation in Bedrock setup flow
Replace p.group() with individual prompts and per-field cancel checks,
matching the pattern used by all other provider setup flows.
* feat: add optional model customization to Anthropic setup
* fix: resolve Docker bind mount permission errors on Linux
Use entrypoint-based UID remapping instead of --user flag so the
container's pentest user matches the host UID/GID, keeping bind-mounted
volumes writable. Git config moved to --system level to survive remapping.
* fix: show resumed workflow ID in splash screen URL
When resuming a workflow, the Temporal Web UI link pointed to the old
(terminated) workflow ID. Now extracts "New Workflow ID" from the resume
header in workflow.log, falling back to the original ID for fresh scans.
* style: fix biome formatting in docker.ts
* fix: align TypeScript config types with JSON Schema
- SuccessCondition.type: use schema values (url_contains,
element_present, url_equals_exactly, text_contains) instead of
stale values (url, cookie, element, redirect)
- Authentication.login_flow: mark optional to match schema which
does not require it
* feat: mark GitHub release as latest during rollback
* fix: use native ARM64 runners for Docker multi-platform builds
Replace QEMU emulation with parallel native builds using a matrix
strategy (ubuntu-latest for amd64, ubuntu-24.04-arm for arm64).
Each platform pushes by digest, then a merge job creates the
multi-arch manifest list before signing with cosign.
* fix: resolve SessionMutex race condition with 3+ concurrent waiters
* fix: skip POSIX permission check on Windows
writeFileSync mode option is ignored on Windows, so config.toml
gets 0o666 and the guard rejects it.
* fix: resolve unsubstituted placeholders in report prompt
Remove unused {{GITHUB_URL}} placeholder and wire up {{AUTH_CONTEXT}}
with structured auth context (login type, username, URL, MFA status).
* fix: remove duplicate environment gate from merge-docker job
Move DOCKERHUB_USERNAME from vars to secrets so merge-docker can access
credentials without its own environment scope. This eliminates the
redundant double approval since build-docker already gates on
release-publish.
* fix: replace POSIX sleep binary with cross-platform async sleep
execFileSync('sleep') is unavailable on Windows. Use node:timers/promises
setTimeout instead, making ensureInfra async.
* fix: use session.json for workflow ID on resume instead of parsing workflow.log
On resume, workflow.log already exists with stale headers from the
previous run. The CLI poll found '====' immediately and extracted the
old workflow ID, producing a wrong Temporal Web UI URL.
Read the workflow ID from session.json instead — the worker writes
resume attempts there atomically. For fresh runs, poll until
originalWorkflowId appears. For resumes, poll until a new
resumeAttempts entry is appended.
* feat: add custom base URL support for Anthropic-compatible proxies
Support ANTHROPIC_BASE_URL + ANTHROPIC_AUTH_TOKEN to route SDK requests
through LiteLLM or any Anthropic-compatible proxy. Adds TUI wizard
option, TOML config mapping, credential validation, and preflight
endpoint reachability check via SDK query.
* fix: remove environment gates and add NPM_TOKEN to publish step
* feat: add beta release and rollback workflows with cosign signing
* fix: remove redundant checkout and pnpm steps from beta release workflow
* docs: normalize README commands to mode-neutral shorthand
Add a substitution note after Quick Start sections so all subsequent
examples use bare `shannon` instead of mixing `./shannon` and
`npx @keygraph/shannon`. Mode-specific commands (build, update,
uninstall) get inline annotations. Also fixes a broken command in the
Custom Base URL section.
* fix: remove redundant `update` command
Image is already auto-pulled by `ensureImage()` during `start` when the
pinned version tag is missing locally. Manual `update` was unnecessary.
* docs: add CLI package README stub
* docs: update README setup instructions for dual CLI modes
* docs: update announcement banner to npx availability
* feat: migrate from MCP tools to CLI based tools (#252)
* feat: migrate from MCP tools to CLI tools
* fix: restore browser action emoji formatters for CLI output
Adapt formatBrowserAction for playwright-cli commands, replacing the old
mcp__playwright__browser_* tool name matching removed during migration.
* fix: mount credential file to fixed container path for Vertex AI
GOOGLE_APPLICATION_CREDENTIALS was forwarded as-is to the container,
causing the relative host path to resolve against the repo mount
instead of the credentials mount. Now both local and npx modes mount
the resolved file to /app/credentials/google-sa-key.json and rewrite
the env var to match.
* feat: add git awareness and optional description field to config
* fix: drop redundant --ipc host flag from worker container
* fix: align announcement banner URL with main branch
* feat: add target URL reachability preflight check (#254)
* Moving asset benchmark graph image to this folder
* Move benchmark results to benchmark repo
Windows Defender flags exploit code in the pentest reports as false positives, forcing every Windows user to add a Defender exclusion just to clone Shannon.
* Updated README
* fix: case-insensitive grep for semantic-release version probe
* fix: harden supply chain security (#255)
* fix: patch smol-toml and tsdown vulnerabilities
Update smol-toml 1.6.0→1.6.1 (DoS via recursive comment parsing) and
tsdown 0.21.2→0.21.5 (picomatch ReDoS + method injection).
* fix: pin all unpinned dependency versions in Dockerfile
Pins subfinder v2.13.0, WhatWeb v0.6.3 (switched from git clone to
release tarball), schemathesis 4.13.0, addressable 2.8.9,
claude-code 2.1.84, and playwright-cli 0.1.1 for reproducible builds.
* fix: pin GitHub Actions to commit SHAs for supply chain security
* fix: pin GitHub Actions to commit SHAs in beta and rollback workflows
This commit is contained in:
@@ -0,0 +1,19 @@
|
||||
/**
|
||||
* `shannon build` command — build the worker Docker image locally.
|
||||
* Only available in local mode (running from cloned repository).
|
||||
*/
|
||||
|
||||
import { buildImage } from '../docker.js';
|
||||
import { isLocal } from '../mode.js';
|
||||
|
||||
export function build(noCache: boolean): void {
|
||||
if (!isLocal()) {
|
||||
console.error('ERROR: Build is only available when running from the Shannon repository');
|
||||
console.error(' (Dockerfile not found in current directory)');
|
||||
console.error('');
|
||||
console.error('For npx usage, run: shannon update');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
buildImage(noCache);
|
||||
}
|
||||
@@ -0,0 +1,106 @@
|
||||
/**
|
||||
* `shannon logs` command — tail a workspace's workflow log.
|
||||
*
|
||||
* Uses chokidar for reliable cross-platform file watching and
|
||||
* bounded synchronous reads to prevent duplicate output.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { watch } from 'chokidar';
|
||||
import { getWorkspacesDir } from '../home.js';
|
||||
|
||||
// Match the exact line the worker writes — anchored to prevent false positives from agent output
|
||||
const COMPLETION_PATTERN = /^Workflow (COMPLETED|FAILED)$/m;
|
||||
|
||||
/** Read a byte range from a file and return it as a UTF-8 string. */
|
||||
function readRange(filePath: string, start: number, end: number): string {
|
||||
const length = end - start;
|
||||
const buffer = Buffer.alloc(length);
|
||||
const fd = fs.openSync(filePath, 'r');
|
||||
try {
|
||||
fs.readSync(fd, buffer, 0, length, start);
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
return buffer.toString('utf-8');
|
||||
}
|
||||
|
||||
/** Resolve a workspace ID to its workflow.log path, or exit with an error. */
|
||||
function resolveLogFile(workspaceId: string): string {
|
||||
const workspacesDir = getWorkspacesDir();
|
||||
|
||||
// 1. Direct match
|
||||
const directPath = path.join(workspacesDir, workspaceId, 'workflow.log');
|
||||
if (fs.existsSync(directPath)) return directPath;
|
||||
|
||||
// 2. Resume workflow ID (e.g. workspace_resume_123)
|
||||
const resumeBase = workspaceId.replace(/_resume_\d+$/, '');
|
||||
if (resumeBase !== workspaceId) {
|
||||
const resumePath = path.join(workspacesDir, resumeBase, 'workflow.log');
|
||||
if (fs.existsSync(resumePath)) return resumePath;
|
||||
}
|
||||
|
||||
// 3. Named workspace ID (e.g. workspace_shannon-123)
|
||||
const namedBase = workspaceId.replace(/_shannon-\d+$/, '');
|
||||
if (namedBase !== workspaceId) {
|
||||
const namedPath = path.join(workspacesDir, namedBase, 'workflow.log');
|
||||
if (fs.existsSync(namedPath)) return namedPath;
|
||||
}
|
||||
|
||||
console.error(`ERROR: Workflow log not found for: ${workspaceId}`);
|
||||
console.error('');
|
||||
console.error('Possible causes:');
|
||||
console.error(" - Workflow hasn't started yet");
|
||||
console.error(' - Workspace ID is incorrect');
|
||||
console.error('');
|
||||
console.error('Check the Temporal Web UI at http://localhost:8233 for workflow details');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
export function logs(workspaceId: string): void {
|
||||
const logFile = resolveLogFile(workspaceId);
|
||||
let position = 0;
|
||||
|
||||
/**
|
||||
* Output any new content appended since the last read.
|
||||
* Returns true when the workflow completion marker is detected.
|
||||
*/
|
||||
function flush(): boolean {
|
||||
try {
|
||||
const { size } = fs.statSync(logFile);
|
||||
if (size <= position) return false;
|
||||
|
||||
const data = readRange(logFile, position, size);
|
||||
process.stdout.write(data);
|
||||
position = size;
|
||||
|
||||
return COMPLETION_PATTERN.test(data);
|
||||
} catch {
|
||||
// File deleted or unreadable — treat as done
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`Tailing workflow log: ${logFile}`);
|
||||
|
||||
// 1. Output existing content
|
||||
if (flush()) {
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// 2. Watch for appended content via chokidar
|
||||
const watcher = watch(logFile, { persistent: true });
|
||||
|
||||
const shutdown = (): void => {
|
||||
watcher.close().finally(() => process.exit(0));
|
||||
// Safety net — force exit if watcher.close() stalls
|
||||
setTimeout(() => process.exit(0), 1000).unref();
|
||||
};
|
||||
|
||||
watcher.on('change', () => {
|
||||
if (flush()) shutdown();
|
||||
});
|
||||
|
||||
process.on('SIGINT', shutdown);
|
||||
}
|
||||
@@ -0,0 +1,350 @@
|
||||
/**
|
||||
* `shn setup` — interactive TUI wizard for one-time credential configuration.
|
||||
*
|
||||
* Walks the user through selecting a provider and entering credentials,
|
||||
* then persists everything to ~/.shannon/config.toml with 0o600 permissions.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import * as p from '@clack/prompts';
|
||||
import { type ShannonConfig, saveConfig } from '../config/writer.js';
|
||||
|
||||
const SHANNON_HOME = path.join(os.homedir(), '.shannon');
|
||||
|
||||
type Provider = 'anthropic' | 'custom_base_url' | 'bedrock' | 'vertex' | 'router';
|
||||
|
||||
export async function setup(): Promise<void> {
|
||||
p.intro('Shannon Setup');
|
||||
|
||||
// 1. Select provider
|
||||
const provider = await p.select({
|
||||
message: 'Select your AI provider',
|
||||
options: [
|
||||
{ value: 'anthropic' as const, label: 'Claude Direct', hint: 'recommended' },
|
||||
{ value: 'custom_base_url' as const, label: 'Custom Base URL', hint: 'proxies, gateways' },
|
||||
{ value: 'bedrock' as const, label: 'Claude via AWS Bedrock' },
|
||||
{ value: 'vertex' as const, label: 'Claude via Google Vertex AI' },
|
||||
{ value: 'router' as const, label: 'Router', hint: 'experimental' },
|
||||
],
|
||||
});
|
||||
if (p.isCancel(provider)) return cancelAndExit();
|
||||
|
||||
const config = await setupProvider(provider as Provider);
|
||||
|
||||
// 2. Save config
|
||||
saveConfig(config);
|
||||
|
||||
const configPath = path.join(SHANNON_HOME, 'config.toml');
|
||||
p.log.success(`Configuration saved to ${configPath}`);
|
||||
p.outro('Run `npx @keygraph/shannon start` to begin a scan.');
|
||||
}
|
||||
|
||||
async function setupProvider(provider: Provider): Promise<ShannonConfig> {
|
||||
switch (provider) {
|
||||
case 'anthropic':
|
||||
return setupAnthropic();
|
||||
case 'custom_base_url':
|
||||
return setupCustomBaseUrl();
|
||||
case 'bedrock':
|
||||
return setupBedrock();
|
||||
case 'vertex':
|
||||
return setupVertex();
|
||||
case 'router':
|
||||
return setupRouter();
|
||||
}
|
||||
}
|
||||
|
||||
// === Provider Setup Flows ===
|
||||
|
||||
async function setupAnthropic(): Promise<ShannonConfig> {
|
||||
const authMethod = await p.select({
|
||||
message: 'Authentication method',
|
||||
options: [
|
||||
{ value: 'api_key' as const, label: 'API Key' },
|
||||
{ value: 'oauth' as const, label: 'OAuth Token' },
|
||||
],
|
||||
});
|
||||
if (p.isCancel(authMethod)) return cancelAndExit();
|
||||
|
||||
const config: ShannonConfig = {};
|
||||
|
||||
if (authMethod === 'oauth') {
|
||||
const token = await promptSecret('Enter your OAuth token');
|
||||
config.anthropic = { oauth_token: token };
|
||||
} else {
|
||||
const apiKey = await promptSecret('Enter your Anthropic API key');
|
||||
config.anthropic = { api_key: apiKey };
|
||||
}
|
||||
|
||||
const customizeModels = await p.confirm({
|
||||
message:
|
||||
'Do you want to change the default models?\n' +
|
||||
' Small - claude-haiku-4-5-20251001\n' +
|
||||
' Medium - claude-sonnet-4-6\n' +
|
||||
' Large - claude-opus-4-6',
|
||||
initialValue: false,
|
||||
});
|
||||
if (p.isCancel(customizeModels)) return cancelAndExit();
|
||||
|
||||
if (customizeModels) {
|
||||
const small = await p.text({
|
||||
message: 'Small model ID',
|
||||
initialValue: 'claude-haiku-4-5-20251001',
|
||||
validate: required('Small model ID is required'),
|
||||
});
|
||||
if (p.isCancel(small)) return cancelAndExit();
|
||||
|
||||
const medium = await p.text({
|
||||
message: 'Medium model ID',
|
||||
initialValue: 'claude-sonnet-4-6',
|
||||
validate: required('Medium model ID is required'),
|
||||
});
|
||||
if (p.isCancel(medium)) return cancelAndExit();
|
||||
|
||||
const large = await p.text({
|
||||
message: 'Large model ID',
|
||||
initialValue: 'claude-opus-4-6',
|
||||
validate: required('Large model ID is required'),
|
||||
});
|
||||
if (p.isCancel(large)) return cancelAndExit();
|
||||
|
||||
config.models = { small, medium, large };
|
||||
}
|
||||
|
||||
return config;
|
||||
}
|
||||
|
||||
async function setupCustomBaseUrl(): Promise<ShannonConfig> {
|
||||
const baseUrl = await p.text({
|
||||
message: 'Endpoint URL',
|
||||
placeholder: 'https://your-proxy.example.com',
|
||||
validate: (value) => {
|
||||
if (!value) return 'Endpoint URL is required';
|
||||
try {
|
||||
new URL(value);
|
||||
} catch {
|
||||
return 'Must be a valid URL';
|
||||
}
|
||||
return undefined;
|
||||
},
|
||||
});
|
||||
if (p.isCancel(baseUrl)) return cancelAndExit();
|
||||
|
||||
const authToken = await promptSecret('Enter the auth token for the custom endpoint');
|
||||
|
||||
const config: ShannonConfig = {
|
||||
custom_base_url: { base_url: baseUrl, auth_token: authToken },
|
||||
};
|
||||
|
||||
const customizeModels = await p.confirm({
|
||||
message:
|
||||
'Do you want to change the default models?\n' +
|
||||
' Small - claude-haiku-4-5-20251001\n' +
|
||||
' Medium - claude-sonnet-4-6\n' +
|
||||
' Large - claude-opus-4-6',
|
||||
initialValue: false,
|
||||
});
|
||||
if (p.isCancel(customizeModels)) return cancelAndExit();
|
||||
|
||||
if (customizeModels) {
|
||||
const small = await p.text({
|
||||
message: 'Small model ID',
|
||||
initialValue: 'claude-haiku-4-5-20251001',
|
||||
validate: required('Small model ID is required'),
|
||||
});
|
||||
if (p.isCancel(small)) return cancelAndExit();
|
||||
|
||||
const medium = await p.text({
|
||||
message: 'Medium model ID',
|
||||
initialValue: 'claude-sonnet-4-6',
|
||||
validate: required('Medium model ID is required'),
|
||||
});
|
||||
if (p.isCancel(medium)) return cancelAndExit();
|
||||
|
||||
const large = await p.text({
|
||||
message: 'Large model ID',
|
||||
initialValue: 'claude-opus-4-6',
|
||||
validate: required('Large model ID is required'),
|
||||
});
|
||||
if (p.isCancel(large)) return cancelAndExit();
|
||||
|
||||
config.models = { small, medium, large };
|
||||
}
|
||||
|
||||
return config;
|
||||
}
|
||||
|
||||
async function setupBedrock(): Promise<ShannonConfig> {
|
||||
const region = await p.text({
|
||||
message: 'AWS Region',
|
||||
placeholder: 'us-east-1',
|
||||
validate: required('AWS Region is required'),
|
||||
});
|
||||
if (p.isCancel(region)) return cancelAndExit();
|
||||
|
||||
const token = await promptSecret('Enter your AWS Bearer Token');
|
||||
|
||||
const small = await p.text({
|
||||
message: 'Small model ID',
|
||||
placeholder: 'us.anthropic.claude-haiku-4-5-20251001-v1:0',
|
||||
validate: required('Small model ID is required'),
|
||||
});
|
||||
if (p.isCancel(small)) return cancelAndExit();
|
||||
|
||||
const medium = await p.text({
|
||||
message: 'Medium model ID',
|
||||
placeholder: 'us.anthropic.claude-sonnet-4-6',
|
||||
validate: required('Medium model ID is required'),
|
||||
});
|
||||
if (p.isCancel(medium)) return cancelAndExit();
|
||||
|
||||
const large = await p.text({
|
||||
message: 'Large model ID',
|
||||
placeholder: 'us.anthropic.claude-opus-4-6',
|
||||
validate: required('Large model ID is required'),
|
||||
});
|
||||
if (p.isCancel(large)) return cancelAndExit();
|
||||
|
||||
return {
|
||||
bedrock: { use: true, region, token },
|
||||
models: { small, medium, large },
|
||||
};
|
||||
}
|
||||
|
||||
async function setupVertex(): Promise<ShannonConfig> {
|
||||
// 1. Collect region and project ID
|
||||
const region = await p.text({
|
||||
message: 'Google Cloud region',
|
||||
placeholder: 'us-east5',
|
||||
validate: required('Region is required'),
|
||||
});
|
||||
if (p.isCancel(region)) return cancelAndExit();
|
||||
|
||||
const projectId = await p.text({
|
||||
message: 'GCP Project ID',
|
||||
validate: required('Project ID is required'),
|
||||
});
|
||||
if (p.isCancel(projectId)) return cancelAndExit();
|
||||
|
||||
// 2. File picker for service account key
|
||||
p.log.info('Select the path to your GCP Service Account JSON key file.');
|
||||
const keySourcePath = await p.path({
|
||||
message: 'Service Account JSON key file',
|
||||
validate: (value) => {
|
||||
if (!value) return 'Path is required';
|
||||
if (!fs.existsSync(value)) return 'File not found';
|
||||
if (!value.endsWith('.json')) return 'Must be a .json file';
|
||||
return undefined;
|
||||
},
|
||||
});
|
||||
if (p.isCancel(keySourcePath)) return cancelAndExit();
|
||||
|
||||
// 3. Copy key to ~/.shannon/ and lock permissions
|
||||
const destPath = path.join(SHANNON_HOME, 'google-sa-key.json');
|
||||
fs.mkdirSync(SHANNON_HOME, { recursive: true });
|
||||
fs.copyFileSync(keySourcePath, destPath);
|
||||
fs.chmodSync(destPath, 0o600);
|
||||
p.log.success(`Key copied to ${destPath} (permissions: 0600)`);
|
||||
|
||||
// 4. Model tiers
|
||||
const models = await p.group({
|
||||
small: () =>
|
||||
p.text({
|
||||
message: 'Small model ID',
|
||||
placeholder: 'claude-haiku-4-5@20251001',
|
||||
validate: required('Small model ID is required'),
|
||||
}),
|
||||
medium: () =>
|
||||
p.text({
|
||||
message: 'Medium model ID',
|
||||
placeholder: 'claude-sonnet-4-6',
|
||||
validate: required('Medium model ID is required'),
|
||||
}),
|
||||
large: () =>
|
||||
p.text({
|
||||
message: 'Large model ID',
|
||||
placeholder: 'claude-opus-4-6',
|
||||
validate: required('Large model ID is required'),
|
||||
}),
|
||||
});
|
||||
if (p.isCancel(models)) return cancelAndExit();
|
||||
|
||||
return {
|
||||
vertex: {
|
||||
use: true,
|
||||
region,
|
||||
project_id: projectId,
|
||||
key_path: destPath,
|
||||
},
|
||||
models: { small: models.small, medium: models.medium, large: models.large },
|
||||
};
|
||||
}
|
||||
|
||||
async function setupRouter(): Promise<ShannonConfig> {
|
||||
const routerProvider = await p.select({
|
||||
message: 'Router provider',
|
||||
options: [
|
||||
{ value: 'openai' as const, label: 'OpenAI' },
|
||||
{ value: 'openrouter' as const, label: 'OpenRouter' },
|
||||
],
|
||||
});
|
||||
if (p.isCancel(routerProvider)) return cancelAndExit();
|
||||
|
||||
const apiKey = await promptSecret(
|
||||
routerProvider === 'openai' ? 'Enter your OpenAI API key' : 'Enter your OpenRouter API key',
|
||||
);
|
||||
|
||||
let defaultModel: string;
|
||||
if (routerProvider === 'openai') {
|
||||
const model = await p.select({
|
||||
message: 'Default model',
|
||||
options: [
|
||||
{ value: 'gpt-5.2' as const, label: 'GPT-5.2' },
|
||||
{ value: 'gpt-5-mini' as const, label: 'GPT-5 Mini' },
|
||||
],
|
||||
});
|
||||
if (p.isCancel(model)) return cancelAndExit();
|
||||
defaultModel = `openai,${model}`;
|
||||
} else {
|
||||
const model = await p.select({
|
||||
message: 'Default model',
|
||||
options: [{ value: 'google/gemini-3-flash-preview' as const, label: 'Google Gemini 3 Flash Preview' }],
|
||||
});
|
||||
if (p.isCancel(model)) return cancelAndExit();
|
||||
defaultModel = `openrouter,${model}`;
|
||||
}
|
||||
|
||||
const router: ShannonConfig['router'] = { default: defaultModel };
|
||||
if (routerProvider === 'openai') {
|
||||
router.openai_key = apiKey;
|
||||
} else {
|
||||
router.openrouter_key = apiKey;
|
||||
}
|
||||
|
||||
return { router };
|
||||
}
|
||||
|
||||
// === Helpers ===
|
||||
|
||||
async function promptSecret(message: string): Promise<string> {
|
||||
const value = await p.password({
|
||||
message,
|
||||
validate: required(`${message.replace(/^Enter /, '')} is required`),
|
||||
});
|
||||
if (p.isCancel(value)) return cancelAndExit();
|
||||
return value;
|
||||
}
|
||||
|
||||
function required(errorMessage: string): (value: string | undefined) => string | undefined {
|
||||
return (value) => {
|
||||
if (!value) return errorMessage;
|
||||
return undefined;
|
||||
};
|
||||
}
|
||||
|
||||
function cancelAndExit(): never {
|
||||
p.cancel('Setup cancelled.');
|
||||
process.exit(0);
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
/**
|
||||
* `shannon start` command — launch a pentest scan.
|
||||
*
|
||||
* Handles both local mode (local build, ./workspaces/, mounted prompts)
|
||||
* and npx mode (Docker Hub pull, ~/.shannon/).
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { ensureImage, ensureInfra, randomSuffix, spawnWorker } from '../docker.js';
|
||||
import { buildEnvFlags, isRouterConfigured, loadEnv, validateCredentials } from '../env.js';
|
||||
import { getCredentialsPath, getWorkspacesDir, initHome } from '../home.js';
|
||||
import { isLocal } from '../mode.js';
|
||||
import { ensureDeliverables, resolveConfig, resolveRepo } from '../paths.js';
|
||||
import { displaySplash } from '../splash.js';
|
||||
|
||||
export interface StartArgs {
|
||||
url: string;
|
||||
repo: string;
|
||||
config?: string;
|
||||
workspace?: string;
|
||||
output?: string;
|
||||
pipelineTesting: boolean;
|
||||
router: boolean;
|
||||
version: string;
|
||||
}
|
||||
|
||||
export async function start(args: StartArgs): Promise<void> {
|
||||
// 1. Initialize state directories and load env
|
||||
initHome();
|
||||
loadEnv();
|
||||
|
||||
// 2. Validate credentials and auto-detect router mode
|
||||
const creds = validateCredentials();
|
||||
if (!creds.valid) {
|
||||
console.error(`ERROR: ${creds.error}`);
|
||||
process.exit(1);
|
||||
}
|
||||
const useRouter = args.router || isRouterConfigured();
|
||||
|
||||
// 3. Resolve paths
|
||||
const repo = resolveRepo(args.repo);
|
||||
const config = args.config ? resolveConfig(args.config) : undefined;
|
||||
ensureDeliverables(repo.hostPath);
|
||||
|
||||
// 4. Ensure workspaces dir is writable by container user (UID 1001)
|
||||
const workspacesDir = getWorkspacesDir();
|
||||
fs.mkdirSync(workspacesDir, { recursive: true });
|
||||
fs.chmodSync(workspacesDir, 0o777);
|
||||
|
||||
// 5. Handle router env
|
||||
if (useRouter) {
|
||||
process.env.ANTHROPIC_BASE_URL = 'http://shannon-router:3456';
|
||||
process.env.ANTHROPIC_AUTH_TOKEN = 'shannon-router-key';
|
||||
}
|
||||
|
||||
// 6. Ensure image (auto-build in dev, pull in npx) and start infra
|
||||
ensureImage(args.version);
|
||||
await ensureInfra(useRouter);
|
||||
|
||||
// 7. Generate unique task queue and container name
|
||||
const suffix = randomSuffix();
|
||||
const taskQueue = `shannon-${suffix}`;
|
||||
const containerName = `shannon-worker-${suffix}`;
|
||||
|
||||
// 8. Generate workspace name if not provided
|
||||
const workspace =
|
||||
args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`;
|
||||
|
||||
// 9. Resolve credentials — mount single file to fixed container path
|
||||
const credentialsPath = getCredentialsPath();
|
||||
const hasCredentials = fs.existsSync(credentialsPath);
|
||||
|
||||
if (hasCredentials) {
|
||||
process.env.GOOGLE_APPLICATION_CREDENTIALS = '/app/credentials/google-sa-key.json';
|
||||
}
|
||||
|
||||
// 10. Resolve output directory
|
||||
const outputDir = args.output ? path.resolve(args.output) : undefined;
|
||||
if (outputDir) {
|
||||
fs.mkdirSync(outputDir, { recursive: true });
|
||||
}
|
||||
|
||||
// 11. Resolve prompts directory (local mode only)
|
||||
const promptsDir = isLocal() ? path.resolve('apps/worker/prompts') : undefined;
|
||||
|
||||
// 12. Display splash screen
|
||||
displaySplash(isLocal() ? undefined : args.version);
|
||||
|
||||
// 13. Spawn worker container
|
||||
const proc = spawnWorker({
|
||||
version: args.version,
|
||||
url: args.url,
|
||||
repo,
|
||||
workspacesDir,
|
||||
taskQueue,
|
||||
containerName,
|
||||
envFlags: buildEnvFlags(),
|
||||
...(config && { config }),
|
||||
...(hasCredentials && { credentials: credentialsPath }),
|
||||
...(promptsDir && { promptsDir }),
|
||||
...(outputDir && { outputDir }),
|
||||
...(workspace && { workspace }),
|
||||
...(args.pipelineTesting && { pipelineTesting: true }),
|
||||
});
|
||||
|
||||
// 14. Wait for workflow to register, then display info
|
||||
proc.on('error', (err) => {
|
||||
console.error(`Failed to start worker: ${err.message}`);
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
// Detect whether this is a fresh workspace or a resume by checking session.json existence
|
||||
const sessionJson = path.join(workspacesDir, workspace, 'session.json');
|
||||
const isResume = fs.existsSync(sessionJson);
|
||||
let initialResumeCount = 0;
|
||||
if (isResume) {
|
||||
try {
|
||||
const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8'));
|
||||
initialResumeCount = session.session?.resumeAttempts?.length ?? 0;
|
||||
} catch {
|
||||
// Corrupted file — worker will handle validation
|
||||
}
|
||||
}
|
||||
|
||||
// Poll for workflow to register in session.json
|
||||
process.stdout.write('Waiting for workflow to start...');
|
||||
let workflowId = '';
|
||||
let started = false;
|
||||
let attempts = 0;
|
||||
const pollInterval = setInterval(() => {
|
||||
attempts++;
|
||||
if (attempts > 60) {
|
||||
clearInterval(pollInterval);
|
||||
process.stdout.write('\n');
|
||||
console.error('Timeout waiting for workflow to start');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
try {
|
||||
const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8'));
|
||||
const resumeAttempts: { workflowId: string }[] = session.session?.resumeAttempts ?? [];
|
||||
|
||||
// Fresh: session.json appears with originalWorkflowId. Resume: new resumeAttempts entry.
|
||||
const ready = isResume ? resumeAttempts.length > initialResumeCount : !!session.session?.originalWorkflowId;
|
||||
|
||||
if (ready) {
|
||||
clearInterval(pollInterval);
|
||||
started = true;
|
||||
|
||||
// Latest workflow ID: last resume attempt, or originalWorkflowId for fresh scans
|
||||
workflowId = resumeAttempts.at(-1)?.workflowId ?? session.session?.originalWorkflowId ?? '';
|
||||
|
||||
// Clear waiting line and show info
|
||||
process.stdout.write('\r\x1b[K');
|
||||
printInfo(args, useRouter, workspace, workflowId, repo.hostPath, workspacesDir);
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
// File doesn't exist yet
|
||||
}
|
||||
process.stdout.write('.');
|
||||
}, 2000);
|
||||
|
||||
// Stop the worker container only if it hasn't started yet
|
||||
let cleaned = false;
|
||||
const cleanup = (): void => {
|
||||
if (cleaned || started) return;
|
||||
cleaned = true;
|
||||
clearInterval(pollInterval);
|
||||
console.log(`\nStopping worker ${containerName}...`);
|
||||
try {
|
||||
execFileSync('docker', ['stop', containerName], { stdio: 'pipe' });
|
||||
} catch {
|
||||
// Container may have already exited
|
||||
}
|
||||
};
|
||||
|
||||
process.on('SIGINT', () => {
|
||||
cleanup();
|
||||
process.exit(0);
|
||||
});
|
||||
process.on('SIGTERM', () => {
|
||||
cleanup();
|
||||
process.exit(0);
|
||||
});
|
||||
process.on('exit', cleanup);
|
||||
}
|
||||
|
||||
function printInfo(
|
||||
args: StartArgs,
|
||||
routerActive: boolean,
|
||||
workspace: string,
|
||||
workflowId: string,
|
||||
repoPath: string,
|
||||
workspacesDir: string,
|
||||
): void {
|
||||
const logsCmd = isLocal() ? `./shannon logs ${workspace}` : `npx @keygraph/shannon logs ${workspace}`;
|
||||
const reportsPath = path.join(workspacesDir, workspace);
|
||||
|
||||
console.log(` Target: ${args.url}`);
|
||||
console.log(` Repository: ${repoPath}`);
|
||||
console.log(` Workspace: ${workspace}`);
|
||||
if (args.config) {
|
||||
console.log(` Config: ${path.resolve(args.config)}`);
|
||||
}
|
||||
if (args.pipelineTesting) {
|
||||
console.log(' Mode: Pipeline Testing');
|
||||
}
|
||||
if (routerActive) {
|
||||
console.log(' Router: Enabled');
|
||||
}
|
||||
console.log('');
|
||||
console.log(' Monitor:');
|
||||
if (workflowId) {
|
||||
console.log(` Web UI: http://localhost:8233/namespaces/default/workflows/${workflowId}`);
|
||||
} else {
|
||||
console.log(' Web UI: http://localhost:8233');
|
||||
}
|
||||
console.log(` Logs: ${logsCmd}`);
|
||||
console.log('');
|
||||
console.log(' Output:');
|
||||
console.log(` Reports: ${reportsPath}/`);
|
||||
console.log('');
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
/**
|
||||
* `shannon status` command — show running workers and Temporal health.
|
||||
*/
|
||||
|
||||
import { isTemporalReady, listRunningWorkers } from '../docker.js';
|
||||
|
||||
export function status(): void {
|
||||
// 1. Temporal health
|
||||
const temporalUp = isTemporalReady();
|
||||
console.log(`Temporal: ${temporalUp ? 'running' : 'not running'}`);
|
||||
if (temporalUp) {
|
||||
console.log(' Web UI: http://localhost:8233');
|
||||
}
|
||||
console.log('');
|
||||
|
||||
// 2. Running workers
|
||||
const workers = listRunningWorkers();
|
||||
if (workers) {
|
||||
console.log('Workers:');
|
||||
console.log(workers);
|
||||
} else {
|
||||
console.log('Workers: none running');
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
/**
|
||||
* `shannon stop` command — stop workers and infrastructure.
|
||||
*/
|
||||
|
||||
import * as p from '@clack/prompts';
|
||||
import { stopInfra, stopWorkers } from '../docker.js';
|
||||
|
||||
export async function stop(clean: boolean): Promise<void> {
|
||||
if (clean) {
|
||||
const confirmed = await p.confirm({
|
||||
message: 'This will stop all running scans and remove the Temporal data. Continue?',
|
||||
});
|
||||
if (p.isCancel(confirmed) || !confirmed) {
|
||||
p.cancel('Aborted.');
|
||||
process.exit(0);
|
||||
}
|
||||
}
|
||||
|
||||
stopWorkers();
|
||||
stopInfra(clean);
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
/**
|
||||
* `shn uninstall` command — remove ~/.shannon/ after confirmation (npx only).
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import * as p from '@clack/prompts';
|
||||
import { stopInfra, stopWorkers } from '../docker.js';
|
||||
|
||||
const SHANNON_HOME = path.join(os.homedir(), '.shannon');
|
||||
|
||||
export async function uninstall(): Promise<void> {
|
||||
p.intro('Shannon Uninstall');
|
||||
|
||||
if (!fs.existsSync(SHANNON_HOME)) {
|
||||
p.log.info('Nothing to remove. Shannon is not configured on this machine.');
|
||||
p.outro('Done.');
|
||||
return;
|
||||
}
|
||||
|
||||
const confirmed = await p.confirm({
|
||||
message: 'This will permanently remove all past scan data, saved configurations, and API keys. Continue?',
|
||||
});
|
||||
if (p.isCancel(confirmed) || !confirmed) {
|
||||
p.cancel('Aborted.');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Stop any running containers first
|
||||
stopWorkers();
|
||||
stopInfra(false);
|
||||
|
||||
fs.rmSync(SHANNON_HOME, { recursive: true, force: true });
|
||||
p.log.success('All Shannon data has been removed.');
|
||||
p.outro('Shannon has been uninstalled. Run `npx @keygraph/shannon setup` to start fresh.');
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
/**
|
||||
* `shannon workspaces` command — list all workspaces.
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import os from 'node:os';
|
||||
import { getWorkerImage } from '../docker.js';
|
||||
import { getWorkspacesDir } from '../home.js';
|
||||
|
||||
export function workspaces(version: string): void {
|
||||
const workspacesDir = getWorkspacesDir();
|
||||
const image = getWorkerImage(version);
|
||||
|
||||
try {
|
||||
execFileSync(
|
||||
'docker',
|
||||
[
|
||||
'run',
|
||||
'--rm',
|
||||
'-v',
|
||||
`${workspacesDir}:/app/workspaces`,
|
||||
'-e',
|
||||
'WORKSPACES_DIR=/app/workspaces',
|
||||
image,
|
||||
'node',
|
||||
'apps/worker/dist/temporal/workspaces.js',
|
||||
],
|
||||
{ stdio: 'inherit', ...(os.platform() === 'win32' && { env: { ...process.env, MSYS_NO_PATHCONV: '1' } }) },
|
||||
);
|
||||
} catch {
|
||||
console.error('ERROR: Failed to list workspaces. Is the Docker image available?');
|
||||
console.error(` Run: docker pull ${image}`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user