farhoodlabs/trebuchet
8
0
Fork 0
Code Issues Pull Requests 5 Actions Packages Projects Releases Wiki Activity

chore: rebrand farhoodliquor → farhoodlabs, API-only mode, split infra

Browse Source 
- Rename org references from farhoodliquor to farhoodlabs in CI workflows
  and GHCR image tags
- Rewrite README for Hightower as API-driven K8s fork of Shannon
- Update CLAUDE.md to reflect API-only deployment model
- Delete docker-compose files (K8s only, no Docker Compose support)
- Delete shannon CLI entry point (API-only going forward)
- Move K8s manifests to farhoodlabs/hightower-infra

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Farhood
2026-04-22 07:19:56 -04:00
parent 0013776646
commit 325eac98ea
19 changed files with 110 additions and 1453 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+25 -110
View File
@@ -1,78 +1,12 @@
# CLAUDE.md
AI-powered penetration testing agent for defensive security analysis. Automates vulnerability assessment by combining reconnaissance tools with AI-powered code analysis.
Hightower is a fork of [Shannon](https://github.com/KeygraphHQ/shannon) by Keygraph — an AI-powered penetration testing agent for defensive security analysis. It wraps Shannon's autonomous pentesting engine with a REST API and Kubernetes deployment tooling.
**Upstream policy:** `apps/cli/` and `apps/worker/` are kept as close to upstream Shannon as possible for backporting. Only `apps/api/`, CI/CD workflows, and infra are Hightower-specific.
## Commands
**Prerequisites:** Docker, AI provider credentials (`.env` for local, `shn setup` or env vars for npx)
### Dual CLI
Shannon supports two CLI modes, auto-detected based on the current working directory:
| | **npx** (`npx @keygraph/shannon`) | **Local** (`./shannon`) |
|---|---|---|
| **Install** | Zero-install via npm | Clone the repo |
| **Image** | Pulled from Docker Hub (`keygraph/shannon:latest`) | Built locally (`shannon-worker`) |
| **State** | `~/.shannon/` | Project directory |
| **Credentials** | `~/.shannon/config.toml` (via `shn setup`) or env vars | `./.env` |
| **Config** | `~/.shannon/config.toml` (via `shn setup`) | N/A |
| **Prompts** | Bundled in Docker image | Mounted from `./apps/worker/prompts/` (live-editable) |
Mode auto-detection: local mode activates when env var `SHANNON_LOCAL=1` is set by the `./shannon` entry point (`apps/cli/src/mode.ts`). Otherwise npx mode.
### npx Quick Start
```bash
# Configure credentials (interactive wizard)
npx @keygraph/shannon setup
# Or export env vars directly (non-interactive / CI)
export ANTHROPIC_API_KEY=your-key
# Run
npx @keygraph/shannon start -u <url> -r /path/to/repo
```
### Local (Development) Quick Start
```bash
# Setup
echo "ANTHROPIC_API_KEY=your-key" > .env
# Build (auto-runs if image missing)
./shannon build
# Run
./shannon start -u <url> -r my-repo
./shannon start -u <url> -r my-repo -c ./apps/worker/configs/my-config.yaml
./shannon start -u <url> -r /any/path/to/repo
```
### Common Commands
```bash
# Setup (npx mode only — one-time credential configuration)
npx @keygraph/shannon setup
# Workspaces & Resume
./shannon start -u <url> -r my-repo -w my-audit # New named workspace
./shannon start -u <url> -r my-repo -w my-audit # Resume (same command)
./shannon workspaces # List all workspaces
# Monitor
./shannon logs <workspace> # Tail workflow log
./shannon status # Show running workers
# Temporal Web UI: http://localhost:8233
# Stop
./shannon stop # Preserves workflow data
./shannon stop --clean # Full cleanup including volumes (confirms first)
# Image management
./shannon build [--no-cache] # Local mode: build worker image
npx @keygraph/shannon uninstall # npx mode: remove ~/.shannon/ (confirms first)
# Build TypeScript (development)
pnpm run build # Build all packages via Turborepo
pnpm run check # Type-check all packages
@@ -82,42 +16,25 @@ pnpm biome:fix # Auto-fix lint, format, and import sorting
**Monorepo tooling:** pnpm workspaces, Turborepo for task orchestration, Biome for linting/formatting. TypeScript compiler options shared via `tsconfig.base.json` at the root. All packages extend it, overriding only `rootDir` and `outDir`. Shared devDependencies (`typescript`, `@types/node`, `turbo`, `@biomejs/biome`) are hoisted to the root workspace.
**Options:** `-c <file>` (YAML config), `-o <path>` (output directory), `-w <name>` (named workspace; auto-resumes if exists), `--pipeline-testing` (minimal prompts, 10s retries), `--router` (multi-model routing via [claude-code-router](https://github.com/musistudio/claude-code-router))
## Architecture
### Monorepo Layout
```
apps/cli/ — @keygraph/shannon (published to npm, bundled with tsdown)
apps/worker/ — @shannon/worker (private, Temporal worker + pipeline logic)
apps/api/ — @shannon/api (Hightower REST API, K8s-native)
apps/cli/ — @keygraph/shannon (upstream CLI, not used in production)
apps/worker/ — @shannon/worker (upstream Temporal worker + pipeline logic)
```
### API Package (`apps/api/`)
Hightower-specific REST API for triggering and managing pentests. Deployed on Kubernetes.
### CLI Package (`apps/cli/`)
Published as `@keygraph/shannon` on npm. Contains only Docker orchestration logic — no Temporal SDK, business logic, or prompts. Bundled with tsdown for single-file ESM output.
- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `logs`, `workspaces`, `status`, `build`, `uninstall`, `info`)
- `apps/cli/src/mode.ts` — Auto-detection: local mode if `SHANNON_LOCAL=1` env var is set
- `apps/cli/src/docker.ts` — Compose lifecycle, image pull/build, ephemeral `docker run` worker spawning
- `apps/cli/src/home.ts` — State directory management (`~/.shannon/` for npx, `./` for local)
- `apps/cli/src/env.ts` — `.env` loading, TOML fallback (npx only) via `apps/cli/src/config/resolver.ts`, credential validation, env flag building
- `apps/cli/src/config/resolver.ts` — Cascading config (npx only): env vars → `~/.shannon/config.toml` (parsed with `smol-toml`)
- `apps/cli/src/config/writer.ts` — TOML serialization and secure file persistence (0o600)
- `apps/cli/src/commands/setup.ts` — Interactive TUI wizard (`@clack/prompts`) for provider credential setup (npx only)
- `apps/cli/src/paths.ts` — Repo/config path resolution (bare name → `./repos/<name>`, or any absolute/relative path)
- `apps/cli/src/commands/` — Command handlers
- `apps/cli/infra/compose.yml` — Bundled Temporal + router compose file for npx mode
- `apps/cli/tsdown.config.ts` — tsdown bundler config
- `shannon` — Node.js entry point (`#!/usr/bin/env node`) that delegates to `apps/cli/dist/index.mjs`
### Docker Architecture
Infra (Temporal + router) runs via `docker-compose.yml`. Workers are ephemeral `docker run --rm` containers, one per scan, each with a unique task queue and isolated volume mounts.
- `docker-compose.yml` — Infra only: `shannon-temporal` (port 7233/8233) and `shannon-router` (port 3456, optional via profile). Network: `shannon-net`
- `Dockerfile` — 2-stage build (builder + Chainguard Wolfi runtime). Uses pnpm. Entrypoint: `CMD ["node", "apps/worker/dist/temporal/worker.js"]`
- No `docker-compose.docker.yml` — host gateway handled via `--add-host` flag in CLI
Upstream Shannon CLI — kept for backporting compatibility. Not used in Hightower's K8s deployment.
### Worker Package (`apps/worker/`)
Upstream Shannon worker — Temporal worker + pipeline logic. Runs as ephemeral K8s Jobs.
- `apps/worker/src/paths.ts` — Centralized path constants (`PROMPTS_DIR`, `CONFIGS_DIR`, `WORKSPACES_DIR`)
- `apps/worker/src/session-manager.ts` — Agent definitions (`AGENTS` record). Agent types in `apps/worker/src/types/agents.ts`
- `apps/worker/src/config-parser.ts` — YAML config parsing with JSON Schema validation
@@ -135,6 +52,7 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig
- `apps/worker/src/temporal/summary-mapper.ts` — Maps `PipelineSummary` to `WorkflowSummary`
- `apps/worker/src/temporal/worker.ts` — Combined worker + client entry point (per-invocation task queue, submits workflow, waits for result)
- `apps/worker/src/temporal/shared.ts` — Types, interfaces, query definitions
### Five-Phase Pipeline
1. **Pre-Recon** (`pre-recon`) — External scans (nmap, subfinder, whatweb) + source code analysis
@@ -143,8 +61,15 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig
4. **Exploitation** (5 parallel agents, conditional) — Exploits confirmed vulnerabilities
5. **Reporting** (`report`) — Executive-level security report
### Docker Images
- `Dockerfile` — Worker image: 2-stage build (builder + Chainguard Wolfi runtime). Uses pnpm. Entrypoint: `CMD ["node", "apps/worker/dist/temporal/worker.js"]`
- `apps/api/Dockerfile` — API image: minimal Alpine build
### Kubernetes Infrastructure
K8s manifests live in a separate repository: [farhoodlabs/hightower-infra](https://github.com/farhoodlabs/hightower-infra).
### Supporting Systems
- **Configuration** — YAML configs in `apps/worker/configs/` with JSON Schema validation (`config-schema.json`). Supports auth settings, MFA/TOTP, and per-app testing parameters. Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `shn setup`)
- **Configuration** — YAML configs in `apps/worker/configs/` with JSON Schema validation (`config-schema.json`). Supports auth settings, MFA/TOTP, and per-app testing parameters
- **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`
- **SDK Integration** — Uses `@anthropic-ai/claude-agent-sdk` with `maxTurns: 10_000` and `bypassPermissions` mode. Browser automation via `playwright-cli` with session isolation (`-s=<session>`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth
- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. Tracks session metrics, per-agent logs, prompts, and deliverables. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive
@@ -171,7 +96,7 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig
- **Modular Error Handling** — `ErrorCode` enum, `Result<T,E>` for explicit error propagation, automatic retry (3 attempts per agent)
- **Services Boundary** — Activities are thin Temporal wrappers; `apps/worker/src/services/` owns business logic, accepts `ActivityLogger`, returns `Result<T,E>`. No Temporal imports in services
- **DI Container** — Per-workflow in `apps/worker/src/services/container.ts`. `AuditSession` excluded (parallel safety)
- **Ephemeral Workers** — Each scan runs in its own `docker run --rm` container with a per-invocation task queue. Temporal routes activities by queue name, so per-scan queues ensure activities never land on a worker with the wrong repo mounted
- **Ephemeral Workers** — Each scan runs in its own container with a per-invocation task queue. Temporal routes activities by queue name, so per-scan queues ensure activities never land on a worker with the wrong repo mounted
### Security
Defensive security tool only. Use only on systems you own or have explicit permission to test.
@@ -223,26 +148,16 @@ Comments must be **timeless** — no references to this conversation, refactorin
## Key Files
**CLI:** `shannon` (entry point), `apps/cli/src/index.ts` (dispatcher), `apps/cli/src/docker.ts` (orchestration), `apps/cli/src/mode.ts` (auto-detection)
**API:** `apps/api/src/` (Hightower REST API), `apps/api/Dockerfile`
**Entry Points:** `apps/worker/src/temporal/workflows.ts`, `apps/worker/src/temporal/activities.ts`, `apps/worker/src/temporal/worker.ts`
**Core Logic:** `apps/worker/src/session-manager.ts`, `apps/worker/src/ai/claude-executor.ts`, `apps/worker/src/config-parser.ts`, `apps/worker/src/services/`, `apps/worker/src/audit/`
**Config:** `docker-compose.yml`, `apps/cli/infra/compose.yml`, `apps/worker/configs/`, `apps/worker/prompts/`, `tsconfig.base.json` (shared compiler options), `turbo.json`, `biome.json`
**Config:** `Dockerfile`, `apps/worker/configs/`, `apps/worker/prompts/`, `tsconfig.base.json` (shared compiler options), `turbo.json`, `biome.json`
**CI/CD:** `.github/workflows/release.yml` (Docker Hub push + npm publish + GitHub release, manual dispatch)
**CI/CD:** `.github/workflows/ci.yml` (type-check, lint, build & push images to GHCR), `.github/workflows/release.yml` (Docker Hub push + GitHub release, manual dispatch)
## Package Installation
Package managers are configured with a minimum release age (7 days). Requires pnpm >= 10.16.0. If `pnpm install` fails due to a package being too new, **do not attempt to bypass it** — report the blocked package to the user and stop.
## Troubleshooting
- **"Repository not found"** — Pass a bare name (`-r my-repo`) for `./repos/my-repo`, or a path (`-r /path/to/repo`) for any directory
- **"Temporal not ready"** — Wait for health check or `docker compose logs temporal`
- **Worker not processing** — Check `docker ps --filter "name=shannon-worker-"`
- **Reset state** — `./shannon stop --clean`
- **Local apps unreachable** — Use `host.docker.internal` instead of `localhost`
- **Missing tools** — Use `--pipeline-testing` to skip nmap/subfinder/whatweb (graceful degradation)
- **Container permissions** — On Linux, may need `sudo` for docker commands