feat: add model tracking and reporting across pipeline

- Track actual model name from router through audit logs, session.json, and query output
- Add router-utils.ts to resolve model names from ROUTER_DEFAULT env var
- Inject model info into final report's Executive Summary section
- Update documentation with supported providers, pricing, and config examples
- Update router-config.json with latest model versions (GPT-5.2, Gemini 2.5, etc.)
This commit is contained in:
ajmallesh
2026-01-15 18:30:19 -08:00
parent d01980ce4b
commit cd04c7a6d2
16 changed files with 312 additions and 56 deletions
+41 -11
View File
@@ -50,7 +50,7 @@ CONFIG=<file> YAML configuration file for authentication and testing pa
OUTPUT=<path> Custom output directory for session folder (default: ./audit-logs/)
PIPELINE_TESTING=true Use minimal prompts and fast retry intervals (10s instead of 5min)
REBUILD=true Force Docker rebuild with --no-cache (use when code changes aren't picked up)
ROUTER=true Route requests through claude-code-router for multi-model support (see limitations below)
ROUTER=true Route requests through claude-code-router for multi-model support
```
### Generate TOTP for Authentication
@@ -262,11 +262,51 @@ The tool should only be used on systems you own or have explicit permission to t
- `shannon` - CLI script for running pentests
- `docker-compose.yml` - Temporal server + worker containers
- `configs/` - YAML configs with `config-schema.json` for validation
- `configs/router-config.json` - Router service configuration for multi-model support
- `prompts/` - AI prompt templates (`vuln-*.txt`, `exploit-*.txt`, etc.)
**Output:**
- `audit-logs/{hostname}_{sessionId}/` - Session metrics, agent logs, deliverables
### Router Mode (Multi-Model Support)
Shannon supports routing Claude Agent SDK requests through alternative LLM providers via [claude-code-router](https://github.com/musistudio/claude-code-router).
**Enable router mode:**
```bash
./shannon start URL=<url> REPO=<path> ROUTER=true
```
**Supported Providers:**
| Provider | Models | Use Case |
|----------|--------|----------|
| OpenAI | `gpt-5.2`, `gpt-5-mini` | Good tool use, balanced cost/performance |
| Gemini | `gemini-2.5-pro` | Long context (1M+ tokens), strong reasoning |
| DeepSeek | `deepseek-ai/DeepSeek-V3`, `deepseek-chat` | Cheapest option for dev/testing |
| OpenRouter | `anthropic/claude-sonnet-4`, `google/gemini-3-pro-preview`, `openai/gpt-5.2` | Multi-provider access via single API |
**Configuration (in .env):**
```bash
# OpenAI
OPENAI_API_KEY=sk-your-key
ROUTER_DEFAULT=openai,gpt-5.2
# Gemini
GEMINI_API_KEY=your-gemini-key
ROUTER_DEFAULT=gemini,gemini-2.5-pro
# DeepSeek (via Together.ai)
DEEPSEEK_API_KEY=your-together-key
ROUTER_DEFAULT=deepseek,deepseek-ai/DeepSeek-V3
# OpenRouter
OPENROUTER_API_KEY=sk-or-your-key
ROUTER_DEFAULT=openrouter,anthropic/claude-sonnet-4
```
**Note:** Shannon is optimized for Anthropic's Claude models. Alternative providers are useful for cost savings during development but may produce varying results.
## Troubleshooting
### Common Issues
@@ -285,16 +325,6 @@ Missing tools can be skipped using `PIPELINE_TESTING=true` mode during developme
- `subfinder` - Subdomain discovery
- `whatweb` - Web technology detection
### Router Mode Limitations
When using `ROUTER=true` to route requests through claude-code-router (e.g., to use OpenAI models):
**Cost tracking shows $0.00**: The Claude Agent SDK expects `total_cost_usd` in the result message, which is Anthropic-specific. OpenAI's API returns token counts in `usage` but not a cost field, and the router doesn't translate this. This is a known limitation of the router, not a Shannon bug.
**Workarounds:**
- Accept $0 costs when using router mode (recommended for dev/testing)
- Use Anthropic directly for production runs where cost tracking matters
- Use external tools like `ccusage` for post-hoc token analysis
### Diagnostic & Utility Scripts
```bash
# View Temporal workflow history