Commit Graph

16 Commits

Author SHA1 Message Date
Dotta 778e775c35 Add secrets provider vaults and remote import (#5429)
## Thinking Path

> - Paperclip orchestrates AI-agent companies and needs secrets handling
to work across local development, hosted operators, and governed agent
execution.
> - The affected subsystem is the company-scoped secrets control plane:
database schema, server services/routes, CLI workflows, and the Secrets
settings UI.
> - The gap was that secrets were local-only and operators could not
manage provider vaults or import existing remote references without
exposing plaintext.
> - This branch adds provider vault configuration plus an AWS Secrets
Manager remote-import path while preserving company boundaries, binding
context, and audit trails.
> - I kept the PR to a single branch PR, removed unrelated
lockfile/package drift, rebased the full branch onto the current
`public-gh/master`, and addressed fresh Greptile findings.
> - The benefit is a reviewable implementation of provider-backed
secrets with focused tests covering provider selection, import
conflicts, deleted secret reuse, rotation guards, and AWS signing
behavior.

## What Changed

- Added provider vault support for company secrets, including provider
config storage, default vault handling, health checks, binding usage,
access events, and remote import preview/commit.
- Added an AWS Secrets Manager provider using SigV4 request signing,
bounded request timeouts, namespace guardrails, cached runtime
credential resolution, and external-reference linking without plaintext
reads.
- Added Secrets UI surfaces for vault management and remote import, plus
CLI/API documentation for setup and operations.
- Stabilized routine webhook secret binding paths and SSH
environment-driver fixture bindings discovered during verification.
- Addressed Greptile and CI findings: no lockfile/package drift,
monotonic migration metadata, disabled-vault default races, soft-deleted
secret hiding/recreate behavior, remove behavior with disabled vaults,
soft-deleted external-reference re-import, non-active rotation guards,
managed-secret soft deletion through PATCH, and per-call AWS SDK
credential client churn.
- Rebased this branch onto `public-gh/master` at `0e1a5828` and
force-pushed with lease to keep this as the single PR for the branch.

## Verification

- `git fetch public-gh master`
- `git rebase public-gh/master`
- `git diff --name-only public-gh/master...HEAD | grep
'^pnpm-lock\.yaml$' || true` confirmed `pnpm-lock.yaml` is not in the PR
diff.
- Confirmed migration ordering: master ends at `0081_optimal_dormammu`;
this PR adds `0082_dry_vision` and
`0083_company_secret_provider_configs`.
- Inspected migrations for repeat safety: new tables/indexes use `IF NOT
EXISTS`; foreign keys are guarded by `DO $$ ... IF NOT EXISTS`; column
additions use `ADD COLUMN IF NOT EXISTS`.
- `pnpm -r typecheck` passed before the Greptile follow-up commits.
- `pnpm test:run` ran the full stable Vitest path before the Greptile
follow-up commits; it completed with 3 timing-related failures under
parallel load: `codex-local-execute.test.ts`,
`cursor-local-execute.test.ts`, and `environment-service.test.ts`.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/codex-local-execute.test.ts
src/__tests__/cursor-local-execute.test.ts
src/__tests__/environment-service.test.ts` passed on targeted rerun
(`24/24`).
- `pnpm build` passed before the Greptile follow-up commits. Vite
reported existing chunk-size/dynamic-import warnings.
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
exec vitest run src/__tests__/secrets-service.test.ts` passed (`26/26`).
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
exec vitest run src/__tests__/aws-secrets-manager-provider.test.ts
src/__tests__/secrets-service.test.ts` passed (`39/39`).
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
typecheck` passed.
- Captured Storybook screenshots from `ui/storybook-static` for visual
review.
- Latest PR checks on `5ca3a5cf`: `policy`, serialized server suites
1/4-4/4, `Canary Dry Run`, `e2e`, `security/snyk`, and `Greptile Review`
pass; aggregate `verify` is still registering the completed child
checks.
- Greptile review loop continued through the latest requested pass; all
Greptile review threads are resolved and the latest `Greptile Review`
check on `5ca3a5cf` passed with 0 comments added.

## Screenshots

Before: the provider-vault and remote-import surfaces did not exist on
`master`; these are after-state screenshots from the Storybook fixtures.

![Secrets
inventory](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/secrets-inventory.png)

![Secret binding
picker](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/secret-binding-picker.png)

![Environment editor with
secrets](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/env-editor-with-secrets.png)

## Risks

- Migration risk: this adds new secret provider tables and extends
existing secret rows. The migrations were checked for monotonic ordering
and idempotent guards, but reviewers should still inspect upgrade
behavior carefully.
- Provider risk: AWS support uses direct SigV4 requests. Automated tests
cover signing, request timeouts, vault-config selection, namespace
guardrails, pending-version archival, sanitized provider errors, and
service-level cleanup paths. A real-vault AWS smoke test remains
deployment validation for an operator with AWS credentials rather than
an unverified merge blocker in this local branch.
- UI risk: the Secrets page and import dialog are large new surfaces;
screenshots are included above for reviewer inspection.
- Verification risk: the full local stable test command hit
parallel-load timing failures, although the exact failed files passed
when rerun directly.
- Operational risk: remote import intentionally avoids plaintext reads;
operators must understand that imported external references resolve at
runtime and may fail if AWS permissions change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5 coding agent with local shell/tool use in the
Paperclip worktree. Exact context-window size was not exposed by the
runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 18:22:17 -05:00
Dotta a77206812e Harden tailnet bind setup 2026-04-11 07:13:41 -05:00
dotta 2a84e53c1b Introduce bind presets for deployment setup
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-11 07:09:07 -05:00
Aditya Sasidhar af97259a9c feat(adapters): add Gemini CLI local adapter support
Signed-off-by: Aditya Sasidhar <telikicherlaadityasasidhar@gmail.com>
2026-03-10 16:46:04 +00:00
Dotta 38cb2bf3c4 fix: add missing disableSignUp to auth config objects in CLI
All auth config literals in the CLI were missing the required
disableSignUp field after it was added to authConfigSchema.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 07:43:47 -05:00
zvictor ffd1631b14 onboard now derives defaults from env vars before writing config 2026-03-05 16:17:26 -03:00
Dotta c145074daf Add configurable automatic database backup scheduling 2026-03-04 18:03:23 -06:00
Dotta f60c1001ec refactor: rename packages to @paperclipai and CLI binary to paperclipai
Rename all workspace packages from @paperclip/* to @paperclipai/* and
the CLI binary from `paperclip` to `paperclipai` in preparation for
npm publishing. Bump CLI version to 0.1.0 and add package metadata
(description, keywords, license, repository, files). Update all
imports, documentation, user-facing messages, and tests accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 08:45:26 -06:00
Forgotten 85c0b9a3dc feat: private hostname guard for authenticated/private mode
Reject requests from unrecognised Host headers when running
authenticated/private. Adds server middleware, CLI `allowed-hostname`
command, config-schema field, and prompt support for configuring
allowed hostnames during onboard/configure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:43:52 -06:00
Forgotten 5b983ca4d3 feat(cli): add deployment mode prompts, auth bootstrap-ceo command, and doctor check
Extend server setup prompts with deployment mode (local_trusted vs
authenticated), exposure (private vs public), bind host, and auth config.
Add auth bootstrap-ceo command that creates a one-time invite URL for the
initial instance admin. Add deployment-auth-check to doctor diagnostics.
Register the new command in the CLI entry point.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 14:40:59 -06:00
Forgotten fdd2ea6157 feat: add storage system with local disk and S3 providers
Introduces a provider-agnostic storage subsystem for file attachments.
Includes local disk and S3 backends, asset/attachment DB schemas, issue
attachment CRUD routes with multer upload, CLI configure/doctor/env
integration, and enriched issue ancestors with project/goal resolution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 10:31:56 -06:00
Forgotten 8f3fc077fa feat(cli): add client commands and home-based local runtime defaults 2026-02-20 07:10:58 -06:00
Forgotten f1b558dcfb CLI: add secrets configuration, doctor check, and path resolver extraction
Add secrets section to onboard, configure, and doctor commands. Doctor
validates local encrypted provider key file and can auto-repair missing
keys. Extract shared path resolution into path-resolver module used by
database and log checks. Show secrets env vars in `paperclip env`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 15:43:59 -06:00
Forgotten 8c2bf0a2e6 Remove contextMode, consolidate wake policies, and default serveUi to true
Drop the unused contextMode field from the agent schema, shared types, validators,
and all UI references. Merge wakeOnOnDemand and wakeOnAutomation into a single
wakeOnDemand toggle. Default serveUi to true and remove the onboarding prompt for it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 16:46:29 -06:00
Forgotten cc24722090 Replace PGlite with embedded-postgres and add startup banner
Switch from PGlite (WebAssembly) to embedded-postgres for zero-config
local development — provides a real PostgreSQL server with full
compatibility. Add startup banner with config summary on server boot.
Improve server bootstrap with auto port detection, database creation,
and migration on startup. Update DATABASE.md, DEVELOPING.md, and
SPEC-implementation.md to reflect the change. Update CLI database
check and prompts. Simplify OnboardingWizard database options.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 11:45:43 -06:00
Forgotten 5306142542 Add CLI package, config file support, and workspace setup
Add cli/ package with initial scaffolding. Add config-schema to shared
package for typed configuration. Add server config-file loader for
paperclip.config.ts support. Register cli in pnpm workspace. Add
.paperclip/ and .pnpm-store/ to gitignore. Minor Companies page fix.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 13:39:47 -06:00