Commit Graph

97 Commits

Author SHA1 Message Date
Dotta 236d11d36f [codex] Add run liveness continuations (#4083)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - Heartbeat runs are the control-plane record of each agent execution
window.
> - Long-running local agents can exhaust context or stop while still
holding useful next-step state.
> - Operators need that stop reason, next action, and continuation path
to be durable and visible.
> - This pull request adds run liveness metadata, continuation
summaries, and UI surfaces for issue run ledgers.
> - The benefit is that interrupted or long-running work can resume with
clearer context instead of losing the agent's last useful handoff.

## What Changed

- Added heartbeat-run liveness fields, continuation attempt tracking,
and an idempotent `0058` migration.
- Added server services and tests for run liveness, continuation
summaries, stop metadata, and activity backfill.
- Wired local and HTTP adapters to surface continuation/liveness context
through shared adapter utilities.
- Added shared constants, validators, and heartbeat types for liveness
continuation state.
- Added issue-detail UI surfaces for continuation handoffs and the run
ledger, with component tests.
- Updated agent runtime docs, heartbeat protocol docs, prompt guidance,
onboarding assets, and skills instructions to explain continuation
behavior.
- Addressed Greptile feedback by scoping document evidence by run,
excluding system continuation-summary documents from liveness evidence,
importing shared liveness types, surfacing hidden ledger run counts,
documenting bounded retry behavior, and moving run-ledger liveness
backfill off the request path.

## Verification

- `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts
server/src/__tests__/run-continuations.test.ts
server/src/__tests__/run-liveness.test.ts
server/src/__tests__/activity-service.test.ts
server/src/__tests__/documents-service.test.ts
server/src/__tests__/issue-continuation-summary.test.ts
server/src/services/heartbeat-stop-metadata.test.ts
ui/src/components/IssueRunLedger.test.tsx
ui/src/components/IssueContinuationHandoff.test.tsx
ui/src/components/IssueDocumentsSection.test.tsx`
- `pnpm --filter @paperclipai/db build`
- `pnpm exec vitest run server/src/__tests__/activity-service.test.ts
ui/src/components/IssueRunLedger.test.tsx`
- `pnpm --filter @paperclipai/ui typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm exec vitest run server/src/__tests__/activity-service.test.ts
server/src/__tests__/run-continuations.test.ts
ui/src/components/IssueRunLedger.test.tsx`
- `pnpm exec vitest run
server/src/__tests__/heartbeat-process-recovery.test.ts -t "treats a
plan document update"`
- `pnpm exec vitest run server/src/__tests__/activity-service.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts -t "activity
service|treats a plan document update"`
- Remote PR checks on head `e53b1a1d`: `verify`, `e2e`, `policy`, and
Snyk all passed.
- Confirmed `public-gh/master` is an ancestor of this branch after
fetching `public-gh master`.
- Confirmed `pnpm-lock.yaml` is not included in the branch diff.
- Confirmed migration `0058_wealthy_starbolt.sql` is ordered after
`0057` and uses `IF NOT EXISTS` guards for repeat application.
- Greptile inline review threads are resolved.

## Risks

- Medium risk: this touches heartbeat execution, liveness recovery,
activity rendering, issue routes, shared contracts, docs, and UI.
- Migration risk is mitigated by additive columns/indexes and idempotent
guards.
- Run-ledger liveness backfill is now asynchronous, so the first ledger
response can briefly show historical missing liveness until the
background backfill completes.
- UI screenshot coverage is not included in this packaging pass;
validation is currently through focused component tests.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5.4, local tool-use coding agent with terminal, git,
GitHub connector, GitHub CLI, and Paperclip API access.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Screenshot note: no before/after screenshots were captured in this PR
packaging pass; the UI changes are covered by focused component tests
listed above.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 06:01:49 -05:00
Dotta a7dc88941b fix(codex-local): avoid fast mode in env probe 2026-04-11 08:33:18 -05:00
Dotta 2d8f97feb0 feat(codex-local): add fast mode support 2026-04-11 08:21:55 -05:00
Dotta c566a9236c fix: harden heartbeat and adapter runtime workflows 2026-04-10 22:26:21 -05:00
dotta 9eaf72ab31 Fix Codex tool-use transcript completion 2026-04-09 06:16:05 -05:00
Brandon Woo 15bd2ef349 fix: recognize missing-rollout Codex resume error as stale session
The Codex CLI can return "no rollout found for thread id ..." when
resuming a heartbeat thread whose rollout has been garbage-collected.
Extend isCodexUnknownSessionError() to match this wording so the
existing single-retry path in execute.ts activates correctly.

Add parse.test.ts covering the new pattern, existing stale-session
wordings, parseCodexJsonl, and a negative case.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-07 10:45:38 +09:00
dotta b9b2bf3b5b Trim resumed comment wake prompts 2026-04-04 18:14:19 -05:00
dotta 91e040a696 Batch inline comment wake payloads
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-04 18:14:19 -05:00
Dotta 19aaa54ae4 Merge branch 'master' into add-gpt-5-4-xhigh-effort 2026-03-31 06:19:26 -05:00
Dotta 5575399af1 Merge pull request #2048 from remdev/fix/codex-rpc-client-spawn-error
fix(codex) rpc client spawn error
2026-03-30 14:24:33 -05:00
dotta b3d61a7561 Clarify manual workspace runtime behavior
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-29 10:55:45 -05:00
dotta cadfcd1bc6 Log resolved adapter command in run metadata
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-29 10:55:26 -05:00
Mikhail Batukhtin dc3aa8f31f test(codex-local): isolate quota spawn test from host CODEX_HOME
After the mocked RPC spawn fails, getQuotaWindows() still calls
readCodexToken(). Use an empty mkdtemp directory for CODEX_HOME for the
duration of the test so we never read ~/.codex/auth.json or call WHAM.
2026-03-29 15:15:37 +03:00
Mikhail Batukhtin c98af52590 test(codex-local): regression for CodexRpcClient spawn ENOENT
Add a Vitest case that mocks `node:child_process.spawn` so the child
emits `error` (ENOENT) after the constructor attaches listeners.
`getQuotaWindows()` must resolve with `ok: false` instead of leaving an
unhandled `error` event on the process.

Register `packages/adapters/codex-local` in the root Vitest workspace.

Document in DEVELOPING.md that a missing `codex` binary should not take
down the API server during quota polling.
2026-03-29 14:43:51 +03:00
Mikhail Batukhtin 01fb97e8da fix(codex-local): handle spawn error event in CodexRpcClient
When the `codex` binary is absent from PATH, Node.js emits an `error`
event on the ChildProcess. Because `CodexRpcClient` only subscribed to
`exit` and `data` events, the `error` event was unhandled — causing
Node to throw it as an uncaught exception and crash the server.

Add an `error` handler in the constructor that rejects all pending RPC
requests and clears the queue. This makes a missing `codex` binary a
recoverable condition: `fetchCodexRpcQuota()` rejects, `getQuotaWindows()`
catches the error and returns `{ ok: false }`, and the server stays up.

The fix mirrors the existing pattern in `runChildProcess`
(packages/adapter-utils/src/server-utils.ts) which already handles
`ENOENT` the same way for the main task execution path.
2026-03-29 14:20:55 +03:00
Devin Foley 80766e589c Clarify docs: skills go to the effective CODEX_HOME, not ~/.codex
The previous documentation parenthetical "(defaulting to ~/.codex/skills/)"
was misleading because Paperclip almost always sets CODEX_HOME to a
per-company managed home.  Update index.ts docs, skills.ts detail string,
and execute.ts inline comment to make the runtime path unambiguous.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-25 20:46:05 -07:00
Devin Foley 4c6b9c190b Fix stale reference to resolveCodexSkillsHome in fallback path
The default fallback in ensureCodexSkillsInjected still referenced the
old function name. Updated to use resolveCodexSkillsDir with shared
home as fallback.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-25 16:09:09 -07:00
Devin Foley f6ac6e47c4 Clarify docs: skills go to $CODEX_HOME/skills/, defaulting to ~/.codex
Addresses Greptile P2 review comment.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-25 16:05:57 -07:00
Devin Foley 623ab1c3ea Fix skill injection to use effective CODEX_HOME, not shared home
The previous commit incorrectly used resolveSharedCodexHomeDir() (~/.codex)
but Codex runs with CODEX_HOME set to a per-company managed home under
~/.paperclip/instances/. Skills injected into ~/.codex/skills/ would not
be discoverable by Codex. Now uses effectiveCodexHome directly.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-25 16:04:53 -07:00
Devin Foley eeec52ad74 Fix Codex skill injection to use ~/.codex/skills/ instead of cwd
The Codex adapter was the only one injecting skills into
<cwd>/.agents/skills/, polluting the project's git repo. All other
adapters (Gemini, Cursor, etc.) use a home-based directory. This
changes the Codex adapter to inject into ~/.codex/skills/ (resolved
via resolveSharedCodexHomeDir) to match the established pattern.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-25 15:55:51 -07:00
Devin Foley fea892c8b3 Merge pull request #1702 from paperclipai/fix/codex-auth-check
fix(codex): check native auth before warning about missing API key
2026-03-24 15:40:20 -07:00
Devin Foley 1696ff0c3f fix(codex): use path.join for auth detail message path
Use path.join instead of string concatenation for the auth.json
fallback path in the detail message, ensuring correct path
separators on Windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 15:27:32 -07:00
Devin Foley 4eecd23ea3 fix(codex): use codexHomeDir() fallback for accurate auth detail path
When adapter config has no CODEX_HOME but process.env.CODEX_HOME is
set, readCodexAuthInfo reads from the process env path. The detail
message now uses codexHomeDir() instead of hardcoded "~/.codex" so
the displayed path always matches where credentials were read from.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 12:46:16 -07:00
Devin Foley 0ce4134ce1 fix(codex): use actual CODEX_HOME in auth detail message
Show the configured CODEX_HOME path instead of hardcoded ~/.codex
when the email fallback message is displayed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 10:54:05 -07:00
Devin Foley 06b85d62b2 test(codex): add coverage for native auth detection in environment probe
Add tests for codex_native_auth_present and codex_openai_api_key_missing
code paths. Also pass adapter-configured CODEX_HOME through to
readCodexAuthInfo so the probe respects per-adapter home directories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 09:51:44 -07:00
dotta 67841a0c6d Remove noisy "Loaded agent instructions file" log from all adapters
Loading an instructions file is normal, expected behavior — not worth
logging to stdout/stderr on every run. Warning logs for failed reads
are preserved.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-24 11:48:43 -05:00
Devin Foley a9dcea023b fix(codex): check native auth before warning about missing API key
The environment test warned about OPENAI_API_KEY being unset even
when Codex was authenticated via `codex auth`. Now checks
~/.codex/auth.json before emitting the warning.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 09:47:48 -07:00
dotta 19154d0fec Clarify Codex instruction sources
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-23 16:57:33 -05:00
dotta d53714a145 fix: manage codex home per company by default 2026-03-20 14:44:27 -05:00
dotta 07757a59e9 Ensure agent home directories exist before use
mkdir -p the CODEX_HOME directory in codex-local adapter and the
agentHome directory in the heartbeat service before passing them to
adapters. This prevents CLI tools from erroring when their home
directory hasn't been created yet. Covers all local adapters that
set AGENT_HOME.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 14:25:18 -05:00
dotta 5140d7b0c4 Merge remote-tracking branch 'public-gh/master' into paperclip-company-import-export
* public-gh/master:
  fix: address greptile follow-up feedback
  docs: clarify quickstart npx usage
  Add guarded dev restart handling
  Fix PAP-576 settings toggles and transcript default
  Add username log censor setting
  fix: use standard toggle component for permission controls

# Conflicts:
#	server/src/routes/agents.ts
#	ui/src/pages/AgentDetail.tsx
2026-03-20 13:28:05 -05:00
dotta 39878fcdfe Add username log censor setting
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 08:50:00 -05:00
dotta d07d86f778 Merge public-gh/master into paperclip-company-import-export 2026-03-20 06:25:24 -05:00
dotta c844ca1a40 Improve orphaned local heartbeat recovery
Persist child-process metadata for local adapter runs, keep detached runs alive when their pid still exists, queue a single automatic retry when the pid is confirmed dead, and clear detached warnings when the original run reports activity again.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 06:05:15 -05:00
dotta 58a3cbd654 Route non-fatal adapter notices to stdout
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 06:05:14 -05:00
dotta b4e06c63e2 Refine codex runtime skills and portability assets 2026-03-19 07:15:36 -05:00
dotta 3b03ac1734 Scope Codex local skills home by company 2026-03-18 14:38:39 -05:00
dotta cfc53bf96b Add unmanaged skill provenance to agent skills
Expose adapter-discovered user-installed skills with provenance metadata, share persistent skill snapshot classification across local adapters, and render unmanaged skills as a read-only section in the agent skills UI.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-18 14:21:50 -05:00
dotta 3689992965 Prune stale Codex skill symlinks
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-18 14:07:24 -05:00
dotta 0036f0e9a1 fix: add npm provenance package metadata 2026-03-17 16:01:48 -05:00
dotta 6ba9aea8ba Add publish metadata for npm provenance 2026-03-17 15:49:42 -05:00
Dotta b5aeae7e22 Merge public-gh/master into paperclip-company-import-export 2026-03-17 10:45:14 -05:00
Dotta 5890b318c4 Namespace company skill identities
Persist canonical namespaced skill keys, split adapter runtime names from skill keys, and update portability/import flows to carry the canonical identity end-to-end.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-16 18:27:20 -05:00
Dotta bb788d8360 Treat Codex bootstrap logs as stdout
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-16 18:26:36 -05:00
Dotta cca086b863 Merge public-gh/master into paperclip-company-import-export 2026-03-16 17:02:39 -05:00
Dotta 76e6cc08a6 feat(costs): add billing, quota, and budget control plane 2026-03-16 15:11:01 -05:00
Sai Shankar 656b4659fc refactor(quota): move provider quota logic into adapter layer, add unit tests
- Extract all Anthropic credential/API logic into claude-local/src/server/quota.ts
- Extract all OpenAI/WHAM credential/API logic into codex-local/src/server/quota.ts
- Add optional getQuotaWindows() to ServerAdapterModule in adapter-utils
- Rewrite quota-windows.ts as a 29-line thin aggregator with zero provider knowledge
- Wire getQuotaWindows into adapter registry for claude-local and codex-local
- Add 47 unit tests covering toPercent, secondsToWindowLabel, WHAM normalization,
  readClaudeToken, readCodexToken, fetchClaudeQuota, fetchCodexQuota, fetchWithTimeout
- Add 8 unit tests covering parseDateRange validation and byProvider pro-rata math

Adding a third provider now requires only touching that provider's adapter.
2026-03-16 15:08:54 -05:00
Dotta 7675fd0856 Fix runtime skill injection across adapters 2026-03-15 07:05:01 -05:00
Dotta 2c35be0212 Merge public-gh/master into paperclip-company-import-export 2026-03-14 21:45:54 -05:00
Dotta 325fcf8505 Merge pull request #864 from paperclipai/fix/agent-home-env
fix: set AGENT_HOME env var for agent processes
2026-03-14 12:44:28 -05:00