Guard cheap recovery model usage (#6371)
## Thinking Path > - Paperclip is the control plane that coordinates AI-agent work through issues, heartbeats, comments, approvals, and auditable recovery paths. > - The affected subsystem is heartbeat/recovery orchestration, especially the optional cheap model profile used for operational recovery overhead. > - Cheap recovery should repair status and liveness, but it must not become the worker lane that writes deliverables, continues source work, or propagates cheap execution hints into downstream retries. > - The gap was that cheap-profile hints could follow recovery wake contexts and assignment overrides farther than intended, making real work eligible to run on the cheap model. > - This pull request separates status-only cheap recovery from normal source-work continuations, adds route guards for deliverable mutations during cheap status-only runs, and documents the invariant. > - The benefit is safer retry/recovery behavior: cheap runs can clean up control-plane state, while any remaining source work resumes through a normal/original model path. ## What Changed - Added recovery model-profile work classes so status-only recovery carries explicit guard context and normal-model continuations scrub cheap hints. - Updated heartbeat, productivity review, liveness continuation, and recovery service wakeups to request cheap only for bounded status-only recovery work. - Blocked cheap status-only recovery runs from writing issue documents, plans, attachments, work products, or assigning downstream work back to `modelProfile: "cheap"`. - Added/updated server tests for cheap profile propagation, artifact/document guards, route authorization, retry scheduling, and successful-run handoff behavior. - Documented the recovery model-profile lane in `doc/SPEC-implementation.md` and `doc/execution-semantics.md`. - After rebasing onto current `public-gh/master`, stabilized the new `InstanceSidebar` plugin-filter tests so the PR check lane stays green. ## Verification - Local: `pnpm exec vitest run --config vitest.config.ts src/services/recovery/model-profile-hint.test.ts src/__tests__/issue-agent-mutation-ownership-routes.test.ts src/__tests__/issue-document-restore-routes.test.ts` from `server/` - 3 files, 37 tests passed after final edits. - Local: `pnpm exec vitest run --config vitest.config.ts src/__tests__/heartbeat-process-recovery.test.ts` from `server/` - 44 tests passed after rerunning the cleanup-sensitive file alone. - Local: `pnpm --filter @paperclipai/ui exec vitest run src/components/InstanceSidebar.test.tsx` - 4 tests passed. - Local: `pnpm --filter @paperclipai/server typecheck` - passed. - Local: `pnpm --filter @paperclipai/ui typecheck` - passed. - PR checks on latest head `6f8c3b1380f5bd872c6f49f6f7188ecf3bb6d263` - all green, including `verify`, build, typecheck, server/general/serialized tests, e2e, Snyk, and policy. - Greptile: pass 3 returned Confidence Score 5/5 with zero unresolved Greptile review threads. ## Risks - Medium risk: recovery behavior is intentionally stricter, so any path that incorrectly relies on cheap recovery to keep doing source work will now need to hand back to a normal-model run. - Low migration risk: no schema changes. - No product UI changes; the UI file touched is a test-only stabilization after rebasing onto current `master`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent, GPT-5 model family (`gpt-5`), tool use and local code execution enabled; context window not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A: no product UI changes) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge
This commit is contained in:
@@ -679,7 +679,13 @@ Behavior:
|
||||
- `thin`: send IDs and pointers only; agent fetches context via API
|
||||
- `fat`: include current assignments, goal summary, budget snapshot, and recent comments
|
||||
|
||||
## 11.5 Scheduler Rules
|
||||
## 11.5 Recovery Model Profiles
|
||||
|
||||
The optional `modelProfiles.cheap` lane is not a retry worker lane. Paperclip may request the cheap profile only for status-only recovery coordination, and those wakes must include guard context that prevents deliverable work and document/plan updates (`allowDeliverableWork: false`, `allowDocumentUpdates: false`, `resumeRequiresNormalModel: true`).
|
||||
|
||||
Failed source-work retries, process-loss retries, transient/scheduled retries, max-turn continuations, source-assignee continuations, and downstream source-work child/requeue/resume contexts must use the normal/original model lane. If cheap recovery repairs liveness while actual work remains, the next live continuation path must be a separate normal-model worker run with cheap hints scrubbed.
|
||||
|
||||
## 11.6 Scheduler Rules
|
||||
|
||||
Per-agent schedule fields in `adapter_config`:
|
||||
|
||||
|
||||
@@ -330,6 +330,12 @@ Recovery rule:
|
||||
|
||||
This is an active-work continuity recovery.
|
||||
|
||||
### 8.3 Recovery model-profile lane
|
||||
|
||||
Cheap model profiles are only for status-only operational recovery overhead. Paperclip may request `modelProfile: "cheap"` for bounded recovery-owner work that updates task liveness, clears bad status, records a disposition, or asks for human/manager intervention. Those wakes must carry guard context such as `allowDeliverableWork: false`, `allowDocumentUpdates: false`, and `resumeRequiresNormalModel: true`.
|
||||
|
||||
Automatic retries that can continue source work must use the original/normal model lane. This includes failed source-work retries, process-loss retries, transient/scheduled retries, max-turn continuations, source-assignee continuations, assigned-todo dispatch recovery, and any run that can update repo files, issue documents, plans, work products, or attachments. When a cheap status-only recovery determines that actual work remains, it must hand back to a normal-model worker run before source work or persistent deliverable updates resume. Cheap recovery hints must be scrubbed from copied retry, resume, child, and downstream source-work contexts.
|
||||
|
||||
## 9. Startup and Periodic Reconciliation
|
||||
|
||||
Startup recovery and periodic recovery are different from normal wakeup delivery.
|
||||
|
||||
@@ -405,6 +405,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
includeIssue?: boolean;
|
||||
runErrorCode?: string | null;
|
||||
runError?: string | null;
|
||||
contextSnapshot?: Record<string, unknown>;
|
||||
}) {
|
||||
const companyId = randomUUID();
|
||||
const agentId = randomUUID();
|
||||
@@ -454,7 +455,9 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
triggerDetail: "system",
|
||||
status: input?.runStatus ?? "running",
|
||||
wakeupRequestId,
|
||||
contextSnapshot: input?.includeIssue === false ? {} : { issueId },
|
||||
contextSnapshot: input?.includeIssue === false
|
||||
? input?.contextSnapshot ?? {}
|
||||
: { ...(input?.contextSnapshot ?? {}), issueId },
|
||||
processPid: input?.processPid ?? null,
|
||||
processGroupId: input?.processGroupId ?? null,
|
||||
processLossRetryCount: input?.processLossRetryCount ?? 0,
|
||||
@@ -765,7 +768,12 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
companyId: input.companyId,
|
||||
reason: "source_scoped_recovery_action",
|
||||
source: "assignment",
|
||||
payload: expect.objectContaining({ modelProfile: "cheap" }),
|
||||
payload: expect.objectContaining({
|
||||
modelProfile: "cheap",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
}),
|
||||
});
|
||||
|
||||
const recoveryRun = recoveryWakeup?.runId
|
||||
@@ -783,6 +791,9 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
sourceIssueId: input.issueId,
|
||||
strandedRunId: input.runId,
|
||||
modelProfile: "cheap",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
});
|
||||
await waitForHeartbeatIdle(db);
|
||||
const sourceIssue = await db
|
||||
@@ -920,6 +931,12 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
it("queues exactly one retry when the recorded local pid is dead", async () => {
|
||||
const { agentId, runId, issueId } = await seedRunFixture({
|
||||
processPid: 999_999_999,
|
||||
contextSnapshot: {
|
||||
modelProfile: "cheap",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
},
|
||||
});
|
||||
const heartbeat = heartbeatService(db);
|
||||
|
||||
@@ -947,7 +964,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
expect(retryRun?.status).toBe("queued");
|
||||
expect(retryRun?.retryOfRunId).toBe(runId);
|
||||
expect(retryRun?.processLossRetryCount).toBe(1);
|
||||
expect(retryRun?.contextSnapshot).toMatchObject({ modelProfile: "cheap" });
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
|
||||
const issue = await db
|
||||
.select()
|
||||
@@ -1253,8 +1270,8 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
expect(retryRun?.scheduledRetryReason).toBe("transient_failure");
|
||||
expect(retryRun?.contextSnapshot).toMatchObject({
|
||||
codexTransientFallbackMode: "same_session",
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
|
||||
const issue = await db
|
||||
.select()
|
||||
@@ -1789,9 +1806,9 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
payload: expect.objectContaining({
|
||||
issueId,
|
||||
mutation: "assigned_todo_liveness_dispatch",
|
||||
modelProfile: "cheap",
|
||||
}),
|
||||
});
|
||||
expect(wakeups[0]?.payload as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
|
||||
const runs = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, agentId));
|
||||
expect(runs).toHaveLength(1);
|
||||
@@ -1801,8 +1818,8 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
taskId: issueId,
|
||||
wakeReason: "issue_assigned",
|
||||
source: "issue.assigned_todo_liveness_dispatch",
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(runs[0]?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
expect((runs[0]?.contextSnapshot as Record<string, unknown>)?.retryReason).toBeUndefined();
|
||||
|
||||
const issue = await db.select().from(issues).where(eq(issues.id, issueId)).then((rows) => rows[0] ?? null);
|
||||
@@ -1909,9 +1926,9 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
payload: expect.objectContaining({
|
||||
issueId: unblocked.issueId,
|
||||
mutation: "assigned_todo_liveness_dispatch",
|
||||
modelProfile: "cheap",
|
||||
}),
|
||||
});
|
||||
expect(unblockedWakeups[0]?.payload as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
const unblockedRuns = await db
|
||||
.select()
|
||||
.from(heartbeatRuns)
|
||||
@@ -1963,7 +1980,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
const retryRun = runs.find((row) => row.id !== runId);
|
||||
expect(retryRun?.id).toBeTruthy();
|
||||
expect((retryRun?.contextSnapshot as Record<string, unknown>)?.retryReason).toBe("assignment_recovery");
|
||||
expect(retryRun?.contextSnapshot).toMatchObject({ modelProfile: "cheap" });
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
if (retryRun) {
|
||||
await waitForRunToSettle(heartbeat, retryRun.id);
|
||||
}
|
||||
@@ -2002,8 +2019,8 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
retryReason: "issue_continuation_needed",
|
||||
retryOfRunId: runId,
|
||||
source: "issue.continuation_recovery",
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
|
||||
const recoveries = await db
|
||||
.select()
|
||||
@@ -2054,7 +2071,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
|
||||
const retryRun = runs.find((row) => row.id !== runId);
|
||||
expect((retryRun?.contextSnapshot as Record<string, unknown>)?.retryReason).toBe("assignment_recovery");
|
||||
expect(retryRun?.contextSnapshot).toMatchObject({ modelProfile: "cheap" });
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
if (retryRun) {
|
||||
await waitForRunToSettle(heartbeat, retryRun.id);
|
||||
}
|
||||
@@ -2296,7 +2313,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
const retryRun = runs.find((row) => row.id !== runId);
|
||||
expect(retryRun?.id).toBeTruthy();
|
||||
expect((retryRun?.contextSnapshot as Record<string, unknown>)?.retryReason).toBe("issue_continuation_needed");
|
||||
expect(retryRun?.contextSnapshot).toMatchObject({ modelProfile: "cheap" });
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
if (retryRun) {
|
||||
await waitForRunToSettle(heartbeat, retryRun.id);
|
||||
}
|
||||
@@ -2786,8 +2803,8 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
retryReason: "issue_continuation_needed",
|
||||
retryOfRunId: runId,
|
||||
source: "issue.productive_terminal_continuation_recovery",
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
|
||||
const wakeups = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, agentId));
|
||||
expect(wakeups).toHaveLength(2);
|
||||
@@ -2854,8 +2871,8 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
retryReason: "issue_continuation_needed",
|
||||
retryOfRunId: runId,
|
||||
source: "issue.productive_terminal_continuation_recovery",
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
});
|
||||
|
||||
it("does not treat a productive terminal run as healthy when in-progress work has no live path", async () => {
|
||||
@@ -2910,8 +2927,8 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
retryReason: "issue_continuation_needed",
|
||||
retryOfRunId: runId,
|
||||
source: "issue.productive_terminal_continuation_recovery",
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
});
|
||||
|
||||
it("does not reconcile user-assigned work through the agent stranded-work recovery path", async () => {
|
||||
|
||||
@@ -286,8 +286,8 @@ describeEmbeddedPostgres("heartbeat bounded retry scheduling", () => {
|
||||
retryOfRunId: sourceRunId,
|
||||
scheduledRetryAttempt: 1,
|
||||
scheduledRetryReason: "transient_failure",
|
||||
contextSnapshot: expect.objectContaining({ modelProfile: "cheap" }),
|
||||
});
|
||||
expect(retryRun?.contextSnapshot as Record<string, unknown>).not.toHaveProperty("modelProfile");
|
||||
expect(retryRun?.scheduledRetryAt?.toISOString()).toBe(expectedDueAt.toISOString());
|
||||
|
||||
const earlyPromotion = await heartbeat.promoteDueScheduledRetries(new Date("2026-04-20T12:01:59.000Z"));
|
||||
|
||||
@@ -13,6 +13,8 @@ const recoveryActionId = "77777777-7777-4777-8777-777777777777";
|
||||
const mockIssueService = vi.hoisted(() => ({
|
||||
addComment: vi.fn(),
|
||||
assertCheckoutOwner: vi.fn(),
|
||||
create: vi.fn(),
|
||||
createChild: vi.fn(),
|
||||
getAttachmentById: vi.fn(),
|
||||
getByIdentifier: vi.fn(),
|
||||
getById: vi.fn(),
|
||||
@@ -46,7 +48,9 @@ const mockDocumentService = vi.hoisted(() => ({
|
||||
}));
|
||||
|
||||
const mockWorkProductService = vi.hoisted(() => ({
|
||||
createForIssue: vi.fn(),
|
||||
getById: vi.fn(),
|
||||
remove: vi.fn(),
|
||||
update: vi.fn(),
|
||||
}));
|
||||
|
||||
@@ -187,21 +191,37 @@ function makeAgent(id: string, overrides: Record<string, unknown> = {}) {
|
||||
};
|
||||
}
|
||||
|
||||
async function createApp(actor: Record<string, unknown>) {
|
||||
function createRunContextDb(contextSnapshot: Record<string, unknown> = {}) {
|
||||
return {
|
||||
transaction: async (callback: (tx: Record<string, never>) => Promise<unknown>) => callback({}),
|
||||
select: vi.fn(() => ({
|
||||
from: vi.fn(() => ({
|
||||
where: vi.fn(() => ({
|
||||
then: async (resolve: (rows: unknown[]) => unknown) =>
|
||||
resolve([{
|
||||
id: ownerRunId,
|
||||
companyId,
|
||||
agentId: ownerAgentId,
|
||||
contextSnapshot,
|
||||
}]),
|
||||
})),
|
||||
})),
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
async function createApp(actor: Record<string, unknown>, db: unknown = createRunContextDb()) {
|
||||
const [{ errorHandler }, { issueRoutes }] = await Promise.all([
|
||||
vi.importActual<typeof import("../middleware/index.js")>("../middleware/index.js"),
|
||||
vi.importActual<typeof import("../routes/issues.js")>("../routes/issues.js"),
|
||||
]);
|
||||
const fakeDb = {
|
||||
transaction: async (callback: (tx: Record<string, never>) => Promise<unknown>) => callback({}),
|
||||
};
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
app.use((req, _res, next) => {
|
||||
(req as any).actor = actor;
|
||||
next();
|
||||
});
|
||||
app.use("/api", issueRoutes(fakeDb as any, mockStorageService as any));
|
||||
app.use("/api", issueRoutes(db as any, mockStorageService as any));
|
||||
app.use(errorHandler);
|
||||
return app;
|
||||
}
|
||||
@@ -262,6 +282,8 @@ describe("agent issue mutation checkout ownership", () => {
|
||||
mockCompanyService.getById.mockReset();
|
||||
mockIssueService.addComment.mockReset();
|
||||
mockIssueService.assertCheckoutOwner.mockReset();
|
||||
mockIssueService.create.mockReset();
|
||||
mockIssueService.createChild.mockReset();
|
||||
mockIssueService.getAttachmentById.mockReset();
|
||||
mockIssueService.getByIdentifier.mockReset();
|
||||
mockIssueService.getById.mockReset();
|
||||
@@ -315,7 +337,9 @@ describe("agent issue mutation checkout ownership", () => {
|
||||
mockIssueService.update.mockReset();
|
||||
mockIssueService.findMentionedAgents.mockReset();
|
||||
mockDocumentService.upsertIssueDocument.mockReset();
|
||||
mockWorkProductService.createForIssue.mockReset();
|
||||
mockWorkProductService.getById.mockReset();
|
||||
mockWorkProductService.remove.mockReset();
|
||||
mockWorkProductService.update.mockReset();
|
||||
mockStorageService.putFile.mockReset();
|
||||
mockStorageService.getObject.mockReset();
|
||||
@@ -337,6 +361,28 @@ describe("agent issue mutation checkout ownership", () => {
|
||||
mockIssueService.getById.mockResolvedValue(makeIssue());
|
||||
mockIssueService.getByIdentifier.mockResolvedValue(null);
|
||||
mockIssueService.assertCheckoutOwner.mockResolvedValue({ adoptedFromRunId: null });
|
||||
mockIssueService.create.mockImplementation(async (_companyId: string, input: Record<string, unknown>) => ({
|
||||
...makeIssue({
|
||||
id: "88888888-8888-4888-8888-888888888888",
|
||||
status: "todo",
|
||||
assigneeAgentId: null,
|
||||
}),
|
||||
...input,
|
||||
companyId,
|
||||
}));
|
||||
mockIssueService.createChild.mockImplementation(async (_parentId: string, input: Record<string, unknown>) => ({
|
||||
issue: {
|
||||
...makeIssue({
|
||||
id: "99999999-9999-4999-8999-999999999999",
|
||||
status: "todo",
|
||||
parentId: issueId,
|
||||
assigneeAgentId: null,
|
||||
}),
|
||||
...input,
|
||||
companyId,
|
||||
},
|
||||
parentBlockerAdded: false,
|
||||
}));
|
||||
mockIssueService.getRelationSummaries.mockResolvedValue({ blockedBy: [], blocks: [] });
|
||||
mockIssueService.listWakeableBlockedDependents.mockResolvedValue([]);
|
||||
mockIssueService.getWakeableParentAfterChildCompletion.mockResolvedValue(null);
|
||||
@@ -378,6 +424,14 @@ describe("agent issue mutation checkout ownership", () => {
|
||||
latestRevisionNumber: 2,
|
||||
},
|
||||
});
|
||||
mockWorkProductService.createForIssue.mockResolvedValue({
|
||||
id: "product-2",
|
||||
issueId,
|
||||
companyId,
|
||||
type: "artifact",
|
||||
provider: "test",
|
||||
title: "Artifact",
|
||||
});
|
||||
mockWorkProductService.getById.mockResolvedValue({
|
||||
id: "product-1",
|
||||
issueId,
|
||||
@@ -391,6 +445,12 @@ describe("agent issue mutation checkout ownership", () => {
|
||||
type: "artifact",
|
||||
title: "Updated",
|
||||
});
|
||||
mockWorkProductService.remove.mockResolvedValue({
|
||||
id: "product-1",
|
||||
issueId,
|
||||
companyId,
|
||||
type: "artifact",
|
||||
});
|
||||
mockStorageService.putFile.mockResolvedValue({
|
||||
provider: "local_disk",
|
||||
objectKey: "issues/upload.txt",
|
||||
@@ -460,6 +520,112 @@ describe("agent issue mutation checkout ownership", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"work product create",
|
||||
(app: express.Express) =>
|
||||
request(app).post(`/api/issues/${issueId}/work-products`).send({
|
||||
type: "artifact",
|
||||
provider: "test",
|
||||
title: "Artifact",
|
||||
}),
|
||||
],
|
||||
["work product update", (app: express.Express) => request(app).patch("/api/work-products/product-1").send({ title: "Blocked" })],
|
||||
["work product delete", (app: express.Express) => request(app).delete("/api/work-products/product-1")],
|
||||
[
|
||||
"attachment upload",
|
||||
(app: express.Express) =>
|
||||
request(app)
|
||||
.post(`/api/companies/${companyId}/issues/${issueId}/attachments`)
|
||||
.attach("file", Buffer.from("report"), { filename: "report.txt", contentType: "text/plain" }),
|
||||
],
|
||||
["attachment delete", (app: express.Express) => request(app).delete("/api/attachments/attachment-1")],
|
||||
])("blocks cheap status-only recovery runs from %s", async (_name, sendRequest) => {
|
||||
const app = await createApp(
|
||||
ownerActor(),
|
||||
createRunContextDb({
|
||||
modelProfile: "cheap",
|
||||
recoveryIntent: "status_only",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await sendRequest(app);
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(403);
|
||||
expect(res.body.error).toContain("Cheap status-only recovery runs cannot update issue documents");
|
||||
expect(mockIssueService.assertCheckoutOwner).toHaveBeenCalledWith(issueId, ownerAgentId, ownerRunId);
|
||||
expect(mockWorkProductService.createForIssue).not.toHaveBeenCalled();
|
||||
expect(mockWorkProductService.update).not.toHaveBeenCalled();
|
||||
expect(mockWorkProductService.remove).not.toHaveBeenCalled();
|
||||
expect(mockStorageService.putFile).not.toHaveBeenCalled();
|
||||
expect(mockStorageService.deleteObject).not.toHaveBeenCalled();
|
||||
expect(mockIssueService.removeAttachment).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"issue create",
|
||||
(app: express.Express) =>
|
||||
request(app).post(`/api/companies/${companyId}/issues`).send({
|
||||
title: "Downstream source work",
|
||||
assigneeAdapterOverrides: { modelProfile: "cheap" },
|
||||
}),
|
||||
],
|
||||
[
|
||||
"child issue create",
|
||||
(app: express.Express) =>
|
||||
request(app).post(`/api/issues/${issueId}/children`).send({
|
||||
title: "Downstream child source work",
|
||||
assigneeAdapterOverrides: { modelProfile: "cheap" },
|
||||
}),
|
||||
],
|
||||
[
|
||||
"issue update",
|
||||
(app: express.Express) =>
|
||||
request(app).patch(`/api/issues/${issueId}`).send({
|
||||
assigneeAdapterOverrides: { modelProfile: "cheap" },
|
||||
}),
|
||||
],
|
||||
])("blocks cheap status-only recovery runs from propagating cheap profile through %s", async (_name, sendRequest) => {
|
||||
const app = await createApp(
|
||||
ownerActor(),
|
||||
createRunContextDb({
|
||||
modelProfile: "cheap",
|
||||
recoveryIntent: "status_only",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await sendRequest(app);
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(403);
|
||||
expect(res.body.error).toContain("cannot assign downstream issue work to the cheap model profile");
|
||||
expect(mockIssueService.create).not.toHaveBeenCalled();
|
||||
expect(mockIssueService.createChild).not.toHaveBeenCalled();
|
||||
expect(mockIssueService.update).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("allows board users to set explicit cheap issue assignee profile overrides", async () => {
|
||||
const app = await createApp(boardActor());
|
||||
|
||||
await request(app)
|
||||
.patch(`/api/issues/${issueId}`)
|
||||
.send({ assigneeAdapterOverrides: { modelProfile: "cheap" } })
|
||||
.expect(200);
|
||||
|
||||
expect(mockIssueService.update).toHaveBeenCalledWith(
|
||||
issueId,
|
||||
expect.objectContaining({
|
||||
assigneeAdapterOverrides: { modelProfile: "cheap" },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("preserves committed issue updates, comments, documents, and work product writes when recovery revalidation fails", async () => {
|
||||
const app = await createApp(ownerActor());
|
||||
|
||||
|
||||
@@ -146,7 +146,34 @@ function registerModuleMocks() {
|
||||
}));
|
||||
}
|
||||
|
||||
async function createApp() {
|
||||
function createRunContextDb(contextSnapshot: Record<string, unknown>) {
|
||||
return {
|
||||
select: vi.fn(() => ({
|
||||
from: vi.fn(() => ({
|
||||
where: vi.fn(() => ({
|
||||
then: async (resolve: (rows: unknown[]) => unknown) =>
|
||||
resolve([{
|
||||
id: "run-1",
|
||||
companyId,
|
||||
agentId: "agent-1",
|
||||
contextSnapshot,
|
||||
}]),
|
||||
})),
|
||||
})),
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
async function createApp(
|
||||
actor: Express.Request["actor"] = {
|
||||
type: "board",
|
||||
userId: "board-user",
|
||||
companyIds: [companyId],
|
||||
source: "local_implicit",
|
||||
isInstanceAdmin: false,
|
||||
},
|
||||
db: unknown = {},
|
||||
) {
|
||||
const [{ issueRoutes }, { errorHandler }] = await Promise.all([
|
||||
vi.importActual<typeof import("../routes/issues.js")>("../routes/issues.js"),
|
||||
vi.importActual<typeof import("../middleware/index.js")>("../middleware/index.js"),
|
||||
@@ -154,16 +181,10 @@ async function createApp() {
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
app.use((req, _res, next) => {
|
||||
(req as any).actor = {
|
||||
type: "board",
|
||||
userId: "board-user",
|
||||
companyIds: [companyId],
|
||||
source: "local_implicit",
|
||||
isInstanceAdmin: false,
|
||||
};
|
||||
(req as any).actor = actor;
|
||||
next();
|
||||
});
|
||||
app.use("/api", issueRoutes({} as any, {} as any));
|
||||
app.use("/api", issueRoutes(db as any, {} as any));
|
||||
app.use(errorHandler);
|
||||
return app;
|
||||
}
|
||||
@@ -315,6 +336,40 @@ describe("issue document revision routes", () => {
|
||||
}));
|
||||
});
|
||||
|
||||
it("blocks cheap status-only recovery runs from restoring issue documents", async () => {
|
||||
mockIssueService.getById.mockResolvedValueOnce({
|
||||
id: issueId,
|
||||
companyId,
|
||||
identifier: "PAP-881",
|
||||
title: "Document revisions",
|
||||
status: "todo",
|
||||
assigneeAgentId: "agent-1",
|
||||
});
|
||||
|
||||
const res = await request(await createApp(
|
||||
{
|
||||
type: "agent",
|
||||
agentId: "agent-1",
|
||||
companyId,
|
||||
runId: "run-1",
|
||||
source: "agent_jwt",
|
||||
},
|
||||
createRunContextDb({
|
||||
modelProfile: "cheap",
|
||||
recoveryIntent: "status_only",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
}),
|
||||
))
|
||||
.post(`/api/issues/${issueId}/documents/plan/revisions/revision-1/restore`)
|
||||
.send({});
|
||||
|
||||
expect(res.status).toBe(403);
|
||||
expect(res.body.error).toContain("Cheap status-only recovery runs cannot update issue documents");
|
||||
expect(mockDocumentsService.restoreIssueDocumentRevision).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("rejects invalid document keys before attempting restore", async () => {
|
||||
const res = await request(await createApp())
|
||||
.post(`/api/issues/${issueId}/documents/INVALID KEY/revisions/revision-1/restore`)
|
||||
|
||||
@@ -76,12 +76,11 @@ describe("run liveness continuations", () => {
|
||||
continuationAttempt: 1,
|
||||
maxContinuationAttempts: DEFAULT_MAX_LIVENESS_CONTINUATION_ATTEMPTS,
|
||||
instruction: "Take the first concrete action now.",
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(decision.payload).not.toHaveProperty("modelProfile");
|
||||
expect(decision.contextSnapshot).toMatchObject({
|
||||
issueId,
|
||||
wakeReason: RUN_LIVENESS_CONTINUATION_REASON,
|
||||
modelProfile: "cheap",
|
||||
livenessContinuationAttempt: 1,
|
||||
livenessContinuationMaxAttempts: DEFAULT_MAX_LIVENESS_CONTINUATION_ATTEMPTS,
|
||||
livenessContinuationSourceRunId: runId,
|
||||
@@ -89,6 +88,7 @@ describe("run liveness continuations", () => {
|
||||
livenessContinuationReason: "Planned without acting",
|
||||
livenessContinuationInstruction: "Take the first concrete action now.",
|
||||
});
|
||||
expect(decision.contextSnapshot).not.toHaveProperty("modelProfile");
|
||||
});
|
||||
|
||||
it("enqueues the second empty_response continuation", () => {
|
||||
|
||||
@@ -7,6 +7,7 @@ import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
activityLog,
|
||||
executionWorkspaces,
|
||||
heartbeatRuns,
|
||||
issueExecutionDecisions,
|
||||
issueRelations,
|
||||
issues as issueRows,
|
||||
@@ -1331,6 +1332,87 @@ export function issueRoutes(
|
||||
return true;
|
||||
}
|
||||
|
||||
function isStatusOnlyCheapRecoveryContext(contextSnapshot: unknown) {
|
||||
if (!contextSnapshot || typeof contextSnapshot !== "object" || Array.isArray(contextSnapshot)) return false;
|
||||
const context = contextSnapshot as Record<string, unknown>;
|
||||
return context.modelProfile === "cheap" &&
|
||||
context.recoveryIntent === "status_only" &&
|
||||
context.allowDeliverableWork === false &&
|
||||
context.allowDocumentUpdates === false &&
|
||||
context.resumeRequiresNormalModel === true;
|
||||
}
|
||||
|
||||
function requestsCheapIssueAssigneeModelProfile(input: { assigneeAdapterOverrides?: unknown }) {
|
||||
const overrides = input.assigneeAdapterOverrides;
|
||||
return !!overrides &&
|
||||
typeof overrides === "object" &&
|
||||
!Array.isArray(overrides) &&
|
||||
(overrides as Record<string, unknown>).modelProfile === "cheap";
|
||||
}
|
||||
|
||||
async function loadActorRunContext(req: Request, companyId: string) {
|
||||
if (req.actor.type !== "agent") return null;
|
||||
const runId = req.actor.runId?.trim();
|
||||
if (!runId) return null;
|
||||
const run = await db
|
||||
.select({
|
||||
id: heartbeatRuns.id,
|
||||
companyId: heartbeatRuns.companyId,
|
||||
agentId: heartbeatRuns.agentId,
|
||||
contextSnapshot: heartbeatRuns.contextSnapshot,
|
||||
})
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, runId))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!run || run.companyId !== companyId || run.agentId !== req.actor.agentId) return null;
|
||||
return run;
|
||||
}
|
||||
|
||||
async function assertCheapRecoveryIssueAssigneeProfileAllowed(
|
||||
req: Request,
|
||||
res: Response,
|
||||
issue: { id?: string; companyId: string },
|
||||
input: { assigneeAdapterOverrides?: unknown },
|
||||
) {
|
||||
if (!requestsCheapIssueAssigneeModelProfile(input)) return true;
|
||||
const run = await loadActorRunContext(req, issue.companyId);
|
||||
if (!run || !isStatusOnlyCheapRecoveryContext(run.contextSnapshot)) return true;
|
||||
|
||||
res.status(403).json({
|
||||
error: "Cheap status-only recovery runs cannot assign downstream issue work to the cheap model profile",
|
||||
details: {
|
||||
issueId: issue.id ?? null,
|
||||
runId: run.id,
|
||||
modelProfile: "cheap",
|
||||
recoveryIntent: "status_only",
|
||||
resumeRequiresNormalModel: true,
|
||||
},
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
async function assertDeliverableMutationAllowedByRunContext(
|
||||
req: Request,
|
||||
res: Response,
|
||||
issue: { id: string; companyId: string },
|
||||
) {
|
||||
const run = await loadActorRunContext(req, issue.companyId);
|
||||
if (!run) return true;
|
||||
if (!isStatusOnlyCheapRecoveryContext(run.contextSnapshot)) return true;
|
||||
|
||||
res.status(403).json({
|
||||
error: "Cheap status-only recovery runs cannot update issue documents, plans, or deliverable artifacts",
|
||||
details: {
|
||||
issueId: issue.id,
|
||||
runId: run.id,
|
||||
modelProfile: "cheap",
|
||||
recoveryIntent: "status_only",
|
||||
resumeRequiresNormalModel: true,
|
||||
},
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
function assertStructuredCommentFieldsAllowed(
|
||||
req: Request,
|
||||
res: Response,
|
||||
@@ -2319,6 +2401,7 @@ export function issueRoutes(
|
||||
}
|
||||
assertCompanyAccess(req, issue.companyId);
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, issue))) return;
|
||||
if (!(await assertDeliverableMutationAllowedByRunContext(req, res, issue))) return;
|
||||
const keyParsed = issueDocumentKeySchema.safeParse(String(req.params.key ?? "").trim().toLowerCase());
|
||||
if (!keyParsed.success) {
|
||||
res.status(400).json({ error: "Invalid document key", details: keyParsed.error.issues });
|
||||
@@ -2523,6 +2606,7 @@ export function issueRoutes(
|
||||
}
|
||||
assertCompanyAccess(req, issue.companyId);
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, issue))) return;
|
||||
if (!(await assertDeliverableMutationAllowedByRunContext(req, res, issue))) return;
|
||||
const keyParsed = issueDocumentKeySchema.safeParse(String(req.params.key ?? "").trim().toLowerCase());
|
||||
if (!keyParsed.success) {
|
||||
res.status(400).json({ error: "Invalid document key", details: keyParsed.error.issues });
|
||||
@@ -2682,6 +2766,7 @@ export function issueRoutes(
|
||||
}
|
||||
assertCompanyAccess(req, issue.companyId);
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, issue))) return;
|
||||
if (!(await assertDeliverableMutationAllowedByRunContext(req, res, issue))) return;
|
||||
const product = await workProductsSvc.createForIssue(issue.id, issue.companyId, {
|
||||
...req.body,
|
||||
projectId: req.body.projectId ?? issue.projectId ?? null,
|
||||
@@ -2725,6 +2810,7 @@ export function issueRoutes(
|
||||
return;
|
||||
}
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, issue))) return;
|
||||
if (!(await assertDeliverableMutationAllowedByRunContext(req, res, issue))) return;
|
||||
const product = await workProductsSvc.update(id, req.body);
|
||||
if (!product) {
|
||||
res.status(404).json({ error: "Work product not found" });
|
||||
@@ -2765,6 +2851,7 @@ export function issueRoutes(
|
||||
return;
|
||||
}
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, issue))) return;
|
||||
if (!(await assertDeliverableMutationAllowedByRunContext(req, res, issue))) return;
|
||||
const removed = await workProductsSvc.remove(id);
|
||||
if (!removed) {
|
||||
res.status(404).json({ error: "Work product not found" });
|
||||
@@ -2998,6 +3085,7 @@ export function issueRoutes(
|
||||
const companyId = req.params.companyId as string;
|
||||
assertCompanyAccess(req, companyId);
|
||||
assertNoAgentHostWorkspaceCommandMutation(req, collectIssueWorkspaceCommandPaths(req.body));
|
||||
if (!(await assertCheapRecoveryIssueAssigneeProfileAllowed(req, res, { companyId }, req.body))) return;
|
||||
if (req.body.assigneeAgentId || req.body.assigneeUserId) {
|
||||
await assertCanAssignTasks(req, companyId);
|
||||
}
|
||||
@@ -3093,6 +3181,7 @@ export function issueRoutes(
|
||||
}
|
||||
assertCompanyAccess(req, parent.companyId);
|
||||
assertNoAgentHostWorkspaceCommandMutation(req, collectIssueWorkspaceCommandPaths(req.body));
|
||||
if (!(await assertCheapRecoveryIssueAssigneeProfileAllowed(req, res, parent, req.body))) return;
|
||||
if (req.body.assigneeAgentId || req.body.assigneeUserId) {
|
||||
await assertCanAssignTasks(req, parent.companyId);
|
||||
}
|
||||
@@ -3239,6 +3328,7 @@ export function issueRoutes(
|
||||
assertCompanyAccess(req, existing.companyId);
|
||||
assertNoAgentHostWorkspaceCommandMutation(req, collectIssueWorkspaceCommandPaths(req.body));
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, existing))) return;
|
||||
if (!(await assertCheapRecoveryIssueAssigneeProfileAllowed(req, res, existing, req.body))) return;
|
||||
|
||||
const actor = getActorInfo(req);
|
||||
const isClosed = isClosedIssueStatus(existing.status);
|
||||
@@ -5261,6 +5351,7 @@ export function issueRoutes(
|
||||
return;
|
||||
}
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, issue))) return;
|
||||
if (!(await assertDeliverableMutationAllowedByRunContext(req, res, issue))) return;
|
||||
|
||||
const company = await companiesSvc.getById(companyId);
|
||||
const attachmentMaxBytes = normalizeIssueAttachmentMaxBytes(company?.attachmentMaxBytes);
|
||||
@@ -5380,6 +5471,7 @@ export function issueRoutes(
|
||||
return;
|
||||
}
|
||||
if (!(await assertAgentIssueMutationAllowed(req, res, issue))) return;
|
||||
if (!(await assertDeliverableMutationAllowedByRunContext(req, res, issue))) return;
|
||||
|
||||
try {
|
||||
await storage.deleteObject(attachment.companyId, attachment.objectKey);
|
||||
|
||||
@@ -2785,7 +2785,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
projectId: input.claimed.projectId,
|
||||
goalId: input.claimed.goalId,
|
||||
assigneeAgentId: input.claimed.assigneeAgentId,
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides(),
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides("status_only"),
|
||||
originKind: RECOVERY_ORIGIN_KINDS.strandedIssueRecovery,
|
||||
originId: input.claimed.id,
|
||||
originFingerprint: `issue_monitor:${input.clearReason}`,
|
||||
@@ -2799,7 +2799,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
triggerDetail: "system",
|
||||
reason: "issue_monitor_recovery_issue",
|
||||
idempotencyKey: `issue-monitor-recovery-issue:${input.claimed.id}:${input.clearReason}:${input.scheduledAtIso}`,
|
||||
payload: withRecoveryModelProfileHint({ issueId: recoveryIssue.id, sourceIssueId: input.claimed.id }),
|
||||
payload: withRecoveryModelProfileHint({ issueId: recoveryIssue.id, sourceIssueId: input.claimed.id }, "status_only"),
|
||||
requestedByActorType: input.actorType,
|
||||
requestedByActorId: input.actorId,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -2807,7 +2807,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
sourceIssueId: input.claimed.id,
|
||||
source: "issue.monitor.recovery_issue",
|
||||
wakeReason: "issue_monitor_recovery_issue",
|
||||
}),
|
||||
}, "status_only"),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -2868,7 +2868,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
serviceName: input.monitor?.serviceName ?? null,
|
||||
timeoutAt: input.monitor?.timeoutAt ?? null,
|
||||
maxAttempts: input.monitor?.maxAttempts ?? null,
|
||||
}),
|
||||
}, "status_only"),
|
||||
requestedByActorType: input.actorType,
|
||||
requestedByActorId: input.actorId,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -2881,7 +2881,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
serviceName: input.monitor?.serviceName ?? null,
|
||||
timeoutAt: input.monitor?.timeoutAt ?? null,
|
||||
maxAttempts: input.monitor?.maxAttempts ?? null,
|
||||
}),
|
||||
}, "status_only"),
|
||||
});
|
||||
|
||||
await logActivity(db, {
|
||||
@@ -4535,7 +4535,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
wakeReason: "missing_issue_comment",
|
||||
retryReason: "missing_issue_comment",
|
||||
missingIssueCommentForRunId: run.id,
|
||||
});
|
||||
}, "status_only");
|
||||
const now = new Date();
|
||||
|
||||
const retryRun = await db.transaction(async (tx) => {
|
||||
@@ -4562,7 +4562,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
issueId,
|
||||
retryOfRunId: run.id,
|
||||
retryReason: "missing_issue_comment",
|
||||
}),
|
||||
}, "status_only"),
|
||||
status: "queued",
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
@@ -4755,7 +4755,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
retryOfRunId: run.id,
|
||||
wakeReason: "process_lost_retry",
|
||||
retryReason: "process_lost",
|
||||
});
|
||||
}, "normal_model");
|
||||
|
||||
const queued = await db.transaction(async (tx) => {
|
||||
const wakeupRequest = await tx
|
||||
@@ -4769,7 +4769,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
payload: withRecoveryModelProfileHint({
|
||||
...(issueId ? { issueId } : {}),
|
||||
retryOfRunId: run.id,
|
||||
}),
|
||||
}, "normal_model"),
|
||||
status: "queued",
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
@@ -5322,7 +5322,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
scheduledRetryAt: schedule.dueAt.toISOString(),
|
||||
...(transientRetryNotBefore ? { transientRetryNotBefore: transientRetryNotBefore.toISOString() } : {}),
|
||||
...(codexTransientFallbackMode ? { codexTransientFallbackMode } : {}),
|
||||
});
|
||||
}, "normal_model");
|
||||
const maxTurnContinuationIdempotencyKey = retryReason === MAX_TURN_CONTINUATION_RETRY_REASON
|
||||
? `max-turn-continuation:${run.companyId}:${issueId ?? "no-issue"}:${run.id}:${schedule.attempt}`
|
||||
: null;
|
||||
@@ -5492,7 +5492,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
scheduledRetryAt: schedule.dueAt.toISOString(),
|
||||
...(transientRetryNotBefore ? { transientRetryNotBefore: transientRetryNotBefore.toISOString() } : {}),
|
||||
...(codexTransientFallbackMode ? { codexTransientFallbackMode } : {}),
|
||||
}),
|
||||
}, "normal_model"),
|
||||
status: "queued",
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
@@ -8562,7 +8562,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
payload: withRecoveryModelProfileHint({
|
||||
issueId: issue.id,
|
||||
retryOfRunId: run.id,
|
||||
}),
|
||||
}, "normal_model"),
|
||||
status: "queued",
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
@@ -8587,7 +8587,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
||||
retryReason,
|
||||
source: recoverySource,
|
||||
retryOfRunId: run.id,
|
||||
}),
|
||||
}, "normal_model"),
|
||||
sessionIdBefore: recoverySessionBefore,
|
||||
retryOfRunId: run.id,
|
||||
updatedAt: now,
|
||||
|
||||
@@ -691,7 +691,7 @@ export function productivityReviewService(db: Db, deps?: { enqueueWakeup?: Enque
|
||||
goalId: evidence.sourceIssue.goalId,
|
||||
billingCode: evidence.sourceIssue.billingCode,
|
||||
assigneeAgentId: ownerAgentId,
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides(),
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides("status_only"),
|
||||
originKind: PRODUCTIVITY_REVIEW_ORIGIN_KIND,
|
||||
originId: evidence.sourceIssue.id,
|
||||
originFingerprint: productivityReviewFingerprint(evidence.sourceIssue.id),
|
||||
@@ -741,7 +741,7 @@ export function productivityReviewService(db: Db, deps?: { enqueueWakeup?: Enque
|
||||
issueId: review.id,
|
||||
sourceIssueId: evidence.sourceIssue.id,
|
||||
trigger: evidence.trigger,
|
||||
}),
|
||||
}, "status_only"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: "productivity_review",
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -751,7 +751,7 @@ export function productivityReviewService(db: Db, deps?: { enqueueWakeup?: Enque
|
||||
source: PRODUCTIVITY_REVIEW_ORIGIN_KIND,
|
||||
sourceIssueId: evidence.sourceIssue.id,
|
||||
productivityReviewTrigger: evidence.trigger,
|
||||
}),
|
||||
}, "status_only"),
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
recoveryAssigneeAdapterOverrides,
|
||||
scrubRecoveryModelProfileHints,
|
||||
withRecoveryModelProfileHint,
|
||||
} from "./model-profile-hint.js";
|
||||
|
||||
describe("recovery model profile policy", () => {
|
||||
it("allows cheap only for status-only recovery and adds guard context", () => {
|
||||
expect(withRecoveryModelProfileHint({ issueId: "issue-1" }, "status_only")).toEqual({
|
||||
issueId: "issue-1",
|
||||
recoveryIntent: "status_only",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
modelProfile: "cheap",
|
||||
});
|
||||
expect(recoveryAssigneeAdapterOverrides("status_only")).toEqual({ modelProfile: "cheap" });
|
||||
});
|
||||
|
||||
it("scrubs inherited cheap hints from normal model source-work retries", () => {
|
||||
expect(withRecoveryModelProfileHint({
|
||||
issueId: "issue-1",
|
||||
retryOfRunId: "run-1",
|
||||
modelProfile: "cheap",
|
||||
recoveryIntent: "status_only",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
}, "normal_model")).toEqual({
|
||||
issueId: "issue-1",
|
||||
retryOfRunId: "run-1",
|
||||
});
|
||||
});
|
||||
|
||||
it("can scrub copied downstream source-work contexts without applying a profile", () => {
|
||||
expect(scrubRecoveryModelProfileHints({
|
||||
taskId: "source-task",
|
||||
modelProfile: "cheap",
|
||||
paperclipModelProfile: { requested: "cheap" },
|
||||
allowDocumentUpdates: false,
|
||||
})).toEqual({ taskId: "source-task" });
|
||||
});
|
||||
});
|
||||
@@ -1,14 +1,65 @@
|
||||
export const RECOVERY_MODEL_PROFILE_KEY = "cheap" as const;
|
||||
|
||||
export type RecoveryModelProfileWorkClass = "status_only" | "normal_model";
|
||||
|
||||
export const STATUS_ONLY_RECOVERY_GUARD_CONTEXT = {
|
||||
recoveryIntent: "status_only",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
} as const;
|
||||
|
||||
const RECOVERY_MODEL_PROFILE_HINT_KEYS = [
|
||||
"modelProfile",
|
||||
"paperclipModelProfile",
|
||||
"recoveryIntent",
|
||||
"allowDeliverableWork",
|
||||
"allowDocumentUpdates",
|
||||
"resumeRequiresNormalModel",
|
||||
] as const;
|
||||
|
||||
type RecoveryModelProfileHintKey = (typeof RECOVERY_MODEL_PROFILE_HINT_KEYS)[number];
|
||||
type WithoutRecoveryModelProfileHints<T> = Omit<T, RecoveryModelProfileHintKey>;
|
||||
|
||||
export function scrubRecoveryModelProfileHints<T extends Record<string, unknown>>(
|
||||
input: T,
|
||||
): WithoutRecoveryModelProfileHints<T> {
|
||||
const output: Record<string, unknown> = { ...input };
|
||||
for (const key of RECOVERY_MODEL_PROFILE_HINT_KEYS) {
|
||||
delete output[key];
|
||||
}
|
||||
return output as WithoutRecoveryModelProfileHints<T>;
|
||||
}
|
||||
|
||||
export function withRecoveryModelProfileHint<T extends Record<string, unknown>>(
|
||||
input: T,
|
||||
): T & { modelProfile: typeof RECOVERY_MODEL_PROFILE_KEY } {
|
||||
workClass: "normal_model",
|
||||
): WithoutRecoveryModelProfileHints<T>;
|
||||
export function withRecoveryModelProfileHint<T extends Record<string, unknown>>(
|
||||
input: T,
|
||||
workClass: "status_only",
|
||||
): WithoutRecoveryModelProfileHints<T> & typeof STATUS_ONLY_RECOVERY_GUARD_CONTEXT & {
|
||||
modelProfile: typeof RECOVERY_MODEL_PROFILE_KEY;
|
||||
};
|
||||
export function withRecoveryModelProfileHint<T extends Record<string, unknown>>(
|
||||
input: T,
|
||||
workClass: RecoveryModelProfileWorkClass,
|
||||
):
|
||||
| WithoutRecoveryModelProfileHints<T>
|
||||
| (WithoutRecoveryModelProfileHints<T> & typeof STATUS_ONLY_RECOVERY_GUARD_CONTEXT & {
|
||||
modelProfile: typeof RECOVERY_MODEL_PROFILE_KEY;
|
||||
}) {
|
||||
if (workClass === "normal_model") {
|
||||
return scrubRecoveryModelProfileHints(input);
|
||||
}
|
||||
|
||||
return {
|
||||
...input,
|
||||
...scrubRecoveryModelProfileHints(input),
|
||||
...STATUS_ONLY_RECOVERY_GUARD_CONTEXT,
|
||||
modelProfile: RECOVERY_MODEL_PROFILE_KEY,
|
||||
};
|
||||
}
|
||||
|
||||
export function recoveryAssigneeAdapterOverrides() {
|
||||
export function recoveryAssigneeAdapterOverrides(_workClass: Extract<RecoveryModelProfileWorkClass, "status_only">) {
|
||||
return { modelProfile: RECOVERY_MODEL_PROFILE_KEY };
|
||||
}
|
||||
|
||||
@@ -166,7 +166,7 @@ export function decideRunLivenessContinuation(input: {
|
||||
instruction:
|
||||
nextAction ??
|
||||
"The previous run ended without concrete progress. Take the first concrete action now or mark the issue blocked with a specific unblock request.",
|
||||
});
|
||||
}, "normal_model");
|
||||
|
||||
return {
|
||||
kind: "enqueue",
|
||||
@@ -184,6 +184,6 @@ export function decideRunLivenessContinuation(input: {
|
||||
livenessContinuationState: livenessState,
|
||||
livenessContinuationReason: livenessReason,
|
||||
livenessContinuationInstruction: payload.instruction,
|
||||
}),
|
||||
}, "normal_model"),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -499,7 +499,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
payload: withRecoveryModelProfileHint({
|
||||
issueId: input.issueId,
|
||||
...(input.retryOfRunId ? { retryOfRunId: input.retryOfRunId } : {}),
|
||||
}),
|
||||
}, "normal_model"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -509,7 +509,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
retryReason: input.retryReason,
|
||||
source: input.source,
|
||||
...(input.retryOfRunId ? { retryOfRunId: input.retryOfRunId } : {}),
|
||||
}),
|
||||
}, "normal_model"),
|
||||
});
|
||||
|
||||
if (queued && input.retryOfRunId) {
|
||||
@@ -535,7 +535,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
payload: withRecoveryModelProfileHint({
|
||||
issueId: issue.id,
|
||||
mutation: "assigned_todo_liveness_dispatch",
|
||||
}),
|
||||
}, "normal_model"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -543,7 +543,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
taskId: issue.id,
|
||||
wakeReason: "issue_assigned",
|
||||
source: "issue.assigned_todo_liveness_dispatch",
|
||||
}),
|
||||
}, "normal_model"),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -650,7 +650,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
payload: withRecoveryModelProfileHint({
|
||||
issueId: candidate.id,
|
||||
mutation: "unassigned_blocker_recovery",
|
||||
}),
|
||||
}, "normal_model"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -658,7 +658,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
taskId: candidate.id,
|
||||
wakeReason: "issue_assigned",
|
||||
source: "issue.unassigned_blocker_recovery",
|
||||
}),
|
||||
}, "normal_model"),
|
||||
});
|
||||
|
||||
if (queued) {
|
||||
@@ -1455,7 +1455,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
goalId: sourceIssue?.goalId ?? null,
|
||||
billingCode: sourceIssue?.billingCode ?? null,
|
||||
assigneeAgentId: ownerAgentId,
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides(),
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides("status_only"),
|
||||
originKind: STALE_ACTIVE_RUN_EVALUATION_ORIGIN_KIND,
|
||||
originId: input.run.id,
|
||||
originRunId: input.run.id,
|
||||
@@ -1501,7 +1501,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
issueId: evaluation.id,
|
||||
staleRunId: input.run.id,
|
||||
sourceIssueId: sourceIssue?.id ?? null,
|
||||
}),
|
||||
}, "status_only"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -1511,7 +1511,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
source: STALE_ACTIVE_RUN_EVALUATION_ORIGIN_KIND,
|
||||
staleRunId: input.run.id,
|
||||
sourceIssueId: sourceIssue?.id ?? null,
|
||||
}),
|
||||
}, "status_only"),
|
||||
});
|
||||
}
|
||||
return { kind: "created" as const, evaluationIssueId: evaluation.id };
|
||||
@@ -1890,7 +1890,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
projectId: input.issue.projectId,
|
||||
goalId: input.issue.goalId,
|
||||
assigneeAgentId: ownerAgentId,
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides(),
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides("status_only"),
|
||||
originKind: STRANDED_ISSUE_RECOVERY_ORIGIN_KIND,
|
||||
originId: input.issue.id,
|
||||
originRunId: input.latestRun?.id ?? null,
|
||||
@@ -1920,7 +1920,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
sourceIssueId: input.issue.id,
|
||||
strandedRunId: input.latestRun?.id ?? null,
|
||||
recoveryCause,
|
||||
}),
|
||||
}, "status_only"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -1931,7 +1931,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
sourceIssueId: input.issue.id,
|
||||
strandedRunId: input.latestRun?.id ?? null,
|
||||
recoveryCause,
|
||||
}),
|
||||
}, "status_only"),
|
||||
});
|
||||
|
||||
return recovery;
|
||||
@@ -2050,7 +2050,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
recoveryActionId: input.action.id,
|
||||
strandedRunId: input.latestRun?.id ?? null,
|
||||
recoveryCause: input.recoveryCause,
|
||||
}),
|
||||
}, "status_only"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -2063,7 +2063,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
sourceIssueId: input.issue.id,
|
||||
strandedRunId: input.latestRun?.id ?? null,
|
||||
recoveryCause: input.recoveryCause,
|
||||
}),
|
||||
}, "status_only"),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3256,7 +3256,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
projectId: recoveryIssue.projectId,
|
||||
goalId: recoveryIssue.goalId,
|
||||
assigneeAgentId: ownerSelection.agentId,
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides(),
|
||||
assigneeAdapterOverrides: recoveryAssigneeAdapterOverrides("status_only"),
|
||||
originKind: RECOVERY_ORIGIN_KINDS.issueGraphLivenessEscalation,
|
||||
originId: input.finding.incidentKey,
|
||||
originFingerprint: livenessRecoveryLeafFingerprint(input.finding),
|
||||
@@ -3342,7 +3342,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
sourceIssueId: issue.id,
|
||||
recoveryIssueId: recoveryIssue.id,
|
||||
incidentKey: input.finding.incidentKey,
|
||||
}),
|
||||
}, "status_only"),
|
||||
requestedByActorType: "system",
|
||||
requestedByActorId: null,
|
||||
contextSnapshot: withRecoveryModelProfileHint({
|
||||
@@ -3353,7 +3353,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
|
||||
sourceIssueId: issue.id,
|
||||
recoveryIssueId: recoveryIssue.id,
|
||||
incidentKey: input.finding.incidentKey,
|
||||
}),
|
||||
}, "status_only"),
|
||||
});
|
||||
|
||||
logger.warn({
|
||||
|
||||
@@ -76,11 +76,17 @@ describe("successful run handoff decision", () => {
|
||||
resumeIntent: true,
|
||||
resumeFromRunId: "run-1",
|
||||
modelProfile: "cheap",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
});
|
||||
expect(decision.contextSnapshot).toMatchObject({
|
||||
wakeReason: FINISH_SUCCESSFUL_RUN_HANDOFF_REASON,
|
||||
handoffRequired: true,
|
||||
modelProfile: "cheap",
|
||||
allowDeliverableWork: false,
|
||||
allowDocumentUpdates: false,
|
||||
resumeRequiresNormalModel: true,
|
||||
});
|
||||
expect(decision.instruction).toContain("Resolve the missing disposition before creating or revising any new artifacts");
|
||||
expect(decision.instruction).toContain("Choose **exactly one** outcome");
|
||||
|
||||
@@ -323,9 +323,9 @@ export function buildSuccessfulRunHandoffInstruction(input: {
|
||||
"3. Mark it `blocked` with first-class blockers (`blockedByIssueIds`) or a clearly named unblock owner/action.",
|
||||
"",
|
||||
"**Is there more work to do?**",
|
||||
`4. Either delegate follow-up work (create/link a follow-up issue and block this one on it, or close this issue if its scope is independently complete) or record an explicit continuation path with \`resumeIntent: true\`, \`resumeFromRunId: ${input.sourceRunId}\`, and a concrete next action.`,
|
||||
`4. Either delegate follow-up work (create/link a follow-up issue and block this one on it, or close this issue if its scope is independently complete) or record an explicit continuation path with \`resumeIntent: true\`, \`resumeFromRunId: ${input.sourceRunId}\`, and a concrete next action. Do not perform the remaining source work in this recovery run; the follow-up/resume wake must use the normal model lane.`,
|
||||
"",
|
||||
"Comments, document revisions, work-product writes, and continuation summaries are supporting evidence only — they do not satisfy this handoff unless the issue state/path also records one valid disposition.",
|
||||
"Comments, document revisions, work-product writes, and continuation summaries are supporting evidence only — they do not satisfy this handoff unless the issue state/path also records one valid disposition. If this wake is status-only recovery, document or plan updates are not allowed.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
@@ -404,7 +404,7 @@ export function decideSuccessfulRunHandoff(input: {
|
||||
resumeFromRunId: run.id,
|
||||
...(input.taskKey ? { taskKey: input.taskKey } : {}),
|
||||
instruction,
|
||||
});
|
||||
}, "status_only");
|
||||
|
||||
return {
|
||||
kind: "enqueue",
|
||||
@@ -418,6 +418,6 @@ export function decideSuccessfulRunHandoff(input: {
|
||||
...payload,
|
||||
wakeReason: FINISH_SUCCESSFUL_RUN_HANDOFF_REASON,
|
||||
livenessState: input.livenessState,
|
||||
}),
|
||||
}, "status_only"),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -72,6 +72,15 @@ async function flushReact() {
|
||||
});
|
||||
}
|
||||
|
||||
async function findPluginLinks(container: HTMLElement, expectedCount: number) {
|
||||
await act(async () => {
|
||||
await vi.waitFor(() => {
|
||||
expect(container.querySelectorAll('a[href^="/instance/settings/plugins/"]')).toHaveLength(expectedCount);
|
||||
});
|
||||
});
|
||||
return Array.from(container.querySelectorAll<HTMLAnchorElement>('a[href^="/instance/settings/plugins/"]'));
|
||||
}
|
||||
|
||||
function renderSidebar(container: HTMLElement) {
|
||||
const queryClient = new QueryClient({
|
||||
defaultOptions: { queries: { retry: false, gcTime: 0 } },
|
||||
@@ -151,8 +160,7 @@ describe("InstanceSidebar", () => {
|
||||
queryClient = rendered.queryClient;
|
||||
await flushReact();
|
||||
|
||||
const pluginLinks = Array.from(container.querySelectorAll('a[href^="/instance/settings/plugins/"]'));
|
||||
expect(pluginLinks).toHaveLength(1);
|
||||
const pluginLinks = await findPluginLinks(container, 1);
|
||||
expect(pluginLinks[0]?.getAttribute("href")).toBe("/instance/settings/plugins/linear");
|
||||
expect(pluginLinks[0]?.textContent).toBe("Linear");
|
||||
});
|
||||
@@ -190,8 +198,7 @@ describe("InstanceSidebar", () => {
|
||||
queryClient = rendered.queryClient;
|
||||
await flushReact();
|
||||
|
||||
const pluginLinks = Array.from(container.querySelectorAll('a[href^="/instance/settings/plugins/"]'));
|
||||
expect(pluginLinks).toHaveLength(1);
|
||||
const pluginLinks = await findPluginLinks(container, 1);
|
||||
expect(pluginLinks[0]?.getAttribute("href")).toBe("/instance/settings/plugins/hybrid");
|
||||
});
|
||||
|
||||
@@ -214,6 +221,7 @@ describe("InstanceSidebar", () => {
|
||||
root = rendered.root;
|
||||
queryClient = rendered.queryClient;
|
||||
await flushReact();
|
||||
await findPluginLinks(container, 1);
|
||||
|
||||
const topLevelLinks = Array.from(
|
||||
container.querySelectorAll<HTMLAnchorElement>('a[href^="/instance/settings/"]'),
|
||||
|
||||
Reference in New Issue
Block a user