Skip to content

test(e2e): cover authorized approval responses - #1824

Open
benpankow wants to merge 1 commit into
ben/hitl-v2-docs-contractsfrom
ben/hitl-v2-authorized-response-e2e
Open

test(e2e): cover authorized approval responses#1824
benpankow wants to merge 1 commit into
ben/hitl-v2-docs-contractsfrom
ben/hitl-v2-authorized-response-e2e

Conversation

@benpankow

Copy link
Copy Markdown
Collaborator

Summary

Adds fixture-owned e2e coverage for the responder-authorized HITL lifecycle introduced by this stack.

The deterministic fixture authenticates the eval driver as a fixed user and defines a tool with explicit request and response policies:

approval: {
  request: always(),
  response: ({ response, responder }) =>
    response.decision === "approve" && responder.principalId === "e2e-approval-responder"
      ? { status: "allowed" }
      : { status: "rejected", safeReason: "Unexpected eval responder." },
}

The eval asserts:

input.requested
→ approval.candidate pending
→ approval.settled approved
→ action.result completed

It uses no external OAuth provider or injected credential, so the result is deterministic and self-contained.

This also adds compiler/runtime normalization coverage for the explicit { request, response } approval object; previously only the function shorthand was accepted by runtime normalization even though the public type allowed the object form.

Depends on #1374.

Validation

  • pnpm --filter agent-tools-hitl typecheck
  • pnpm --filter eve typecheck
  • pnpm guard:invariants
  • focused authored-definition unit tests

@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
eve-docs Error Error Aug 8, 2026 1:16am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
eve-docs-4759 Skipped Skipped Aug 8, 2026 1:16am

const parked = await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
const approval = t.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });

@vercel vercel Bot Aug 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

t.respond(...) is called with a bare input-response object instead of an array, so the HITL approval is posted as a malformed inputResponses body and the server rejects the turn with 400.

Fix on Vercel

afterApproval.event("authorization.required", { count: 1 });

const startIndex = t.events.length;
const callback = await t.target.fetch(required.data.authorization.url);

@vercel vercel Bot Aug 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OAuth eval passes an absolute authorization URL to t.target.fetch, which treats it as a route path and produces a mangled 404 URL, causing the eval to fail.

Fix on Vercel

@benpankow
benpankow force-pushed the ben/hitl-v2-docs-contracts branch from ea75d7c to 64d990f Compare August 7, 2026 23:20
@benpankow
benpankow force-pushed the ben/hitl-v2-authorized-response-e2e branch from f5fb14d to a87306e Compare August 7, 2026 23:20
Signed-off-by: benpankow <ben.pankow@vercel.com>
@benpankow
benpankow force-pushed the ben/hitl-v2-docs-contracts branch from 40f7697 to 7c3c57c Compare August 8, 2026 01:15
@benpankow
benpankow force-pushed the ben/hitl-v2-authorized-response-e2e branch from 92de737 to 25f3f9d Compare August 8, 2026 01:15
@vercel
vercel Bot temporarily deployed to Preview – eve-docs-4759 August 8, 2026 01:15 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants