Skip to content

Agents - Move agent test library to BC apps#8114

Open
nikolakukrika wants to merge 4 commits into
mainfrom
features/MoveAITestLibraryOutOfInternalFolder-BCApps
Open

Agents - Move agent test library to BC apps#8114
nikolakukrika wants to merge 4 commits into
mainfrom
features/MoveAITestLibraryOutOfInternalFolder-BCApps

Conversation

@nikolakukrika
Copy link
Copy Markdown
Contributor

@nikolakukrika nikolakukrika commented May 12, 2026

What & why

Moving agent test library to BC Apps

Linked work

Fixes AB#629478

@nikolakukrika nikolakukrika requested review from a team as code owners May 12, 2026 09:24
@github-actions github-actions Bot added this to the Version 29.0 milestone May 12, 2026
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AL Documentation Audit

Documentation gaps were detected in the following apps:

  • Agent-Test-Library: 0% documentation coverage

To generate documentation, run /al-docs init or /al-docs update using GitHub Copilot CLI or Claude Code.
This review is for awareness to help keep documentation in sync with code changes. It is okay to dismiss this request.

…res/MoveAITestLibraryOutOfInternalFolder-BCApps
begin
AgentTaskMessage.CalcFields(Content);
AgentTaskMessage.Content.CreateInStream(ContentInStream, GetDefaultEncoding());
ContentInStream.Read(ContentText);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟠\ High\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 1}}$

InStream.Read truncates multi-line messages

InStream.Read(ContentText) reads only until the next newline character — it stops at the first \n. Agent messages routinely span multiple lines, so any content after the first line is silently discarded and will never appear in test output.

Recommendation:

  • Replace ContentInStream.Read(ContentText) with ContentInStream.ReadText(ContentText) to read the full blob content until end-of-stream.
Suggested change
ContentInStream.Read(ContentText);
ContentInStream.ReadText(ContentText);

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

begin
EnsureIsTest();
AgentTask.ReadIsolation := IsolationLevel::ReadCommitted;
AgentTask.SetFilter(Status, '<>%1', AgentTask.Status::Paused);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 1}}$

StopTasks overwrites terminal task statuses

The filter <>%1 AgentTask.Status::Paused selects every task that is not Paused — including already-Completed and "Stopped by System" tasks. StopTask then overwrites their status to Stopped by User, silently corrupting the final state that tests or diagnostics may later inspect.

Recommendation:

  • Filter to only actionable (running) statuses — Ready, Scheduled, and Running — so completed tasks are left untouched.
Suggested change
AgentTask.SetFilter(Status, '<>%1', AgentTask.Status::Paused);
AgentTask.SetFilter(Status, '%1|%2|%3',
AgentTask.Status::Ready,
AgentTask.Status::Scheduled,
AgentTask.Status::Running);

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

AgentTestContext: Codeunit "Agent Test Context";
begin
EnsureIsTest();
UserInterventionRequestEntry.Get(AgentTask.ID, AgentTask."Last Log Entry ID");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 1}}$

ContinueTaskAndWait assumes task is paused

ContinueTaskAndWait calls UserInterventionRequestEntry.Get(AgentTask.ID, AgentTask."Last Log Entry ID") without first asserting that the task is actually in a Paused state requiring user intervention. If called on a Running or Completed task, the Get will either fail with a cryptic table-not-found error or return an unrelated log entry and corrupt the task state.

Recommendation:

  • Add a guard at the start of the method that errors with a descriptive message if the task does not require user intervention.
Suggested change
UserInterventionRequestEntry.Get(AgentTask.ID, AgentTask."Last Log Entry ID");
if not RequiresUserIntervention(AgentTask) then
Error('Task is not paused waiting for user intervention. Current status: %1.', AgentTask.Status);
UserInterventionRequestEntry.Get(AgentTask.ID, AgentTask."Last Log Entry ID");

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why


FromInput := QueryInput.ElementExists(FromTok, HasFrom);
if not HasFrom then
exit;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 1}}$

Silent false return when 'from' element missing

When the YAML query lacks a from element, CreateTaskFromQueryAndWait executes a bare exit (returns false) after AgentTaskBuilder.Initialize has already been called, leaving the AgentTask output parameter uninitialised. The caller receives false with no diagnostic, making it look like the task ran and failed rather than was never submitted.

Recommendation:

  • Replace the silent exit with an Error() call (or Assert.Fail) so the missing required field is surfaced immediately.
Suggested change
exit;
if not HasFrom then
Error('Task input query must contain a ''from'' element.');

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

end;

SuggestionsInput := ExpectedInterventionRequest.ElementExists(SuggestionsTok, SuggestionsExist);
if SuggestionsExist then
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 1}}$

ValidateInterventionRequest missing suggestion count check

The public ValidateInterventionRequest verifies that every expected suggestion code exists in the actual set, but never checks that the actual count equals the expected count. A test can therefore pass silently even when the agent returns extra unexpected suggestions — unlike ValidateInterventionDetails which does assert count equality.

Recommendation:

  • Add the count assertion after the per-suggestion loop, mirroring the check in ValidateInterventionDetails.
Suggested change
if SuggestionsExist then
// After the for loop:
if SuggestionsExist then
Assert.AreEqual(
SuggestionsInput.GetElementCount(),
TempSuggestion.Count(),
StrSubstNo(SuggestionCountMismatchErr,
SuggestionsInput.GetElementCount(), TempSuggestion.Count()));

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

CopilotCapability: Codeunit "Copilot Capability";
ModuleInfo: ModuleInfo;
begin
CopilotCapability.RegisterCapability(Capability, '');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Style} \quad \color{gray}{\texttt{\small Iteration\ 1}}$

RegisterCopilotCapabilityWithAppId silently no-ops on failure

RegisterCapability registers the capability under ModuleInfo.Id, then a CopilotSettings.Get(Capability, ModuleInfo.Id) tries to find that record to rename it to AppId. If the Get fails (e.g. the capability was already registered under a different module ID), the if block is skipped entirely: the capability ends up registered but inactive and under the wrong AppId, with no error surfaced to the caller.

Recommendation:

  • Add an else Error(...) branch after the if CopilotSettings.Get to surface the failure, or verify the pre-condition before calling RegisterCapability.
Suggested change
CopilotCapability.RegisterCapability(Capability, '');
if CopilotSettings.Get(Capability, ModuleInfo.Id) then begin
CopilotSettings.Rename(Capability, AppId);
CopilotSettings.Status := CopilotSettings.Status::Active;
CopilotSettings.Modify();
Commit();
end else
Error('Failed to locate registered capability %1 under module %2.', Capability, ModuleInfo.Id);

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why

@aholstrup1 aholstrup1 closed this May 20, 2026
@aholstrup1 aholstrup1 reopened this May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants