test(crawler): add contract and parser coverage for crawler plugin#628
Open
anshul23102 wants to merge 4 commits into
Open
test(crawler): add contract and parser coverage for crawler plugin#628anshul23102 wants to merge 4 commits into
anshul23102 wants to merge 4 commits into
Conversation
Two separate CI regressions were introduced by commits 0e03877 and a2a7e02: Backend lint (F821 - Undefined name 'db') workflows.py._run_workflow() calls get_target_policy(db, ...) but 'db' was never acquired in that method. tick() obtains 'db' but does not pass it into _run_workflow(). Fixed by adding db = await get_db() at the top of _run_workflow(). Frontend unit test failures (3 tests) ToolConfig.tsx now calls listTargetPolicies(), listCredentialProfiles(), and listSessionProfiles() inside its useEffect via Promise.all. Tests that only mocked the original 3-4 API functions caused Promise.all to reject (unmocked vi.fn() returns undefined, not a Promise), making setServerLimits never execute and breaking max/min attribute assertions. Workflows.tsx changed emptySteps to include an execution_context object in each step. The createWorkflow assertion expected the old shape. Fixes applied: - ToolConfigDynamic.test.tsx: add listTargetPolicies, listCredentialProfiles, listSessionProfiles, getSettings to vi.mock factory and beforeEach mocks; update startTask assertion to accept the new 5th executionContext argument - ToolConfigTimeout.test.tsx: add the three new API functions to vi.mock factory and beforeEach mocks so Promise.all resolves correctly - Workflows.test.tsx: update createWorkflow expectation to include execution_context in the steps array
…mpliance
{ items: [] } was inferred as { items: never[] }, which does not satisfy
NamedResourceList<T> (requires items: T[] and total: number). Added total: 0
to all three mock returns so TypeScript accepts the fixture without casting.
Add backend test suite for the crawler plugin that loads the real plugins/crawler/metadata.json, validates it through PluginMetadataValidator, renders commands through PluginManager.build_command(), and calls the real plugins.crawler.parser.parse() directly. Assertions are tied to the actual plugin contract: - engine.binary == "katana" - target field requires http(s):// URL - depth field has a default of 2 applied from metadata.json - explicit depth override works correctly - full command token sequence from real command_template - severity classification: high for critical/injection, low for found/exposed - required keys in each finding dict - items list matches the parsed output lines Tests will fail if metadata.json, command_template, or parser.py drift. Closes utksh1#494
anshul23102
force-pushed
the
test/494-crawler-plugin-coverage
branch
from
a36e42e to
ac1eabf
Compare
build_command drops the unresolved {target} token instead of returning None.
Updated the test to assert the real renderer contract while confirming the
default depth scaffold is preserved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #494
Adds backend contract and parser tests for the
crawlerplugin. All tests load the realplugins/crawler/metadata.json, validate through the projectPluginMetadataValidator, render commands throughPluginManager.build_command(), and callplugins.crawler.parser.parse()directly.What Is Tested
Metadata contract
metadata.jsonexists and is valid JSONPluginMetadataValidatorpasses without errorsidmatches directory name (crawler)katanatargetfield is declared, required, and enforceshttps?://URL patterndepthfield is optional with a default of2customandparser.pyexistsCommand rendering (real
PluginManager)["katana", "-u", "<target>", "-depth", "2", "-silent"]2frommetadata.jsonis applied whendepthis omitteddepthoverride replaces the default correctlytargetreturnsNonePluginManagerParser contract (real
parser.py)findings,count,itemskeyscountequalslen(findings)title,category,severity,description,remediation,metadatacritical/injectionkeywords classified ashighseverityfound/exposed/detectedkeywords classified aslowseverityitemslist matches all non-empty lines from outputWhy These Tests Will Catch Regressions
katanais replaced inengine.binary,test_crawler_engine_is_katanafailscommand_templatetokens change order,test_crawler_command_full_token_sequencefailsdepthdefault is removed,test_crawler_command_uses_default_depthfailsparser.pystops returningfindings, every parser test failsProgram
This contribution is submitted under NSoC'26 (Nexus Spring of Code 2026).
Please apply labels:
type:testing,level:intermediate,area:backend,area:plugins