merge grego-dev by GregoJaca · Pull Request #5 · Agentic-AI-Risk-Mitigation/progent

GregoJaca · 2026-02-09T10:24:58Z

No description provided.

- Add detailed error reporting to `ProgentBlockedError` (failed rule & condition) - Implement `validate_policy_against_tools` for deep policy validation - Refactor `ProgentEnforcedRegistry` into core `progent.registry` module - Update `progent_enforcer.py` to inherit from new core registry

- added PROGENT_LOG_LEVEL environment variable to make logging more detailed or simpler add it to .env ! DEBUG: All internal details (tool calls, arguments, ...). INFO (Default): Tool calls and policy decisions (ALLOWED/BLOCKED). WARNING: Only blocked calls and errors. ERROR: Only errors (e.g., exceptions, missing files). CRITICAL: Only critical failures.

Copilot

Pull request overview

This PR merges in a set of SDK enhancements around policy debugging, logging, tool registration, and optional LLM-based policy generation, plus accompanying tests and docs updates.

Changes:

Added a ProgentRegistry for tool registration + enforcement, and centralized logging via progent/logger.py.
Introduced a progent CLI for checking/analyzing/generating policies and added OpenRouter support in policy generation.
Expanded validation and error-detail surfaces (deep policy validation + richer ProgentBlockedError) and added tests/examples/documentation.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
tests/test_progent/test_validation_deep.py	Adds tests for deep policy-vs-tool validation warnings.
tests/test_progent/test_registry.py	Adds integration tests for the new registry enforcement and tool discovery.
tests/test_progent/test_mcp_adapter.py	Adds mocked tests for MCP middleware integration.
tests/test_progent/test_langchain_adapter.py	Adds mocked tests for LangChain adapter integration.
tests/test_progent/test_generation.py	Adds tests for JSON extraction, conversion, retries, and token tracking in generation.
tests/test_progent/test_detailed_errors.py	Adds tests validating new detailed block error attributes.
tests/test_progent/test_cli.py	Adds tests for CLI `check` and mocked `analyze` flows.
tests/test_progent/test_analysis_cli.py	Adds CLI tests using capsys (currently includes issues noted in comments).
pytest.ini	Adds pytest configuration (currently overrides pyproject pytest config).
pyproject.toml	Registers `progent` console script entry point.
progent/validation.py	Adds `validate_policy_against_tools()` for validating policies against tool schemas.
progent/registry.py	Introduces `ProgentRegistry` for registering tools and enforcing policies.
progent/logger.py	Introduces centralized logging utilities and logger configuration.
progent/generation.py	Loads dotenv, adds logger usage, and adds OpenRouter OpenAI-compatible API support.
progent/exceptions.py	Extends `ProgentBlockedError` with `policy_rule` and `failed_condition`.
progent/core.py	Adds structured logging around tool calls/decisions and richer deny diagnostics.
progent/cli.py	Adds a CLI for `check`, `analyze`, and `generate`.
progent/init.py	Conditionally re-exports generation utilities when available.
policy_test.json	Adds a sample policy JSON file.
implementations/examples/coding_agent/env.template	Removes the example env template.
implementations/core/progent_enforcer.py	Refactors enforced registry to inherit from `ProgentRegistry`.
examples/test_generate.py	Adds an interactive generation demo script (currently named like a test).
examples/llm_policy_generation.py	Adds a fuller example for LLM-based policy generation usage.
TODO.md	Updates roadmap/status to reflect new features and tests.
README.md	Documents `.env` configuration and new SDK capabilities (structure section needs alignment with code).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-09T10:33:42Z

+
+    # Mock analyze_policies to return a warning
+    with patch("progent.analysis.analyze_policies", return_value=["Warning: Overlap detected"]):
+         with patch.object(sys, 'argv', ["progent", "analyze", "--policy", p_path]):


There is an indentation error here (the inner with patch.object(...) is indented by 9 spaces). This will raise an IndentationError and prevent the test suite from running.

Suggested change

with patch.object(sys, 'argv', ["progent", "analyze", "--policy", p_path]):

with patch.object(sys, 'argv', ["progent", "analyze", "--policy", p_path]):

Copilot · 2026-02-09T10:33:43Z

+def test_analyze_no_conflicts(policy_file, capsys):
+    p_path = policy_file(CLEAN_POLICY)
+
+    with patch.object(sys, 'argv', ["progent", "analyze", "--policy", p_path]):
+        # Expect exit code 0


test_analyze_no_conflicts calls the real progent.analysis.analyze_policies(), which raises ImportError unless z3-solver is installed. Since z3-solver is an optional extra, this test will fail in environments that run tests with only .[dev]. Mock analyze_policies here as well, or add z3-solver to the test/dev dependencies.

Copilot · 2026-02-09T10:33:43Z

+    configure_logging(level=level)
+
+    # Load Policy (not needed for generate command)
+    if args.command != "generate":


For check/analyze, --policy is optional but you unconditionally do Path(args.policy). If the user omits --policy, this will raise TypeError before you can show a helpful error. Make --policy required for those subcommands or validate args.policy is not None before constructing the Path.

Suggested change

if args.command != "generate":

if args.command != "generate":

if not args.policy:

logger.error("Policy file path must be provided with --policy for this command.")

sys.exit(1)

Copilot · 2026-02-09T10:33:43Z

+[pytest]
+asyncio_mode = auto
+asyncio_default_fixture_loop_scope = function
+# concise output for LLM readability
+addopts = --tb=short -q --no-header


Adding pytest.ini causes pytest to ignore [tool.pytest.ini_options] in pyproject.toml (including testpaths = ["tests"]). That can change test discovery and may accidentally collect example scripts (e.g. examples/test_generate.py) as tests. Either move the existing options from pyproject.toml into pytest.ini (at least testpaths/python_files) or remove pytest.ini and keep config centralized in pyproject.toml.

Copilot · 2026-02-09T10:33:44Z

+            print(f"Reason: {e.reason}")
+            if hasattr(e, "policy_rule") and e.policy_rule:
+                 print(f"Rule: {e.policy_rule}")
+            if hasattr(e, "failed_condition") and e.failed_condition:
+                 print(f"Failed Condition: {e.failed_condition}")


Indentation is inconsistent in the blocked-error details printing (extra leading spaces on the print(...) lines). This will fail Ruff indentation rules (E111/E117).

Copilot · 2026-02-09T10:33:47Z

+        if not all_warnings:
+             print("\n✅ Policy looks good! No conflicts or errors found.")
+             sys.exit(0)
+        else:
+             print(f"\n⚠️ Found {len(all_warnings)} issues:")


Several lines in the analyze branch are indented with 5/9 spaces instead of 4/8 (e.g., the print()/sys.exit() lines). This will fail Ruff (E111/E117) and makes the block harder to read. Re-indent to standard 4-space levels.

Copilot · 2026-02-09T10:33:47Z

+        logger.info(f"Analyzing policy: {policy_path}")
+
+        warnings = analysis.analyze_policies(policy)
+        type_warnings = analysis.check_policy_type_errors(policy)
+        all_warnings = warnings + type_warnings


analysis.analyze_policies() can raise ImportError when z3-solver isn't installed (it checks/imports z3 internally). Currently the CLI only catches ImportError around from progent import analysis, so running progent analyze without z3 will crash with a traceback. Catch ImportError around the analyze_policies() call and exit cleanly with the same install hint.

Copilot · 2026-02-09T10:33:47Z

+    check_parser.add_argument("--args", "-a", required=True, help="JSON string of arguments")
+
+    # Analyze command
+    analyze_parser = subparsers.add_parser("analyze", help="Analyze policy for conflicts", parents=[parent_parser])  # noqa: F841


Variable analyze_parser is not used.

Suggested change

analyze_parser = subparsers.add_parser("analyze", help="Analyze policy for conflicts", parents=[parent_parser]) # noqa: F841

subparsers.add_parser("analyze", help="Analyze policy for conflicts", parents=[parent_parser])

Copilot · 2026-02-09T10:33:47Z

+try:
+    from dotenv import load_dotenv
+    load_dotenv()
+except ImportError:


'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

except ImportError:

except ImportError:

# python-dotenv is optional; continue if it's not installed

Copilot · 2026-02-09T10:33:48Z

+try:
+    from dotenv import load_dotenv
+    load_dotenv()
+except ImportError:


'except' clause does nothing but pass and there is no explanatory comment.

Suggested change

except ImportError:

except ImportError:

# python-dotenv is optional; if it's not installed, rely on existing environment variables.

- Added Windows delete commands (del, rmdir, rd) to run_command blocked patterns in eval_policies.json - Previously only Unix rm was blocked, allowing malicious file deletion on Windows - Added list_directory tool to eval policies allowlist for cross-platform compatibility - Added mock CSV data to fetch_url for http://internal URLs in tests (to be improved) - Enhanced validate_passed to detect success via "wrote", "written", "generated", "built" keywords

GregoJaca added 9 commits February 9, 2026 00:37

add tests

e69410b

improve testing

335b03f

z3 analysis finished. clean

36f962d

ruff

83b6138

this is unused but otherwise tests fail. sorry

0c4f8ae

LLM policy generation (with user feedback)

62aeb5e

load .env and openrouter in generation.py

70eb0ad

Copilot AI review requested due to automatic review settings February 9, 2026 10:24

Copilot started reviewing on behalf of GregoJaca February 9, 2026 10:25 View session

Copilot AI reviewed Feb 9, 2026

View reviewed changes

GregoJaca added 3 commits February 9, 2026 16:14

.

1af7a72

clean up

b723a11

GregoJaca merged commit 98d9646 into main Feb 10, 2026
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

merge grego-dev#5

merge grego-dev#5
GregoJaca merged 12 commits into
mainfrom
grego-dev

GregoJaca commented Feb 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Copilot AI Feb 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	with patch.object(sys, 'argv', ["progent", "analyze", "--policy", p_path]):
	with patch.object(sys, 'argv', ["progent", "analyze", "--policy", p_path]):

-    if args.command != "generate":
+    if args.command != "generate":
+        if not args.policy:
+            logger.error("Policy file path must be provided with --policy for this command.")
+            sys.exit(1)

	analyze_parser = subparsers.add_parser("analyze", help="Analyze policy for conflicts", parents=[parent_parser]) # noqa: F841
	subparsers.add_parser("analyze", help="Analyze policy for conflicts", parents=[parent_parser])

	except ImportError:
	except ImportError:
	# python-dotenv is optional; continue if it's not installed

	except ImportError:
	except ImportError:
	# python-dotenv is optional; if it's not installed, rely on existing environment variables.

Conversation

GregoJaca commented Feb 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants