Skip to content

fix: override strategist post_checks in design_workflow() with dual-format sentinels - #1324

Open
crqu wants to merge 1 commit into
akashgit:mainfrom
crqu:factory/run-46c04ac1
Open

fix: override strategist post_checks in design_workflow() with dual-format sentinels#1324
crqu wants to merge 1 commit into
akashgit:mainfrom
crqu:factory/run-46c04ac1

Conversation

@crqu

@crqu crqu commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Closes #1323

Changes

  • Override the strategist node's post_checks in design_workflow() to use must_contain=["### Phase 1", "### Hypotheses"], accepting either phased plans (new projects) or improvement hypotheses (existing projects) via grep -qE OR semantics
  • Placement after the reads loop (L643-647) preserves the study-combined.md reads update via model_copy
  • Rename and update test test_design_workflow_inherits_post_checkstest_design_workflow_overrides_post_checks to reflect the override behavior and assert ### Hypotheses instead of ### Architecture

…ormat sentinels

Design mode serves two contexts: new projects (phased plans with
"### Phase 1") and existing projects (improvement hypotheses with
"### Hypotheses"). The inherited build_workflow sentinel required
"### Architecture", which only appears in new-project plans, causing
false verification failures on existing projects.

Override the strategist's post_checks in design_workflow() to use
must_contain=["### Phase 1", "### Hypotheses"], which leverages
checks_to_bash() OR semantics (grep -qE with | alternation) to
accept either format.

Closes akashgit#1323
@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.87%. Comparing base (c2161fa) to head (e01ab7e).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1324      +/-   ##
==========================================
+ Coverage   84.83%   84.87%   +0.03%     
==========================================
  Files         231      231              
  Lines       25766    25785      +19     
  Branches     4117     4120       +3     
==========================================
+ Hits        21859    21885      +26     
+ Misses       2928     2923       -5     
+ Partials      979      977       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gx-ai-architect

Copy link
Copy Markdown
Collaborator

@ceo-review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Factory Review: KEEP

Verdict: KEEP
Reason: QA: CLEAN — 5746 tests pass, 0 failures, lint clean. Code review 7/7 PASS. Adversarial QA 6/6 criteria VERIFIED. Precheck HALT on score_direction (no eval baseline, not a code regression).

QA Analysis

Adversarial QA Report

PR: #1324
Title: fix: override strategist post_checks in design_workflow() with dual-format sentinels
Detected project type: Library (Python CLI + workflow engine)
Date: 2026-08-19

Smoke Test

Command:

uv run pytest tests/test_models.py tests/test_guards.py tests/test_runners.py -x -q --tb=short -k 'not (BobAuth or preflight_error_unchanged)'

Output: 202 passed, 3 deselected in 6.31s
Result: PASS

Test Plan

The PR overrides the strategist post_checks in design_workflow() so it uses ["### Phase 1", "### Hypotheses"] instead of inheriting the build workflow's ["### Phase 1", "### Architecture"]. This is a dual-format sentinel fix — design mode produces plans with ### Hypotheses headers, not ### Architecture.

Acceptance criteria derived from the commit:

  1. design_workflow() strategist has must_contain=["### Phase 1", "### Hypotheses"]
  2. build_workflow() strategist retains must_contain=["### Phase 1", "### Architecture"]
  3. Build workflow is not mutated as a side effect of calling design_workflow()
  4. just_plan=True variant also gets the override
  5. SKILL.md export includes the correct sentinels
  6. No regressions in workflow or verification tests

Feature Tests

Criterion 1: design_workflow strategist has correct dual-format sentinels

Status: VERIFIED
Command:

uv run python3 -c "
from factory.workflow.definitions import design_workflow
dwf = design_workflow()
d_strat = dwf.nodes['strategist']
print('DESIGN strategist post_checks must_contain:', d_strat.post_checks[0].must_contain)
assert '### Hypotheses' in d_strat.post_checks[0].must_contain
assert '### Phase 1' in d_strat.post_checks[0].must_contain
assert '### Architecture' not in d_strat.post_checks[0].must_contain
print('PASS')
"

Output:

DESIGN strategist post_checks must_contain: ['### Phase 1', '### Hypotheses']
PASS

Criterion 2: build_workflow strategist retains original sentinels

Status: VERIFIED
Command:

uv run python3 -c "
from factory.workflow.definitions import build_workflow
bwf = build_workflow()
b_strat = bwf.nodes['strategist']
print('BUILD strategist post_checks must_contain:', b_strat.post_checks[0].must_contain)
assert '### Architecture' in b_strat.post_checks[0].must_contain
assert '### Phase 1' in b_strat.post_checks[0].must_contain
print('PASS')
"

Output:

BUILD strategist post_checks must_contain: ['### Phase 1', '### Architecture']
PASS

Criterion 3: build_workflow is not mutated by design_workflow()

Status: VERIFIED
Command:

uv run python3 -c "
from factory.workflow.definitions import design_workflow, build_workflow
bwf1 = build_workflow()
_ = design_workflow()
bwf2 = build_workflow()
assert bwf1.nodes['strategist'].post_checks[0].must_contain == bwf2.nodes['strategist'].post_checks[0].must_contain
assert '### Architecture' in bwf2.nodes['strategist'].post_checks[0].must_contain
print('PASS: build_workflow is not mutated by design_workflow()')
"

Output:

PASS: build_workflow is not mutated by design_workflow()

Criterion 4: just_plan=True variant also gets the override

Status: VERIFIED
Command:

uv run python3 -c "
from factory.workflow.definitions import design_workflow
plan_wf = design_workflow(just_plan=True)
strat = plan_wf.nodes['strategist']
print('PLAN mode must_contain:', strat.post_checks[0].must_contain)
assert '### Hypotheses' in strat.post_checks[0].must_contain
assert '### Phase 1' in strat.post_checks[0].must_contain
print('PASS')
"

Output:

PLAN mode must_contain: ['### Phase 1', '### Hypotheses']
PASS

Criterion 5: SKILL.md export includes correct sentinels

Status: VERIFIED
Command:

uv run python3 -c "
from factory.workflow.definitions import design_workflow
from factory.workflow.skill_export import workflow_to_skill_md
wf = design_workflow()
skill = workflow_to_skill_md(wf)
assert 'Hypotheses' in skill
assert 'Phase 1' in skill
print('PASS: SKILL.md export includes dual-format sentinels')
"

Output:

PASS: SKILL.md export includes dual-format sentinels

Criterion 6: No regressions in verification or workflow tests

Status: VERIFIED
Command:

uv run pytest tests/test_verification.py -v --tb=short

Output: 40 passed in 0.17s

Command:

uv run pytest tests/ -v --tb=short -k "workflow"

Output: 821 passed, 9 skipped, 4682 deselected in 13.93s

Edge Case Tests

min_size and must_exist preserved in override

Status: VERIFIED
Command:

uv run python3 -c "
from factory.workflow.definitions import design_workflow
wf = design_workflow()
strat = wf.nodes['strategist']
assert strat.post_checks[0].min_size == 200
assert strat.post_checks[0].must_exist is True
assert strat.post_checks[0].path == '.factory/strategy/current.md'
print('PASS: all fields preserved')
"

Output:

PASS: all fields preserved

Acceptance Criteria Verification

# Criterion Status
1 design_workflow strategist uses ["### Phase 1", "### Hypotheses"] VERIFIED
2 build_workflow strategist retains ["### Phase 1", "### Architecture"] VERIFIED
3 build_workflow not mutated by design_workflow() call VERIFIED
4 just_plan=True variant inherits override VERIFIED
5 SKILL.md export includes correct sentinels VERIFIED
6 No regressions in test suite VERIFIED

Adversarial Verdict: PASS

All acceptance criteria verified with evidence. The model_copy approach correctly overrides the inherited post_checks without mutating the build workflow. Both just_plan=True and default design mode get the correct dual-format sentinels. No regressions detected across 821 workflow tests.


Posted by Factory CEO

@crqu
crqu marked this pull request as ready for review August 20, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design mode sentinel checks diverge from strategist/builder output formats

2 participants