Skip to content

python-docs.yml: docstring-coverage gate is a silent no-op when interrogate isn't installed #243

Description

@williaby

Summary

The Check docstring coverage step in the reusable python-docs.yml workflow silently does nothing on any downstream repo that does not have interrogate installed in its uv environment. The step looks like an enforced quality gate but never actually measures coverage, so the --fail-under threshold is not enforced anywhere it can't spawn the binary.

Location

.github/workflows/python-docs.yml, build job:

      - name: Check docstring coverage
        env:
          SRC_DIR: ${{ inputs.source-directory }}
          DOCSTRING_THRESHOLD: ${{ inputs.docstring-threshold }}
        run: |
          echo "📝 Checking docstring coverage..."
          uv run interrogate "$SRC_DIR" \
            --fail-under="$DOCSTRING_THRESHOLD" \
            --verbose || echo "⚠️  Docstring coverage below threshold"

Root cause

The step installs deps with uv sync --all-extras (step above it) but interrogate is not guaranteed to be a project dependency. When it is absent, uv run interrogate fails with:

error: Failed to spawn: `interrogate`
  Caused by: No such file or directory (os error 2)

The || echo "⚠️ Docstring coverage below threshold" catch turns that spawn failure into a step success. So there is no distinction between:

  • interrogate ran and coverage was below threshold, and
  • interrogate never ran at all.

Both paths pass silently. The intended --fail-under gate is effectively dead in the second case.

Why this is not just "install interrogate downstream"

At least one consuming repo (ByronWilliamsCPA/homelab-infra) deliberately removed interrogate because it still pulls in the transitive py package (CVE-2022-42969). Verified on PyPI: even the current interrogate 1.7.0 lists py in requires_dist, so re-adding it reintroduces the flagged CVE and would fail that repo's pip-audit gate. Any repo that made the same tradeoff hits this silent no-op.

Impact

  • Repos that removed interrogate for the CVE: the docstring gate is permanently inert, but reads as present/green.
  • The ⚠️ warning is buried in logs and does not surface as a failed check, so the gap is invisible on the PR checks UI.

Suggested remediations (pick one)

  1. Run interrogate in an isolated tool env, decoupled from the project's locked deps and pip-audit surface:
    run: uvx interrogate "$SRC_DIR" --fail-under="$DOCSTRING_THRESHOLD" --verbose \
         || echo "⚠️  Docstring coverage below threshold"
    (uvx/uv tool run fetches interrogate ephemerally; it never lands in the repo's lockfile.)
  2. Gate the step on an input, e.g. enforce-docstrings: false default, so repos that opt out don't run a no-op step at all.
  3. Distinguish "not installed" from "ran below threshold": probe for the binary first and emit a distinct, visible warning (or a hard failure) when the gate cannot run, so a dead gate is never mistaken for a passing one.

Option 1 is the smallest change and also fixes the CVE tension, since interrogate no longer needs to be a project dependency.

How this surfaced

Found while running /pr-review on ByronWilliamsCPA/homelab-infra (the error: Failed to spawn: interrogate line was initially mistaken for the cause of a red Doc Build; it is actually non-fatal). Filed for org-workflow tracking.

Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions