Pin generate-homeval workflow Python dependencies to lockfile#158
Merged
jeancochrane merged 2 commits intoMar 25, 2026
Merged
Conversation
jeancochrane
commented
Mar 25, 2026
|
|
||
| env: | ||
| PYTHONUNBUFFERED: "1" | ||
| UV_SYSTEM_PYTHON: 1 |
Member
Author
There was a problem hiding this comment.
We don't need to set UV_SYSTEM_PYTHON anymore because we're going to use uv run to run our Python scripts. uv run uses a virtual environment, so it doesn't need to install packages into the system site-packages.
|
|
||
| - name: Install Python requirements | ||
| run: uv pip install . | ||
| run: uv sync --frozen |
Member
Author
There was a problem hiding this comment.
The --frozen option will ensure that uv does not try to look for more recent versions, and instead installs the exact versions listed in the lockfile (source).
wagnerlmichael
approved these changes
Mar 25, 2026
wagnerlmichael
left a comment
wagnerlmichael
left a comment
Member
There was a problem hiding this comment.
Nice job pinpointing the probable package env cause
jeancochrane
deleted the
jeancochrane/pin-workflow-python-deps-to-lockfile
branch
This was referenced May 14, 2026
jeancochrane
added a commit
that referenced
this pull request
May 14, 2026
In #158 we tweaked the `generate-homeval` workflow to ensure it installs the exact Python package versions listed in `scripts/generate_homeval/uv.lock`, but we didn't touch the setup instructions in the README, so those instructions continue to list installation commands that will ignore the lockfile. This PR tweaks those instructions to make sure the commands install the package versions listed in the uv lockfile. I tested this change by removing my virtualenv, following the new instructions to recreate it, and confirming that A) the package versions match the lockfile and B) a `generate_homeval.py` call continues to work: ```bash cd scripts/generate_homeval rm -rf .venv uv sync --frozen source .venv/bin/activate cd ../.. python3 scripts/generate_homeval/generate_homeval.py --run-id 2025-06-14-flamboyant-rob --pin 01011000040000 10112040080000 ```
jeancochrane
added a commit
to ccao-data/model-res-avm
that referenced
this pull request
May 20, 2026
… lockfile (#499) As we found in ccao-data/homeval#158, the way we use `uv pip install` in the `test` workflow in this repo will cause uv to ignore `python/uv.lock` when installing dependencies. Instead, uv will decide which dependency versions to install based on the contents of `python/pyproject.toml`, but the dependencies in that config file are not locked to specific versions and hashes like they are in `python/uv.lock`. This PR updates the `test` workflow that runs our Python tests to use uv commands that will ensure we use the exact versions of packages pinned in `python/uv.lock`. We don't need to update the modeling pipeline itself to use different commands because we use the reticulate uv integration to manage Python dependencies for us. This code has a bit more background: https://github.com/ccao-data/model-res-avm/blob/3cbee466261aadb6350f067685193060b73d3324/pipeline/04-interpret.R#L12-L25 Unfortunately, this means that reticulate won't use `python/uv.lock` either. However, this is [the recommended way of using the reticulate uv integration](https://posit.co/blog/reticulate-1-41), and [`py_require` doesn't include an option to respect a lockfile](https://rstudio.github.io/reticulate/reference/py_require.html), so we would need to roll our own solution to start using the lockfile. I think it's better to use the first-party reticulate uv integration and accept the possibility of different package patch versions than it is to roll our own solution and lock down exact package versions, but I'm open to pushback on that decision. We don't need a corresponding condo PR for this res PR because the condo PR doesn't have any Python code, so it doesn't use uv.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It turns out the command we use to install Python dependencies in the
generate-homevalworkflow (uv pip install .) will ignore auv.lockfile and install dependency versions as listed inpyproject.tomlinstead. This is causing the workflow to fail due to a breaking change in a recent pandas version.I was able to get the workflow working again by switching the install step to use
uv syncanduv runso that uv will use the exact dependency versions listed inuv.lock.These changes led to a successful staging build whose logs you can inspect here. The changes might be confusing if you're not used to working with uv, so I'm happy to talk them through out loud if it would be helpful.