[feat] Filter SimpleStorage nodes by custom resource - #151
Conversation
Allow SimpleStorage to restrict hard round-robin placement to alive Ray nodes with a positive custom resource while preserving the all-alive-node default when the option is unset. Closes Ascend#150 Signed-off-by: nataliekung <4054699+nataliekung@users.noreply.github.com>
|
Could a maintainer please approve the pending workflows for this fork PR? The focused local suite passed (6 tests), and an independent review found no blockers. |
| # Recommended: >= 2 x number of nodes so each node hosts multiple units. | ||
| num_data_storage_units: 16 | ||
| # Optional Ray custom resource required on storage nodes. null keeps all alive nodes eligible. | ||
| node_resource: null |
There was a problem hiding this comment.
I suggest we rename the config as required_node_resource
There was a problem hiding this comment.
Renamed it to required_node_resource throughout the config, implementation, error message, and tests. Thanks for the suggestion.
There was a problem hiding this comment.
Renamed it to required_node_resource throughout the config, implementation, error message, and tests. Thanks for the suggestion.
Signed-off-by: nataliekung <4054699+nataliekung@users.noreply.github.com>
|
I have no further comments. You can turn the PR as ready for review if you have no more further changes. |
CLA Signature Guide@nataliekung , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
1 similar comment
CLA Signature Guide@nataliekung , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
3 participants
Summary
backend.SimpleStorage.node_resourceplacement filtering.ValueErrorwhen none match.Compatibility
node_resourcedefaults tonull, so existing configurations and placement behavior are unchanged. The filter only affectsSimpleStorage; other storage backends are unchanged.Closes #150
Tests
RAY_ADDRESS=local uv run python -m pytest -q tests/test_simple_storage_scheduling.py tests/test_simple_storage_unit.py tests/test_async_simple_storage_manager.py tests/e2e/test_e2e_lifecycle_consistency.py tests/e2e/test_kv_interface_e2e.py(109 passed, 2 skipped)uv run pre-commit run --all-files --show-diff-on-failure --color=neveruv run python -m compileall -q transfer_queue tutorial tests