utils.disk: Added comprehensive disk cleanup utilities#6312
utils.disk: Added comprehensive disk cleanup utilities#6312maramsmurthy wants to merge 1 commit into
Conversation
|
Verified these changes by adding it to tests in a CR flow TestSuite TestRun Summary host_io_disk_cleanup_bonnie_bonnie Run Successfully executed host_io_disk_cleanup_rawread_rawread Run Successfully executed host_io_disk_cleanup_tiobench_tiobench Run Successfully executed host_io_disk_cleanup_disktest_disktest Run Successfully executed host_io_disk_cleanup_fiotest_fio Run Successfully executed host_io_disk_cleanup_softwareraid_softwareraid Run Successfully executed host_io_disk_cleanup_ltp_fsstress_ltp_fsstress Run Successfully executed host_io_disk_cleanup_ltp_fs_ltp_fs Run Successfully executed host_io_disk_cleanup_parallel_dd_parallel_dd Run Successfully executed host_io_disk_cleanup_fs_mark_fs_mark Run Successfully executed host_io_disk_cleanup_ioping_ioping Run Successfully executed host_io_disk_cleanup_iozone_iozone Run Successfully executed host_io_disk_cleanup_lvsetup_lvsetup Run Successfully executed No issues observed during cleanup and no breakage to the CR flow. |
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive disk management and cleanup utilities in avocado/utils/disk.py, including multipath normalization, dependency resolution, LVM/RAID teardown, and metadata wiping. The review feedback highlights several critical issues, including incorrect kernel name mapping for RAID slaves, dangerous substring matching during device unmounting, ordering bugs in LVM and RAID cleanup sequences, broken retry logic in metadata wiping due to stderr redirection, and potential multi-line parsing issues when retrieving volume groups.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6312 +/- ##
==========================================
- Coverage 71.69% 70.52% -1.18%
==========================================
Files 206 206
Lines 23480 23922 +442
==========================================
+ Hits 16835 16870 +35
- Misses 6645 7052 +407 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
dfe50bb to
dcffe4e
Compare
This enables consistent disk cleanup across all storage validation tests. Functions added: - cleanup_disks(): Main API for disk cleanup operations - normalize_multipath_devices(): Map devices to multipath - build_device_dependencies(): Build cleanup dependency graph - cleanup_raid_arrays(): Remove software RAID arrays - cleanup_lvm_volumes(): Remove LVM logical volumes - cleanup_lvm_groups(): Remove LVM volume groups - cleanup_lvm_physical_volumes(): Remove LVM physical volumes - unmount_filesystems(): Unmount all filesystems on devices - wipe_disk_metadata(): Clear filesystem and RAID metadata - zero_disk_start(): Zero first 100MB of disks - And 8 additional helper functions Code Quality Improvements (addressing review feedback): Exception Handling: - Replaced bare 'except Exception:' with specific exception types (OSError, ValueError, KeyError, TimeoutError) in 4 locations - Added debug logging for caught exceptions to improve troubleshooting Magic Numbers: - Extracted 20 hardcoded values to module-level constants (timeouts, retry counts, buffer sizes) for better maintainability - Updated 15+ locations to use named constants instead of magic numbers Robustness Improvements: - Fixed variable shadowing in normalize_multipath_devices() - Eliminated TOCTOU race conditions in 3 locations - Added proper error handling for filesystem operations - Improved handling of permission errors and missing paths All improvements maintain zero functional impact for normal operation while significantly improving code robustness and maintainability. Testing: - Validated with 13 test suites in avocado-misc-tests - 203 tests PASS (bonnie, rawread, tiobench, disktest, fiotest, softwareraid, ltp_fsstress, ltp_fs, parallel_dd, fs_mark, ioping, lvsetup) - Zero regressions introduced - Code quality improvements verified with no behavior changes Signed-off-by: Maram Srimannarayana Murthy <msmurthy@linux.vnet.ibm.com>
dcffe4e to
6f8341a
Compare
|
Hi @maramsmurthy is it a good idea to perhaps contribute this directly to https://github.com/avocado-framework/aautils/tree/main/autils? This adds a lot of code that is meant to be migrated to begin with. What do you think? |
Hi @pevogam, given that this change is currently part of our CR flow, we have a dependency on the existing implementation at this point. That said, I’m open to migrating it to aautils in the future if needed. Please let me know as part of the migration plan if you expect any changes in this code, so we can align early and avoid rework later. |
I assume by your projects depending on this implementation you mean that you use a patch or modified package in some way? Because the pull request is not merged in the repo and as such is not deployed with the default packages built here. If this then integrated as a patch or anything modular then perhaps it maybe we adjusted later on to simply patch on top of aautils?
Glad to hear this, I would say at this point the aautils is still fairly unexplored in terms of standardization so I would say bolder changes like this will also be merged less conservatively there (meaning that I don't expect any further changes to be needed unless you could of course provide some unit tests which would be much appreciated). |
Hi @pevogam, Thank you for the detailed feedback and for being open to future migration possibilities. I'd like to proceed with avocado.utils for this PR, as it maintains consistency with the current codebase and avoids introducing dependencies on aautils while it's still in the exploration phase. I completely understand your perspective on standardization. Before we finalize the approach, I'd also like to get @PraveenPenguin opinion on this, especially regarding the long-term direction and whether there are any specific migration considerations we should keep in mind. In the meantime, if you'd like me to make any other improvements to strengthen this implementation, please let me know and I'll be happy to incorporate them. Looking forward to hearing from both of you! |
|
Alright, let's also hear the opinion of @PraveenPenguin on this. Maybe also same for @harvey0100 but if they are too busy and turn around within enough given time we may find another way forward together on this. |
This enables consistent disk cleanup across all storage validation tests.
Functions added: