Lift fallback for PyPI package names to obtain_file

[Flamefire]

Followup to #4943

When the download URLs include a PYPI source and the download filename looks like a pypi source where the package name contains dashes then retry downloading with dashes replaced by underscores.

This is more generic than #4943 as it handles e.g. PythonPackage easyconfigs, not just extensions with defaulted names.

Done in 2 steps for easier checking:

1. Factor out a "wrapper" for download_file as a EasyBlock method from obtain_file without any significant changes
   Small change to dry-run logging as "extensions typically have custom source URLs specified, only mention first" didn't seem to match behavior so removed the comment and add the URL to the initial message instead of a second one
2. Call this again if we have a PYPI URL and at least 2 dashes in the download filename (last is assumed to be the version delimiter)

This is broader than #4943

• Handle not only extensions but any obtain_file calls, especially top-level easyconfig sources
• Trigger when PYPI URL appears anywhere in the URLs, not only in first position
• Trigger when there are at least 2 dashes not only when the extensions source name is defaulted and also for dots and combinations of dashes, dots and underscores. Specs say runs of those should be replaced by single underscore

Issue I had was that at this point we do not know anymore where the (download)filename came from and which template values (for name & version) we should use as extensions temporarily update the easyconfig templates.

Example easyconfig PR that makes use of this: easybuilders/easybuild-easyconfigs#25682

[boegel]

@smoors Are you up for taking a look at this?

No rush, it's a bit too late to get this in for EasyBuild v5.3.0...

[Flamefire]

@smoors can you look into this now that EB 5.3 is released? The changed PyTorch easyconfigs need to be converted to PythonBundle and IIRC the fetch_step isn't run for extensions so the test_step will fail

[smoors]

now that the retry is moved to obtain_file, would be good to add a test? also for the download_file function?

[smoors]

actually, according to the specs, we should also replace '.' with '_'
https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode

not sure if it should already be done in this PR?

[Flamefire]

Done

[Flamefire]

I confused the 2 PRs: The more important one is #5162 to get the test file. But also getting this one ready. Writing the test might be the worst...

[Flamefire]

now that the retry is moved to obtain_file, would be good to add a test? also for the download_file function?

That was tricky to do. But I noticed an unrelated bug (with progress bars) which I fixed in #5162

Used the same approach now here too: unittest.mock to avoid actually downloading anything and being able to check calls.
This actually caught a few cases that were not right: E.g. [.-_] doesn't match - but is easy to overlook.

Rebased to latest develop too.

[Flamefire]

@smoors Do you still want a unit test for EasyBlock.download_file?

It isn't too complicated and the main parts are covered by this and 2 more obtain_file tests.

Missing:

• Alternative URL formats
• Derived PyPI URL usage
• dry-run
• handling IOError raised from ft.download_file