Skip to content

nix-prefetch: fix prefetching when using the hash key - #190735

Merged
mweinelt merged 1 commit into
NixOS:masterfrom
winterqt:nix-prefetch-hash-key
Sep 11, 2022
Merged

nix-prefetch: fix prefetching when using the hash key#190735
mweinelt merged 1 commit into
NixOS:masterfrom
winterqt:nix-prefetch-hash-key

Conversation

@winterqt

Copy link
Copy Markdown
Member
Description of changes

Backporting msteen/nix-prefetch#43 since it seems that the author/maintainer is MIA, and this is somewhat important.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@ofborg
ofborg Bot requested a review from msteen September 11, 2022 01:31
@ofborg ofborg Bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Sep 11, 2022
@mweinelt

Copy link
Copy Markdown
Member

Thanks, works for me as well.

@mweinelt
mweinelt merged commit dc53ae8 into NixOS:master Sep 11, 2022
@winterqt
winterqt deleted the nix-prefetch-hash-key branch September 11, 2022 01:31
@zowoq

zowoq commented Sep 11, 2022

Copy link
Copy Markdown
Contributor
❯ nix-update gdu
$ nix eval --json --impure --extra-experimental-features nix-command --expr (
    let
      inputs = (if (builtins.hasAttr "overlays" (builtins.functionArgs (import ./.))) then { overlays = []; } else { });
    in
    with import ./. inputs;
    let
      pkg = gdu;
      raw_version_position = builtins.unsafeGetAttrPos "version" pkg;

      position = if pkg ? isRubyGem then
        raw_version_position
      else
        builtins.unsafeGetAttrPos "src" pkg;
    in {
      name = pkg.name;
      old_version = (builtins.parseDrvName pkg.name).version;
      inherit raw_version_position;
      filename = position.file;
      line = position.line;
      urls = pkg.src.urls or null;
      url = pkg.src.url or null;
      rev = pkg.src.url.rev or null;
      hash = pkg.src.outputHash or null;
      mod_sha256 = pkg.modSha256 or null;
      vendor_sha256 = pkg.vendorSha256 or null;
      cargo_sha256 = pkg.cargoHash or pkg.cargoSha256 or null;
      tests = builtins.attrNames (pkg.passthru.tests or {});
    })
fetch https://github.com/dundee/gdu/releases.atom
Update 5.16.0 -> 5.17.0 in /Users/zowoq/src/nixpkgs/pkgs/tools/system/gdu/default.nix
$ nix-prefetch (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).gdu
The package gdu-5.17.0 will be fetched as follows:
> fetchFromGitHub {
>   hash = "sha256-e2fZJ1HRoi2uMyCWgjPIXs6Q6neg8XEqtC8YcUev+60=";
>   owner = "dundee";
>   repo = "gdu";
>   rev = "v5.17.0";
> }

$ nix-prefetch { sha256 }: (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).gdu.go-modules.overrideAttrs (_: { vendorSha256 = sha256; })
error: anonymous function at (string):5:140 called without required argument 'sha256'

       at /nix/store/awg2xzkv7j8brs1r6ibvgzj7wlj75b5b-nix-prefetch-0.4.1/lib/nix-prefetch/prefetcher.nix:77:21:

           76|       else throw "The --fetchurl option does not work with builtin fetchers.";
           77|     in f // { drv = f f.args; };
             |                     ^
           78|
(use '--show-trace' to show detailed location information)
Traceback (most recent call last):
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/bin/.nix-update-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/__init__.py", line 224, in main
    package = update(options)
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 154, in update
    update_go_vendor_hash(opts, package.filename, package.vendor_sha256)
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 102, in update_go_vendor_hash
    target_hash = nix_prefetch([expr])
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 77, in nix_prefetch
    res = run(["nix-prefetch"] + cmd, extra_env=extra_env)
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/utils.py", line 35, in run
    return subprocess.run(
  File "/nix/store/c4i6xn02gfljkh7n84f0qm2nbk3a5bd7-python3-3.10.6/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix-prefetch', '{ sha256 }: (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).gdu.go-modules.overrideAttrs (_: { vendorSha256 = sha256; })']' returned non-zero exit status 1.

@winterqt

winterqt commented Sep 11, 2022

Copy link
Copy Markdown
Member Author

@zowoq Can you try with msteen/nix-prefetch@fbe879e applied instead? This fixes prefetching that expression, but I want to make sure I didn't break anything else in the process. (I believe) this fixes the original issue properly, as opposed to the original patch that just happened to work in the cases hexa and I tested. Thanks!

@zowoq

zowoq commented Sep 11, 2022

Copy link
Copy Markdown
Contributor

Previous one is fixed but found another:

❯ nix-update less --version 600
$ nix eval --json --impure --extra-experimental-features nix-command --expr (
    let
      inputs = (if (builtins.hasAttr "overlays" (builtins.functionArgs (import ./.))) then { overlays = []; } else { });
    in
    with import ./. inputs;
    let
      pkg = less;
      raw_version_position = builtins.unsafeGetAttrPos "version" pkg;

      position = if pkg ? isRubyGem then
        raw_version_position
      else
        builtins.unsafeGetAttrPos "src" pkg;
    in {
      name = pkg.name;
      old_version = (builtins.parseDrvName pkg.name).version;
      inherit raw_version_position;
      filename = position.file;
      line = position.line;
      urls = pkg.src.urls or null;
      url = pkg.src.url or null;
      rev = pkg.src.url.rev or null;
      hash = pkg.src.outputHash or null;
      mod_sha256 = pkg.modSha256 or null;
      vendor_sha256 = pkg.vendorSha256 or null;
      cargo_sha256 = pkg.cargoHash or pkg.cargoSha256 or null;
      tests = builtins.attrNames (pkg.passthru.tests or {});
    })
Update 608 -> 600 in /Users/zowoq/src/nixpkgs/pkgs/tools/misc/less/default.nix
$ nix-prefetch (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).less
error: hash '02f2d9d6hyf03va28ip620gjc6rf4aikmdyk47h7frqj18pbx6m6' does not include a type, nor is the type otherwise known from context
(use '--show-trace' to show detailed location information)
Traceback (most recent call last):
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/bin/.nix-update-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/__init__.py", line 224, in main
    package = update(options)
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 149, in update
    update_src_hash(opts, package.filename, package.hash)
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 90, in update_src_hash
    target_hash = nix_prefetch([expr])
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 77, in nix_prefetch
    res = run(["nix-prefetch"] + cmd, extra_env=extra_env)
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/utils.py", line 35, in run
    return subprocess.run(
  File "/nix/store/c4i6xn02gfljkh7n84f0qm2nbk3a5bd7-python3-3.10.6/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix-prefetch', '(import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).less']' returned non-zero exit status 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants