Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apple_sdk: broken link update #73984

Merged
merged 1 commit into from Nov 27, 2019
Merged

Conversation

dredozubov
Copy link
Contributor

@dredozubov dredozubov commented Nov 23, 2019

Motivation for this change

Apple DevSDK for OS X Sierra(10.12) became unavailable recently, so trying to build apple_sdk leads to this:

% nix build nixpkgs.darwin.apple_sdk
builder for '/nix/store/k08c0q6vmv8swrkibqap961bkysfg7g8-DevSDK_OSX1012.pkg.drv' failed with exit code 1; last 7 log lines:

  trying http://swcdn.apple.com/content/downloads/28/09/091-29862/pafhn2u002b9slnrxzy9p86rpedycnjhb5/DevSDK_OSX1012.pkg
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  curl: (22) The requested URL returned error: 404 Not Found
  error: cannot download DevSDK_OSX1012.pkg from any mirror
cannot build derivation '/nix/store/s0advspxkj2nmpw9valj66vxx1cfjh1p-MacOS_SDK-10.12.drv': 1 dependencies couldn't be built
[0 built (1 failed)]
error: build of '/nix/store/s0advspxkj2nmpw9valj66vxx1cfjh1p-MacOS_SDK-10.12.drv' failed
Things done

DevSDK link updated to the one still available. I'm leaving #73744 as a discussion for a possible DevSDK update.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @NixOS/darwin-maintainers


This change is Reviewable

Copy link
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't work for me:

# cat test.nix
with import <nixpkgs> { };

fetchurl {
  url =
    "http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg";
  sha256 = "1sggc70rypqwcjwr7ciavw8sczwll16cwqxdxrbw7r2qvy3b0nhx";
}
# nix-build test.nix --option substituters ""
these derivations will be built:
  /nix/store/x76mzld15zspnyf312g52hmxhqz5hqkg-DevSDK_OSX1012.pkg.drv
building '/nix/store/x76mzld15zspnyf312g52hmxhqz5hqkg-DevSDK_OSX1012.pkg.drv'...

trying http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20.4M  100 20.4M    0     0  11.3M      0  0:00:01  0:00:01 --:--:-- 11.3M
hash mismatch in fixed-output derivation '/nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg':
  wanted: sha256:1sggc70rypqwcjwr7ciavw8sczwll16cwqxdxrbw7r2qvy3b0nhx
  got:    sha256:13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih
error: build of '/nix/store/x76mzld15zspnyf312g52hmxhqz5hqkg-DevSDK_OSX1012.pkg.drv' failed

@LnL7
Copy link
Member

LnL7 commented Nov 23, 2019

Hmm that's odd it does for me.

checking outputs of '/nix/store/hxv0x4fc7yd7vgv1vqji9izjy7950fs7-DevSDK_OSX1012.pkg.drv'...

trying http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20.4M  100 20.4M    0     0   9.9M      0  0:00:02  0:00:02 --:--:--  9.9M
warning: rewriting hashes in '/nix/store/10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg'; cross fingers
/nix/store/10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg

@veprbl
Copy link
Member

veprbl commented Nov 23, 2019

# nix-prefetch-url "http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg"
[20.5 MiB DL]
path is '/nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg'
13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih
# du /nix/store/10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg /nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg
21696   /nix/store/10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg
21008   /nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg

Once I extract, the contents appear to be identical:

# sha256sum *.pkg                                                                                                         :(
1d5ab086df58e4c357eead63ce4ca0947fa611df2ab293b9641c5f9fc161efe9  10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg
30ea9857e79adb7ed03d089015e6cdd72407ed3307780ecf58038db33419b88f  hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg
# for f in *.pkg; do mkdir "$f"-x; pushd "$f"-x; xar -x -f ../$f; popd; done
# ls -R *-x
10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg-x:
Bom  PackageInfo  Payload

hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg-x:
Bom  PackageInfo  Payload
# sha256sum *-x/*
df44eefb3e68a17aad30c45b3e391a996ac7e99525f8f62d0b78106061fde1f7  10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg-x/Bom
f04d0ee0bedbfbf84c7069251fe4d74db41a92b12e64e5e0b5dbbb10d06977f4  10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg-x/PackageInfo
384e7085144eecc9c9977811f8dbd128ecf596275f87d0e6c641c0153495425e  10336122d689dkkbs2qax503zf0s7jfq-DevSDK_OSX1012.pkg-x/Payload
df44eefb3e68a17aad30c45b3e391a996ac7e99525f8f62d0b78106061fde1f7  hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg-x/Bom
f04d0ee0bedbfbf84c7069251fe4d74db41a92b12e64e5e0b5dbbb10d06977f4  hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg-x/PackageInfo
384e7085144eecc9c9977811f8dbd128ecf596275f87d0e6c641c0153495425e  hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg-x/Payload

@veprbl
Copy link
Member

veprbl commented Nov 23, 2019

Should we run xar in postFetch?

@veprbl
Copy link
Member

veprbl commented Nov 23, 2019

Running nix-prefetch-url on machines in the other parts of the world I still get

# nix-prefetch-url "http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg"
[20.5 MiB DL]
path is '/nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg'
13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih
# nix-prefetch-url "http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg"
path is '/nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg'
13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih

@LnL7
Copy link
Member

LnL7 commented Nov 23, 2019

Are you sure you reproduced it correctly? This is exactly what happens in the stdenv.

$ nix-build '<nix/fetchurl.nix>' --argstr url "http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg" --argstr sha256 0000000000000000000000000000000000000000000000000000 --option substituters ''
these derivations will be built:
  /nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv
building '/nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv'...
hash mismatch in fixed-output derivation '/nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg':
  wanted: sha256:0000000000000000000000000000000000000000000000000000
  got:    sha256:13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih
error: build of '/nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv' failed

@veprbl
Copy link
Member

veprbl commented Nov 23, 2019

Are you sure you reproduced it correctly? This is exactly what happens in the stdenv.

$ nix-build '<nix/fetchurl.nix>' --argstr url "http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg" --argstr sha256 0000000000000000000000000000000000000000000000000000 --option substituters ''
these derivations will be built:
  /nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv
building '/nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv'...
hash mismatch in fixed-output derivation '/nix/store/hxgqrabm865wl8nglyp6cw2g858nf0b2-DevSDK_OSX1012.pkg':
  wanted: sha256:0000000000000000000000000000000000000000000000000000
  got:    sha256:13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih
error: build of '/nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv' failed

This agrees with what I observe:

  wanted: sha256:1sggc70rypqwcjwr7ciavw8sczwll16cwqxdxrbw7r2qvy3b0nhx
  got:    sha256:13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih

@LnL7
Copy link
Member

LnL7 commented Nov 23, 2019

Oh I see. Perhaps I didn't test correctly yesterday, we can change the hash and target staging instead.

@dredozubov
Copy link
Contributor Author

dredozubov commented Nov 24, 2019

Weird, it worked for me. I was able to build rustc and darwin.apple_sdk yesterday with nix build nixpkgs.rustc -I nixpkgs=. with this branch. At the same time I can see the sha256 mismatch with 13xq34sb7383b37hwy076gnhf96prpk1b4087p87xnwswxbrisih if I run it today. I don't know how is that possible, but I'll amend the branch to reflect that nevertheless.

On the side note, if I run nix-build '<nix/fetchurl.nix>' --argstr url "http://swcdn.apple.com/content/downloads/33/36/041-90419-A_7JJ4H9ZHO2/xs88ob5wjz6riz7g6764twblnvksusg4ps/DevSDK_OSX1012.pkg" --argstr sha256 0000000000000000000000000000000000000000000000000000 --option substituters '', I get:

these derivations will be built:
warning: Nix search path entry '/Users/dr/.nix-defexpr/channels' does not exist, ignoring
these derivations will be built:
  /nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv
building '/nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv'...
objc[24910]: +[__NSPlaceholderArray initialize] may have been in progress in another thread when fork() was called.
objc[24910]: +[__NSPlaceholderArray initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
builder for '/nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv' failed due to signal 6 (Abort trap: 6)
error: build of '/nix/store/9fsd57072mqsja7sir5a8yvx3djy0dkk-DevSDK_OSX1012.pkg.drv' failed

which I haven't ever seen before.

@veprbl
Copy link
Member

veprbl commented Nov 26, 2019

@dredozubov Could you please rebase to staging?

@veprbl veprbl added this to WIP in Staging via automation Nov 27, 2019
@veprbl veprbl added the 9.needs: port to stable A PR needs a backport to the stable release. label Nov 27, 2019
@veprbl
Copy link
Member

veprbl commented Nov 27, 2019

@GrahamcOfBorg eval

@dredozubov You did exactly what we asked, thank you. The issue with reviewers is a known one. One possible workaround is to first force push a common commit of both master and staging into your PR branch, then change the target branch of your PR on github (at this time your PR will have no changes), then you push current staging + your changes back to your PR branch. This time made me realise that we should really document this procedure.

@veprbl
Copy link
Member

veprbl commented Nov 27, 2019

Found an issue on PR retargeting procedure #48491

Staging automation moved this from WIP to Ready Nov 27, 2019
@veprbl veprbl merged commit 7233afc into NixOS:staging Nov 27, 2019
Staging automation moved this from Ready to Done Nov 27, 2019
@veprbl veprbl self-assigned this Nov 27, 2019
veprbl pushed a commit that referenced this pull request Nov 27, 2019
@veprbl veprbl added 8.has: port to stable A PR already has a backport to the stable release. and removed 9.needs: port to stable A PR needs a backport to the stable release. labels Nov 27, 2019
@veprbl veprbl removed their assignment Jan 7, 2020
rvem pushed a commit to serokell/nixpkgs that referenced this pull request Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants