Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

init libdmg-hfsplus, use instead of xpwn to fix Linux xcode extraction #34503

Closed
wants to merge 2 commits into from
Closed

init libdmg-hfsplus, use instead of xpwn to fix Linux xcode extraction #34503

wants to merge 2 commits into from

Conversation

dtzWill
Copy link
Member

@dtzWill dtzWill commented Feb 1, 2018

See commit messages.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

TODO:

  • Test on Darwin
  • Compare against current extraction w/Darwin to ensure things are good.
  • Possibly avoid Darwin rebuild by only using these tools on Linux

@dtzWill
libdmg-hfsplus: init at 2016-01-31
e5ddac3 
Using this fork to fix extraction on Linux.
@dtzWill
xcode: use libdmg-hfsplus for extraction instead of xpwn
7906631 
* xpwn is primarily about ...xpwn.  Understandable :).
* libdmg-hfsplus focuses on dmg, hfs, hdutil tools
* Generally same authors, at least in terms of "upstream".

The tools are the same, just nothing else included.

Motivation for this is to allow unpacking xcode on Linux,
instead of the current behavior of crashing.

There are multiple forks of this but this is the only
one that works successfully on Linux.

Also, specify tools as nativeBuildInputs.
@matthewbauer
Copy link
Member

If this doesn't work out, you should also try undmg which I think can do the same thing.

@dtzWill
Copy link
Member Author

dtzWill commented Feb 1, 2018

Okay, thanks! Do you know if it preserves permissions + symlinks? p7zip can extract but loses permissions :(.

@matthewbauer
Copy link
Member

I believe so but it's been a while since I've looked through it. It's based on the XPWN sources though (simplified though to make it like unzip) so it should be fairly close to the same behavior.

@dtzWill
Copy link
Member Author

dtzWill commented Feb 9, 2018 •
edited

I mistakenly believed these were part of Darwin stdenv, but that appears to not be the case. Phew :).

Hmm I suppose I can't ask Borg to test extraction on Darwin since it'll need the xcode dmg (non-free, and must be manually acquired). Boo. :)

@dtzWill
Copy link
Member Author

dtzWill commented Feb 9, 2018

@GrahamcOfBorg build libdmg-hfsplus

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Partial log (click to expand)

Install the project...
-- Install configuration: "Release"
-- Installing: /nix/store/75vpizdb95j716kp2ffci9linhdvm0w7-libdmg-hfsplus-2016-01-31/bin/dmg
-- Installing: /nix/store/75vpizdb95j716kp2ffci9linhdvm0w7-libdmg-hfsplus-2016-01-31/bin/hdutil
-- Installing: /nix/store/75vpizdb95j716kp2ffci9linhdvm0w7-libdmg-hfsplus-2016-01-31/bin/hfsplus
post-installation fixup
strip is /nix/store/5a88zk3jgimdmzg8rfhvm93kxib3njf9-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/75vpizdb95j716kp2ffci9linhdvm0w7-libdmg-hfsplus-2016-01-31/bin
patching script interpreter paths in /nix/store/75vpizdb95j716kp2ffci9linhdvm0w7-libdmg-hfsplus-2016-01-31
/nix/store/75vpizdb95j716kp2ffci9linhdvm0w7-libdmg-hfsplus-2016-01-31

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31
shrinking /nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31/bin/hfsplus
shrinking /nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31/bin/dmg
shrinking /nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31/bin/hdutil
strip is /nix/store/5qj61lcvzlap87rf6blvf8p577d482bv-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31/bin 
patching script interpreter paths in /nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31
checking for references to /tmp/nix-build-libdmg-hfsplus-2016-01-31.drv-0 in /nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31...
/nix/store/mzshr8xjkkgrby3g58nn38difcfzqi0p-libdmg-hfsplus-2016-01-31

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31
shrinking /nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31/bin/hfsplus
shrinking /nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31/bin/hdutil
shrinking /nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31/bin/dmg
strip is /nix/store/xmpjypwjmp2qi1chs5kr0hacnh161ls4-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31/bin
patching script interpreter paths in /nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31
checking for references to /build in /nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31...
/nix/store/qyzzpr2571pfjm6naxkka4wmk3q7lfdq-libdmg-hfsplus-2016-01-31

@dtzWill
Copy link
Member Author

dtzWill commented Feb 19, 2018

So much success! :)

@dtzWill dtzWill mentioned this pull request Mar 22, 2018
Closed
@dtzWill
Copy link
Member Author

dtzWill commented Mar 22, 2018

Would it make sense to bump xcode version, as part of #37403? (perhaps as a follow-up, but along with?)

@matthewbauer
Copy link
Member

matthewbauer commented Mar 22, 2018 •
edited

Would it make sense to bump xcode version, as part of #37403?

I'm having more difficulties than expected with that. We ended up having to modify "stdenv" via an update to Libsystem. Since Xcode is part of the old "impure" Apple stuff, it can probably be updated separately.

@dtzWill
Copy link
Member Author

dtzWill commented Nov 2, 2018

Does this seem useful still, or should it be retired?

@dtzWill
Copy link
Member Author

dtzWill commented Nov 21, 2018

Think we have our answer :). Can revive if seems useful in the future!

@dtzWill dtzWill closed this Nov 21, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewbauer matthewbauer matthewbauer approved these changes

Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@dtzWill @matthewbauer @GrahamcOfBorg