Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

now-cli: init at 11.4.6 #48193

Merged
merged 2 commits into from Oct 13, 2018
Merged

now-cli: init at 11.4.6 #48193

merged 2 commits into from Oct 13, 2018

Conversation

brendan-hall
Copy link
Contributor

Motivation for this change

Adds Zeit's now-cli, per #36395 (sorry @giodamelio)
Uses the binary distribution, which presented some challenges as it's essentially an archive of a nodejs, and all the js files, with hardcoded offsets to said files.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@brendan-hall
Copy link
Contributor Author

I was quite surprised to find that genericBuild can't handle plain .gz files.
Have I written the unpack correctly? I couldn't find anything stating how to leave the downloaded files or the newly unpacked.

@grahamc
Copy link
Member

grahamc commented Oct 11, 2018

@GrahamcOfBorg build now-cli

@GrahamcOfBorg
Copy link

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: now-cli

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: now-cli

Partial log (click to expand)

8+0 records out
8 bytes copied, 4.8854e-05 s, 164 kB/s
8+0 records in
8+0 records out
8 bytes copied, 3.9593e-05 s, 202 kB/s
shrinking RPATHs of ELF executables and libraries in /nix/store/s8jy89dbd5pxqxzmqr7ldjbdm6mbh2ng-now-cli-11.4.6
shrinking /nix/store/s8jy89dbd5pxqxzmqr7ldjbdm6mbh2ng-now-cli-11.4.6/bin/now
patching script interpreter paths in /nix/store/s8jy89dbd5pxqxzmqr7ldjbdm6mbh2ng-now-cli-11.4.6
checking for references to /build in /nix/store/s8jy89dbd5pxqxzmqr7ldjbdm6mbh2ng-now-cli-11.4.6...
/nix/store/s8jy89dbd5pxqxzmqr7ldjbdm6mbh2ng-now-cli-11.4.6

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: now-cli

Partial log (click to expand)

8+0 records out
8 bytes copied, 6.638e-05 s, 121 kB/s
8+0 records in
8+0 records out
8 bytes copied, 5.774e-05 s, 139 kB/s
shrinking RPATHs of ELF executables and libraries in /nix/store/7c67q0zg3qcsf9ssg4sf2a5696s3f3bb-now-cli-11.4.6
shrinking /nix/store/7c67q0zg3qcsf9ssg4sf2a5696s3f3bb-now-cli-11.4.6/bin/now
patching script interpreter paths in /nix/store/7c67q0zg3qcsf9ssg4sf2a5696s3f3bb-now-cli-11.4.6
checking for references to /build in /nix/store/7c67q0zg3qcsf9ssg4sf2a5696s3f3bb-now-cli-11.4.6...
/nix/store/7c67q0zg3qcsf9ssg4sf2a5696s3f3bb-now-cli-11.4.6

PRELUDE=$(expr 4096 + $PRELUDE)

echo -n $PAYLOAD | dd of=$out/bin/now bs=1 seek=$PAYLOAD_offset conv=notrunc
echo -n $PRELUDE | dd of=$out/bin/now bs=1 seek=$PRELUDE_offset conv=notrunc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use a function to abstract these two patches


#TODO: calcuate required adjustment?
PAYLOAD=$(expr 4096 + $PAYLOAD)
PRELUDE=$(expr 4096 + $PRELUDE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of these hardcoded numbers, how resilient are these for future updates? Also how did you get these numbers? Would be worth a short comment on that (the 20, 22 and 4096).

I guess it's fine, as long as it runs, if people really want this package.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found these numbers by experimentation, and looking at the file with xxd.

I did however just notice patchelf --debug which gives the following output:

$ patchelf --debug --set-interpreter /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/ld-linux-x86-64.so.2 ./now-test
patching ELF file `./now-test'
Kernel page size is 4096 bytes
replacing section `.interp' with size 80
this is an executable
using replaced section `.interp'
last replaced is 1
looking at section `.interp'
first reserved offset/addr is 0x254/0x400254
first page is 0x400000
needed space is 648
needed space is 704
needed pages is 1
changing alignment of program header 2 from 2097152 to 4096
changing alignment of program header 3 from 2097152 to 4096
clearing first 4068 bytes
rewriting section `.interp' from offset 0x1238 (size 28) to offset 0x270 (size 80)
rewriting symbol table section 5
rewriting symbol table section 38

Does that give any insight as to how safe the patch is?

# with zeit/pkg to be run on nixos.

preFixup = let
libPath = lib.makeLibraryPath [stdenv.cc.cc.lib];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makeLibraryPath already takes the .lib output if it exists, so you can remove that.

@@ -0,0 +1,68 @@
{ stdenv, lib, makeWrapper, fetchurl }:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makeWrapper is unused

@brendan-hall
Copy link
Contributor Author

@infinisil RE Your review: How's that?

# [...]
# PRELUDE_POSITION = '1234 ' | 0
# ^-----20-chars-----^^------22-chars------^
# ^-- grep points here
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's been a while but here is an alternative solution, maybe its useful

perl -pe 's/(var PAYLOAD_POSITION = .)(\d+)/$1 . ($2 + 4096)/e; s/(var PRELUDE_POSITION = .)(\\d+)/$1 . ($2 + 4096)/e;' < tailwind-freebsd-x64 > tailwind-damien

This relies on rpath injection causing a 4096 byte extension obviously.

works on FreeBSD.

Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing left to do for the future is to add macOS support too. This is looking good as it is now. when you squash the last commit I'll merge this.

@Hinidu
Copy link
Contributor

Hinidu commented Oct 12, 2018

Guys this is awesome - you've just saved my weekend!
I had exactly the same problem with packaging new version of Unity #34399
I've just copied your zeit/pkg fix and it finally works now!
It can't be a coincidence 😄

@brendan-hall
Copy link
Contributor Author

brendan-hall commented Oct 12, 2018

Unfortunately, I don't have a Mac. How much of the existing work can be shared with a macOS system? (I know that Zeit provide a macOS binary explicitly)

@Hinidu I'm lightly horrified that Unity has the same issues as package did.

@infinisil infinisil merged commit 9179c14 into NixOS:master Oct 13, 2018
@Hinidu
Copy link
Contributor

Hinidu commented Oct 16, 2018

@brendan-hall they use zeit/pkg for Unity Package Manager since version 2017.4 I suppose. And the editor can't launch without it.

@Hinidu Hinidu mentioned this pull request Oct 17, 2018
9 tasks
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/unrelatable-error-when-working-with-patchelf/12043/3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants