Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chicken: fix cross-compilation #104775

Closed
wants to merge 1 commit into from
Closed

chicken: fix cross-compilation #104775

wants to merge 1 commit into from

Conversation

siraben
Copy link
Member

@siraben siraben commented Nov 24, 2020 •
edited

Motivation for this change
  • nix -build -A pkgs.chicken (build == host == target == x86_64-apple-darwin)
  • nix-build --argstr system 'x86_64-linux' -A pkgs.chicken (build == host == target)
  • nix-build --argstr system 'x86_64-linux' -A pkgs.pkgsCross.remarkable1.chicken (build != host == target)
  • nix-build --argstr system 'x86_64-linux' -A pkgs.pkgsCross.remarkable1.buildPackages.chicken (build == host != target)
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@siraben siraben marked this pull request as ready for review November 24, 2020 15:13
@siraben
Copy link
Member Author

siraben commented Nov 24, 2020

This is why nix-build --argstr system 'x86_64-linux' -A pkgs.chicken fails, this is the defaults.make file in the source.

ifdef HOSTSYSTEM
C_COMPILER ?= $(HOSTSYSTEM)-gcc
CXX_COMPILER ?= $(HOSTSYSTEM)-g++
LIBRARIAN ?= $(HOSTSYSTEM)-ar # problematic line
else
C_COMPILER ?= gcc
CXX_COMPILER ?= g++
LIBRARIAN ?= ar
endif

The cross-compiled gcc is named x86_64-unknown-linux-gnu-gcc-ar, but the script looks for x86_64-unknown-linux-gnu-ar which fails.

@siraben siraben force-pushed the chicken-cross branch 2 times, most recently from 19b1c4b to 53b0d43 Compare November 24, 2020 17:18
@ofborg ofborg bot requested a review from corngood November 24, 2020 19:02
r-burns
r-burns requested changes Nov 25, 2020
View reviewed changes
Copy link
Contributor

@r-burns r-burns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting this error when building (without parallelism) on non-nixos x86_64

x86_64-unknown-linux-gnu-gcc  -shared -Wl,-soname,libchicken.so.11 \
  -o libchicken.so library.o eval.o read-syntax.o repl.o data-structures.o pathname.o port.o file.o extras.o lolevel.o tcp.o srfi-4.o continuation.o posixunix.o internal.o irregex.o scheduler.o debugger-client.o profiler.o stub.o expand.o modules.o chicken-syntax.o chicken-ffi-syntax.o build-version.o runtime.o -lm -ldl
/nix/store/p3kn26g5nhmx6spn37ar3mn0xjbzks3g-binutils-2.31.1/bin/ld: cannot find crti.o: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [rules.make:190: libchicken.so] Error 1

pkgs/development/compilers/chicken/5/chicken.nix Outdated Show resolved Hide resolved
@siraben siraben changed the title chicken: enable cross-compilation chicken: fix cross-compilation Nov 25, 2020
@siraben
Copy link
Member Author

siraben commented Nov 25, 2020

Using the compiler on the target doesn't seem to work,

reMarkable: ~/ /nix/store/j9s0lgiszr2sy8x270hdhgjqsab432ni-chicken-5.2.0-armv7l-unknown-linux-gnueabihf/bin/csc hello.scm                                                            
sh: armv7l-unknown-linux-gnueabihf-armv7l-unknown-linux-gnueabihf-gcc: command not found

Error: shell command terminated with non-zero exit status 32512: 'armv7l-unknown-linux-gnueabihf-armv7l-unknown-linux-gnueabihf-gcc' 'hello.c' -o 'hello.o' -c  -fno-strict-aliasing -fwrapv -DHAVE_CHICKEN_CONFIG_H -DC_ENABLE_PTABLES -Os -fomit-frame-pointer -I/nix/store/j9s0lgiszr2sy8x270hdhgjqsab432ni-chicken-5.2.0-armv7l-unknown-linux-gnueabihf/include/chicken

@siraben
Copy link
Member Author

siraben commented Nov 25, 2020 •
edited

@r-burns I can't replicate that. Could you try again?

EDIT:

x86_64-unknown-linux-gnu-gcc  -shared -Wl,-soname,libchicken.so.11 \
  -o libchicken.so library.o eval.o read-syntax.o repl.o data-structures.o pathname.o port.o file.o extras.o lolevel.o tcp.o srfi-4.o continuation.o posixunix.o internal.o irregex.o scheduler.o debugger-client.o profiler.o stub.o expand.o modules.o chicken-syntax.o chicken-ffi-syntax.o build-version.o runtime.o -lm -ldl
/nix/store/p3kn26g5nhmx6spn37ar3mn0xjbzks3g-binutils-2.31.1/bin/ld: cannot find crti.o: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [rules.make:190: libchicken.so] Error 1
builder for '/nix/store/ymlyvk9b4zpyp3v8gc7g0gqilv2wpg8q-chicken-5.2.0.drv' failed with exit code 2
error: build of '/nix/store/ymlyvk9b4zpyp3v8gc7g0gqilv2wpg8q-chicken-5.2.0.drv' on 'ssh://nix-linuxkit' failed: builder for '/nix/store/ymlyvk9b4zpyp3v8gc7g0gqilv2wpg8q-chicken-5.2.0.drv' failed with exit code 2
builder for '/nix/store/ymlyvk9b4zpyp3v8gc7g0gqilv2wpg8q-chicken-5.2.0.drv' failed with exit code 1
cannot build derivation '/nix/store/vjwxd2qxl2d7cxwb2kla8s9zcs394sfq-chicken-5.2.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/vjwxd2qxl2d7cxwb2kla8s9zcs394sfq-chicken-5.2.0.drv' failed

@siraben
Copy link
Member Author

siraben commented Nov 25, 2020

Relevant section of the docs on cross-compilation

==== Building the target libraries

First you need a version of the runtime system ({{libchicken}}),
compiled for the target system. Obtain and unpack a tarball of the
CHICKEN sources, or check out the code from the official code
repository, then build the libraries and necessary development files:

 make ARCH= \
     PREFIX=/usr \
     PLATFORM=linux \
     HOSTSYSTEM=arm-none-linux-gnueabi \
     DESTDIR=$HOME/target \
     TARGET_FEATURES="-no-feature x86 -feature arm" \
     install

@Ericson2314
Copy link
Member

@siraben
I think we should make a patch that's something like

ifneq TARGETSYSTEM HOSTSYSTEM
TOOL_PREFIX ?= TARGETSYSTEM
else
TOOL_PREFIX ?=
endif

C_COMPILER ?= $(TOOL_PREFIX)gcc
CXX_COMPILER ?= $(TOOL_PREFIX)g++
LIBRARIAN ?= $(TOOL_PREFIX)ar # problematic line

But even that I suspect will not be enough, as the will need to distinguish between the tools used at runtime, and the tools used at build time, if, for example, they want to build libchicken in one go.

How slow is this build? A related good thing we can do is always build libchicken separately. This means moving bootstrapping logic from their build system to ours, which is good.

@veprbl veprbl added the 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on label Dec 3, 2020
@siraben siraben marked this pull request as draft December 7, 2020 17:10
@siraben
Copy link
Member Author

siraben commented Jun 16, 2021

Hey @Mindavi, if you're free, could you also take a look at cross-compilation of CHICKEN as well? Let me know if you have any questions.

@Mindavi
Copy link
Contributor

Mindavi commented Jun 16, 2021

I'm running into this issue:

#116207 / #116504.

(oh, makeWrapper should be moved to nativeBuildInputs, good to know 👍 )

Haven't looked into this much further yet, that's for another day :).

@Artturin Artturin mentioned this pull request Nov 9, 2021
Closed
12 tasks
@Artturin
Copy link
Member

Artturin commented Nov 9, 2021

#145194

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 19, 2022
@ParetoOptimalDev ParetoOptimalDev mentioned this pull request Aug 25, 2022
Closed
@nagy
Copy link
Member

nagy commented Jul 27, 2023

I have implemented this in #245420 . I am going to proclaim, that everything here has been done and close this PR.

@nagy nagy closed this Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@veprbl veprbl veprbl left review comments

@r-burns r-burns r-burns requested changes

@corngood corngood Awaiting requested review from corngood

Assignees
No one assigned
Labels
2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@siraben @Ericson2314 @Mindavi @Artturin @nagy @veprbl @r-burns