Allow postgresql to cross compile #44083
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dezgeg
reviewed
Jul 25, 2018
| patches = [ ./shtool.patch ]; | ||
|
|
||
| preConfigure = '' | ||
| export ac_cv_va_copy=yes |
There was a problem hiding this comment.
I think conventionally in Nixpkgs, this sort of stuff is not exported but added to configureFlags instead.
There was a problem hiding this comment.
Thats the problem. This configure script does not allow you to fix the test results on va_copy. Please advise a better approach, if any.
There was a problem hiding this comment.
@dezgeg means passing the foo=var directly to configure, which does work for autoconf-based configure scripts.
dezgeg
reviewed
Jul 25, 2018
| @@ -14,8 +14,15 @@ let | |||
| outputs = [ "out" "lib" "doc" "man" ]; | |||
| setOutputFlags = false; # $out retains configureFlags :-/ | |||
|
|
|||
| combinedLibXML2 = symlinkJoin { | |||
There was a problem hiding this comment.
Hmm, why is this needed? Why doesn`t
nativeBuildInputs = [ libxml2 ];
work?
There was a problem hiding this comment.
Possibly, but I just realised that the upstream broke cross-compilation capability, possibly due to #44068 by @Ericson2314. I will close this PR for now.
There was a problem hiding this comment.
@dezgeg I tried. However, the configure of postgresql expects the libxml2 path to contain both its headers and its static libraries.
We would either patch the configure, or just supply the combined libxml2.
There was a problem hiding this comment.
If you pass it as a buildInput, its dev outputs will be passed with the headers, and the -L to the libs added by cc-wrapper will probably leave the configure script unable to notice the difference. If you make it just a nativeBuildInput you will not get the -L.
There was a problem hiding this comment.
@Ericson2314 I just confirmed that if libxml2 is added to buildInputs, the static library is visible to the cross compilers. However, the headers are not picked up by the configure script.
Sorry that I am new to this cross compiling business. Should we modify CFLAGS to point to the headers?
There was a problem hiding this comment.
Currently libxml2 export headers in the include/libxml2 folder in the Nix store, but the CFLAGS is pointing to include only. When configure is resolving #include <libxml/parser.h>, it is looking at the include folder only and this fails the configure checks.
The build works if I point the headers to include/libxml2. The question now is whether libxml2 Nix expression evaluating correctly.
|
Cross compilation is broken. Close for now and will come back later. |
|
Cross compilation is fixed again. |
| @@ -22,14 +22,18 @@ let | |||
|
|
|||
| makeFlags = [ "world" ]; | |||
|
|
|||
| preConfigure = '' | |||
| export CFLAGS="$CFLAGS -I${libxml2.dev}/include/libxml2" | |||
There was a problem hiding this comment.
I think we usually prefer to do this like:
NIX_CFLAGS_COMPILE = "-I${libxml2.dev}/include/libxml2";
Mostly this is because not all build systems recognize CFLAGS.
| ++ lib.optional (!stdenv.isDarwin) "--with-ossp-uuid"; | ||
| ++ lib.optional stdenv.isDarwin "--with-uuid=e2fs" | ||
| ++ lib.optional (!stdenv.isDarwin) "--with-ossp-uuid" | ||
| ++ lib.optional stdenv.isCross "--with-system-tzdata=${tzdata}"; |
There was a problem hiding this comment.
This one you can probably do unconditionally.
Mic92
reviewed
Jul 30, 2018
| @@ -22,14 +22,18 @@ let | |||
|
|
|||
| makeFlags = [ "world" ]; | |||
|
|
|||
| preConfigure = '' | |||
| export NIX_CFLAGS_COMPILE="$NIX_CFLAGS_COMPILE -I${libxml2.dev}/include/libxml2" | |||
There was a problem hiding this comment.
Does postgresql also picks this up, when pkgconfig is included in nativeBuildInputs?
There was a problem hiding this comment.
Now I see why this fails:
https://github.com/postgres/postgres/blob/master/configure.in#L896
It needs xml2-config compiled for the build platform.
There was a problem hiding this comment.
Instead of preConfigure, this can be exported in nix instead:
# needed for cross-compiling, since xml2-config is not compiled for the build platform
NIX_CFLAGS_COMPILE = [ "-I${libxml2.dev}/include/libxml2" ];There was a problem hiding this comment.
The long-term solution would be pkg-config: However this would been an upstream fix: libarchive/libarchive#407
There was a problem hiding this comment.
I see. I will try to raise this issue in postgresql mailing list.
There was a problem hiding this comment.
Actually xml2-config is built correctly but it doesn't get into the PATH since we don't add binaries from dev output to PATH. This is something we should do in stdenv since many other packages will benefit from that as well.
There was a problem hiding this comment.
Should I do it in a separate pull request? Or this one?
There was a problem hiding this comment.
I think for now this is fine.
There was a problem hiding this comment.
Yes I agree it can be done later.
Let's however conditionalize it on cross, so that someone doesn't break it by noticing it native-compiles fine with it.
|
@GrahamcOfBorg build pkgsCross.raspberryPi.postgresql |
|
Success on x86_64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
Failure on x86_64-darwin (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
No attempt on aarch64-linux (full log) The following builds were skipped because they don't evaluate on aarch64-linux: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
No attempt on x86_64-linux (full log) The following builds were skipped because they don't evaluate on x86_64-linux: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
matthewbauer
reviewed
Aug 2, 2018
| patches = [ ./shtool.patch ]; | ||
|
|
||
| configureFlags = [ | ||
| "ac_cv_va_copy=yes" |
There was a problem hiding this comment.
@Ericson2314 Configure flags is also above
Ericson2314
force-pushed
the
cross-compiling-postgresql
branch
from
ccc2357
to
f539c30
Compare
|
@GrahamcOfBorg build pkgsCross.raspberryPi.postgresql |
|
retroactively testing oops (I missed that above, but I also figured at least native builds would continue to work). |
|
Success on x86_64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql The following builds were skipped because they don't evaluate on x86_64-linux: retroactively... Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
Cool :) |
|
Failure on x86_64-darwin (full log) Attempted: pkgsCross.raspberryPi.postgresql The following builds were skipped because they don't evaluate on x86_64-darwin: retroactively... Partial log (click to expand)
|
|
Failure on x86_64-darwin (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql The following builds were skipped because they don't evaluate on aarch64-linux: retroactively... Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
@Ericson2314 this was a mass-rebuild and should have gone to staging. |
|
Sorry @FRidh. I missed that completely. |
9 tasks
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Mar 7, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Mar 7, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Mar 8, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Mar 9, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Mar 16, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Mar 17, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Mar 17, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Apr 3, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Apr 4, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
Currently,
PostgreSQLdoes not cross compile due to not listinglibXML2headers as a native build dependency properly, andlibossp-uuidas a dependency ofPostgreSQLdoes not cross compile because it does not respectstripcommand setting made byconfigure.Things done
sandboxinnix.confon non-NixOS)nix-shell -p nox --run "nox-review wip"./result/bin/)nix path-info -Sbefore and after)