Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] singular: run autoreconfHook #34724

Closed
wants to merge 3 commits into from
Closed

[WIP] singular: run autoreconfHook #34724

wants to merge 3 commits into from

Conversation

teto
Copy link
Member

@teto teto commented Feb 8, 2018

Motivation for this change

I was trying to install sage on nixos-unstable but the current singular package complains if you use a different version of automake:

aclocal.m4:70: AM_PROG_AR is expanded from...
configure.ac:25: the top level
configure.ac:23: error: version mismatch.  This is Automake 1.15.1,
configure.ac:23: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:23: comes from Automake 1.15.  You should recreate
configure.ac:23: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.15' is probably too old.
         You should only need it if you modified 'Makefile.am' or
         'configure.ac' or m4 files included by 'configure.ac'.
         The 'automake' program is part of the GNU Automake package:
         <http://www.gnu.org/software/automake>
         It also requires GNU Autoconf, GNU m4 and Perl in order to run:
         <http://www.gnu.org/software/autoconf>
         <http://www.gnu.org/software/m4/>
         <http://www.perl.org/>

run autoreconfHook allows for more flexibility.

Using it ends up with this problem currently.

prompt> nix-build -A singular ~/nixpkgs                                                                             
these derivations will be built:
  /nix/store/x25q7qjygyyl02w1p7lz3agp52m98c4x-singular-3-1-7.drv
building path(s) ‘/nix/store/vjsfmv8a7a2xfa2dfmlnl94lcqnwiv4i-singular-3-1-7’
unpacking sources
unpacking source archive /nix/store/rafrq97i4wmpnks005jb1dnsifhrjk8s-Singular-3-1-7.tar.gz
source root is Singular-3-1-7
setting SOURCE_DATE_EPOCH to timestamp 1407340755 of file Singular-3-1-7/warn_not_found.sh
patching sources
autoreconfPhase
autoreconf: Entering directory `.'
autoreconf: configure.in: not using Gettext
autoreconf: running: aclocal --force 
aclocal: warning: autoconf input should be named 'configure.ac', not 'configure.in'
configure.in:425: error: m4_defn: undefined macro: _AC_LANG
../../lib/autoconf/lang.m4:107: AC_LANG_POP is expanded from...
../../lib/autoconf/lang.m4:134: AC_LANG_RESTORE is expanded from...
configure.in:425: the top level

Should I propose to rename the file upstream @raskin ?

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@teto
Copy link
Member Author

teto commented Feb 8, 2018

@timokau I just realized that the nix singular package was in fact not used by sage.
Also you wrote in sage/default.nix:

# - consider writing a script to convert spkgs to nix packages, similar to vim
#   or cabal2nix. This would allow a more efficient and "cleaner" build, greater
#   flexibility and the possibility to select which dependencies to add and which
#   to remove. It would also allow to use system packages for some dependencies
#   and recompile others (optimized for the system) without recompiling everything.

Is there a way to do it manually, i.e. replace a single spkg by a system one ?

@timokau
Copy link
Member

timokau commented Feb 8, 2018

I'm not sure how to tell sage to use a system package. I haven't tried because most packages are patched and I think even if we could manually convert all to system packages, it wouldn't be maintainable.

I could imagine simply adding singular as a buildInput could work.

Is there a reason you want to use system singular? And does the build failure here occur when you try to build sage or the nixpkgs singular?

@teto
Copy link
Member Author

teto commented Feb 8, 2018

In this case mostly because it would allow me to fix the autoreconf incompatibility. It might be hard to maintain sage with system packages without package pinning but having the possibility to plug some in manually to fix this kind of bug would be nice. Also because right now a nix-build will compile everything from scratch, that's crazy. The alternative of patching sage even more doesn't look sexy.
When looking at this doc, one can specify ver yfew out of tree packages apparently (grep for SAGE_MP_LIBRARY) http://doc.sagemath.org/html/en/installation/source.html#make-targets

@timokau
Copy link
Member

timokau commented Feb 8, 2018

In this case mostly because it would allow me to fix the autoreconf incompatibility.

How does that incompatibility affect sage? I'm not sure if I understand the problem.

It might be hard to maintain sage with system packages without package pinning but having the possibility to plug some in manually to fix this kind of bug would be nice.

I agree, that would be nice.

Also because right now a nix-build will compile everything from scratch, that's crazy. The alternative of patching sage even more doesn't look sexy.

Yes, thats annoying. But then again, the sage developers themselves strongly advise against using all system packages: https://wiki.sagemath.org/faq#Wouldn.27t_it_be_way_better_if_Sage_did_not_ship_as_a_gigantic_bundle.3F-1
I think really the best would be to continue using sages package declarations, but build them using nix (through some kind of automatic package generation, as mentioned in that comment).
I probably won't implement that though. If you're interested to do that, I'd be glad to help.

For what its worth, sage should soon be in the binary cache. At the moment its not because of an issue with the nixUnstable hydra uses, which will be fixed as soon as staging is merged into master. See the last few comments of #34387.

When looking at this doc, one can specify ver yfew out of tree packages apparently (grep for SAGE_MP_LIBRARY) http://doc.sagemath.org/html/en/installation/source.html#make-targets

As far as I know, Debian manages to build sage with only system dependencies. If you're interested, thats probably where to look.
Archlinux too, but I'm not sure to which degree they use system dependencies.

@teto
Copy link
Member Author

teto commented Feb 8, 2018

Looking at https://github.com/cschwan/sage-on-gentoo/blob/master/sci-mathematics/sage/sage-8.1-r2.ebuild shows that one needs to patch the sage script (that's baaad).
I am adding a hostSingular ? false parameter to build with host singular.
And a shellHook to export out=/tmp into sage otherwise the nix-shell tries to write in the store.

@timokau
Copy link
Member

timokau commented Feb 8, 2018

Looking at https://github.com/cschwan/sage-on-gentoo/blob/master/sci-mathematics/sage/sage-8.1-r2.ebuild shows that one needs to patch the sage script (that's baaad).

Yeah everything needs patches. I don't understand why sage makes it so much more complicated than other projects, but I'm sure they have reasons.

I am adding a hostSingular ? false parameter to build with host singular.

Cool, thats a good first step for system packages. Good luck!

And a shellHook to export out=/tmp into sage otherwise the nix-shell tries to write in the store.

I'm not sure if that's what you mean, but the build deliberately happens in $out (in the nix store) instead of the standard /tmp build dir, since sage doesn't seperate make and make install.

Edit: I'm still not quite sure if theres some problem with the singular build in sage. Could you clarify that?

@teto
Copy link
Member Author

teto commented Feb 9, 2018

I'm not sure if that's what you mean, but the build deliberately happens in $out (in the nix store) instead of the standard /tmp build dir, since sage doesn't seperate make and make install.
Oh interesting thanks for the tip. how do you debug the package in a nix-shell then ? Is it possible to install in /tmp (so that nix-shell can run as plain user) and just cp -r the whole folder to $out during the installPhase ?

Edit: I'm still not quite sure if theres some problem with the singular build in sage. Could you clarify that?
The latest automake stops when it sees configure.in instead of configure.ac. Thus the singular package should rename it to be compatible with newer automake.

@timokau
Copy link
Member

timokau commented Feb 9, 2018

Oh interesting thanks for the tip. how do you debug the package in a nix-shell then ?

Most of the time I don't, since the whole build process is so messy and hard to keep track of.

Instead I just add debug statements to the build and inspect the log and the build dir afterwards.

If you want to though you can either point $out at a folder you can write to (it will work, you just can't do it for the actual installation) or if you're feeling adventurous temporarily remount the store as rw.

Is it possible to install in /tmp (so that nix-shell can run as plain user) and just cp -r the whole folder to $out during the installPhase ?

Its not that easy, since the build result will have references to the build dir. You'd have to patch all those references afterwards.

The latest automake stops when it sees configure.in instead of configure.ac. Thus the singular package should rename it to be compatible with newer automake.

So checking out the lastest nixos-unstable and nix-building sage doesn't work for you? Because it worked for me the last time I tried. When was that update to automake?

(Side note: Quotes start with > on github)

Current package complains if you use a different version of automake:
aclocal.m4:70: AM_PROG_AR is expanded from...
configure.ac:25: the top level
configure.ac:23: error: version mismatch.  This is Automake 1.15.1,
configure.ac:23: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:23: comes from Automake 1.15.  You should recreate
configure.ac:23: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.15' is probably too old.
         You should only need it if you modified 'Makefile.am' or
         'configure.ac' or m4 files included by 'configure.ac'.
         The 'automake' program is part of the GNU Automake package:
         <http://www.gnu.org/software/automake>
         It also requires GNU Autoconf, GNU m4 and Perl in order to run:
         <http://www.gnu.org/software/autoconf>
         <http://www.gnu.org/software/m4/>
         <http://www.perl.org/>

run autoreconfHook allows for more flexibility.
@teto
Copy link
Member Author

teto commented May 21, 2018

dropping as #39981 seems to cover everything

@teto teto closed this May 21, 2018
@teto teto deleted the singular branch May 13, 2020 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants