idris: Recurse into attrs, fix build and remove unneeded overrides #43444

infinisil · 2018-07-12T23:15:33Z

The gmp issue has long been fixed and closed
The preBuild thing was never used afaik, it works no problem without
it, especially since checks don't run (To get checks to work we'd have
to compile the libraries into idris, which are currently nicely
separated)
The dependencies overrides aren't needed anymore

Motivation for this change

Originally just removing the hacks, but after rebasing on master I discovered that since 69cd09f it wouldn't build at all. This cleanup surprisingly (or not) just happens to fix it :).

In the second commit I also made idris packages to be included in the packages hydra builds and nix-env finds. I hope that's okay, the ~80 idris packages don't change very often and it would be very nice to have them cached because Idris can take a while to build them (and you currently don't even have any progress reporting! Will be fixed with my idris-lang/Idris-dev#4498 in the next Idris version though).

Ping @brainrape @mpickering @peti

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

- The gmp issue has long been fixed and closed - The preBuild thing was never used afaik, it works no problem without it, especially since checks don't run (To get checks to work we'd have to compile the libraries into idris, which are currently nicely separated) - The dependencies overrides aren't needed anymore

GrahamcOfBorg · 2018-07-12T23:38:48Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: idris

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-12T23:58:47Z

Success on x86_64-linux (full log)

Attempted: idris

Partial log (click to expand)

post-installation fixup
moving /nix/store/ir96n2xz98hvnniz8y3fb6vscvc8isif-base-1.3.0/doc to /nix/store/ir96n2xz98hvnniz8y3fb6vscvc8isif-base-1.3.0/share/doc
shrinking RPATHs of ELF executables and libraries in /nix/store/ir96n2xz98hvnniz8y3fb6vscvc8isif-base-1.3.0
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/ir96n2xz98hvnniz8y3fb6vscvc8isif-base-1.3.0
checking for references to /tmp/nix-build-base-1.3.0.drv-0 in /nix/store/ir96n2xz98hvnniz8y3fb6vscvc8isif-base-1.3.0...
building path(s) ‘/nix/store/46kzhiyifqmaipg40jkgl0wsgc9zkmkj-idris-1.3.0’
/nix/store/gdskxkj82lir32qdwlrxfkb7czx1x5ba-idris-1.3.0/nix-support:
propagated-build-inputs: /nix/store/ir96n2xz98hvnniz8y3fb6vscvc8isif-base-1.3.0/nix-support/propagated-build-inputs
/nix/store/46kzhiyifqmaipg40jkgl0wsgc9zkmkj-idris-1.3.0

matthewbauer · 2018-07-13T18:46:23Z

Is there a way to add packages to Hydra without making them show up in nix-env? I think one package set uses this but I can’t remember which one.

I think if we do recurseIntoAttrs for idris, we should make sure all of the names are prefixed properly with something like idris-. A lot of times there will be conflicts with global vs language-specific package names. This can screw up things like nix-env -u.

infinisil · 2018-07-13T20:33:38Z

@matthewbauer Good point, I didn't think of that, will fix that

infinisil · 2018-07-13T20:45:58Z

@matthewbauer Fixed that

@GrahamcOfBorg build idrisPackages.recursion_schemes

GrahamcOfBorg · 2018-07-13T20:46:16Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: idrisPackages.recursion_schemes

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-13T21:05:33Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: idris

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-13T21:08:05Z

Success on x86_64-linux (full log)

Attempted: idrisPackages.recursion_schemes

Partial log (click to expand)

Installing 00recursion_schemes-idx.ibc to /nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19/libs/recursion_schemes
Attempting to install IdrisDocs for recursion_schemes in: /nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19/doc/recursion_schemes
Warning: Ignoring empty or non-existing namespace 'Data.Functor.Foldable'
post-installation fixup
moving /nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19/doc to /nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19/share/doc
shrinking RPATHs of ELF executables and libraries in /nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19
checking for references to /tmp/nix-build-idris-recursion_schemes-2018-01-19.drv-0 in /nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19...
/nix/store/69pz3sy4s2qynr3wmnlmky4gj8kj1y19-idris-recursion_schemes-2018-01-19

GrahamcOfBorg · 2018-07-13T21:54:37Z

Success on x86_64-linux (full log)

Attempted: idris

Partial log (click to expand)

post-installation fixup
moving /nix/store/7maq8cmih3v4y3n1cxi7w01azixl35mz-idris-base-1.3.0/doc to /nix/store/7maq8cmih3v4y3n1cxi7w01azixl35mz-idris-base-1.3.0/share/doc
shrinking RPATHs of ELF executables and libraries in /nix/store/7maq8cmih3v4y3n1cxi7w01azixl35mz-idris-base-1.3.0
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/7maq8cmih3v4y3n1cxi7w01azixl35mz-idris-base-1.3.0
checking for references to /build in /nix/store/7maq8cmih3v4y3n1cxi7w01azixl35mz-idris-base-1.3.0...
building '/nix/store/snxbhx1g6c3s487snvgdnkilzsn5m107-idris-1.3.0.drv'...
/nix/store/gdskxkj82lir32qdwlrxfkb7czx1x5ba-idris-1.3.0/nix-support:
propagated-build-inputs: /nix/store/7maq8cmih3v4y3n1cxi7w01azixl35mz-idris-base-1.3.0/nix-support/propagated-build-inputs
/nix/store/kl2hz7vl0zsb4g9bnap09ba1jddy117p-idris-1.3.0

peti · 2018-07-14T08:06:32Z

Is there a way to add packages to Hydra without making them show up in nix-env?

You can add the name of the attribute set at the bottom of pkgs/top-level/release.nix, like "haskellPackages" does it.

infinisil · 2018-07-15T23:42:30Z

Is this good to merge?

peti

The clean up is very desirable to have, obviously. I'm not sure whether Idris packages should show up with nix-env -qa. Are these packages useful to users, i.e. can you install nix-env -i idris-foo and do something useful with it? If you can, then showing them on the index seems fine. If Idris packages need to be installed using a wrapper, like Haskell packages do, then showing them on the index make little sense IMHO.

mpickering · 2018-07-16T08:20:00Z

The Idris packages need to be installed using a wrapper like the Haskell packages do.

peti

Please don't use recurseIntoAttrs. It seems like a bad idea to show packages to users in nix-env that they actually can't use in nix-env.

Hydra builds can be enabled by adding an appropriate entry to release.nix, similar to this one: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/release.nix#L168

brainrake · 2018-07-16T13:55:41Z

Peti, thanks for the review and the above great points.
Idris packages with binaries (directly useful for users) can be added to pkgs/top-level/all-packages.nix, like other languages.

infinisil · 2018-07-16T14:00:49Z

@brainrape Are there any idris packages with binaries though? And are the binaries even built and installed into $out/bin?

infinisil · 2018-07-16T14:05:29Z

@peti I removed recurseIntoAttrs and added it to the mentioned release.nix

GrahamcOfBorg · 2018-07-16T14:20:22Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: idris

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-16T15:22:33Z

Success on x86_64-linux (full log)

Attempted: idris

Partial log (click to expand)

post-installation fixup
moving /nix/store/6kyaw6cjsa91zlmn3pyhirmsxcflz60x-idris-base-1.3.0/doc to /nix/store/6kyaw6cjsa91zlmn3pyhirmsxcflz60x-idris-base-1.3.0/share/doc
shrinking RPATHs of ELF executables and libraries in /nix/store/6kyaw6cjsa91zlmn3pyhirmsxcflz60x-idris-base-1.3.0
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/6kyaw6cjsa91zlmn3pyhirmsxcflz60x-idris-base-1.3.0
checking for references to /build in /nix/store/6kyaw6cjsa91zlmn3pyhirmsxcflz60x-idris-base-1.3.0...
building '/nix/store/ac7z28g05kp9n901806xwvsx9kmak5w8-idris-1.3.0.drv'...
/nix/store/l46ky638p24vmh75k8jvdv6lzxi6md07-idris-1.3.0/nix-support:
propagated-build-inputs: /nix/store/6kyaw6cjsa91zlmn3pyhirmsxcflz60x-idris-base-1.3.0/nix-support/propagated-build-inputs
/nix/store/8pzm0gd2kgzdshddrqwgi1g6scjwi10w-idris-1.3.0

TealG · 2018-07-16T15:35:08Z

There are use cases for idris packages with binaries, however, build-idris-package currently does not allow installing these in $out/bin (idris --install only installs libs). It'd be useful if build-idris-package would allow specifying additional commands at the end of the installPhase for this purpose.

infinisil requested a review from peti as a code owner July 12, 2018 23:15

GrahamcOfBorg added 6.topic: haskell 8.has: package (new) 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Jul 12, 2018

infinisil mentioned this pull request Jul 13, 2018

idris package build failure #43493

Closed

TealG mentioned this pull request Jul 14, 2018

Idris 1.3.0 build using Nix 2.0 typedefs/typedefs#20

Closed

6 tasks

peti reviewed Jul 16, 2018

View reviewed changes

peti requested changes Jul 16, 2018

View reviewed changes

infinisil added 2 commits July 16, 2018 16:00

idrisPackages: build with hydra

cb12f60

idris: prefix package derivation names with "idris-"

1d94e4c

infinisil force-pushed the idris-cleanup branch from 2388fbd to 1d94e4c Compare July 16, 2018 14:04

peti approved these changes Jul 16, 2018

View reviewed changes

peti merged commit 9a32daf into NixOS:master Jul 16, 2018

infinisil deleted the idris-cleanup branch July 16, 2018 20:14

infinisil mentioned this pull request Nov 10, 2018

.github/CODEOWNERS: Add infinisil to idris-modules #50183

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

idris: Recurse into attrs, fix build and remove unneeded overrides #43444

idris: Recurse into attrs, fix build and remove unneeded overrides #43444

infinisil commented Jul 12, 2018

GrahamcOfBorg commented Jul 12, 2018

GrahamcOfBorg commented Jul 12, 2018

matthewbauer commented Jul 13, 2018

infinisil commented Jul 13, 2018

infinisil commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

peti commented Jul 14, 2018 via email •

edited

infinisil commented Jul 15, 2018

peti left a comment

mpickering commented Jul 16, 2018

peti left a comment

brainrake commented Jul 16, 2018 •

edited

infinisil commented Jul 16, 2018 •

edited

infinisil commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

TealG commented Jul 16, 2018

idris: Recurse into attrs, fix build and remove unneeded overrides #43444

idris: Recurse into attrs, fix build and remove unneeded overrides #43444

Conversation

infinisil commented Jul 12, 2018

Motivation for this change

Things done

GrahamcOfBorg commented Jul 12, 2018

GrahamcOfBorg commented Jul 12, 2018

matthewbauer commented Jul 13, 2018

infinisil commented Jul 13, 2018

infinisil commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

GrahamcOfBorg commented Jul 13, 2018

peti commented Jul 14, 2018 via email • edited

infinisil commented Jul 15, 2018

peti left a comment

Choose a reason for hiding this comment

mpickering commented Jul 16, 2018

peti left a comment

Choose a reason for hiding this comment

brainrake commented Jul 16, 2018 • edited

infinisil commented Jul 16, 2018 • edited

infinisil commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

TealG commented Jul 16, 2018

peti commented Jul 14, 2018 via email •

edited

brainrake commented Jul 16, 2018 •

edited

infinisil commented Jul 16, 2018 •

edited