all-packages.nix: remove `res.` (next stage super) #55061

danbst · 2019-02-01T23:30:14Z

A bit more consistent version of #51527

A bit more consistent version of NixOS#51527

infinisil · 2019-02-01T23:44:15Z

Huh, shouldn't this be a no-op? ofborg lists a bunch of derivations that seem to have changed

danbst · 2019-02-02T06:32:20Z

Same "huh" here, but it turned out pcre16 is different from res.pcre16 (and qt depends on pcre16)

$ nix-diff  '/nix/store/c2xiw43k59a1h8jnpmiz9q6304ahfs54-pcre16-8.42.drv' '/nix/store/5q4j7rairip8qj9p19sdjza2rbwy4bzf-pcre16-8.42.drv'
- /nix/store/c2xiw43k59a1h8jnpmiz9q6304ahfs54-pcre16-8.42.drv:{out}
+ /nix/store/5q4j7rairip8qj9p19sdjza2rbwy4bzf-pcre16-8.42.drv:{out}
• The builders do not match
    - /nix/store/n9acaakxahkv1q3av11l93p7rgd4xqsf-bootstrap-tools/bin/bash
    + /nix/store/vs6d2fjkl4kb3jb7rwibsd76k9v2n4xy-bash-4.4-p23/bin/bash
• The set of input names do not match:
    - bootstrap-stage4-stdenv-linux
    - bootstrap-tools
    + bash-4.4-p23
    + stdenv-linux
• The input named `pcre-8.42.tar.bz2` differs
  - /nix/store/npj2j3yvh75f6ghiix6sdy49yia4bqf6-pcre-8.42.tar.bz2.drv:{out}
  + /nix/store/v37dhldm7mf5sgrmbvnjp2lc047sbkzn-pcre-8.42.tar.bz2.drv:{out}
  • The platforms do not match
      - builtin
      + x86_64-linux
  • The builders do not match
      - builtin:fetchurl
      + /nix/store/vs6d2fjkl4kb3jb7rwibsd76k9v2n4xy-bash-4.4-p23/bin/bash
  • The arguments do not match
      + -e
      + /nix/store/qjk2h301faawvbnhv5azhyiaw6vrqalg-builder.sh
  • The set of input names do not match:
      + bash-4.4-p23
      + curl-7.63.0
      + mirrors-list
      + stdenv-linux
  • The set of input sources do not match:
      + /nix/store/qjk2h301faawvbnhv5azhyiaw6vrqalg-builder.sh

so, res.pcre was a bootstrap package, but self.pcre is not. I think it is fine to merge this.

Ericson2314 · 2019-02-02T20:47:13Z

It's the other way around, res.pcre is not the bootstrap package. This is probably because stdenv overrides are applied after all-packages.nix, so res.pcre bypasses it. This change then has the side affect of avoiding a pointless rebuild of pcre.

Ericson2314 · 2019-02-02T20:48:41Z

@roberth I'd like to apologize to you for not so analyzing your PR like I said I would. I have had less time for Nixpkgs things lately but still I should follow up on anything I said I'd do.

roberth · 2019-02-03T10:40:30Z

@Ericson2314 no problem. It seems that there are plenty more important things to do, even when considering just Nixpkgs.

delroth · 2019-02-03T12:32:20Z

This PR broke nixos/tests/transmission.nix:

error: while evaluating the attribute 'buildCommand' of the derivation 'vm-test-run-transmission' at /home/delroth/work/nixpkgs/nixos/lib/testing.nix:53:7:
while evaluating the attribute 'buildCommand' of the derivation 'nixos-test-driver-transmission' at /home/delroth/work/nixpkgs/pkgs/build-support/trivial-builders.nix:7:14:
while evaluating the attribute 'buildCommand' of the derivation 'nixos-vm' at /home/delroth/work/nixpkgs/pkgs/build-support/trivial-builders.nix:7:14:
while evaluating the attribute 'activationScript' of the derivation 'nixos-system-machine-19.03.git.56b7912' at /home/delroth/work/nixpkgs/nixos/modules/system/activation/top-level.nix:102:5:
while evaluating the attribute 'system.activationScripts.script' at /home/delroth/work/nixpkgs/nixos/modules/system/activation/activation-script.nix:68:9:
while evaluating 'textClosureMap' at /home/delroth/work/nixpkgs/lib/strings-with-deps.nix:70:35, called from /home/delroth/work/nixpkgs/nixos/modules/system/activation/activation-script.nix:89:18:
while evaluating 'id' at /home/delroth/work/nixpkgs/lib/trivial.nix:14:5, called from undefined position:
while evaluating the attribute 'text' at /home/delroth/work/nixpkgs/nixos/modules/system/activation/activation-script.nix:9:5:
while evaluating the attribute 'text' at /home/delroth/work/nixpkgs/lib/strings-with-deps.nix:77:38:
while evaluating the attribute 'sources' of the derivation 'etc' at /home/delroth/work/nixpkgs/nixos/modules/system/etc/etc.nix:12:5:
while evaluating anonymous function at /home/delroth/work/nixpkgs/nixos/modules/system/etc/etc.nix:20:20, called from undefined position:
while evaluating the attribute 'source' at undefined position:
while evaluating anonymous function at /home/delroth/work/nixpkgs/lib/modules.nix:75:45, called from undefined position:
while evaluating the attribute 'value' at /home/delroth/work/nixpkgs/lib/modules.nix:338:9:
while evaluating the option `environment.etc.systemd/system.source':
while evaluating the attribute 'mergedValue' at /home/delroth/work/nixpkgs/lib/modules.nix:370:5:
while evaluating anonymous function at /home/delroth/work/nixpkgs/lib/modules.nix:370:32, called from /home/delroth/work/nixpkgs/lib/modules.nix:370:19:
while evaluating 'check' at /home/delroth/work/nixpkgs/lib/types.nix:245:15, called from /home/delroth/work/nixpkgs/lib/modules.nix:371:10:
while evaluating the attribute 'buildCommand' of the derivation 'system-units' at /home/delroth/work/nixpkgs/pkgs/build-support/trivial-builders.nix:7:14:
while evaluating the attribute 'buildInputs' of the derivation 'udisks-2.8.1' at /home/delroth/work/nixpkgs/pkgs/os-specific/linux/udisks/2-default.nix:11:3:
while evaluating the attribute 'buildInputs' of the derivation 'libblockdev-2.20' at /home/delroth/work/nixpkgs/pkgs/development/libraries/libblockdev/default.nix:9:3:
while evaluating the attribute 'buildInputs' of the derivation 'volume_key-0.3.11' at /home/delroth/work/nixpkgs/pkgs/development/libraries/volume-key/default.nix:8:3:
while evaluating the attribute 'configureFlags' of the derivation 'gpgme-1.12.0' at /home/delroth/work/nixpkgs/pkgs/development/libraries/gpgme/default.nix:16:3:
while evaluating 'callPackageWith' at /home/delroth/work/nixpkgs/lib/customisation.nix:108:35, called from /home/delroth/work/nixpkgs/pkgs/top-level/all-packages.nix:2948:13:
while evaluating 'makeOverridable' at /home/delroth/work/nixpkgs/lib/customisation.nix:67:24, called from /home/delroth/work/nixpkgs/lib/customisation.nix:112:8:
while evaluating anonymous function at /home/delroth/work/nixpkgs/pkgs/tools/security/gnupg/22.nix:1:1, called from /home/delroth/work/nixpkgs/lib/customisation.nix:69:12:
while evaluating the attribute 'pinentry_ncurses' at /home/delroth/work/nixpkgs/pkgs/top-level/all-packages.nix:4808:3:
infinite recursion encountered, at /home/delroth/work/nixpkgs/pkgs/top-level/all-packages.nix:4808:22

edit: Also seems to impact a few more tests (simple.nix, memcached.nix, etc.)

delroth · 2019-02-03T12:41:47Z

This seems to be caused by an interaction with overlays. In nixos/modules/config/no-x-libs.nix:

    nixpkgs.overlays = singleton (const (super: {
      pinentry = super.pinentry_ncurses;
    }));

danbst · 2019-02-03T13:32:06Z

@delroth crap. However this reveal many other ways when overlay override would cause infinite recursion. For example:

$ nix-instantiate -E 'with import <nixpkgs> { overlays = [(self: super: { 
    ttfautohint = self.ttfautohint-nox;
  })]; }; ttfautohint'
error: infinite recursion encountered, at /nix/store/94cbas4j0w0132qhxa5qn7zybrss3ymm-nixpkgs-19.03pre167327.11cf7d6e1ff/nixpkgs/pkgs/top-level/all-packages.nix:5926:21
(use '--show-trace' to show detailed location information)

$ nix-instantiate -E 'with import <nixpkgs> { overlays = [(self: super: { 
    ttfautohint = super.ttfautohint-nox;
  })]; }; ttfautohint'
error: infinite recursion encountered, at /nix/store/94cbas4j0w0132qhxa5qn7zybrss3ymm-nixpkgs-19.03pre167327.11cf7d6e1ff/nixpkgs/pkgs/top-level/all-packages.nix:5926:21
(use '--show-trace' to show detailed location information)

There should be some principal approach to fix this (never use .override in all-packages.nix?)

cc @nbp

This reverts commit 3ec0eb4. See NixOS#55061 (comment)

This reverts commit 3ec0eb4. See #55061 (comment)

danbst · 2019-02-03T13:56:15Z

Should have read this commit first d1a2853

Ericson2314 · 2019-02-04T01:34:57Z

Yes If I understand the issue correctly that commit has a pretty bogus workaround. A good rule for the "overrides and default assignment problem" is to make a fooReal which is overriden instead. So pinentry_ncurses could instead override pinentryReal, which is never flat-out replaced in any overlay.

oxij · 2019-02-21T03:01:09Z

Another attempt #56119 by me.

Tested with nix-env -qaf . The res -> super changes are necessary because res points to the result of the current overlay, but some of the packages were moved to the previous overlay, which can be accessed with super. This is a fundamental problem to be fixed, see NixOS#55061 and related issues The schleuder change is necessary because the ./default.nix file referenced ./., but there's also another package in ./bin. The tool couldn't move the gemfiles in ./. because The schleuder change is necessary because the tool isn't smart enough and this is a special case of the default.nix reading gemfiles from a ./. reference and there being both schleuder in pkgs/tools/security/schleuder and schleuder-cli in pkgs/tools/security/schleuder/cli

Tested with nix-env -qaf . The res -> super changes are necessary because res points to the result of the current overlay, but some of the packages were moved to the previous overlay, which can be accessed with super. This is a fundamental problem to be fixed, see NixOS#55061 and related issues The schleuder change is necessary because the ./default.nix file referenced ./., but there's also another package in ./bin. The tool couldn't move the gemfiles in ./. because The schleuder change is necessary because the tool isn't smart enough and this is a special case of the default.nix reading gemfiles from a ./. reference and there being both schleuder in pkgs/tools/security/schleuder and schleuder-cli in pkgs/tools/security/schleuder/cli Fix eval

Tested with nix-env -qaf . The res -> super changes are necessary because res points to the result of the current overlay, but some of the packages were moved to the previous overlay, which can be accessed with super. This is a fundamental problem to be fixed, see NixOS#55061 and related issues The schleuder change is necessary because the ./default.nix file referenced ./., but there's also another package in ./bin. The tool couldn't move the gemfiles in ./. because The schleuder change is necessary because the tool isn't smart enough and this is a special case of the default.nix reading gemfiles from a ./. reference and there being both schleuder in pkgs/tools/security/schleuder and schleuder-cli in pkgs/tools/security/schleuder/cli

all-packages.nix: remove res. (next stage super)

3ec0eb4

A bit more consistent version of NixOS#51527

infinisil approved these changes Feb 1, 2019

View reviewed changes

danbst mentioned this pull request Feb 1, 2019

[WIP] all-packages.nix: Use pkgs ("self") instead of res (next stage super) #51527

Closed

10 tasks

GrahamcOfBorg added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Feb 1, 2019

Ericson2314 merged commit c6bb6dd into NixOS:master Feb 2, 2019

danbst deleted the remove-res branch February 3, 2019 00:00

danbst added a commit to danbst/nixpkgs that referenced this pull request Feb 3, 2019

Revert "all-packages.nix: remove res. (next stage super)"

7d725a9

This reverts commit 3ec0eb4. See NixOS#55061 (comment)

danbst mentioned this pull request Feb 3, 2019

Revert "all-packages.nix: remove res. (next stage super)" #55125

Merged

danbst added a commit that referenced this pull request Feb 3, 2019

Revert "all-packages.nix: remove res. (next stage super)" (#55125)

b251a36

This reverts commit 3ec0eb4. See #55061 (comment)

danbst mentioned this pull request Feb 4, 2019

RFE: Evaluate NixOS tests on each PR NixOS/ofborg#315

Open

oxij mentioned this pull request Feb 21, 2019

all-packages.nix: cleanup, remove res. (next stage super) #56119

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

all-packages.nix: remove `res.` (next stage super) #55061

all-packages.nix: remove `res.` (next stage super) #55061

danbst commented Feb 1, 2019

infinisil commented Feb 1, 2019

danbst commented Feb 2, 2019

Ericson2314 commented Feb 2, 2019

Ericson2314 commented Feb 2, 2019

roberth commented Feb 3, 2019

delroth commented Feb 3, 2019 •

edited

delroth commented Feb 3, 2019

danbst commented Feb 3, 2019

danbst commented Feb 3, 2019

Ericson2314 commented Feb 4, 2019

oxij commented Feb 21, 2019

all-packages.nix: remove res. (next stage super) #55061

all-packages.nix: remove res. (next stage super) #55061

Conversation

danbst commented Feb 1, 2019

infinisil commented Feb 1, 2019

danbst commented Feb 2, 2019

Ericson2314 commented Feb 2, 2019

Ericson2314 commented Feb 2, 2019

roberth commented Feb 3, 2019

delroth commented Feb 3, 2019 • edited

delroth commented Feb 3, 2019

danbst commented Feb 3, 2019

danbst commented Feb 3, 2019

Ericson2314 commented Feb 4, 2019

oxij commented Feb 21, 2019

all-packages.nix: remove `res.` (next stage super) #55061

all-packages.nix: remove `res.` (next stage super) #55061

delroth commented Feb 3, 2019 •

edited