Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

idea: move meta to each package #45948

Closed
wants to merge 1 commit into from
Closed

Conversation

grahamc
Copy link
Member

@grahamc grahamc commented Sep 2, 2018

nix edit boost

currently edits generic.nix, leaving the user a bit confused about where
a specific boost version is defined. This is because Nix internally uses
builtins.unsafeGetAttrPos "description" package.meta to find the file
a package is from.

Additionally, I think it is possible for OfBorg to ping maintainers if the
maintainers are listed closer to the expressions being modified.

A con is obviously the duplication of meta fields.

A pro is the simplification of the meta: not having to do version comparisons in a generic location, for example.

cc @vcunat @peti @dezgeg @edolstra @Ericson2314 @dtzWill

What do y'all think? This PR is for Boost-only, but we could easily apply the same idea and improve the experience for many other packages with a "generic" home:

"position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89"
      "position": "pkgs/development/libraries/boost/generic.nix:89",
      "position": "pkgs/development/libraries/botan/generic.nix:46",
      "position": "pkgs/development/libraries/botan/generic.nix:46",

      "position": "pkgs/servers/nosql/cassandra/generic.nix:65"
      "position": "pkgs/servers/nosql/cassandra/generic.nix:65"
      "position": "pkgs/servers/nosql/cassandra/generic.nix:65"
      "pkgs/development/libraries/celt/generic.nix:17"
      "position": "pkgs/development/libraries/celt/generic.nix:17"
      "position": "pkgs/development/libraries/celt/generic.nix:17"
      "position": "pkgs/tools/filesystems/ceph/generic.nix:171"
      "position": "pkgs/tools/filesystems/ceph/generic.nix:171"
      "position": "pkgs/development/libraries/db/generic.nix:48"
      "position": "pkgs/development/libraries/db/generic.nix:48"
      "position": "pkgs/development/libraries/db/generic.nix:48"
      "position": "pkgs/data/sgml+xml/schemas/xml-dtd/docbook/generic.nix:4"
      "position": "pkgs/data/sgml+xml/schemas/xml-dtd/docbook/generic.nix:4"
      "position": "pkgs/data/sgml+xml/schemas/xml-dtd/docbook/generic.nix:4"
      "position": "pkgs/data/sgml+xml/schemas/xml-dtd/docbook/generic.nix:4"
      "position": "pkgs/data/sgml+xml/schemas/xml-dtd/docbook/generic.nix:4"

      "position": "pkgs/development/interpreters/elixir/generic-builder.nix:65"
      "position": "pkgs/development/interpreters/elixir/generic-builder.nix:65"
      "position": "pkgs/development/interpreters/elixir/generic-builder.nix:65"

      "position": "pkgs/development/interpreters/erlang/generic-builder.nix:107"
      "position": "pkgs/development/interpreters/erlang/generic-builder.nix:107"
      "position": "pkgs/development/interpreters/erlang/generic-builder.nix:107"
      "position": "pkgs/development/interpreters/erlang/generic-builder.nix:107"
      "position": "pkgs/development/interpreters/erlang/generic-builder.nix:107"
      "position": "pkgs/development/interpreters/erlang/generic-builder.nix:107"
      "position": "pkgs/development/interpreters/erlang/generic-builder.nix:107"
      "position": "pkgs/development/libraries/ffmpeg/generic.nix:196"
      "position": "pkgs/development/libraries/ffmpeg/generic.nix:196"
      "position": "pkgs/development/libraries/ffmpeg/generic.nix:196"
      "position": "pkgs/misc/gnuk/generic.nix:48"
      "position": "pkgs/misc/gnuk/generic.nix:48",
      "position": "pkgs/misc/gnuk/generic.nix:48",
 21 58.8M   21 12.8M    0     0   170k      0  0:05:53  0:01:17  0:04:36  205k      "description": "A generic backend for GNUstep",
      "position": "pkgs/development/libraries/gnutls/generic.nix:71"
      "position": "pkgs/development/libraries/gnutls-kdh/generic.nix:78"
      "position": "pkgs/development/go-modules/generic/default.nix:79"
      "position": "pkgs/development/go-modules/generic/default.nix:79"
      "position": "pkgs/development/go-modules/generic/default.nix:79"
 "position": "pkgs/development/go-modules/generic/default.nix:79"
      "position": "pkgs/development/go-modules/generic/default.nix:79"
      "position": "pkgs/development/go-modules/generic/default.nix:79"
      "position": "pkgs/development/go-modules/generic/default.nix:79"
      "position": "pkgs/development/compilers/mono/generic.nix:90",
      "position": "pkgs/development/compilers/mono/generic.nix:90",
      "position": "pkgs/development/compilers/mono/generic-cmake.nix:87",
      "position": "pkgs/development/compilers/mono/generic-cmake.nix:87",
      "position": "pkgs/development/compilers/mono/generic-cmake.nix:87",
      "position": "pkgs/development/compilers/mono/generic-cmake.nix:87",

(and more)

> nix edit boost

currently edits generic.nix, leaving the user a bit confused about where
a specific boost version is defined.

Additionally, I think it is possible for OfBorg to ping maintainers if the
maitainers are listed closer to the expressions being modified
@grahamc
Copy link
Member Author

grahamc commented Sep 2, 2018

cc @LnL7

@grahamc grahamc changed the title ideao: move meta to each package idea: move meta to each package Sep 2, 2018
@grahamc grahamc requested a review from peti September 2, 2018 12:45
@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: boost

Partial log (click to expand)

these paths will be fetched (1.61 MiB download, 11.83 MiB unpacked):
  /nix/store/34mngyv1fiqjnzzmjgj0dnf7w6153jia-boost-1.67_0
copying path '/nix/store/34mngyv1fiqjnzzmjgj0dnf7w6153jia-boost-1.67_0' from 'https://cache.nixos.org'...
/nix/store/34mngyv1fiqjnzzmjgj0dnf7w6153jia-boost-1.67_0

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: boost

Partial log (click to expand)

these paths will be fetched (2.49 MiB download, 15.68 MiB unpacked):
  /nix/store/4wgjhlpfl6p9ji8v2ajkx4llsmxy772s-boost-1.67_0
copying path '/nix/store/4wgjhlpfl6p9ji8v2ajkx4llsmxy772s-boost-1.67_0' from 'https://nix-cache.s3.amazonaws.com'...
/nix/store/4wgjhlpfl6p9ji8v2ajkx4llsmxy772s-boost-1.67_0

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: boost

Partial log (click to expand)

these paths will be fetched (2.48 MiB download, 17.38 MiB unpacked):
  /nix/store/qky9i4xq2qzv65kfy6lx8rj2s874q2lj-boost-1.67_0
copying path '/nix/store/qky9i4xq2qzv65kfy6lx8rj2s874q2lj-boost-1.67_0' from 'https://cache.nixos.org'...
/nix/store/qky9i4xq2qzv65kfy6lx8rj2s874q2lj-boost-1.67_0

@dezgeg
Copy link
Contributor

dezgeg commented Sep 2, 2018

This is because Nix internally uses builtins.unsafeGetAttrPos "description" package.meta to find the file a package is from.

I wonder if you'd get better results simpler by tweaking this logic. Maybe src would be a better attribute?

@LnL7
Copy link
Member

LnL7 commented Sep 2, 2018

One could argue that it makes sense to generalise certain meta fields like maintainers or platforms. This is done in a few places, but there's no real benefit in doing that IMHO.

@dezgeg I don't think using src will yield in better results, certain groups of packages don't require configuring a source because the derivation has enough information.

@Ericson2314
Copy link
Member

I never understood why we even had the position tricks with --show-trace. So I'm not at all informed here, but this seems fine to me.

@grahamc
Copy link
Member Author

grahamc commented Sep 2, 2018

@Ericson2314 hat do you mean position tricks?

@vcunat
Copy link
Member

vcunat commented Sep 2, 2018

If we do the transition to define separate version attributes (in majority of cases at least), we could use position of that as a better approximation. Hopefully that would be good enough in itself, so that we wouldn't need to enforce restrictions like this. The separate version attributes seem generally liked, even without this as an argument.

@Ericson2314
Copy link
Member

@graham I mean the use of builtins.unsafeGetAttrPos.

@infinisil
Copy link
Member

Not in favor of this, the duplicated meta is very ugly. I propose this instead, which has the same effect:

diff --git a/pkgs/development/libraries/boost/generic.nix b/pkgs/development/libraries/boost/generic.nix
index 617484e5a40..37ca77d674f 100644
--- a/pkgs/development/libraries/boost/generic.nix
+++ b/pkgs/development/libraries/boost/generic.nix
@@ -18,7 +18,7 @@
 # Attributes inherit from specific versions
 , version, src
 , ...
-}:
+}@attrs:
 
 # We must build at least one type of libraries
 assert enableShared || enableStatic;
@@ -99,6 +99,7 @@ stdenv.mkDerivation {
     homepage = http://boost.org/;
     description = "Collection of C++ libraries";
     license = stdenv.lib.licenses.boost;
+    position = builtins.unsafeGetAttrPos "version" attrs;
 
     platforms = (if versionOlder version "1.59" then remove "aarch64-linux" else id) platforms.unix;
     maintainers = with maintainers; [ peti wkennington ];
diff --git a/pkgs/stdenv/generic/make-derivation.nix b/pkgs/stdenv/generic/make-derivation.nix
index 8af6d0e0400..5415ae3339c 100644
--- a/pkgs/stdenv/generic/make-derivation.nix
+++ b/pkgs/stdenv/generic/make-derivation.nix
@@ -264,12 +264,12 @@ rec {
               hasOutput = out: builtins.elem out outputs;
             in [( lib.findFirst hasOutput null (["bin" "out"] ++ outputs) )];
         }
-        // attrs.meta or {}
         # Fill `meta.position` to identify the source location of the package.
         // lib.optionalAttrs (pos != null) {
           position = pos.file + ":" + toString pos.line;
         # Expose the result of the checks for everyone to see.
-        } // {
+        } // attrs.meta or {}
+        // {
           available = validity.valid
                    && (if config.checkMetaRecursively or false
                        then lib.all (d: d.meta.available or true) references

That is, enable the meta definition to override the default position value.

@infinisil
Copy link
Member

Although, it doesn't have the same effect, because the result of that will be an attrset

{ column = 3; file = "/home/infinisil/src/nixpkgs/pkgs/development/libraries/boost/1.55.nix"; line = 4; }

instead of a string. But tbh, I think this should be an attrset to begin with, it's easy for tools to construct a string from the attrset, but the inverse isn't easy and requires parsing.

@Mic92 Mic92 mentioned this pull request Sep 3, 2018
7 tasks
@grahamc
Copy link
Member Author

grahamc commented Sep 4, 2018

I found a way around this for ofborg :)

@grahamc grahamc closed this Sep 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants