dconf2nix: init at 0.5.0 #94891

gvolpe · 2020-08-07T20:17:56Z

Motivation for this change

To make the dconf2nix package available on Nixpkgs.

Things done

gvolpe · 2020-08-07T20:50:18Z

Apparently, I'm not a trusted user: https://gist.github.com/GrahamcOfBorg/755c9a6d20cccee353811b1df5fb430a

nix-env failed:
warning: ignoring the user-specified setting 'restrict-eval', because it is a restricted setting and you are not a trusted user

If there's anything I should fix, I'd appreciate any help 🙏

maintainers/maintainer-list.nix

pkgs/development/tools/haskell/dconf2nix/default.nix

jonringer · 2020-08-07T22:22:51Z

Apparently, I'm not a trusted user: https://gist.github.com/GrahamcOfBorg/755c9a6d20cccee353811b1df5fb430a
nix-env failed:
warning: ignoring the user-specified setting 'restrict-eval', because it is a restricted setting and you are not a trusted user
If there's anything I should fix, I'd appreciate any help pray

This is because you're doing a IFD (import from derivation)

gvolpe · 2020-08-07T22:24:49Z

Thanks for the quick review @jonringer ! I'll address your comments soon.

This is because you're doing a IFD (import from derivation)

Does this have any implications? Should I do it differently?

jonringer

The main issue is that you're trying to evaluate ANOTHER derivation from the src fixed output derivation.

Please move all the relevant packaging detail from upstream to here.

@cdepillabout please help with haskell packaging :)

pkgs/development/tools/haskell/dconf2nix/default.nix

gvolpe · 2020-08-07T22:37:16Z

I think I addressed all your comments, @jonringer , thanks again!

The main issue is that you're trying to evaluate ANOTHER derivation from the src fixed output derivation.

Not sure exactly how I should solve this, any help would be much appreciated 🙏

gvolpe · 2020-08-07T22:40:01Z

Maybe doing somethink like this?

prePatch = ''
  rm $src/default.nix 
'';

cdepillabout · 2020-08-08T02:12:58Z

pkgs/development/haskell-modules/non-hackage-packages.nix

@@ -10,6 +10,8 @@ self: super: {
  multi-ghc-travis = throw ("haskellPackages.multi-ghc-travis has been renamed"
    + " to haskell-ci, which is now on hackage");

+  dconf2nix = self.callPackage ../tools/haskell/dconf2nix { };


Rather than trying to build this package as a non-hackage-package, it would be easier for us if you were to just release it to Hackage.

If dconf2nix was released to Hackage, then the Haskell tooling here in Nixpkgs will pull in the latest version every time you make a new release to Hackage. In most cases, you won't have to do anything to maintain it here in nixpkgs. It will just be available by default.

Also, releasing to Hackage generally helps the wider Hackage community, since non-nixpkg users have an easy way to get your package.

I know about the trade-offs but if we think about it, this is a tool for Nix users so I'd expect anyone using it to install it via Nix and not by other means.

OTOH I don't want to maintain a Hackage package, that's why I added myself as a maintainer here.

If you don't upload to Hackage, it will probably be more work for you to maintain it here in nixpkgs, but as long as you're willing to deal with that extra work, it is fine.

Check out how haskellPackages.spago is packaged in nixpkgs, and model the packaging for dconf2nix on that. Spago is another tool written in Haskell that is not on Hackage.

I might consider it in the long run if this doesn't work out but for now I just want to make it available on Nixpkgs as a top-level package, so the PR must exist regardless of being on Hackage or not.

I looked at the definition of vaultenv to have an idea about how to package it but it's also my first time doing it so I'm not really sure what else I'm missing. I'll have a look at Spago too 👍

gvolpe · 2020-08-08T12:07:51Z

I don't really understand how Spago is built 😕 . I see it has a similar derivation but on the Github repository there's no default.nix file.

OTOH vaultenv has a very simple default.nix in the Github repository but it uses Static Haskell Nix which is only supported by Stack AFAIU and I'm using Cabal.

I think the issue here with my derivation is due to my default.nix calling cabal2nix? If so, any suggestions on what's the approach I should follow? Because I need that default.nix to build the project locally and on the CI build, and the shell.nix for my local dev env.

Any help much appreciated 🙇

cdepillabout · 2020-08-08T12:21:03Z

@gvolpe You can basically copy the spago directory here in nixpkgs and re-use it for dconf2nix.

Basically, you'll need a dconf2nix.nix file that is generated by manually calling cabal2nix. You'll probably also want a default.nix file that adds some meta values to haskellPackages.dconf2nix, as well as doing things like generating optparse-applicative autocomplete stuff. Finally, you'll need an update.sh script that will make it easy to update the dconf2nix.nix file.

I see [spago] has a similar derivation but on the Github repository there's no default.nix file.

Yeah, there are derivations for building Haskell packages here in nixpkgs, but there are no derivations on most of the upstream Haskell packages. In general you will want to look for how to package stuff here in nixpkgs, not in the upstream repos.

I think the issue here with my derivation is due to my default.nix calling cabal2nix? If so, any suggestions on what's the approach I should follow? Because I need that default.nix to build the project locally and on the CI build, and the shell.nix for my local dev env.

Yes, the problem is that you're using IFD: https://blog.hercules-ci.com/2019/08/30/native-support-for-import-for-derivation/

The approach that you should follow is the exact same approach that spago is using here in nixpkgs.

Because Hydra doesn't allow IFD, you will need to duplicate the Nix files here in nixpkgs and your upstream repo. This is somewhat unfortunate, but it is the current state of affairs here in nixpkgs.

gvolpe · 2020-08-08T14:18:12Z

Thanks @cdepillabout , I'll have some time tomorrow to look into it.

In general you will want to look for how to package stuff here in nixpkgs, not in the upstream repos.

FWIW I'm not packing it per-se but I'm taking advantages of Cachix and Github Actions. Isn't there something like fetchTarball but that can ignore the default.nix files so it avoids IFD? That seems like a rather common scenario.

Also, the current default.nix in this PR builds fine locally by running nix-build -A dconf2nix. Is there a way to verify there is no IFD happening? So I can test it locally.

Thanks again for your time!

gvolpe · 2020-08-08T16:48:32Z

I tried to follow how spago is built but now I'm no longer sure how to build it locally. nix-build -A dconf2nix no longer works due to having some dependencies in the new default.nix (more or less copied from spagos):

{ haskell, haskellPackages, lib, runCommand }:

let
  dconf2nix =
    haskell.lib.justStaticExecutables
      (haskell.lib.overrideCabal haskellPackages.dconf2nix (oldAttrs: {
        maintainers = (oldAttrs.maintainers or []) ++ [
          lib.maintainers.cdepillabout
        ];
      }));
in

dconf2nix.overrideAttrs (oldAttrs: {
  passthru = (oldAttrs.passthru or {}) // {
    updateScript = ./update.sh;

    # These tests can be run with the following command.  The tests access the
    # network, so they cannot be run in the nix sandbox.  sudo is needed in
    # order to change the sandbox option.
    #
    # $ sudo nix-build -A dconf2nix.passthru.tests --option sandbox relaxed
    #
    tests =
      runCommand
        "dconf2nix-tests"
        {
          __noChroot = true;
          nativeBuildInputs = [
            dconf2nix
          ];
        }
        ''
          dconf2nix -v
        '';
  };
})

pkgs/development/tools/haskell/dconf2nix/default.nix

cdepillabout · 2020-08-09T01:05:35Z

I tried to follow how spago is built but now I'm no longer sure how to build it locally. nix-build -A dconf2nix no longer works due to having some dependencies in the new default.nix (more or less copied from spagos)

Would you be able to push what you have so far? The code snippet you pasted looks fine for the most part, so I am having trouble seeing what you're having trouble with.

gvolpe · 2020-08-09T09:24:11Z

Sure @cdepillabout , I just pushed what I have, based on the spago derivation. I don't know how to test it, though.

gvolpe · 2020-08-09T09:28:43Z

The command sudo nix-build -A spago.passthru.tests --option sandbox relaxed (located in the comments of update.sh) fails for me with the same error I mentioned above due to default.nix having some dependencies.

error: cannot auto-call a function that has an argument without a default value ('haskell')

cdepillabout · 2020-08-09T10:59:31Z

@gvolpe Here's what I needed to do to get this to build:

iff --git a/pkgs/development/haskell-modules/non-hackage-packages.nix b/pkgs/development/haskell-modules/non-hackage-packages.nix
index dd6a13f23c2..8801f1f1ddd 100644
--- a/pkgs/development/haskell-modules/non-hackage-packages.nix
+++ b/pkgs/development/haskell-modules/non-hackage-packages.nix
@@ -10,7 +10,7 @@ self: super: {
   multi-ghc-travis = throw ("haskellPackages.multi-ghc-travis has been renamed"
     + " to haskell-ci, which is now on hackage");
 
-  dconf2nix = self.callPackage ../tools/haskell/dconf2nix { };
+  dconf2nix = self.callPackage ../tools/haskell/dconf2nix/dconf2nix.nix { };
 
   # https://github.com/channable/vaultenv/issues/1
   vaultenv = self.callPackage ../tools/haskell/vaultenv { };
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index f3ec07d8534..dcff5fadd66 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -2921,7 +2921,7 @@ in
 
   dclxvi = callPackage ../development/libraries/dclxvi { };
 
-  dconf2nix = haskellPackages.dconf2nix;
+  dconf2nix = callPackage ../development/tools/haskell/dconf2nix { };
 
   dcraw = callPackage ../tools/graphics/dcraw { };

gvolpe · 2020-08-09T12:23:33Z

Wow, I feel dumb I didn't see that! Thanks @cdepillabout, I'll try it out as soon as I get home 🙇

gvolpe · 2020-08-09T13:41:40Z

@cdepillabout what command do you run to build dconf2nix? I made the changes you suggested but I'm still not able to build it by running nix-build -A dconf2nix or sudo nix-build -A dconf2nix.passthru.tests --option sandbox relaxed.

gvolpe · 2020-08-09T13:57:13Z

Oh nevermind, it builds fine. I was running the command not from the top-level directory... Thanks a lot for your help!

cdepillabout

There are a few other small fixes that can be made:

diff --git a/pkgs/development/tools/haskell/dconf2nix/default.nix b/pkgs/development/tools/haskell/dconf2nix/default.nix
index 3bcd4e2e1f3..cbaa47b6556 100644
--- a/pkgs/development/tools/haskell/dconf2nix/default.nix
+++ b/pkgs/development/tools/haskell/dconf2nix/default.nix
@@ -5,7 +5,7 @@ let
     haskell.lib.justStaticExecutables
       (haskell.lib.overrideCabal haskellPackages.dconf2nix (oldAttrs: {
         maintainers = (oldAttrs.maintainers or []) ++ [
-          lib.maintainers.cdepillabout
+          lib.maintainers.gvolpe
         ];
       }));
 in
@@ -14,23 +14,19 @@ dconf2nix.overrideAttrs (oldAttrs: {
   passthru = (oldAttrs.passthru or {}) // {
     updateScript = ./update.sh;
 
-    # These tests can be run with the following command.  The tests access the
-    # network, so they cannot be run in the nix sandbox.  sudo is needed in
-    # order to change the sandbox option.
-    #
-    # $ sudo nix-build -A dconf2nix.passthru.tests --option sandbox relaxed
+    # These tests can be run with the following command.
     #
+    # $ nix-build -A dconf2nix.passthru.tests
     tests =
       runCommand
         "dconf2nix-tests"
         {
-          __noChroot = true;
           nativeBuildInputs = [
             dconf2nix
           ];
         }
         ''
-          dconf2nix -v
+          dconf2nix > $out
         '';
   };
 })

You don't need to run the tests with the relaxed sandbox, because it appears that just running dconf2nix produces output. I'm assuming this is a good enough check to see if dconf2nix is compiled correctly, but if there is a better one please feel free to use it.

I've made sure that dconf2nix builds, the update.sh script works as expected, and the tests pass. Once you make these above changes, this looks good and I will go ahead and merge it in!

edit: Oh, and usually other maintainers appreciate if you squash your commits, and make sure that the commit adding you as a maintainer comes before the commit where you add dconf2nix.

cdepillabout

Looks good, thanks for working through this with us!

gvolpe · 2020-08-10T07:25:17Z

Thank you for being patient @cdepillabout ! 🙇

gvolpe requested a review from cdepillabout as a code owner August 7, 2020 20:17

gvolpe mentioned this pull request Aug 7, 2020

Submit package to nixpkgs nix-community/dconf2nix#18

Closed

ofborg bot added the 6.topic: haskell label Aug 7, 2020

jonringer reviewed Aug 7, 2020

View reviewed changes

pkgs/development/tools/haskell/dconf2nix/default.nix Outdated Show resolved Hide resolved

gvolpe mentioned this pull request Aug 7, 2020

maintainers: add gvolpe #94900

Closed

10 tasks

gvolpe force-pushed the development/tools/dconf2nix branch from 27b36a8 to 60e398b Compare August 7, 2020 22:35

cdepillabout reviewed Aug 8, 2020

View reviewed changes

jtojnar reviewed Aug 9, 2020

View reviewed changes

pkgs/development/tools/haskell/dconf2nix/default.nix Outdated Show resolved Hide resolved

ofborg bot added 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Aug 9, 2020

cdepillabout requested changes Aug 9, 2020

View reviewed changes

gvolpe force-pushed the development/tools/dconf2nix branch from e904a40 to 7d0ae2b Compare August 9, 2020 15:48

gvolpe added 2 commits August 9, 2020 17:58

maintainers: add gvolpe

04989d7

dconf2nix: init at 0.5.0

b987a31

gvolpe force-pushed the development/tools/dconf2nix branch from 7d0ae2b to b987a31 Compare August 9, 2020 15:59

cdepillabout approved these changes Aug 10, 2020

View reviewed changes

cdepillabout merged commit 1ae9ecf into NixOS:master Aug 10, 2020

gvolpe deleted the development/tools/dconf2nix branch August 10, 2020 07:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dconf2nix: init at 0.5.0 #94891

dconf2nix: init at 0.5.0 #94891

gvolpe commented Aug 7, 2020 •

edited

gvolpe commented Aug 7, 2020 •

edited

jonringer commented Aug 7, 2020

gvolpe commented Aug 7, 2020

jonringer left a comment

gvolpe commented Aug 7, 2020

gvolpe commented Aug 7, 2020 •

edited

cdepillabout Aug 8, 2020 •

edited

gvolpe Aug 8, 2020

cdepillabout Aug 8, 2020 •

edited

gvolpe Aug 8, 2020 •

edited

gvolpe commented Aug 8, 2020

cdepillabout commented Aug 8, 2020 •

edited

gvolpe commented Aug 8, 2020

gvolpe commented Aug 8, 2020 •

edited

cdepillabout commented Aug 9, 2020

gvolpe commented Aug 9, 2020

gvolpe commented Aug 9, 2020

cdepillabout commented Aug 9, 2020

gvolpe commented Aug 9, 2020

gvolpe commented Aug 9, 2020

gvolpe commented Aug 9, 2020

cdepillabout left a comment •

edited

cdepillabout left a comment

gvolpe commented Aug 10, 2020

dconf2nix: init at 0.5.0 #94891

dconf2nix: init at 0.5.0 #94891

Conversation

gvolpe commented Aug 7, 2020 • edited

Motivation for this change

Things done

gvolpe commented Aug 7, 2020 • edited

jonringer commented Aug 7, 2020

gvolpe commented Aug 7, 2020

jonringer left a comment

Choose a reason for hiding this comment

gvolpe commented Aug 7, 2020

gvolpe commented Aug 7, 2020 • edited

cdepillabout Aug 8, 2020 • edited

Choose a reason for hiding this comment

gvolpe Aug 8, 2020

Choose a reason for hiding this comment

cdepillabout Aug 8, 2020 • edited

Choose a reason for hiding this comment

gvolpe Aug 8, 2020 • edited

Choose a reason for hiding this comment

gvolpe commented Aug 8, 2020

cdepillabout commented Aug 8, 2020 • edited

gvolpe commented Aug 8, 2020

gvolpe commented Aug 8, 2020 • edited

cdepillabout commented Aug 9, 2020

gvolpe commented Aug 9, 2020

gvolpe commented Aug 9, 2020

cdepillabout commented Aug 9, 2020

gvolpe commented Aug 9, 2020

gvolpe commented Aug 9, 2020

gvolpe commented Aug 9, 2020

cdepillabout left a comment • edited

Choose a reason for hiding this comment

cdepillabout left a comment

Choose a reason for hiding this comment

gvolpe commented Aug 10, 2020

gvolpe commented Aug 7, 2020 •

edited

gvolpe commented Aug 7, 2020 •

edited

gvolpe commented Aug 7, 2020 •

edited

cdepillabout Aug 8, 2020 •

edited

cdepillabout Aug 8, 2020 •

edited

gvolpe Aug 8, 2020 •

edited

cdepillabout commented Aug 8, 2020 •

edited

gvolpe commented Aug 8, 2020 •

edited

cdepillabout left a comment •

edited