mkShell: add builder #30975

zimbatm · 2017-10-30T15:53:51Z

Motivation for this change

For nix-shell-only scenarios, this is a small optimization to make it nicer such that:

{ pkgs ? import <nixpkgs> {} }:
pkgs.stdenv.mkDerivation {
  name = "forced-to-add-but-i-dont-care";
  src = null; # irrelevant
}

Is simplified to:

{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell { }

It also fails when trying to build it directly:

$ nix-build shell.nix
...
This derivation is only meant to be used in nix-shell, aborting
...

And finally, it supports the scenario where one wants to compose the shell with different inputs from various packages:

{ pkgs ? import <nixpkgs> {}}:
pkgs.mkShell {
  mergeInputs = [ pkgs.hello pkgs.gnutar ];
}

zimbatm · 2017-10-30T15:56:54Z

For now I am looking for feedback on the following things:

should the shellHooks be merged as well?
where should I put the documentation for that?
is there any other kind of feature that you often use in your shells?

zimbatm · 2017-10-30T15:57:11Z

It would be nice if the shellHooks where composable

orivej · 2017-10-30T16:30:07Z

pkgs/build-support/shell-builder.nix

+
+  buildInputs = mergeInputs' "buildInputs";
+  nativeBuildInputs = mergeInputs' "nativeBuildInputs";
+  propagatedBuildInputs = mergeInputs' "propagatedBuildInputs";


You are missing propagatedNativeBuildInputs.

Thanks, I didn't know this existed, but makes sense. Fixed in b111860 which will be squashed later

gilligan · 2017-10-30T17:44:06Z

@zimbatm Could you also add documentation to the manual for this?

Edit: I see you asked where to document.. good question ;-)

gilligan · 2017-11-08T14:34:46Z

bump - i would love to see this merged. What's the status?

orivej · 2017-11-08T15:40:23Z

pkgs/top-level/all-packages.nix

@@ -301,6 +301,8 @@ with pkgs;
      inherit kernel rootModules allowMissing;
    };

+  mkShell = callPackage ../build-supports/shell-builder.nix { };


This should be build-support, not build-supports.

I'd suggest moving it into ../build-support/mkshell/default.nix anyway. Quiet a few derivations there are already single file.

orivej · 2017-11-08T15:57:09Z

pkgs/build-support/shell-builder.nix

+  buildInputs ? [],
+  nativeBuildInputs ? [],
+  propagatedBuildInputs ? [],
+  ...


You are missing propagatedNativeBuildInputs here (or not, and the others are not needed too?)

edolstra · 2017-11-08T16:16:28Z

pkgs/build-support/shell-builder.nix

+# A special kind of derivation that is only meant to be consumed by the
+# nix-shell.
+{
+  mergeInputs ? [], # a list of derivations whose inputs will be merged


This does not explain what it means for an input to be "merged". (And it's not clear to me from your description.)

Also, linguistically, mergeInputs sounds like a Boolean variable, i.e. whether or not to merge inputs, not "list of things to be merged".

How about inputsFrom? a list of derivations whose inputs will be made available to the environment

avnik · 2017-11-09T11:18:17Z

Nice,
can it be a base for myEnvFun replacement?

zimbatm · 2017-12-20T23:58:03Z

I realize that the doc needs to be improved, will do another round tomorrow.

vaibhavsagar · 2018-01-19T02:49:38Z

Is this coming to nixos-17.09?

zimbatm · 2018-01-19T10:14:45Z

This is a new feature so it will land in 18.04

CMCDragonkai · 2018-07-05T01:59:40Z

The inputsFrom is so intriguing.

I'm wondering if one major usecase is say you have multiple projects each with their own shell.nix, and you could create 1 master shell.nix that merges all subproject's shell.nix in case you wanted to work on all the subprojects together.

@zimbatm Could you enumerate what other usecases you envision for inputsFrom?

Also with regards to merging shell hook, this is definitely something that would be cool as well as long as the shell hooks were written in a way that didn't conflict with each other. My project's shell hook tends to be creating relevant environment variables. So they could be automerged. However I'd imagine in some cases you wouldn't want to auto merge them. Maybe some function that can explicitly compose the shell hooks based on the inputsFrom.

zimbatm · 2018-07-05T11:22:03Z

Yes that's exactly the use-case that I had in mind. Imagine you have a monorepo with multiple services, each in their own folder with their own default.nix that you can nix-shell and nix-build. On the top-level you can then add a shell.nix with, mkShell { inputsFrom = [ serviceA serviceB ]; to pull all the dependencies together.

shellHooks are a bit more tricky because they also tend to assume things about $PWD. For example if you have a node package it might add $PWD/node_modules/.bin to the PATH. But that only works if you start the shell within the frontend service for example.

CMCDragonkai · 2018-07-07T09:35:35Z

How should mkShell be used in comparison to higher order shell functions like haskell.lib.buildStackProject or pythonPackages.buildPythonApplication that already set things up relevant to stack or python build environment? Could potentially mkShell be used to eventually compose a stack and python environment together in case you're working with Python and Haskell at the same time?

zimbatm · 2018-07-07T14:34:21Z

I don't have a satisfying answer and would be interested in figuring this out. Aside from changing the PATH it's going to be hard to prove that two shells are going to compose nicely and that the application is commutative.

One idea is to require each environment to provide a function that takes the project root as an argument. Eg: nodePackage.shellHook could be executed to be setup for the $PWD/frontend. But then it doesn't work if there are two nodejs projects in the same repo.

For now I just use mkShell to pull project-global dependencies like deployment tooling and when the $language build tools can just be imported into the PATH and knows how to do the right thing based on $PWD alone. And if necessary I compose the things by hand by writing my own shellHook. It's more work but also creates less surprises in the event of a nixpkgs update.

nixos-discourse · 2022-10-09T01:26:02Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-shell-tips-tricks-and-best-practices/22332/1

zimbatm force-pushed the mkShell branch from 1497fdc to a6eb34a Compare October 30, 2017 15:55

orivej reviewed Oct 30, 2017

View reviewed changes

orivej reviewed Nov 8, 2017

View reviewed changes

edolstra reviewed Nov 8, 2017

View reviewed changes

zimbatm force-pushed the mkShell branch from 912d6e1 to db6136f Compare November 25, 2017 14:48

mkShell: add builder

a5eabe9

zimbatm force-pushed the mkShell branch from db6136f to a5eabe9 Compare December 20, 2017 23:41

zimbatm merged commit adc5c9b into master Dec 20, 2017

zimbatm deleted the mkShell branch December 20, 2017 23:42

garbas mentioned this pull request Jan 4, 2018

2018/01 weekly newsletter, collecting content NixOS/nixos-weekly#41

Merged

toraritte mentioned this pull request Jul 15, 2019

mkShell needs more documentation in the manual #58624

Open

veprbl mentioned this pull request Jan 12, 2021

gcc_multi build input ignored #107504

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mkShell: add builder #30975

mkShell: add builder #30975

zimbatm commented Oct 30, 2017

zimbatm commented Oct 30, 2017

zimbatm commented Oct 30, 2017

orivej Oct 30, 2017

zimbatm Oct 30, 2017

gilligan commented Oct 30, 2017 •

edited

Loading

gilligan commented Nov 8, 2017

orivej Nov 8, 2017

orivej Nov 8, 2017 •

edited

Loading

zimbatm Nov 25, 2017

orivej Nov 8, 2017 •

edited

Loading

zimbatm Nov 25, 2017

edolstra Nov 8, 2017

zimbatm Nov 25, 2017

avnik commented Nov 9, 2017

zimbatm commented Dec 20, 2017

vaibhavsagar commented Jan 19, 2018

zimbatm commented Jan 19, 2018

CMCDragonkai commented Jul 5, 2018 •

edited

Loading

zimbatm commented Jul 5, 2018

CMCDragonkai commented Jul 7, 2018

zimbatm commented Jul 7, 2018 •

edited

Loading

nixos-discourse commented Oct 9, 2022

mkShell: add builder #30975

mkShell: add builder #30975

Conversation

zimbatm commented Oct 30, 2017

Motivation for this change

zimbatm commented Oct 30, 2017

zimbatm commented Oct 30, 2017

orivej Oct 30, 2017

Choose a reason for hiding this comment

zimbatm Oct 30, 2017

Choose a reason for hiding this comment

gilligan commented Oct 30, 2017 • edited Loading

gilligan commented Nov 8, 2017

orivej Nov 8, 2017

Choose a reason for hiding this comment

orivej Nov 8, 2017 • edited Loading

Choose a reason for hiding this comment

zimbatm Nov 25, 2017

Choose a reason for hiding this comment

orivej Nov 8, 2017 • edited Loading

Choose a reason for hiding this comment

zimbatm Nov 25, 2017

Choose a reason for hiding this comment

edolstra Nov 8, 2017

Choose a reason for hiding this comment

zimbatm Nov 25, 2017

Choose a reason for hiding this comment

avnik commented Nov 9, 2017

zimbatm commented Dec 20, 2017

vaibhavsagar commented Jan 19, 2018

zimbatm commented Jan 19, 2018

CMCDragonkai commented Jul 5, 2018 • edited Loading

zimbatm commented Jul 5, 2018

CMCDragonkai commented Jul 7, 2018

zimbatm commented Jul 7, 2018 • edited Loading

nixos-discourse commented Oct 9, 2022

gilligan commented Oct 30, 2017 •

edited

Loading

orivej Nov 8, 2017 •

edited

Loading

orivej Nov 8, 2017 •

edited

Loading

CMCDragonkai commented Jul 5, 2018 •

edited

Loading

zimbatm commented Jul 7, 2018 •

edited

Loading