-
-
Notifications
You must be signed in to change notification settings - Fork 15.5k
dockerTools: Add buildNixShellImage
#78967
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
b62ddc9
to
4751e94
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Builds and functions as expected on NixOS. Thanks for the documentation!
is it possible to mount source as a volume?
should make |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice to add a test in nixos/tests/docker-tools.nix
. I think building and executing hello would be perfect ;)
, drv | ||
, tag ? null | ||
# Extra packages available inside the image | ||
, extraPackages ? [] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about calling this contents
? It seems to me it behaves like the contents
attribute of our other docker tools.
let | ||
|
||
finalDrv = drv.overrideAttrs (old: { | ||
nativeBuildInputs = old.nativeBuildInputs or [] ++ extraPackages; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not just adding extraPackages
in the contents list?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm it's not the same thing: contents
are added to the /
path, but extraPackages
are things that are added to PATH
(and other env vars) later by stdenv's setup-hook. So by using nativeBuildInputs
all kinds of dependencies should work, though now I'm debating whether that's even needed. Maybe it's needed for man
support or so though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIUC, this is to allow a user to add extra tooling in its container. I think, in other images, this use case is covered by the contents
attribute, even if it is less convenient than what you are doing (because enviroment variables are not set). In order to have a coherent dockerTools
API, I would prefer to expose the dockerTools.buildLayeredImage
contents
and env
attributes: if the user needs extra tooling, it would be still possible to add packages with contents and set some environment variables with env
. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thinking about this again, I feel like extraPackages
should be removed, because people can just use .overrideAttrs
on the derivation they pass in instead.
Similarly, I don't think an env
is needed, because this can also be achieved by setting that in the derivation in multiple ways already:
{
VAR = "VAL";
shellHook = ''
export VAR=VAL
'';
}
I think the idea of passing through contents
is a good one though, I'll change it as such
Hello, I'm a bot and I thank you in the name of the community for your contributions. Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human. If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do: If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list. If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past. If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments. Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel. |
still important to me |
Me too. |
This tool looks really nice for preserving development environment for teams that don't universally use nix! However, given that nix is introducing flakes and Thanks. |
This would be amazing! |
|
||
# Pass --login such that /etc/profile gets loaded | ||
# Use bashInteractive for tab completion support and such | ||
config.Entrypoint = [ "${bashInteractive}/bin/bash" "--login" ]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is different from the bash
image, which has this entrypoint https://github.com/tianon/docker-bash/blob/master/5.1/docker-entrypoint.sh
When I try this in a normal session I get
$ bash --login echo foo
/run/current-system/sw/bin/echo: /run/current-system/sw/bin/echo: cannot execute binary file
Which seems to be a cryptic way of telling us that it's refusing to interpret an ELF executable as a script.
Passing a script does work
$ bash --login <(echo "echo hello") asdf
hello
So I think it should be like the official bash entrypoint.
The Cmd
should probably be ["bash"]
.
We'll need a test for this. You can add cases to nixos/tests/docker-tools.nix
, based on a new example you can add to pkgs/build-support/docker/examples.nix
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure I get the motivation. nix-shell
is bash-based, but it's not an image for the purpose of executing bash. In this case, we need some way to source stdenv and execute shellHook when it starts, which I'm now doing by passing --login
and writing the shell init to /etc/profile
. Currently this allows you to either do docker run -it <image>
, then run genericBuild
, or you can also do
docker run -it <image> -c genericBuild
And this works, which I don't think would be the case with your suggestion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree we seem to have different use cases in mind, but we can probably support both.
- use the nix-shell image to build something
- use the nix-shell image as a developer environment
For building, we should invoke the builder
with args
and not wrongly assume bashInteractive
.
For interactive use, I do think we should emulate the official bash image behavior.
Can't we support both?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I now changed it to have no entrypoint at all, however there's bash, all of coreutils and a special buildDerivation
binary in the PATH
. In addition, the Cmd
is set to [ "bash" ]
. I managed to get around the --login
requirement by using ~/.bashrc
instead. This means by default you'll get a nix-shell
environment, but you can also do any of
$ docker run -it <image> # nix-shell
$ docker run -it <image> bash # nix-shell
$ docker run -it <image> buildDerivation # build the derivation directly
$ docker run -it <image> bash -c 'buildDerivation && ls $out/bin' # builds derivation and lists the binaries it produced
# We can't just use `toString` on all derivation attributes because that | ||
# would not put path literals in the closure. So we explicitly copy | ||
# those into the store here | ||
stringValue = value: if builtins.typeOf value == "path" then "${value}" | ||
else if builtins.isList value then toString (map stringValue value) | ||
else toString value; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we're basically emulating the nix sandbox env setup here. Structured attrs and passAsFile are missing, but that's probably ok for a start.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, didn't want to bother with structuredAttrs, but I guess I can issue a warning when it's enabled
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An assertion now triggers when __structuredAttrs is set. The current code can't handle __structuredAttrs at all really.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about passAsFile
though?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah that didn't work before, but I implemented support for it now :)
cd /home | ||
'' + extraInit; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
extraInit
seems like a rather arbitrary term that doesn't have a well-understood purpose. Can't we remove it and rely on the user extending shellHook
beforehand, or setting an appropriate Cmd
?
cd /home
breaks the WorkingDir
config item.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I'm removing extraInit
in the next iteration. Can also be done with just shellHook
.
WorkingDir
can't be configured in the current version anyways
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With docker you can always configure any config attribute afterwards.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now using WorkingDir
directly to specify this, instead of cd
ing
This also readds support for those arguments to buildLayeredImage, which was removed in 560201d
4751e94
to
840b464
Compare
In addition to above things, I also changed the derivation output variables to point directly at the correct |
''; | ||
|
||
# A mapping from output name to the nix store path where they should end up | ||
outputPaths = lib.genAttrs drv.outputs (output: builtins.unsafeDiscardStringContext drv.${output}.outPath); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these paths writable in the store?
We should have a test case for this as well.
It would be nice to have symlinks to these paths in a fixed location for derived images to use.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are indeed writable. Perhaps I could create /home/user/outputs/* again, symlinking into the store
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now created symlinks from /home/user/outputs/*
into the /nix/store
paths
Result of |
840b464
to
085fd09
Compare
085fd09
to
1d8ecd8
Compare
@infinisil What else needs to be done for this? |
@3noch @infinisil mostly just tests by the looks of it. |
For the documentation, it is using Naively I'd just use the backward-compatibility tool to "expose" my |
@yipengsun You could move the |
exec ${lib.escapeShellArg (stringValue drv.drvAttrs.builder)} ${lib.escapeShellArgs (map stringValue drv.drvAttrs.args)} | ||
''; | ||
|
||
in buildLayeredImage { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I want to override some settings in this config. I also want to use streamLayeredImage
. So really it makes sense to just return the attrset directly and let the user decide where the image is actually realized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could use (buildLayeredImage args).stream
to solve that part of the problem.
I feel the increasing number of interfaces for building docker images is starting to become a problem, so I'm not excited about having a function that returns arguments for another function. I'd like to experiment with a more powerful interface during Ocean Sprint, so I'll know more in a month.
inherit uid gid; | ||
|
||
# By default just starts a shell | ||
config.Cmd = [ "bash" ]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should probably be easily overridable. I want to override it in my use case as I don't want to only be able to run containers interactively. Using shellHook
to run a task means it only works when .bashrc
is loaded which means it must be interactive.
I marked this as stale due to inactivity. → More info |
This is important to get in. |
This needs tests. |
@@ -496,4 +496,123 @@ buildImage { | |||
Creating base files like <literal>/etc/passwd</literal> or <literal>/etc/login.defs</literal> is necessary for shadow-utils to manipulate users and groups. | |||
</para> | |||
</section> | |||
<section xml:id="ssec-pkgs-dockerTools-buildNixShellImage"> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that file not yet converted to markdown?
I don't have any interest in this anymore. I think this would be really nice to get in, but I'll leave it up to someone that actually wants to use it ;). Anybody feel free to use the code from this PR |
Check out the new #172736, where I'm continuing work on this function |
Motivation for this change
Edit: An updated and improved version of this is being continued in #172736
This adds
buildNixShellImage
, which can be used to create a Docker image with an environment to build a derivation, similar to anix-shell
, but much purer.This PR was sponsored by Niteo ✨
Ping @grahamc @nlewo @danieldk
Things done
pkgs.hello
,pkgs.openssl
andpkgs.id3v2