Jupyterhub #31871

ixxie · 2017-11-20T19:27:27Z

Motivation for this change

Initial packaging of Jupyterhub; also packaged the dependency python-oauth2.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

adisbladis · 2017-11-21T03:23:35Z

pkgs/development/python-modules/jupyterhub/default.nix

+buildPythonPackage rec {
+  pname = "jupyterhub";
+  version = "0.8.1";
+  name = "${pname}-${version}";


Since 78ed9da name defaults to pname + version so this is not required anymore.

adisbladis · 2017-11-21T03:23:53Z

pkgs/development/python-modules/python-oauth2/default.nix

+buildPythonPackage rec {
+  pname = "python-oauth2";
+  version = "1.0.1";
+  name = "${pname}-${version}";


Same here, pname not required.

adisbladis · 2017-11-21T03:24:15Z

pkgs/development/python-modules/python-oauth2/default.nix

+    sha256 = "0a1d0qnlgm07wq9r9bbm5jqkqry73w34m87p0141bk76lg7bb0sm";
+  };
+
+  doCheck = false;


Please add a comment why you are skipping tests

adisbladis · 2017-11-21T03:24:45Z

pkgs/development/python-modules/jupyterhub/default.nix

+      traitlets
+    ];
+
+  doCheck = false;


Please add a comment why you are skipping tests (or even better, run the tests)

adisbladis · 2017-11-21T03:26:16Z

pkgs/development/python-modules/jupyterhub/default.nix

+
+  buildInputs =
+    [
+      requests


requests should probably also be in propagatedBuildInputs.

adisbladis · 2017-11-21T03:31:44Z

pkgs/development/python-modules/jupyterhub/default.nix

+  buildInputs =
+    [
+      requests
+      nodejs-8_x


This looks a bit funky to me. Don't you need any nodePackages?

adisbladis · 2017-11-21T03:35:01Z

pkgs/development/python-modules/jupyterhub/default.nix

+    sha256 = "100cf18d539802807a45450d38fefbb376cf1c810f3b1b31be31638829a5c69c";
+  };
+
+  preBuild = "mkdir .home; HOME=$PWD/.home";


I would replace this with HOME=`mktemp -d`

adisbladis · 2017-11-21T03:36:57Z

Please squash the commits and change commit messages according to https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md

ixxie · 2017-11-22T19:04:00Z

Will sort it by the weekend hopefully!

cstrahan · 2017-11-22T22:13:17Z

I made the suggested changes, fixed a problem with pamela, and took care of packaging the configurable-http-proxy dependency here: #31950

ixxie · 2017-12-25T15:23:39Z

@FRidh - I merged #31950 into my own branch and made the requested changes.

grahamc · 2017-12-26T17:33:55Z

@GrahamcOfBorg eval

(sorry, master was broken and is now fixed)

aborsu · 2018-01-05T01:42:53Z

@ixxie @FRidh I made a service around jupyterhub here.
It's almost functional, I still need to create a basic kernel in tthe pre-start part of the service.

I'm not going to make a pull-request as I believe two open requests is already more than enough but feel free to incorporate my changes or if you want, I can add my changes to your PR.

ixxie · 2018-01-07T20:51:44Z

@adisbladis @FRidh - after much discussion and wrangling with a complicated commit history, I think I managed to address the required fixes to #31950 and pushed them back into this PR. Hopefully its now in order. Note that squashing was not as easy in all cases without losing contribution history, so this is about as good as I can get it.

@aborsu - that is awesome but I think that should become a separate PR - one for the package and one for the module -no?

aborsu · 2018-01-08T08:16:54Z

@ixxie, I'll get started as soon as this gets merged.

ixxie · 2018-01-17T19:40:53Z

@adisbladis @FRidh - I refreshed this once again since it went stale.

FRidh

Python part looks good to me. What is needed:

approval from a node package set maintainer
squash and name the commits according to the contributing guidelines

FRidh · 2018-01-18T10:23:11Z

cc @svanderburg regarding Node package set

ixxie · 2018-01-18T14:35:43Z

@FRidh - I had great difficulty squashing the commits because - ignorant as I was of good git practices - my commits mix up multiple packages; also, I had trouble when it came to squashing without loosing authorship information of @cstrahan's contributions. Any ideas on how to proceed?

aborsu · 2018-01-18T20:09:56Z

@ixxie if I can give you a little hand.

git rebase --interactive 6f5cf6b^

Will start an intarctive rebase just before your first commit.
Select the following:

pick 6f5cf6b90af Added myself (ixxie) as maintainer.
s 6d3515bc869 Initial commit of the Jupyterhub package (not building).
s a10221a7e82 First draft of pythonPackages.python-oauth2
s ae159093a6b Got python-oauth2 to build by setting doCheck = false.
s da7f0a10832 Got jupyterhub to build by tweaking depenendencies and phases.
s 77f31d6b1ea Style cleanup for jupyterhub python module.
s 2301b3871eb jupyterhub: init at 0.8.1
pick 94ef6d2c661 pythonPackages.pamela: fix resolution of libpam.so
d 8497568a75d jupyterhub: various tweaks/fixes
d 66a5b853cad Rebuilt Node packages.
d cd1e91a876f Rebuilt node packages again.

# Rebase 9be54c61a2c..cd1e91a876f onto 9be54c61a2c (11 commands)
#
# Commands:
# p, pick = use commit
# r, reword = use commit, but edit the commit message
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
# f, fixup = like "squash", but discard this commit's log message
# x, exec = run command (the rest of the line) using shell
# d, drop = remove commit
#

We are squashing all of the commits about jupyterhub together
and keeping the commit for pythonPackages.pamela: fix resolution of libpam.so

notes:

author information will be merged
we are dropping last 3 commits we will cherry-pick it later

This should go smoothly, and you should see this:

# This is a combination of 7 commits.
# This is the 1st commit message:

Added myself (ixxie) as maintainer.

# This is the commit message #2:

Initial commit of the Jupyterhub package (not building).

# This is the commit message #3:

First draft of pythonPackages.python-oauth2

# This is the commit message #4:

Got python-oauth2 to build by setting doCheck = false.

# This is the commit message #5:

Got jupyterhub to build by tweaking depenendencies and phases.

# This is the commit message #6:

Style cleanup for jupyterhub python module.

# This is the commit message #7:

jupyterhub: init at 0.8.1

Delete everything but the last line (This will be your final commit message).
And it should then apply the pamela fix.

Once it is done, we will cherry-pick @cstrahan's commit.

git-cherry pick df4b90b

Of course you get conflicts. (Don't worry)
pkgs/development/node-packages/node-packages-v6.nix is a generated file (You should not edit it by hand !!!)
So let's generate it (it will take a while):

cd pkgs/development/node-packages/
./generate.sh

Once this is over, add it and continue the cherry-pick:

git add node-packages-v6.nix
git cherry-pick --continue

and voilà !
It should look like there aborsu/nixpkgs#ixxie-jupyterhub

ixxie · 2018-01-19T13:57:39Z

@aborsu thanks for the tips! I followed your instructions, more or less, and it looks better at least. Hopefully @FRidh and @adisbladis approve.

aborsu · 2018-01-19T18:19:58Z

pkgs/development/node-packages/composition-v4.nix

@@ -1,4 +1,4 @@
-# This file has been generated by node2nix 1.5.1. Do not edit!


@ixxie If you had followed my instructions this would not be here. (Just saying ;) )

Dang! I did regenerate the files though, so I am not sure what went wrong...

Note the only deviation I made from your instructions is not to drop "jupyterhub: various tweaks/fixes" because it included some stuff that was needed for this to work. Then running ./generate.sh produced both a new node-packages-v4.nix and a node-packages-v6.nix.

@ixxie Your version of the commit with the message 'jupyterhub: various tweaks/fixes' had the commit starting with 8497568a75d while the one I told you to cherry-pick df4b90b was the one made by cstrahan.

In your version, you have changes to multiple files which are actually not the ones made by cstrahan but happened when you tried to deal with the original rebase.

If you go look at cstrahan's original commit df4b90b you will notice that he only modified 4 files

composition-v4.nix (which is only a newline at end of file.)

node-packages-v6.json (need to keep)

node-packages-v6.nix (Which is generated)

jupyterhub/default.nix (where he uses the node packages)

All of the other file changed in your commit come from the rebase and should not be there.

FRidh · 2018-01-20T13:35:29Z

I suggest using git reset --soft HEAD~3 and creating new commits that are according to the guidelines.

ixxie · 2018-01-20T16:54:40Z

@FRidh I believe now everything should at long last be in order. @cstrahan - I am deeply sorry but I didn't manage to preserve the authorship history correctly.

FRidh · 2018-01-20T19:19:37Z

@GrahamcOfBorg build python2.pkgs.jupyterhub python3.pkgs.jupyterhub

GrahamcOfBorg

Failure for system: x86_64-darwin

Package ‘linux-pam-1.3.0’ in /Users/graham/nix-borg/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/pkgs/os-specific/linux/pam/default.nix:53 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg

Failure for system: x86_64-linux

/tmp/nix-build-python3.6-jupyterhub-0.8.1.drv-0/jupyterhub-0.8.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1
strip is /nix/store/mdyy001q67hiks0g24ra53z7ckm4jfr4-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/lib  /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/bin 
patching script interpreter paths in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1
checking for references to /tmp/nix-build-python3.6-jupyterhub-0.8.1.drv-0 in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1...
wrapping `/nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/bin/jupyterhub-singleuser'...
wrapping `/nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/bin/jupyterhub'...
error: build of ‘/nix/store/b6b9xhljbc9sgk500lfhb7w0pysjk6lp-python2.7-jupyterhub-0.8.1.drv’ failed

GrahamcOfBorg

Failure for system: aarch64-linux

/build/jupyterhub-0.8.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1
strip is /nix/store/jwz859pxqj7sl2dbwvmxkx68jp774izb-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1/lib  /nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1/bin
patching script interpreter paths in /nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1
checking for references to /build in /nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1...
wrapping `/nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1/bin/jupyterhub'...
wrapping `/nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1/bin/jupyterhub-singleuser'...
error: build of '/nix/store/l3yg9vrlsr4mzx3p8s377gd6dc4dr9rq-python2.7-jupyterhub-0.8.1.drv' failed

FRidh

ERROR: JupyterHub requires Python version 3.4 or above.
builder for '/nix/store/7nx99i1gw60h56fwddyw3317k71iaxfi-python2.7-jupyterhub-0.8.1.drv' failed with exit code 1
error: build of '/nix/store/7nx99i1gw60h56fwddyw3317k71iaxfi-python2.7-jupyterhub-0.8.1.drv' failed

jupyterhub needs a

disabled = pythonOlder "3.4";

FRidh · 2018-01-20T19:34:26Z

@GrahamcOfBorg build python3.pkgs.jupyterhub

GrahamcOfBorg

Failure for system: x86_64-linux

error: while evaluating ‘callPackageWith’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/lib/customisation.nix:113:35, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/pkgs/top-level/python-packages.nix:9520:16:
while evaluating ‘makeOverridable’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/lib/customisation.nix:72:24, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/lib/customisation.nix:117:8:
undefined variable ‘pythonOlder’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/pkgs/development/python-modules/jupyterhub/default.nix:113:14

GrahamcOfBorg

Failure for system: aarch64-linux

error: while evaluating 'callPackageWith' at /var/lib/gc-of-borg/nix-test-rs-3/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-3/lib/customisation.nix:113:35, called from /var/lib/gc-of-borg/nix-test-rs-3/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-3/pkgs/top-level/python-packages.nix:9520:16:
while evaluating 'makeOverridable' at /var/lib/gc-of-borg/nix-test-rs-3/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-3/lib/customisation.nix:72:24, called from /var/lib/gc-of-borg/nix-test-rs-3/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-3/lib/customisation.nix:117:8:
undefined variable 'pythonOlder' at /var/lib/gc-of-borg/nix-test-rs-3/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-3/pkgs/development/python-modules/jupyterhub/default.nix:113:14

FRidh · 2018-01-20T19:45:25Z

@GrahamcOfBorg build python3.pkgs.jupyterhub

GrahamcOfBorg

Failure for system: x86_64-darwin

Package ‘linux-pam-1.3.0’ in /Users/graham/nix-borg/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/pkgs/os-specific/linux/pam/default.nix:53 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg

Success for system: aarch64-linux

/nix/store/1aiid8gz93mh27zm46dh6g4xkdmlkn02-python3.6-jupyterhub-0.8.1

GrahamcOfBorg

Success for system: x86_64-linux

/tmp/nix-build-python3.6-jupyterhub-0.8.1.drv-0/jupyterhub-0.8.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1
strip is /nix/store/mdyy001q67hiks0g24ra53z7ckm4jfr4-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/lib  /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/bin 
patching script interpreter paths in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1
checking for references to /tmp/nix-build-python3.6-jupyterhub-0.8.1.drv-0 in /nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1...
wrapping `/nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/bin/jupyterhub'...
wrapping `/nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1/bin/jupyterhub-singleuser'...
/nix/store/pr65n8611y7y5hknfcdkkdnni934z4dv-python3.6-jupyterhub-0.8.1

cstrahan · 2018-01-26T06:04:53Z

I am deeply sorry but I didn't manage to preserve the authorship history correctly.

@ixxie Don't sweat it -- nice work! :)

ixxie requested a review from FRidh as a code owner November 20, 2017 19:27

cstrahan self-assigned this Nov 20, 2017

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Nov 20, 2017

adisbladis requested changes Nov 21, 2017

View reviewed changes

FRidh added 6.topic: python 2.status: work-in-progress labels Nov 21, 2017

cstrahan mentioned this pull request Nov 22, 2017

jupyterhub - the sequel #31950

Closed

8 tasks

GrahamcOfBorg added 10.rebuild-linux: 11-100 and removed 10.rebuild-linux: 1-10 labels Dec 26, 2017

FRidh reviewed Jan 18, 2018

View reviewed changes

aborsu reviewed Jan 19, 2018

View reviewed changes

FRidh mentioned this pull request Jan 20, 2018

pythonPackages.oauthenticator & dependencies #34079

Merged

8 tasks

Matan Shenhav and others added 3 commits January 20, 2018 13:54

pythonPackage.python-oauth2: init at 1.0.1

c8300a0

pythonPackages.pamela: fix resolution of libpam.so

850799e

nodePackages.clean-css: init at 4.1.9

55f966b

Matan Shenhav added 3 commits January 20, 2018 15:36

nodePackages.configurable-http-proxy: init at 3.1.1

5bdc471

nodePackages.less: init at 2.7.3

845071c

nodePackages.less-plugin-clean-css: init at 1.5.1

400e9d0

GrahamcOfBorg reviewed Jan 20, 2018

View reviewed changes

FRidh approved these changes Jan 20, 2018

View reviewed changes

FRidh self-assigned this Jan 20, 2018

GrahamcOfBorg reviewed Jan 20, 2018

View reviewed changes

FRidh requested changes Jan 20, 2018

View reviewed changes

GrahamcOfBorg reviewed Jan 20, 2018

View reviewed changes

pythonPackages.jupyterhub: init at 0.8.1

e8ff1e0

FRidh merged commit fc1c0fe into NixOS:master Jan 20, 2018

GrahamcOfBorg reviewed Jan 20, 2018

View reviewed changes

ixxie deleted the jupyterhub branch January 20, 2018 20:18

Janik-Haag added the 12. first-time contribution label Jun 12, 2023

dotlambda mentioned this pull request Aug 19, 2023

configurable-http-proxy: use buildNpmPackage #250203

Merged

12 tasks

		@@ -1,4 +1,4 @@
		# This file has been generated by node2nix 1.5.1. Do not edit!

Jupyterhub #31871

Jupyterhub #31871

Conversation

ixxie commented Nov 20, 2017 • edited

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adisbladis commented Nov 21, 2017

ixxie commented Nov 22, 2017

cstrahan commented Nov 22, 2017

ixxie commented Dec 25, 2017 • edited

grahamc commented Dec 26, 2017

aborsu commented Jan 5, 2018 • edited

ixxie commented Jan 7, 2018

aborsu commented Jan 8, 2018

ixxie commented Jan 17, 2018

FRidh left a comment

Choose a reason for hiding this comment

FRidh commented Jan 18, 2018

ixxie commented Jan 18, 2018

aborsu commented Jan 18, 2018

ixxie commented Jan 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FRidh commented Jan 20, 2018

ixxie commented Jan 20, 2018

FRidh commented Jan 20, 2018

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

FRidh left a comment

Choose a reason for hiding this comment

FRidh commented Jan 20, 2018

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

FRidh commented Jan 20, 2018

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

cstrahan commented Jan 26, 2018

ixxie commented Nov 20, 2017 •

edited

ixxie commented Dec 25, 2017 •

edited

aborsu commented Jan 5, 2018 •

edited