Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/jupiter: add a kernel jsFile option #61177

Conversation

jraygauthier
Copy link
Member

@jraygauthier jraygauthier commented May 9, 2019

Motivation for this change

Required in order to get haskell syntax highlighting when using ihaskell as a jupyter kernel.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Relates to IHaskell/IHaskell#920 and IHaskell/IHaskell#1027

@jraygauthier
Copy link
Member Author

@aborsu / @sveitser: Does this seems fine to any of you?

@sveitser
Copy link
Contributor

I have never used this functionality of jupyter. Happy to test it if you can provide an example.

@jraygauthier
Copy link
Member Author

@sveitser Here's the config I use:

  services.jupyter.enable = true;
  services.jupyter.password = "'sha1:1b961dc713fb:88483270a63e57d18d43cf337e629539de1436ba'";
  services.jupyter.kernels = {
    python3 = let
      env = (pkgs.python3.withPackages (pythonPackages: with pythonPackages; [
        ipykernel
        pandas
        scikitlearn
      ]));
    in {
      displayName = "Python 3 for machine learning";
      argv = [
        "${env.interpreter}"
        "-m"
        "ipykernel_launcher"
        "-f"
        "{connection_file}"
      ];
      language = "python";
      logo32 = env + "/${env.sitePackages}/ipykernel/resources/logo-32x32.png";
      logo64 = env + "/${env.sitePackages}/ipykernel/resources/logo-64x64.png";
    };

    haskell = let
      callPackage = lib.callPackageWith pkgs;
      ihaskell = ./ihaskell-with-pkgs-unstable {};
      ihaskellEnv = ihaskell.ihaskellEnv;
      ihaskellWrapperSh = ihaskell.ihaskellWrapperSh;

    in {
      displayName = "Haskell";
      argv = [
        "${ihaskellWrapperSh}/bin/ihaskell-wrapper"
        "kernel"
        "--ghclib"
        "${ihaskellEnv}/lib/ghc-${ihaskellEnv.version}"
        # "--debug"
        "+RTS"
        "-M3g"
        "-N2"
        "-RTS"
        "{connection_file}"

      ];
      jsFile = ihaskell.ihaskellJsFile;
      logo64 = ihaskell.ihaskellLogo64;
      language = "haskell";
    };
  };

with ./ihaskell-with-pkgs-unstable defined as:

{ lib }:
  let
    ihaskellNixpkgsCfg = {
      # allowBroken = true;

      packageOverrides = super: let self = super.pkgs; in
        let
            # See pkgs/development/haskell-modules/lib.nix for available helpers.
            overrideCabal = super.haskell.lib.overrideCabal;
            unmarkBroken = drv: super.haskell.lib.overrideCabal drv (drv: { broken = false; });
            doJailbreak = super.haskell.lib.doJailbreak;
            overrideSrc = super.haskell.lib.overrideSrc;
            addSetupDepend = super.haskell.lib.addSetupDepend;
        in rec
      {
        haskell = super.haskell // {
          packageOverrides = self: super: rec {
            /*
            haskell-src-meta = super.haskell-src-meta.overrideScope (self: super: { haskell-src-exts = self.haskell-src-exts_1_21_0; });
            ihaskell-charts = doJailbreak (unmarkBroken super.ihaskell-charts);
            Chart = overrideCabal super.Chart (drv: {
              broken = false;
              jailbreak = true;
              # preConfigure = ''
              #   sed -i ihaskell-charts -E \
              #     -e 's#(lens[^<]+)< *4.17#\1<4.50#g'
              # '';
            });
            */
            static-canvas = doJailbreak (unmarkBroken super.static-canvas);

            Chart = self.callHackageDirect {
              pkg = "Chart";
              ver = "1.9.1";
              sha256 = "0aiq9r78yhma1xzhwl2xlzwqb2c59k6a4jjvdpninwm6lngac3s7";
            } {};
            Chart-cairo = self.callHackageDirect {
              pkg = "Chart-cairo";
              ver = "1.9.1";
              sha256 = "0ambbkxbndjqxr0c4pysn87b3z0f8zpgvldj4qjbyazadna9k807";
            } {};
          };
        };
      };
    };
    # ihaskellNixpkgs = pkgs;

    ihaskellNixpkgsSrc = builtins.fetchTarball {
      url = "https://github.com/NixOS/nixpkgs-channels/archive/2df17ecf1b748379a349f763f376bad173bff83b.tar.gz";
      sha256 = "10s6p0i3g4jkmdgfh3zb73baapks6gwcfl2p9hc3j490ly2wa0kv";
    };
    ihaskellNixpkgs = import "${ihaskellNixpkgsSrc}" { config = ihaskellNixpkgsCfg; };

    ihaskellCleanSource = name: type: let
      baseName = baseNameOf (toString name);
    in lib.cleanSourceFilter name type && !(
      (type == "directory" && (builtins.elem baseName [ ".stack-work" "dist"])) ||
      builtins.any (lib.flip lib.hasSuffix baseName) [ ".hi" ".ipynb" ".sock" ".yaml" ".yml" ]
    );
    ihaskellSourceFilter = src: name: type: let
      relPath = lib.removePrefix (toString src + "/") (toString name);
    in ihaskellCleanSource name type && !( builtins.any (lib.flip lib.hasPrefix relPath) [
      "notebooks" "demo" "docker" "image"
    ]);


    ihaskellRepoLocal = builtins.filterSource (ihaskellSourceFilter ../../IHaskell) ../../IHaskell;
    ihaskellRepoRemote = builtins.fetchTarball {
      url = "https://github.com/jraygauthier/IHaskell/archive/534c0ac45544d7a79c07027678a04eb4b40f2bc3.tar.gz";
      sha256 = "07x0y7w2iscs9607shrxsx9iyjgjg2pcq8fdzrx43iipn4376q9a";
    };

    # ihaskellRepo = ihaskellRepoLocal;
    ihaskellRepo = ihaskellRepoRemote;

    ihaskellSystemPackages = pkgs: [
    ];

    ihaskellPackages = hps: with hps; [
      ihaskell-blaze
      ihaskell-diagrams
      ihaskell-charts

      # [ihaskell-display: IHaskell display instances for basic types
      # ](http://hackage.haskell.org/package/ihaskell-display)
      # > Deprecated. in favor of ihaskell
      # ihaskell-display

      # Error: HList
      # ihaskell-aeson
      # Error: HList
      # ihaskell-rlangqq
      ihaskell-gnuplot
      ihaskell-graphviz
      ihaskell-hatex
      ihaskell-juicypixels
      ihaskell-magic
      # Error: No instance for (Control.Monad.Fail.MonadFail Axis)
      # ihaskell-plot
      ihaskell-static-canvas
      /*
      # Compiles but takes a lot of time to do so.
      # Also:
      # [ihaskell-widgets runs with no error, however, displays no widget ·
      # Issue #870 · gibiansky/IHaskell](https://github.com/gibiansky/IHaskell/issues/870)
      ihaskell-widgets
      */
    ];

    ihaskellRelease86 = (import "${ihaskellRepo}/release-8.6.nix" {
      nixpkgs = ihaskellNixpkgs;
      packages = ihaskellPackages;
      systemPackages = ihaskellSystemPackages;
    });
  in
ihaskellRelease86

@jraygauthier
Copy link
Member Author

There is ihaskell's *.js file: https://github.com/gibiansky/IHaskell/blob/master/html/kernel.js

@jraygauthier
Copy link
Member Author

For the curious, here's the PR that added passthrough for ihaskell release IHaskell/IHaskell#1027

@teto
Copy link
Member

teto commented May 16, 2019

Thanks for your work ! Will look into it this week end probably.
Btw, I've recently tried https://github.com/tweag/jupyterWith and was really impressed how well it worked.

@jraygauthier
Copy link
Member Author

@teto Nice, thanks for the ref. I wasn't aware of this jupyterWith project. This is pretty impressive.

@teto teto self-assigned this May 16, 2019
@jraygauthier jraygauthier force-pushed the jrg/nixos_jupyter_js_file_option branch from c944652 to d8b8ac8 Compare August 23, 2019 14:33
@jraygauthier
Copy link
Member Author

@teto: Just rebased on latest. Still interested in testing this?

@teto
Copy link
Member

teto commented Aug 23, 2019

It's a bit hard to test and the jupyterWith project seemed so much better than nixpkgs implementation that I kinda lost interest (had developed some local jupyter improvements as well). I hope they merged theirs. Eventually I can merge this as is if it passes the tests. Low-risk PR.

@jraygauthier jraygauthier force-pushed the jrg/nixos_jupyter_js_file_option branch from d8b8ac8 to c4ad72b Compare August 23, 2019 15:33
@jraygauthier
Copy link
Member Author

Alright.

Should be building fine now:

$ nix-shell -p nix-review --run "nix-review rev HEAD"
$ git fetch --force https://github.com/NixOS/nixpkgs master:refs/nix-review/0
$ git worktree add /home/rgauthier/.cache/nix-review/rev-c4ad72bcb8e822bd7fe5aa978f6d35b3b9dae587/nixpkgs a4083fb4551ce6dda68ac684a2ca951d2fa3ab72
Preparing worktree (detached HEAD a4083fb4551)
HEAD is now at a4083fb4551 Merge pull request #67236 from Chiiruno/dev/general
$ nix-env -f /home/rgauthier/.cache/nix-review/rev-c4ad72bcb8e822bd7fe5aa978f6d35b3b9dae587/nixpkgs -qaP --xml --out-path --show-trace
$ git merge --no-commit c4ad72bcb8e822bd7fe5aa978f6d35b3b9dae587
Automatic merge went well; stopped before committing as requested
$ nix-env -f /home/rgauthier/.cache/nix-review/rev-c4ad72bcb8e822bd7fe5aa978f6d35b3b9dae587/nixpkgs -qaP --xml --out-path --show-trace --meta
$ nix build --no-link --keep-going --max-jobs 4 --option build-use-sandbox true -f /home/rgauthier/.cache/nix-review/rev-c4ad72bcb8e822bd7fe5aa978f6d35b3b9dae587/build.nix
[8 built, 676 copied (3620.9 MiB), 823.9 MiB DL]
3 package were build:
jupyter sage sageWithDoc

@Profpatsch
Copy link
Member

cc @MMesch maybe we can add that to jupiterWith? Or even push jupiterWith to nixpkgs?

@stale
Copy link

stale bot commented Jul 24, 2020

Hello, I'm a bot and I thank you in the name of the community for your contributions.

Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do:

If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list.

If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past.

If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments.

Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 24, 2020
@jraygauthier
Copy link
Member Author

Using jupiterWith myself nowadays.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants