Add haskell-ide-engine. #32947

shlevy · 2017-12-21T19:38:57Z

Fixes #31697
References haskell/haskell-ide-engine#357

shlevy · 2017-12-21T19:39:23Z

@Fresheyeball @imalsogreg Can you test and see if this fits the bill of your bounty?

Fresheyeball · 2017-12-21T19:50:26Z

@shlevy While I love you forever for putting hie into nixpkgs. That was not the issue with the bounty. This is the issue with the bounty: haskell/haskell-ide-engine#357

shlevy · 2017-12-21T19:53:51Z

@Fresheyeball Yes, but if you pop haskell-ide-engine from the same package set as your package into your nix-shell env I believe this should work.

Fresheyeball · 2017-12-21T19:56:20Z

@shlevy ok, let me test.

peti

Looks good to me. I'd be happy if we can avoid checking the patch files into our repo, though. Is that possible?

peti · 2017-12-21T20:05:28Z

pkgs/development/haskell-modules/configuration-ghc-8.2.x.nix

@@ -71,4 +71,12 @@ self: super: {
  # https://github.com/aristidb/aws/issues/238
  aws = doJailbreak super.aws;

+  # There's no 8.2 testsuite


The changes to configuration-ghc-8.2.x.nix seem to be orthogonal to the addition of the HIE package set. Can you please submit those fixes separately (or commit them directly) to take them out of this PR for easier reviewing?

peti · 2017-12-21T20:06:41Z

pkgs/development/haskell-modules/configuration-ghc-8.2.x.nix

+  # There's no 8.2 testsuite
+  ghc-exactprint = dontCheck super.ghc-exactprint;
+
+  # Missing FlexibleContexts in test suite


Is upstream aware of this issue? If they are, do you have a link to the upstream ticket so that we can see the current state of this bug?

I'll open a ticket, thanks.

peti · 2017-12-21T20:07:43Z

pkgs/development/haskell-modules/configuration-ghc-8.2.x.nix

+  # Missing FlexibleContexts in test suite
+  monad-memo = dontCheck super.monad-memo;
+
+  haddock-api = super.haddock-api.override


Is this override specific to GHC 8.2.2? It looks like it might belong into configuration-nix.nix or configuration-common.nix.

Right, I confused myself about the order of overrides, I didn't want to overwrite the old versions uses in 7.10 and 7.8. Will fix.

peti · 2017-12-21T20:09:08Z

pkgs/development/haskell-modules/initial-packages.nix

@@ -0,0 +1,2 @@
+args@{ pkgs, stdenv, callPackage }: self:
+  (import ./hackage-packages.nix args self) // (import ./hie-packages.nix args self)


Please reverse the order of that set update to make sure that no entry in hie-packages can ever replace an entry of the same name in hackage-packages.

peti · 2017-12-21T20:09:41Z

pkgs/development/haskell-modules/patches/haddock-library-system-attoparsec.patch

@@ -0,0 +1,46 @@
+diff -Naur haddock-library-1.4.4-orig/haddock-library.cabal haddock-library-1.4.4/haddock-library.cabal


Are those patch files maybe available for download from somewhere so that we don't have to check them into our repository?

This one isn't available elsewhere. I should probably open an issue upstream, this is only needed because install doesn't work right yet with sublibraries.

peti · 2017-12-21T20:10:08Z

pkgs/development/haskell-modules/patches/hie-build-plugin-haddock.patch

@@ -0,0 +1,23 @@
+commit ae0fb7a79a8912cf7a40bee418db239a7188805c


Ditto. Can this file be downloaded from somewhere else?

This is from a PR, I'll fetch it

peti · 2017-12-21T20:10:16Z

pkgs/development/haskell-modules/patches/hie-testsuite.patch

@@ -0,0 +1,40 @@
+diff --git a/test/HaRePluginSpec.hs b/test/HaRePluginSpec.hs


Ditto. Can this file be downloaded from somewhere else?

This is a workaround, not a real fix, and doesn't exist anywhere yet.

shlevy · 2017-12-21T20:13:59Z

pkgs/development/haskell-modules/configuration-ghc-8.2.x.nix

@@ -71,4 +71,12 @@ self: super: {
  # https://github.com/aristidb/aws/issues/238
  aws = doJailbreak super.aws;

+  # There's no 8.2 testsuite


shlevy · 2017-12-21T20:14:11Z

pkgs/development/haskell-modules/configuration-ghc-8.2.x.nix

+  # There's no 8.2 testsuite
+  ghc-exactprint = dontCheck super.ghc-exactprint;
+
+  # Missing FlexibleContexts in test suite


I'll open a ticket, thanks.

shlevy · 2017-12-21T20:15:19Z

pkgs/development/haskell-modules/configuration-ghc-8.2.x.nix

+  # Missing FlexibleContexts in test suite
+  monad-memo = dontCheck super.monad-memo;
+
+  haddock-api = super.haddock-api.override


Right, I confused myself about the order of overrides, I didn't want to overwrite the old versions uses in 7.10 and 7.8. Will fix.

shlevy · 2017-12-21T20:15:53Z

pkgs/development/haskell-modules/hie-packages.nix

+    HaRe = self.HaRe_hie;
+in
+  { # This isn't really hie-specific, should probably go in cabal2nix
+    haddock-library_1_4_4 = callPackage


@peti What's the right way to add this to hackage-packages.nix?

Just add it to the extra-packages section in pkgs/development/haskell-modules/configuration-hackage2nix.yaml. Then it will appear in the main package set within a couple of days (i.e. whenever I think of merging haskell-updates into master).

shlevy · 2017-12-21T20:16:05Z

pkgs/development/haskell-modules/initial-packages.nix

@@ -0,0 +1,2 @@
+args@{ pkgs, stdenv, callPackage }: self:
+  (import ./hackage-packages.nix args self) // (import ./hie-packages.nix args self)


shlevy · 2017-12-21T20:17:02Z

pkgs/development/haskell-modules/patches/haddock-library-system-attoparsec.patch

@@ -0,0 +1,46 @@
+diff -Naur haddock-library-1.4.4-orig/haddock-library.cabal haddock-library-1.4.4/haddock-library.cabal


This one isn't available elsewhere. I should probably open an issue upstream, this is only needed because install doesn't work right yet with sublibraries.

shlevy · 2017-12-21T20:17:11Z

pkgs/development/haskell-modules/patches/hie-build-plugin-haddock.patch

@@ -0,0 +1,23 @@
+commit ae0fb7a79a8912cf7a40bee418db239a7188805c


This is from a PR, I'll fetch it

shlevy · 2017-12-21T20:17:23Z

pkgs/development/haskell-modules/patches/hie-testsuite.patch

@@ -0,0 +1,40 @@
+diff --git a/test/HaRePluginSpec.hs b/test/HaRePluginSpec.hs


This is a workaround, not a real fix, and doesn't exist anywhere yet.

Fixed fixable comments.

shlevy · 2017-12-22T15:14:37Z

@peti good now?

shlevy · 2017-12-23T14:50:42Z

This is now on top of #32993 to avoid the need to patch haddock-library

domenkozar · 2017-12-24T12:28:02Z

@shlevy are hie-packages.nix hand-crafted?

shlevy · 2017-12-24T15:01:39Z

@domenkozar Yes, though the bulk of it could be scripted pretty easily if updates become obnoxious

vcunat · 2017-12-26T15:20:02Z

Add to the next haskell-update iteration to avoid double re-building (and re-checking)?

shlevy · 2017-12-26T15:34:48Z

@vcunat I think that applies more to #32993 (which this PR is on top of) than this, but yes.

vcunat · 2017-12-26T16:00:35Z

Right, I didn't notice a commit was from a different PR.

peti

Great work! Please feel free to merge. I have added one more minor nit that you may want to address beforhand.

peti · 2017-12-29T14:49:23Z

pkgs/development/haskell-modules/configuration-common.nix

+  # https://github.com/haskell/cabal/issues/4969
+  haddock-library_1_4_4 = dontHaddock
+    (super.haddock-library_1_4_4.override
+       { # The 'attoparsec' haddock-library depends on is its own


This issue is fixed in 40a3d77. cabal2nix-1.7.1 no longer generates the bogus dependency.

michalrus · 2017-12-29T16:45:12Z

Silly question: will this handle Cabal@Nix projects, too? 😻 I.e. the Nix (not NixOS) part in haskell/haskell-ide-engine#357.

shlevy · 2017-12-29T18:12:28Z

@michalrus I don't really understand what you mean by Cabal@Nix projects, but the point of this PR was to address that issue 😄 . The idea is that your nix-shell (or nix-buffer, or whatever) env will include haskell-ide-engine from the same package set as your actual project, so you know it matches.

michalrus · 2017-12-30T13:27:01Z

Pure Cabal projects with deps managed by Nix. =]

Awesome, so it should just work. ♥

Thank you! ♥ ♥

Finally I can get rid of Intero and those hacks, which are ±useless, cf. michalrus/intero-nix-shim#7.

peti · 2017-12-30T17:18:02Z

haskell-ide-engine does not compile because the following dependency fails: https://hydra.nixos.org/build/66889657

imalsogreg · 2017-12-31T03:39:51Z

@shlevy I'm seeing

...
building path(s) ‘/nix/store/gbi2cwqamac15insqhb8yzl338bwmzk0-haddock-api-2.18.1’, ‘/nix/store/gwq9pbhfq9z5349zyakb2bn7x5pcjf80-haddock-api-2.18.1-data’, ‘/nix/store/vcj4h08c41yasfdhxh4i92as1r8jmal0-haddock-api-2.18.1-doc’
setupCompilerEnvironmentPhase
Build with /nix/store/3b1axvsyr53f4f5nihvpxv3p4hskb3j7-ghc-8.2.2.
ln: failed to create symbolic link '/nix/store/gbi2cwqamac15insqhb8yzl338bwmzk0-haddock-api-2.18.1/lib/links/libHShaddock-library-1.4.4-BXedb9tEAJEKpUW5xu3aRK-attoparsec-ghc8.2.2.dylib': File exists
ln: failed to create symbolic link '/nix/store/gbi2cwqamac15insqhb8yzl338bwmzk0-haddock-api-2.18.1/lib/links/libHShaddock-library-1.4.4-G8iPfxLjJ1KJncaQAcQooZ-ghc8.2.2.dylib': File exists
builder for ‘/nix/store/zjq2bjsnvn6mg50izlnav0mkwjg6m93y-haddock-api-2.18.1.drv’ failed with exit code 1
building path(s) ‘/nix/store/9wcrbw3w1dgmy0g209jxs3adfykmi2fm-happy-1.19.8’, ‘/nix/store/x7adjlzcz42dhmzw60ixf9m7s84lwjvy-happy-1.19.8-doc’
cannot build derivation ‘/nix/store/ksb034msgmaq1pcbayij0vw4ny0gmdhl-hie-haddock-0.1.0.0.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/8jw9zx2l5id8lfgcg563w54mv8dvgzk0-haskell-ide-engine-0.1.0.0.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/8jw9zx2l5id8lfgcg563w54mv8dvgzk0-haskell-ide-engine-0.1.0.0.drv’ failed

(nix-env -f ~/nixpkgs -iA haskellPackages.haskell-ide-engine -j1) on macos.

shlevy · 2018-01-01T01:28:41Z

@peti This works as of 89e882f . Testing on latest master now.

shlevy · 2018-01-01T01:31:47Z

@Ericson2314 any idea about the issue @imalsogreg is seeing? Seems to have something to do with the dynamic loading stuff on darwin.

shlevy · 2018-01-01T01:36:02Z

@peti Ah, this is due to the lts bump. Will take a look soon.

Ericson2314 · 2018-01-01T07:08:37Z

@shelvy I think that's a Darwin impurity from previous runs.

shlevy · 2018-01-01T15:13:47Z

@Ericson2314 ? What impure dir is being referenced there?

Ericson2314 · 2018-01-01T22:50:57Z

@shlevy this sounds to be true, but I swear this hit this when working on the darwin dynamic linking stuff. The previous attempt earlier in the build (say preConfigure) would but a a bunch of symlinks in there to avoid using as many RPATHs`. If the first build failed then on the second attempt it would fail because for some reason $out wasn't cleaned out and the symlinks would still be there.

shlevy · 2018-01-02T04:36:49Z

@Ericson2314 That would be a huge bug in nix if true...

shlevy · 2018-01-02T16:40:39Z

@imalsogreg I think it should be fixed in #33333

shlevy · 2018-01-02T16:40:57Z

(but there's still the hoogle issue to fix 😄 )

imalsogreg · 2018-01-02T18:10:23Z

@shlevy @Ericson2314 Looking at the error message from cabal trying to solve version constraints, I tried (a) patching hoogle 5.0.15 to accept haskell-src-exts_1_19_1 and (b) dropping the nixpkgs override that sets haskell-src-exts to 1.20.1.

This worked on my ubuntu machine. But the patch to hoogle introduces some ugly CPP to allow the newest hoogle to support both haskell-src-exts versions. So I'm wondering, is there another way to support hie-hoogle here?

shlevy · 2018-01-02T20:40:27Z

@imalsogreg I'll fix that side of things in a later PR.

imalsogreg · 2018-01-02T23:36:12Z

@shlevy making sure I understand: I should not try to get the hoogle patch (which loosens its haskell-src-exts deps to include 1.19.1) merged?

shlevy · 2018-01-03T16:44:38Z

Oh, if you can get it merged upstream that'd also be good.

shlevy · 2018-01-03T20:02:26Z

@imalsogreg should be fixed in 707cfbf

peti · 2018-01-18T16:21:52Z

@shlevy, according to https://hydra.nixos.org/job/nixpkgs/haskell-updates/haskellPackages.haskell-ide-engine.x86_64-linux/all, the package hasn't seen a successful build yet, apparently because the test suite assumes network access. Would you mind taking a look at that issue?

shlevy · 2018-01-18T17:02:47Z

f12f2ed

michalrus · 2018-01-22T16:59:17Z

Hmm, I can’t build it on current unstable, 5402412:

Warning:
    This package indirectly depends on multiple versions of the same package. This is very likely to cause a compile failure.
      package ghc-mod-core (ghc-mod-core-5.9.0.0) requires haskell-src-exts-1.19.1-GkJUFo8Rp3b1KlAdoTXU6c
      package hlint (hlint-2.0.15-DnfSXCOnlvhJh25E4Cb1Bg) requires haskell-src-exts-1.20.1-835K5nW7Qg0K3DUFrUYhiW
      package haskell-src-exts-util (haskell-src-exts-util-0.2.1.2-4UhAxVuyQajDfxD9gJGXOC) requires haskell-src-exts-1.20.1-835K5nW7Qg0K3DUFrUYhiW

*** abort because of serious configure-time warning from Cabal
builder for ‘/nix/store/bkry8lqkj605av0ybmi1qismb42h5j4w-ghc-mod-core-5.9.0.0.drv’ failed with exit code 1
cannot build derivation ‘/nix/store/mk2i2g7lgg4f7lwpf0m1jbr2idv282f8-haskell-ide-engine-0.1.0.0.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/mk2i2g7lgg4f7lwpf0m1jbr2idv282f8-haskell-ide-engine-0.1.0.0.drv’ failed

michalrus · 2018-01-23T18:20:13Z

@shlevy ^

shlevy · 2018-01-24T04:35:36Z

I'll work on this but I'm not sure when I'll have time.

shlevy requested a review from peti December 21, 2017 19:38

shlevy mentioned this pull request Dec 21, 2017

Nix / Nixos support haskell/haskell-ide-engine#357

Closed

GrahamcOfBorg added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Dec 21, 2017

peti previously requested changes Dec 21, 2017

View reviewed changes

shlevy commented Dec 21, 2017

View reviewed changes

shlevy force-pushed the haskell-ide-engine branch from 5d8ae27 to de14451 Compare December 22, 2017 15:13

shlevy requested a review from peti December 22, 2017 15:14

shlevy force-pushed the haskell-ide-engine branch from de14451 to f4fabf5 Compare December 23, 2017 14:49

GrahamcOfBorg added 10.rebuild-darwin: 501+ 10.rebuild-linux: 501+ and removed 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Dec 23, 2017

vcunat added the 6.topic: haskell label Dec 26, 2017

peti self-assigned this Dec 28, 2017

peti approved these changes Dec 29, 2017

View reviewed changes

Add haskell-ide-engine

961907d

shlevy force-pushed the haskell-ide-engine branch from f4fabf5 to 961907d Compare December 29, 2017 19:41

shlevy merged commit 961907d into NixOS:master Dec 29, 2017

imalsogreg mentioned this pull request Jan 2, 2018

allow haskell-src-exts 1.19 ndmitchell/hoogle#232

Merged

infinisil mentioned this pull request Jan 31, 2018

haskell-ide-engine broken #34447

Closed

		@@ -0,0 +1,2 @@
		args@{ pkgs, stdenv, callPackage }: self:
		(import ./hackage-packages.nix args self) // (import ./hie-packages.nix args self)

		@@ -0,0 +1,46 @@
		diff -Naur haddock-library-1.4.4-orig/haddock-library.cabal haddock-library-1.4.4/haddock-library.cabal

		@@ -0,0 +1,23 @@
		commit ae0fb7a79a8912cf7a40bee418db239a7188805c

		@@ -0,0 +1,40 @@
		diff --git a/test/HaRePluginSpec.hs b/test/HaRePluginSpec.hs

Add haskell-ide-engine. #32947

Add haskell-ide-engine. #32947

Conversation

shlevy commented Dec 21, 2017

shlevy commented Dec 21, 2017

Fresheyeball commented Dec 21, 2017

shlevy commented Dec 21, 2017

Fresheyeball commented Dec 21, 2017

peti left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shlevy commented Dec 22, 2017

shlevy commented Dec 23, 2017

domenkozar commented Dec 24, 2017

shlevy commented Dec 24, 2017

vcunat commented Dec 26, 2017

shlevy commented Dec 26, 2017

vcunat commented Dec 26, 2017

peti left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michalrus commented Dec 29, 2017

shlevy commented Dec 29, 2017

michalrus commented Dec 30, 2017

peti commented Dec 30, 2017

imalsogreg commented Dec 31, 2017

shlevy commented Jan 1, 2018

shlevy commented Jan 1, 2018

shlevy commented Jan 1, 2018

Ericson2314 commented Jan 1, 2018

shlevy commented Jan 1, 2018

Ericson2314 commented Jan 1, 2018

shlevy commented Jan 2, 2018

shlevy commented Jan 2, 2018

shlevy commented Jan 2, 2018

imalsogreg commented Jan 2, 2018

shlevy commented Jan 2, 2018

imalsogreg commented Jan 2, 2018

shlevy commented Jan 3, 2018

shlevy commented Jan 3, 2018

peti commented Jan 18, 2018

shlevy commented Jan 18, 2018

michalrus commented Jan 22, 2018 • edited

michalrus commented Jan 23, 2018

shlevy commented Jan 24, 2018

michalrus commented Jan 22, 2018 •

edited