Unbreak ghcide and bump haskell-lsp version #75449

turion · 2019-12-10T17:22:37Z

Motivation for this change

Things done

Notify maintainers

cc @

cdepillabout · 2019-12-11T01:15:02Z

pkgs/development/haskell-modules/configuration-hackage2nix.yaml

-  - haskell-lsp ==0.15.0.0
-  - haskell-lsp-types ==0.15.0.0
+  - haskell-lsp ==0.17.0.0
+  - haskell-lsp-types ==0.17.0.0


In general, the Haskell stuff in nixpkgs tracks the latest LTS release. We don't override these package versions with our own updated versions (since this could break a lot of stuff).

If you look at the top of this list here, you can see that these packages are locked to versions from the latest LTS. (peti updates these weekly, you can watch live if you're interested: https://www.twitch.tv/peti343)

It appears that the latest version of ghcide requires haskell-lsp ==0.17.*, haskell-lsp-types ==0.17.*, which is later than the version from LTS-14.

However, hackage2nix generates the latest versions of packages in the LTS as well, so haskellPackages.haskell-lsp_0_18_0_0 and haskellPackages.haskell-lsp-types_0_18_0_0 should be available.

I suggest trying to override ghcide to use these two package versions and then jailbreaking the cabal file.

Here's an example of hnix overriding package versions as well as using doJailbreak:

nixpkgs/pkgs/development/haskell-modules/configuration-common.nix

Lines 230 to 233 in 63f6681

hnix =

generateOptparseApplicativeCompletion "hnix" (

dontCheck (doJailbreak (super.hnix.override { these = self.these_0_7_6; }))

);

Could you update this PR to remove the above lines from configuration-hackage2nix.yaml, and instead adding an entry for ghcide to configuration-common.nix?

I suggest trying to override ghcide to use these two package versions and then jailbreaking the cabal file.

Oh, and let me know if this doesn't work. There is a way to force hackage2nix to generate arbitrary package versions (so we could make it generate haskellPackages.haskell-lsp_0_17_0_0 for ghcide).

Thanks for your explanations! Do I have to run hackage2nix locally to test my changes?

@turion No, peti runs hackage2nix locally, so you don't have to do it in this PR.

Unfortunately ghcide does not build with haskell-lsp-0.18. We probably need to make a haskellPackages.haskell-lsp_0_17_0_0 , but I don't know how. In parallel, I'll start and (send a PR to) upgrade ghcide to build with a newer haskell-lsp.

I see. I'm working towards 1. then first :)

ghcide with haskell-lsp-0.18 is released. I don't really understand how this change gets here. Do I just wait and rebase?

@turion

I don't really understand how this change gets here.

I think there is an automatic script that updates the haskell-updates branch with recent package versions from Hackage. (Or maybe it is semi-automatic and peti has to run it by hand)

You can see the current state of haskell-updates here (including a bunch of commits with the message hackage-packages.nix: automatic Haskell package set update):

#75429

At some point (maybe today or tomorrow?) one of these Haskell package set updates should contain the new version of ghcide. At that point you should be able to rebase this PR on haskell-updates and use the new version of haskell-lsp.

Ok. Thanks for all your explanations!

Is this whole workflow documented somewhere? Just in case peti goes on a 2 month vacation and someone needs to pick up the reign ;)

It is somewhat documented in the hackage2nix Readme:

https://github.com/NixOS/cabal2nix/tree/master/hackage2nix

There is also a youtube video showing the steps:

https://www.youtube.com/watch?v=qX0mgtSm360

Also, peti streams live these updates, so if you watch him doing it once or twice, you should get a good idea of how to do it:

https://discourse.nixos.org/t/please-join-the-new-weekly-haskell-updates-video-livestream/4420

peti

Can you please re-base this PR on top of a current version of haskell-updates to resolve the merge conflicts?

This update was generated by hackage2nix v2.15.0-9-g650d530 from Hackage revision commercialhaskell/all-cabal-hashes@dc5b0e7.

turion · 2019-12-18T10:47:04Z

Can you please re-base this PR on top of a current version of haskell-updates to resolve the merge conflicts?

@peti Yes, since the conflicts are only internal to haskell-updates. But it seems that you added a commit that obsoletes my PR. Also, it seems that it doesn't work anymore because haskell-lsp has advanced to 0.19.0.0 in the meantime.

turion · 2019-12-18T12:18:56Z

Another dependency, hie-bios, is broken on haskell-updates. Something with the test suite. I'll see whether I can fix it easily.

turion · 2019-12-18T12:33:06Z

Another dependency, hie-bios, is broken on haskell-updates. Something with the test suite. I'll see whether I can fix it easily.

Seems nontrivial to fix. I'll add dontCheck for the time being. This should be removed once there is an acceptable answer to haskell/hie-bios#25 (comment)

turion · 2019-12-18T13:11:38Z

Fixing hie-bios is separately in #75869.

turion · 2019-12-18T13:29:07Z

I'm a bit out of ideas. The upstream version for haskell-lsp is 0.19 by now, and ghcide is once again behind. It does not compile with the newer haskell-lsp, that's the latest thing I tried now. Do I have to pin the version of haskell-lsp at 0.18 somewhere? Is this a manual process that gets stuck every time upstream versions get out of sync?

mpickering · 2019-12-18T13:45:50Z

@turion You are discovering why people use haskell.nix.

cdepillabout · 2019-12-18T13:46:54Z

@turion What you can do is send a PR adding a line to this list:

nixpkgs/pkgs/development/haskell-modules/configuration-hackage2nix.yaml

Lines 2475 to 2536 in c8c7398

    
           extra-packages: 
        
             - aeson < 0.8                         # newer versions don't work with GHC 7.6.x or earlier 
        
             - aeson-pretty < 0.8                  # required by elm compiler 
        
             - apply-refact < 0.4                  # newer versions don't work with GHC 8.0.x 
        
             - aws ^>= 0.18                        # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - binary > 0.7 && < 0.8               # keep a 7.x major release around for older compilers 
        
             - binary > 0.8 && < 0.9               # keep a 8.x major release around for older compilers 
        
             - blank-canvas < 0.6.3                # more recent versions depend on base-compat-batteries == 0.10.* but we're on base-compat-0.9.* 
        
             - Cabal == 2.2.*                      # required for jailbreak-cabal etc. 
        
             - Cabal == 2.4.*                      # required for cabal-install etc. 
        
             - colour < 2.3.4                      # newer versions don't support GHC 7.10.x 
        
             - conduit >=1.1 && <1.3               # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - conduit-extra >=1.1 && <1.3         # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - containers < 0.5                    # required to build alex with GHC 6.12.3 
        
             - control-monad-free < 0.6            # newer versions don't compile with anything but GHC 7.8.x 
        
             - dbus <1                             # for xmonad-0.26 
        
             - deepseq == 1.3.0.1                  # required to build Cabal with GHC 6.12.3 
        
             - generic-deriving == 1.10.5.*        # new versions don't compile with GHC 7.10.x 
        
             - gloss < 1.9.3                       # new versions don't compile with GHC 7.8.x 
        
             - haddock == 2.22.*                   # required on GHC 8.0.x 
        
             - haddock-api == 2.22.*               # required on GHC 7.8.x 
        
             - happy <1.19.6                       # newer versions break Agda 
        
             - happy == 1.19.9                     # for purescript 
        
             - haskell-gi-overloading == 0.0       # gi-* packages use this dependency to disable overloading support 
        
             - haskell-src-exts == 1.19.*          # required by hindent and structured-haskell-mode 
        
             - hinotify == 0.3.9                   # for xmonad-0.26: https://github.com/kolmodin/hinotify/issues/29 
        
             - hoogle == 5.0.14                    # required by hie-hoogle 
        
             - html-conduit ^>= 1.2                # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - http-conduit ^>= 2.2                # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - inline-c < 0.6                      # required on GHC 8.0.x 
        
             - inline-c-cpp < 0.2                  # required on GHC 8.0.x 
        
             - lens-labels == 0.1.*                # required for proto-lens-descriptors 
        
             - mainland-pretty == 0.6.2.*          # required for tensorflow-opgen-0.1.0.0 
        
             - mtl < 2.2                           # newer versions require transformers > 0.4.x, which we cannot provide in GHC 7.8.x 
        
             - mtl-prelude < 2                     # required for to build postgrest on mtl 2.1.x platforms 
        
             - network == 2.6.3.1                  # newer versions don't compile with GHC 7.4.x and below 
        
             - network == 3.0.*                    # required by network-bsd, HTTP, and many others (2019-04-30) 
        
             - parallel == 3.2.0.3                 # newer versions don't work with GHC 6.12.3 
        
             - patience ^>= 0.1                    # required by chell-0.4.x 
        
             - persistent >=2.5 && <2.8            # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - persistent-sqlite < 2.7             # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - primitive == 0.5.1.*                # required to build alex with GHC 6.12.3 
        
             - proto-lens == 0.2.*                 # required for tensorflow-proto-0.1.x on GHC 8.2.x 
        
             - proto-lens-protobuf-types == 0.2.*  # required for tensorflow-proto-0.1.x on GHC 8.2.x 
        
             - proto-lens-protoc == 0.2.*          # required for tensorflow-proto-0.1.x on GHC 8.2.x 
        
             - QuickCheck < 2                      # required by test-framework-quickcheck and its users 
        
             - resolv == 0.1.1.2                   # required to build cabal-install-3.0.0.0 with pre ghc-8.8.x 
        
             - resourcet ==1.1.*                   # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - seqid < 0.2                         # newer versions depend on transformers 0.4.x which we cannot provide in GHC 7.8.x 
        
             - seqid-streams < 0.2                 # newer versions depend on transformers 0.4.x which we cannot provide in GHC 7.8.x 
        
             - split < 0.2                         # newer versions don't work with GHC 6.12.3 
        
             - tar < 0.4.2.0                       # later versions don't work with GHC < 7.6.x 
        
             - these == 0.7.6                      # required by hnix 0.6.1 
        
             - transformers == 0.4.3.*             # the latest version isn't supported by mtl yet 
        
             - vector < 0.10.10                    # newer versions don't work with GHC 6.12.3 
        
             - xml-conduit ^>= 1.7                 # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - yesod ^>= 1.4                       # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - yesod-core < 1.5                    # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - yesod-form < 1.5                    # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - yesod-persistent < 1.5              # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - yesod-static ^>= 1.5                # pre-lts-11.x versions neeed by git-annex 6.20180227 
        
             - yesod-test ^>= 1.5                  # pre-lts-11.x versions neeed by git-annex 6.20180227

like

- haskell-lsp == 0.18.0.0 # for ghcide-0.4 (or whatever is the correct ghcide version)

Once this gets merge into haskell-updates and the update script run, it will create a haskell-lsp_0_18_0_0 derivation. This will exist as long as the haskell-lsp == 0.18.0.0 line exists.

You should be able to use this package in an override to ghcide. That will hopefully get ghcide building.

turion · 2019-12-18T14:29:10Z

@turion You are discovering why people use haskell.nix.

Hehe, yes ;) but I haven't given up on nixpkgs becoming good enough for standard usage yet.

turion · 2019-12-18T14:39:24Z

@cdepillabout I guess I could, but this is quite a manual and tiresome cycle. I'll just wait until upstream has released a new version. https://github.com/digital-asset/ghcide/issues/277

This whole workflow feels like it's going to break every time e.g. haskell-lsp releases a newer version than ghcide can handle. Is that correct? How would this situation ever stabilize?

cdepillabout · 2019-12-18T15:44:38Z

@turion Sorry this is somewhat of a manual/annoying process.

This whole workflow feels like it's going to break every time e.g. haskell-lsp releases a newer version than ghcide can handle. Is that correct?

For the most part, yes.

How would this situation ever stabilize?

There are some cases where the situation would stabilize:

On release branches of nixpkgs.

release branches of nixpkgs don't get updates to the Haskell package versions. So if you get ghcide working on one of the release branches, it will probably stay working forever.
If ghcide and haskell-lsp are released to stackage (and end up in an LTS release).

nixpkgs tracks the latest LTS release, so packages from the LTS releases are almost always working, because they are all "known compatible".

If the upstream authors of ghcide and haskell-lsp don't want to get compatible versions into stackage, maybe you could?
If you add specific versions of haskell-lsp and ghcide to extra-packages in configuration-hackage2nix.nix as suggested above.

This will generate versions of haskell-lsp and ghcide like haskell-lsp_0_18_0_0 and ghcide_0_4_0_0 (or whatever) that you can use in overrides and are known to work together.

These should continue working at least until other dependencies get too new.
If ghcide and haskell-lsp stop frequently releasing new versions to Hackage.

Unfortunately, the Haskell stuff in nixpkgs doesn't work too well for new packages that aren't in Stackage and have frequent releases to Hackage.

As suggested by mpickering, creating an overlay with known package versions (or using a tool like haskell.nix to do this for you) might be the easiest solution.

Creating a ghcide.nix github repo that other people could also use might be helpful for the community as well.

turion · 2019-12-18T16:03:31Z

@cdepillabout Thank again you for your detailed explanations! It certainly helps me to understand the process and to more effectively contribute.

If ghcide and haskell-lsp are released to stackage (and end up in an LTS release).

Ah, good point! So LTS in Haskell on nixpkgs always refers to stackage LTS, I'm assuming now. There is a ticket tracking that: https://github.com/digital-asset/ghcide/issues/144
This sounds like the best option for me then. I'll see whether I can help with that.

peti · 2019-12-20T19:52:36Z

I have attempted to fix the ghcide build in Nixpkgs, too, but it is quite a mess. Honestly, I think it's much easier to work with upstream to fix ghcide's build with recent versions of its dependencies than trying to build it in Nixpkgs with old ones.

cdepillabout · 2019-12-21T00:26:41Z

I think @mdorman might have gotten ghcide working in #76103.

domenkozar · 2019-12-21T14:12:02Z

This is why I maintain https://github.com/hercules-ci/ghcide-nix

peti · 2019-12-27T11:17:07Z

@turion, do you still intend to update this PR? Or can it be closed?

turion requested review from basvandijk and cdepillabout as code owners December 10, 2019 17:22

ofborg bot added 6.topic: haskell 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Dec 10, 2019

turion force-pushed the haskell-updates_ghcide branch from da9ccdc to 7461625 Compare December 10, 2019 20:21

cdepillabout requested changes Dec 11, 2019

View reviewed changes

cdepillabout mentioned this pull request Dec 11, 2019

hprotoc: fix broken build #75473

Closed

10 tasks

peti force-pushed the haskell-updates branch 2 times, most recently from 14b0fb3 to 9c01e12 Compare December 13, 2019 19:49

peti requested changes Dec 18, 2019

View reviewed changes

cdepillabout and others added 2 commits December 18, 2019 11:06

haskellPackages.stack2cabal: mark unbroken

09b6fbb

hackage-packages.nix: automatic Haskell package set update

8bee1b9

This update was generated by hackage2nix v2.15.0-9-g650d530 from Hackage revision commercialhaskell/all-cabal-hashes@dc5b0e7.

peti force-pushed the haskell-updates branch from 3814616 to 8bee1b9 Compare December 18, 2019 10:07

WIP: Unbreak ghcide and bump haskell-lsp version

6417091

turion force-pushed the haskell-updates_ghcide branch from 7461625 to 6417091 Compare December 18, 2019 12:36

WIP: Try to build ghcide with haskell-lsp-0.19

68c96cb

peti force-pushed the haskell-updates branch from 6484351 to 1f79abf Compare December 20, 2019 19:53

peti force-pushed the haskell-updates branch 2 times, most recently from 1f7024e to 7d2cc64 Compare December 26, 2019 12:07

peti mentioned this pull request Dec 27, 2019

Update Haskell Package Set #75857

Merged

3 tasks

peti force-pushed the haskell-updates branch from d3a751a to aa46cdd Compare December 27, 2019 19:35

peti force-pushed the haskell-updates branch from aa46cdd to fdd64fb Compare December 27, 2019 19:37

peti closed this in 4d403a3 Dec 27, 2019

turion mentioned this pull request Jan 1, 2021

Please release version compatible with haskell-lsp-0.18 haskell/haskell-language-server#1034

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unbreak ghcide and bump haskell-lsp version #75449

Unbreak ghcide and bump haskell-lsp version #75449

turion commented Dec 10, 2019

cdepillabout Dec 11, 2019 •

edited

cdepillabout Dec 11, 2019

turion Dec 11, 2019

cdepillabout Dec 11, 2019

turion Dec 11, 2019 •

edited

turion Dec 12, 2019

turion Dec 12, 2019

cdepillabout Dec 12, 2019

turion Dec 13, 2019

cdepillabout Dec 13, 2019

peti left a comment

turion commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

mpickering commented Dec 18, 2019

cdepillabout commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

cdepillabout commented Dec 18, 2019 •

edited

turion commented Dec 18, 2019

peti commented Dec 20, 2019

cdepillabout commented Dec 21, 2019

domenkozar commented Dec 21, 2019

peti commented Dec 27, 2019

	hnix =
	generateOptparseApplicativeCompletion "hnix" (
	dontCheck (doJailbreak (super.hnix.override { these = self.these_0_7_6; }))
	);

Unbreak ghcide and bump haskell-lsp version #75449

Unbreak ghcide and bump haskell-lsp version #75449

Conversation

turion commented Dec 10, 2019

Motivation for this change

Things done

Notify maintainers

cdepillabout Dec 11, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turion Dec 11, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peti left a comment

Choose a reason for hiding this comment

turion commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

mpickering commented Dec 18, 2019

cdepillabout commented Dec 18, 2019

turion commented Dec 18, 2019

turion commented Dec 18, 2019

cdepillabout commented Dec 18, 2019 • edited

turion commented Dec 18, 2019

peti commented Dec 20, 2019

cdepillabout commented Dec 21, 2019

domenkozar commented Dec 21, 2019

peti commented Dec 27, 2019

cdepillabout Dec 11, 2019 •

edited

turion Dec 11, 2019 •

edited

cdepillabout commented Dec 18, 2019 •

edited