Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redo: add doc to ghc wrapper #77523

Closed

Conversation

matthewbauer
Copy link
Member

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

This reverts commit 93aabab.
This reverts commit e915608.

Based on change in master at d1bd0fb
opened in NixOS#77442
This reverts commit 2395ac6.

Added allowCollisions to fix the issue mentioned in NixOS#77442
@@ -57,6 +57,8 @@ symlinkJoin {
# as a dedicated drv attribute, like `compiler-name`
name = ghc.name + "-with-packages";
paths = paths ++ [ghc];
extraOutputsToInstall = ["doc"];
allowCollisions = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignoring all file collisions in the ghc-wrapper feels like a hack. I'd rather not do this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's due to buildEnv propagating the dependencies. Not sure how to turn that off. I'd actually prefer to set allowCollisions here regardless because that is what symlinkJoin does by default.

For instance, compare:

{ pkgs ? import <nixpkgs> {} }:

pkgs.symlinkJoin {
  name = "test";
  paths = [
    (pkgs.writeTextDir "/share/doc/uuid-types-1.0.3/html/uuid-types.haddock" "a")
    (pkgs.writeTextDir "/share/doc/uuid-types-1.0.3/html/uuid-types.haddock" "b")
  ];
}

vs.

{ pkgs ? import <nixpkgs> {} }:

pkgs.buildEnv {
  name = "test";
  paths = [
    (pkgs.writeTextDir "/share/doc/uuid-types-1.0.3/html/uuid-types.haddock" "a")
    (pkgs.writeTextDir "/share/doc/uuid-types-1.0.3/html/uuid-types.haddock" "b")
  ];
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to treat collisions as an error. Because they usually are.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how to avoid the regression then. The two packages given in the revert (cabal-plan & aeson), depend on two different versions of uuid-types:

$ nix-store -qR $(nix-build '<nixpkgs>' -A haskellPackages.cabal-plan) | grep uuid-types
/nix/store/vs6xvan33d5jmhaybwkddmdfljigm050-uuid-types-1.0.3-doc
/nix/store/pv27lwjz900idmyrf0hykhfr95a2yk44-uuid-types-1.0.3
$ nix-store -qR $(nix-build '<nixpkgs>' -A haskellPackages.aeson) | grep uuid-types
/nix/store/s9babnwzfpq4vj02p20070pxfc6zpilx-uuid-types-1.0.3-doc
/nix/store/ciqb2va7qbhzb0b0s5ak289pnks3bfvd-uuid-types-1.0.3

We could just add map (lib.getOutput "doc") to the symlinkJoin, if that seems better.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how to remedy that issue. My problem with the "simple solution" is this: If different versions of a package exist, then these versions differ in the set of dependencies they link or flags they've been compiled with or something like that. Arguably, those differences may very well manifest in the API and in the generated documentation, so I would assume that those foobar.haddock files might differ, too. Ignoring collisions means that we'll just pick a random one and link it into the generated store path. But that doesn't feel right. The foobar.haddock file you'll get should be bundled in some obvious way with the library that it belongs to. This means that doc hierarchy might need $out baked into its directory name, maybe? Then there would be no more file collisions, because the *.haddock files would no longer be ambiguous. Does that make sense?

@stale
Copy link

stale bot commented Aug 5, 2020

Hello, I'm a bot and I thank you in the name of the community for your contributions.

Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do:

If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list.

If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past.

If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments.

Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Aug 5, 2020
@SuperSandro2000 SuperSandro2000 marked this pull request as draft November 29, 2020 04:32
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Nov 29, 2020
@stale
Copy link

stale bot commented Jun 2, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 2, 2021
sternenseemann added a commit to sternenseemann/nixpkgs that referenced this pull request Jan 14, 2023
* Will make it so that GHC.Paths's docdir NIX_GHC_DOCDIR points to an
  actual directory.

* Documentation of all packages in the environment is available in
  `$out/share/doc`.

This has previously been attempted in NixOS#76842 and reverted in NixOS#77442,
since documentation can collide when the libraries wouldn't (thanks to
the hash in the lib filename). `symlinkJoin` allows collision, so this
solution should be akin to NixOS#77523 (minus `buildEnv`, one step at a
time).

Collision in the documentation only happen if the dependency closure of
the given packages has more than one different derivation for the same
library of the very same version. I'm personally inclined not to claim
that our infrastructure does anything sensible in this case.
Additionally, the documentation is likely largely the same in such
cases (unless it is heavily patched).

Resolves NixOS#150666.
Resolves NixOS#76837.
Closes NixOS#150968.
Closes NixOS#77523.
sternenseemann added a commit to sternenseemann/nixpkgs that referenced this pull request Jan 14, 2023
* Will make it so that GHC.Paths's docdir NIX_GHC_DOCDIR points to an
  actual directory.

* Documentation of all packages in the environment is available in
  `$out/share/doc`.

This has previously been attempted in NixOS#76842 and reverted in NixOS#77442,
since documentation can collide when the libraries wouldn't (thanks to
the hash in the lib filename). `symlinkJoin` allows collision, so this
solution should be akin to NixOS#77523 (minus `buildEnv`, one step at a
time). `installDocumentation = false` restores the old behavior.

Collision in the documentation only happen if the dependency closure of
the given packages has more than one different derivation for the same
library of the very same version. I'm personally inclined not to claim
that our infrastructure does anything sensible in this case.
Additionally, the documentation is likely largely the same in such
cases (unless it is heavily patched).

Resolves NixOS#150666.
Resolves NixOS#76837.
Closes NixOS#150968.
Closes NixOS#77523.
lunik1 pushed a commit to lunik1/nixpkgs that referenced this pull request Jan 29, 2023
* Will make it so that GHC.Paths's docdir NIX_GHC_DOCDIR points to an
  actual directory.

* Documentation of all packages in the environment is available in
  `$out/share/doc`.

This has previously been attempted in NixOS#76842 and reverted in NixOS#77442,
since documentation can collide when the libraries wouldn't (thanks to
the hash in the lib filename). `symlinkJoin` allows collision, so this
solution should be akin to NixOS#77523 (minus `buildEnv`, one step at a
time). `installDocumentation = false` restores the old behavior.

Collision in the documentation only happen if the dependency closure of
the given packages has more than one different derivation for the same
library of the very same version. I'm personally inclined not to claim
that our infrastructure does anything sensible in this case.
Additionally, the documentation is likely largely the same in such
cases (unless it is heavily patched).

Resolves NixOS#150666.
Resolves NixOS#76837.
Closes NixOS#150968.
Closes NixOS#77523.
xanderio pushed a commit to xanderio/nixpkgs that referenced this pull request Feb 13, 2023
* Will make it so that GHC.Paths's docdir NIX_GHC_DOCDIR points to an
  actual directory.

* Documentation of all packages in the environment is available in
  `$out/share/doc`.

This has previously been attempted in NixOS#76842 and reverted in NixOS#77442,
since documentation can collide when the libraries wouldn't (thanks to
the hash in the lib filename). `symlinkJoin` allows collision, so this
solution should be akin to NixOS#77523 (minus `buildEnv`, one step at a
time). `installDocumentation = false` restores the old behavior.

Collision in the documentation only happen if the dependency closure of
the given packages has more than one different derivation for the same
library of the very same version. I'm personally inclined not to claim
that our infrastructure does anything sensible in this case.
Additionally, the documentation is likely largely the same in such
cases (unless it is heavily patched).

Resolves NixOS#150666.
Resolves NixOS#76837.
Closes NixOS#150968.
Closes NixOS#77523.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: haskell 10.rebuild-darwin: 1-10 10.rebuild-linux: 11-100
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants