Pass hoogle-local's buildCommand as a file. #67228
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is necessary when a very large number of packages are included in the package database. Without this change, setting up the build environment will fail, since the environment will be too large. This simply applies the technique mentioned in NixOS#25057 to solve this problem.
|
CC @ttuegel |
cdepillabout
approved these changes
Aug 26, 2019
There was a problem hiding this comment.
LGTM
You can look at the produced buildCommand like the following:
$ nix-shell -E 'with (import ./. {}); haskellPackages.hoogleLocal { packages = with haskellPackages; [ conduit lens ])'
$ # now in nix-shell
$ cat $buildCommandPath
mkdir -p $out/share/doc/hoogle
echo importing builtin packages
for docdir in /nix/store/v0na4j4j950mz5ax4knm13nqvx0ifpwv-ghc-8.6.5-doc/share/doc/ghc*/html/libraries"/"*; do
name="$(basename $docdir)"
if [[ -d $docdir ]]; then
ln -sfn $docdir $out/share/doc/hoogle/$name
fi
done
echo importing other packages
ln -sfn /nix/store/2c2xbjq6q3n5w90r4dz9fcsp89svvvnm-reflection-2.1.4-doc/share/doc/reflection-2.1.4/html "$out/share/doc/hoogle/reflection"
ln -sfn /nix/store/phw92832z7mr8l2j5g4axl41pyzcr9na-parallel-3.2.2.0-doc/share/doc/parallel-3.2.2.0/html "$out/share/doc/hoogle/parallel"
ln -sfn /nix/store/30qrbnwkk522cz612xqd8rq067ibxlxs-invariant-0.5.3-doc/share/doc/invariant-0.5.3/html "$out/share/doc/hoogle/invariant"
ln -sfn /nix/store/kkc22mvsn30hms8i2vdkppci5y70pwj4-void-0.7.3-doc/share/doc/void-0.7.3/html "$out/share/doc/hoogle/void"
ln -sfn /nix/store/hf565np4xfnwsnvhz16l3p1sj4aw0yhc-adjunctions-4.4-doc/share/doc/adjunctions-4.4/html "$out/share/doc/hoogle/adjunctions"
ln -sfn /nix/store/x953z3iwcr5hi5dvvbc641n0yqa0sjjw-kan-extensions-5.2-doc/share/doc/kan-extensions-5.2/html "$out/share/doc/hoogle/kan-extensions"
ln -sfn /nix/store/s2vz7q1mp64myblzqv74xz8m6dllq2l9-transformers-base-0.4.5.2-doc/share/doc/transformers-base-0.4.5.2/html "$out/share/doc/hoogle/transformers-base"
ln -sfn /nix/store/vi8f5bgw1v1lngkblhng2rr0q4xw7hyl-semigroupoids-5.3.2-doc/share/doc/semigroupoids-5.3.2/html "$out/share/doc/hoogle/semigroupoids"
ln -sfn /nix/store/r8chmvi7056mc8sazj116ys23f6q7658-semigroups-0.18.5-doc/share/doc/semigroups-0.18.5/html "$out/share/doc/hoogle/semigroups"
ln -sfn /nix/store/7nag5j4nq8fcf86rhq3pq1nb17sll0l3-profunctors-5.3-doc/share/doc/profunctors-5.3/html "$out/share/doc/hoogle/profunctors"
ln -sfn /nix/store/60nkgv37q0p1mwlhfy6qiwb15n8jjls0-free-5.1.1-doc/share/doc/free-5.1.1/html "$out/share/doc/hoogle/free"
ln -sfn /nix/store/zdxjnchdsp3rndsipl0hq1xdn0dym6vb-StateVar-1.2-doc/share/doc/StateVar-1.2/html "$out/share/doc/hoogle/StateVar"
ln -sfn /nix/store/7p68c5sl5hx1jdvawcca6nzl3sfmmswc-contravariant-1.5.2-doc/share/doc/contravariant-1.5.2/html "$out/share/doc/hoogle/contravariant"
ln -sfn /nix/store/nqp41zy77w6lab1dcd7lywsv6w6db4fy-call-stack-0.1.0-doc/share/doc/call-stack-0.1.0/html "$out/share/doc/hoogle/call-stack"
ln -sfn /nix/store/38zn560bs5vl1rx6n8qb8mc13p3kx0ij-th-abstraction-0.3.1.0-doc/share/doc/th-abstraction-0.3.1.0/html "$out/share/doc/hoogle/th-abstraction"
ln -sfn /nix/store/f9yragyz2mnw2g52h5qvcspmh15i00cy-tagged-0.8.6-doc/share/doc/tagged-0.8.6/html "$out/share/doc/hoogle/tagged"
ln -sfn /nix/store/71fli0qilzgs7d633ix3nq0lw4yqlr0m-distributive-0.6-doc/share/doc/distributive-0.6/html "$out/share/doc/hoogle/distributive"
ln -sfn /nix/store/55ggzyqsjliq94gph75mwqp1qqji3w97-comonad-5.0.5-doc/share/doc/comonad-5.0.5/html "$out/share/doc/hoogle/comonad"
ln -sfn /nix/store/3lq5hcd7gdqc4af11783r2xsf0rpz75y-bifunctors-5.5.4-doc/share/doc/bifunctors-5.5.4/html "$out/share/doc/hoogle/bifunctors"
ln -sfn /nix/store/dnv9x88661qwh4f0rk6yc2mjzdw9mmiq-base-orphans-0.8.1-doc/share/doc/base-orphans-0.8.1/html "$out/share/doc/hoogle/base-orphans"
ln -sfn /nix/store/21q9nqn98kxj8vama2hg270xkghd9ip4-lens-4.17.1-doc/share/doc/lens-4.17.1/html "$out/share/doc/hoogle/lens"
ln -sfn /nix/store/273wpfqpqiszhv0hv0b0yqjf7v3nl3qr-unliftio-core-0.1.2.0-doc/share/doc/unliftio-core-0.1.2.0/html "$out/share/doc/hoogle/unliftio-core"
ln -sfn /nix/store/sh96cr7ml71sybnsnb9az2fhiapv96vk-resourcet-1.2.2-doc/share/doc/resourcet-1.2.2/html "$out/share/doc/hoogle/resourcet"
ln -sfn /nix/store/ynnkkslbkb76w09c7cghv35l3ba6zwiw-vector-algorithms-0.8.0.1-doc/share/doc/vector-algorithms-0.8.0.1/html "$out/share/doc/hoogle/vector-algorithms"
ln -sfn /nix/store/9mhfr02lq903ivzrqby09wdswgs2fc6z-primitive-0.6.4.0-doc/share/doc/primitive-0.6.4.0/html "$out/share/doc/hoogle/primitive"
ln -sfn /nix/store/y1sggx2ll77lm9ws1gnzpbhywdi0xw4h-vector-0.12.0.3-doc/share/doc/vector-0.12.0.3/html "$out/share/doc/hoogle/vector"
ln -sfn /nix/store/35qjrnz4ssaa697qw1rzlq77wwji3vi5-unordered-containers-0.2.10.0-doc/share/doc/unordered-containers-0.2.10.0/html "$out/share/doc/hoogle/unordered-containers"
ln -sfn /nix/store/llg7wcvmsz7gnp6kb3j9zy2789sk451z-split-0.2.3.3-doc/share/doc/split-0.2.3.3/html "$out/share/doc/hoogle/split"
ln -sfn /nix/store/5fxd3x3hpx9aff7l6xx8fhjammzkiid4-hashable-1.2.7.0-doc/share/doc/hashable-1.2.7.0/html "$out/share/doc/hoogle/hashable"
ln -sfn /nix/store/wxhf009p4g50w951b86rg40rqp8sc28n-mono-traversable-1.0.11.0-doc/share/doc/mono-traversable-1.0.11.0/html "$out/share/doc/hoogle/mono-traversable"
ln -sfn /nix/store/qg0mfbik67jns09lpxhb9wkmziw43zh8-transformers-compat-0.6.5-doc/share/doc/transformers-compat-0.6.5/html "$out/share/doc/hoogle/transformers-compat"
ln -sfn /nix/store/a21p5483s3fpkr8cjqf0jgss875akf7n-exceptions-0.10.2-doc/share/doc/exceptions-0.10.2/html "$out/share/doc/hoogle/exceptions"
ln -sfn /nix/store/dxz51yq6ldp45gc2f8ipnrh4qzxqz7gr-conduit-1.3.1.1-doc/share/doc/conduit-1.3.1.1/html "$out/share/doc/hoogle/conduit"
echo building hoogle database
hoogle generate --database $out/share/doc/hoogle/default.hoo --local=$out/share/doc/hoogle
echo building haddock index
# adapted from GHC's gen_contents_index
cd $out/share/doc/hoogle
args=
for hdfile in $(ls -1 *"/"*.haddock | grep -v '/ghc\.haddock' | sort); do
name_version=`echo "$hdfile" | sed 's#/.*##'`
args="$args --read-interface=$name_version,$hdfile"
done
/nix/store/ljz8yyc2jcq2zjbj3a51idvbq8zn9sn4-ghc-8.6.5/bin/haddock --gen-index --gen-contents -o . \
-t "Haskell Hierarchical Libraries" \
-p /nix/store/v0na4j4j950mz5ax4knm13nqvx0ifpwv-ghc-8.6.5-doc/share/doc/ghc*/html/libraries/prologue.txt \
$args
echo finishing up
mkdir -p $out/bin
substitute /nix/store/l0dx1n01vnk671fqbbxa53dn9y4cbdcf-hoogle-local-wrapper.sh $out/bin/hoogle \
--subst-var out --subst-var-by shell /nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23/bin/bash \
--subst-var-by hoogle /nix/store/r54mjhn1fk8bgrv04jrghs64a540v00b-hoogle-5.0.17.10
chmod +x $out/bin/hoogleI believe everything is as expected. You can see that without this change, the ln lines can get very long, and probably overflow the max length for environment variables.
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
cc @infinisil Also, this PR could probably be changed to be based on the |
|
Base changed to |
|
cc @peti and @matthewbauer for this as well. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is necessary when a very large number of packages are included in the
package database. Without this change, setting up the build environment will
fail, since the environment will be too large.
This simply applies the technique mentioned in #25057 to solve this problem.