Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

findlib: make scripts using #use "topfind" work #45799

Closed
wants to merge 1 commit into from

Conversation

xplat
Copy link
Contributor

@xplat xplat commented Aug 30, 2018

Motivation for this change

Up to now packages that use Ocaml as a scripting language have needed special handholding to work in nix (see pkgs/development/ocaml-modules/topkg for instance, because topfind which is used by most of them to find libraries is installed as part of this package, not ocaml itself, and therefore isn't in the default library search path for the Ocaml toplevel. This patch solves that issue by adding a setup hook to rewrite uses of topfind to use the full path.

This improves the situation for #16085 although it does not fully address interactive use.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@vbgl
Copy link
Contributor

vbgl commented Aug 31, 2018

Thanks for working on this.

If this makes topkg work out of the box, can you please simplify the topkg.run command accordingly as part of this PR; thanks.

As a minor remark, I find the wording “make [it] work” a bit misleading: it still does not work but is automatically rewritten into something that does; right?

@xplat xplat changed the title findlib: make #use "topfind" work findlib: make scripts using #use "topfind" work Aug 31, 2018
@xplat
Copy link
Contributor Author

xplat commented Aug 31, 2018

OK, I did this. Tried running nox-review but my nix daemon ran out of memory :(

@vbgl
Copy link
Contributor

vbgl commented Sep 1, 2018

@GrahamcOfBorg build ocamlPackages.decompress

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: ocamlPackages.decompress

Partial log (click to expand)

unpacking source archive /nix/store/lrdsqf4imhdzm99x0j28azzfiwr65m2k-source
source root is source
patching sources
configuring
no configure script, doing nothing
building
Cannot find file topfind.
Unknown directive `require'.
builder for '/nix/store/r88r3z2g1qi2152bmwyv5qg4crmj6j92-ocaml4.06.1-decompress-0.6.drv' failed with exit code 2
error: build of '/nix/store/r88r3z2g1qi2152bmwyv5qg4crmj6j92-ocaml4.06.1-decompress-0.6.drv' failed

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: ocamlPackages.decompress

Partial log (click to expand)

source root is source
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
no configure script, doing nothing
building
Cannot find file topfind.
Unknown directive `require'.
builder for '/nix/store/k2wx9wamx947ryq5sy2d680w1ly4rxmc-ocaml4.06.1-decompress-0.6.drv' failed with exit code 2
error: build of '/nix/store/k2wx9wamx947ryq5sy2d680w1ly4rxmc-ocaml4.06.1-decompress-0.6.drv' failed

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: ocamlPackages.decompress

Partial log (click to expand)

unpacking source archive /nix/store/lrdsqf4imhdzm99x0j28azzfiwr65m2k-source
source root is source
patching sources
configuring
no configure script, doing nothing
building
Cannot find file topfind.
Unknown directive `require'.
builder for '/nix/store/9jlrvqjg1052mk4alb44xa9rxfzh8rmp-ocaml4.06.1-decompress-0.6.drv' failed with exit code 2
error: build of '/nix/store/9jlrvqjg1052mk4alb44xa9rxfzh8rmp-ocaml4.06.1-decompress-0.6.drv' failed

@xplat
Copy link
Contributor Author

xplat commented Sep 1, 2018

Ugh, thanks for finding that ... looks like the substituter needs to be a bit more generous with the syntax it allows...

@Mic92
Copy link
Member

Mic92 commented Sep 2, 2018

You can also try nix-review instead of nox-review, which takes less memory for evaluation.

@xplat
Copy link
Contributor Author

xplat commented Sep 4, 2018

Hm. So I fixed decompress, but digestif fails because it has a #directory "pkg" directive before #use "topfind". This intentionally doesn't get patched because there could be a topfind in the new directory that would take precedence. So I'll have to fix packages like this individually I guess...

@vbgl
Copy link
Contributor

vbgl commented Sep 4, 2018

I’m starting to wonder whether this is a good thing to do (to rewrite source code so as to turn relative paths into absolute ones). It does not really help.

Thinking aloud: maybe a way to address this issue would be to acknowledge that findlib belongs to OCaml and ship topfind & co. as part of the standard library…

@xplat
Copy link
Contributor Author

xplat commented Sep 4, 2018

Honestly i'm not too against that idea. Another alternative would be to have a wrapper of ocaml in findlib, but that runs into PATH ordering issues; another yet is to have a wrapper in ocaml itself that lets setup hooks add options, like the C compilers use. Unfortunately the way ocaml handles include path options makes that one needlessly difficult in some ways...

I really only want these scripts to work out of the box, I am not wedded to this method.

@Mic92
Copy link
Member

Mic92 commented Sep 4, 2018

Make sense to have a wrapper, than people do not have to patch there source code as well.

@nixos-discourse
Copy link

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/ocaml-utop-on-nixos-not-detecting-libraries/1587/5

@mmahut
Copy link
Member

mmahut commented Aug 12, 2019

Are there any updates on this pull request, please?

@stale
Copy link

stale bot commented Jun 1, 2020

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the
    related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
    irc.freenode.net.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 1, 2020
@SuperSandro2000
Copy link
Member

Closing due to inactivity. Feel free to reopen the discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants