Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bundlerApp: add support for wildcards [WIP] #64238

Closed
wants to merge 1 commit into from

Conversation

peterhoeg
Copy link
Member

Motivation for this change

We currently quote the executable we link to in a bundlerApp which is neat because it supports files with spaces, but the problem is that we then cannot use wildcards in the bundlerApp.exes property.

None of the existing bundlerApp users in nixpkgs has executables with spaces in their names.

Supporting regular wildcards, means we can do stuff like this:
exes = [ "check-*" "metrics-*" ]; which is great when one is using a script to automatically package a lot of ruby plugins for sensu.

Cc: @manveru @cstrahan @lilyball

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@alyssais
Copy link
Member

alyssais commented Jul 3, 2019

I’m personally -1 on this, because I don’t think a high-level Nix function should be exposing low level shell functionality. I won’t block it if somebody else thinks it’s a good idea though.

@lilyball
Copy link
Member

lilyball commented Jul 3, 2019

I'm not a fan of this. I agree that it's unexpected for the entries in the exes property to actually be arbitrary unquoted shell inputs.

If anything, we should be updating this code to use the library function for correctly quoting shell strings (I forget offhand what it is but it's in there) instead of just assuming a single quote is sufficient. Granted, an exe with a single quote in it would be rather surprising, but it would currently break the code.

If there's a compelling reason to want to support wildcards then we should expose that as a separate property.

@zimbatm
Copy link
Member

zimbatm commented Jul 4, 2019

For gems where the list varies a lot it's nice to be able to pattern-match. Is there any other way to implement this that doesn't rely on forwarding the logic to bash?

@zimbatm
Copy link
Member

zimbatm commented Jul 4, 2019

Maybe add an exesPrefix argument that would escape the prefix with the globbing added to the end?

${(lib.concatMapStrings (x: "ln -s '${basicEnv}/bin/${x}'* $out/bin/${x};\n") exesPrefix)}

@peterhoeg peterhoeg changed the title bundlerApp: do not quote so we can use wildcards bundlerApp: do not quote so we can use wildcards [WIP] Jul 5, 2019
@peterhoeg peterhoeg changed the title bundlerApp: do not quote so we can use wildcards [WIP] bundlerApp: add support for wildcards [WIP] Jul 5, 2019
@peterhoeg
Copy link
Member Author

Let me recap:

  1. Everybody agrees that the current situation is not ideal due to quoting possibly breaking
  2. The current code will also happily create a symlink to an executable that doesn't exist
  3. The proposed PR doesn't cut it as we blur the lines between bash and nix by writing bash globs in nix
  4. Instead of changing the functionality of exes, introduce a new one.

I'll address points 2 to 4 and resubmit.

@zimbatm
Copy link
Member

zimbatm commented Jul 7, 2019

I think there is also an issue if the prefix matches multiple results. It might be best to use something like this;

set -o nullglob
${(lib.concatMapStrings (x: "for file in '${basicEnv}/bin/${x}'* ; do ln -s "$file" "$out/bin/$(basename "$file")"; done;\n") exesPrefix)}

@lilyball
Copy link
Member

lilyball commented Jul 7, 2019

ln supports having multiple source files. Really the problem with just using the glob as-is is if it matches none at all.

@peterhoeg
Copy link
Member Author

I'm going to be using find .

@lilyball
Copy link
Member

lilyball commented Jul 8, 2019

How is that any better than the shell for loop version that @zimbatm suggested?

@peterhoeg
Copy link
Member Author

For 2 reasons:

  1. We have to worry about whitespaces with a for loop, and
  2. The nullglob option is a bash thing where as find is POSIX, so let's use the standard option.

@zimbatm
Copy link
Member

zimbatm commented Jul 9, 2019

I don't mind either way, as long as it's correct it doesn't matter that much

@stale
Copy link

stale bot commented Jun 1, 2020

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the
    related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
    irc.freenode.net.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 1, 2020
@peterhoeg peterhoeg self-assigned this Jun 2, 2020
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 2, 2020
@ryantm ryantm marked this pull request as draft October 23, 2020 03:06
@stale
Copy link

stale bot commented Apr 26, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Apr 26, 2021
@peterhoeg
Copy link
Member Author

I'm going to go ahead and drop this.

Don't really have a need for it any longer.

@peterhoeg peterhoeg closed this Mar 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants