Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

makeWrapper: quote variables #31497

Merged
merged 1 commit into from Nov 14, 2017
Merged

Conversation

abbradar
Copy link
Member

Motivation for this change

Fix things like --prefix FOO ' ' 'bar baz'

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@orivej
Copy link
Contributor

orivej commented Nov 10, 2017

Could you post an example of the resulting wrapper before and after this change?

@abbradar
Copy link
Member Author

Before:

export NIX_CFLAGS_COMPILE=-isystem /nix/store/3cv0q9g9zyg5alzzz6072rhdmvzhk5yy-cuda-floatn.h/include${NIX_CFLAGS_COMPILE:+ }$NIX_CFLAGS_COMPILE

After:

export NIX_CFLAGS_COMPILE="-isystem /nix/store/3cv0q9g9zyg5alzzz6072rhdmvzhk5yy-cuda-floatn.h/include${NIX_CFLAGS_COMPILE:+ }$NIX_CFLAGS_COMPILE"

@orivej
Copy link
Contributor

orivej commented Nov 11, 2017

This swaps no quoting for approximately correct quoting. Please do not add double quotes, use "${value@Q}" and "${separator@Q}" instead of $value and $separator. The result will look like this:

export NIX_CFLAGS_COMPILE='-isystem /nix/store/3cv0q9g9zyg5alzzz6072rhdmvzhk5yy-cuda-floatn.h/include'${NIX_CFLAGS_COMPILE:+' '}$NIX_CFLAGS_COMPILE

@abbradar abbradar force-pushed the quote-makewrapper branch 2 times, most recently from 3387d98 to cfd9693 Compare November 11, 2017 10:15
@abbradar
Copy link
Member Author

abbradar commented Nov 11, 2017

Excellent, thank you for the catch! To be honest I didn't know about @Q Bash feature before, it makes many things which I have in mind simpler.

Can you review this version? It passes my new test for me with the following wrapper:

export FOO='ahaha $ \'
export BAR='ahaha $ \'
export BAZ='ahaha $ \'
export QUZ='ahaha $ \'
export FOO='bar '\''$baz'${FOO:+' '}$FOO
export BAR=$BAR${BAR:+'' ''}'bar '\''$baz'
export BAZ=$'test1\n test3\ntest4'${BAZ:+$}$BAZ
export QUZ=$QUZ${QUZ:+$}$'test1\n test3\ntest4'

@abbradar
Copy link
Member Author

@grahamc Bot forgot to remove old labels after a force-push.

@abbradar
Copy link
Member Author

abbradar commented Nov 11, 2017

How can I quote @Q in a commit message so that to not ping @Q the member with it? @q, sorry for the noise!

EDIT: fixed it with a rewording for now.

@@ -63,9 +63,9 @@ makeWrapper() {
n=$((n + 3))
if test -n "$value"; then
if test "$p" = "--suffix"; then
echo "export $varName=\$$varName\${$varName:+$separator}$value" >> "$wrapper"
echo "export $varName=\$$varName\${$varName:+'${separator@Q}'}${value@Q}" >> "$wrapper"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These ' should not be added.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, leftover. Fixed.

This uses Bash ${foo@Q} feature to quote values properly, which allows us to
handle values containing spaces, dollars etc.

Thanks orivej for the idea!
@orivej
Copy link
Contributor

orivej commented Nov 11, 2017

To be honest I didn't know about @Q Bash feature before, it makes many things which I have in mind simpler.

I've learnt about it not long ago; before that I'd have suggested to use $(printf %q "$var") which achieves the same a bit more verbosely.

Copy link
Contributor

@orivej orivej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now this looks good. (But I've not tested it.)

@orivej
Copy link
Contributor

orivej commented Nov 14, 2017

I have set up a Hydra cluster and tested this. Thank you!

@orivej orivej merged commit cd0e2f5 into NixOS:staging Nov 14, 2017
@orivej
Copy link
Contributor

orivej commented Nov 17, 2017

Somehow I missed some failures… This is causing a bit of trouble in staging in cases such as --set CC "$""{CC:-$CC""}" and --set RETROFE_PATH "\${RETROFE_PATH:-\$PWD}". I'm going to introduce --set-default for the first kind of usage (set to a constant unless already set) and --set-eval for the latter (set to an arbitrary expression).
/cc @vcunat

orivej added a commit to orivej/nixpkgs that referenced this pull request Nov 17, 2017
After NixOS#31497 starter quoting all values, there arouse the need to left some
values evaluated.

`--set-default var value` expands to `export var=${var-value}`, where value is
not evaluated and literally assigned to var unless it is already set.

`--set-eval var value` expands to `export var=$(eval echo value)`, where value
is evaluated by `eval`.
@orivej orivej mentioned this pull request Nov 17, 2017
8 tasks
@vcunat
Copy link
Member

vcunat commented Nov 18, 2017

The docs refer to the source files for such details, so I see no direct need to update the manual. It seems too low-level detail for release notes, too, not affecting any "normal" uses of the wrappers.

@orivej
Copy link
Contributor

orivej commented Nov 20, 2017

I've reviewed all usages of makeWrapper / wrapProgram in Nixpkgs and found 6 additional expressions that had to be updated in fb703ad (and eb3d207). It is not unlikely that when this reaches master there will be a couple runtime breakages invisible on Hydra and missed during the manual review.

@infinisil
Copy link
Member

Well that explains why a few of my wrappers suddenly stopped working..

kamilchm added a commit to kamilchm/nixpkgs that referenced this pull request Dec 8, 2017
running pinta after NixOS#31497
results in:

```
Unhandled Exception:
System.IO.FileNotFoundException: Could not load file or assembly 'glib-sharp, Version=2.12.0.0, Culture=neutral, PublicKeyToken=35e10195dab3c99f' or one of its dependencies.
File name: 'glib-sharp, Version=2.12.0.0, Culture=neutral, PublicKeyToken=35e10195dab3c99f'
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'glib-sharp, Version=2.12.0.0, Culture=neutral, PublicKeyToken=35e10195dab3c99f' or one of its dependencies.
File name: 'glib-sharp, Version=2.12.0.0, Culture=neutral,
PublicKeyToken=35e10195dab3c99f'
```
orivej added a commit that referenced this pull request Jan 9, 2018
orivej added a commit that referenced this pull request Jan 9, 2018
Does not rebuild `mercurial`, only `mercurialFull`.

Fixes #33625 after #31497
orivej added a commit that referenced this pull request Jan 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants