Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetchpatch: quote excludes #43538

Merged
merged 4 commits into from Jul 19, 2018
Merged

fetchpatch: quote excludes #43538

merged 4 commits into from Jul 19, 2018

Conversation

timokau
Copy link
Member

@timokau timokau commented Jul 14, 2018

The excludes argument of fetchpatch is based on the -x flag of
filterdiff. That flag takes a "pattern" as an argument, which is similar
to a bash glob. For example 'some/path/*' would be acceptable and
exclude all files in some/path.

Currently those excludes are passed without quoting, leading to
unexpected things (the shell expanding the glob).

This shouldn't break anything, since nobody actually uses anything but plain paths in excludes yet. It shouldn't cause any rebuilds either, since the result of fetchpatch doesn't change and the hashes are fixed.

@vcunat @globin

There used to also be a includes that had the same error, but that patch (by @Ericson2314) was shortly after reverted (by mistake? the commit message doesn't indicate why).

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@dotlambda
Copy link
Member

This shouldn't break anything, since nobody actually uses anything but plain paths in excludes yet.

I used a * just a few days ago: af1313e

@Ericson2314
Copy link
Member

@volth Good call. @timokau It was part of #41420 which was revert. Feel free to revert the revert, but the newer version of that PR doesn't use it so it will be dead code.

Ericson2314 and others added 2 commits July 15, 2018 09:56
…h not be non-empty.

This commit was originally introduced as part of NixOS#41420 and then
reverted with the rest of that PR. However there was no reason to revert
his particular commit.
Excludes and includes are implemented by passing the parameters to the
respective flags of `filterdiff`. Those were passed unescaped until now.
Since those flags expect patterns (similar to shell globs), something
like `/some/path/*` might be used to exclude or include all files in
some path. Without escaping the shell would expand the `*`, leading to
unexpected behaviour.
@timokau
Copy link
Member Author

timokau commented Jul 15, 2018

@dotlambda Oh I only grepped through master. But the patch fetched in that commit actually ends up to be empty. That has happened to me more often than I'd like and I recognize the hash by now 😁

So that shows that we should probably apply this to 18.03 too.

@volth Good point. Changed.

@Ericson2314 I think it makes sense to offer both, so I included that.

Copy link
Member

@vcunat vcunat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good. Personally I'd use an assert instead of meta.broken for this.

@vcunat
Copy link
Member

vcunat commented Jul 15, 2018

Perhaps we might additionally block empty result of fetchpatch somehow, either make the derivation fail or recognize the hash at evaluation time.

Since this is probably never the desired case and has led to actual
issues, see the comments at:

NixOS@af1313e

This might also happen when pulling a patch from GitHub or a similar web
interface without explicitly selecting the "raw" format.
@timokau
Copy link
Member Author

timokau commented Jul 15, 2018

I have added another commit that checks for empty files. It is a bit verbose because I check at every step, but I think the improved error messages are worth it.

@timokau
Copy link
Member Author

timokau commented Jul 15, 2018

While we're at it: I think it would be neat to add an option to reverse a patch. That can already be done with ${args.postFetch or ""} right now though and could be considered feature creep. What do you think?

@vcunat
Copy link
Member

vcunat commented Jul 16, 2018

The ability to revert a patch sounds natural, though it seems rarely useful in nixpkgs to me.

@timokau
Copy link
Member Author

timokau commented Jul 16, 2018

It would be useful if a bug was introduced by some commit and the commit isn't reversed yet upstream. I've needed that at least once for sage and ended up downloading & reversing the patch. But its not all that common.

@timokau
Copy link
Member Author

timokau commented Jul 16, 2018

Since its usually easier to get feedback about something after implementing than it is to get feedback for ideas, I went ahead and implemented it.

@matthewbauer matthewbauer merged commit fddd90e into NixOS:master Jul 19, 2018
@vcunat
Copy link
Member

vcunat commented Jul 19, 2018

Borg reported this changes the sage* builds, but I suppose @timokau knows about that.

@dotlambda
Copy link
Member

Borg reported this changes the sage* builds

That's because @timokau immediately made use of the new revert feature: https://github.com/NixOS/nixpkgs/pull/43538/files#diff-5bd819821ddc9ac4a654e7ea45752448.

@timokau timokau deleted the fetchpatch-fix branch July 19, 2018 11:58
vcunat pushed a commit that referenced this pull request Jul 22, 2018
(cherry picked from commit fddd90e)
This seems safe enough.  It solves a bug in a conservative way;
it also adds features, possibly easing cherry-picks of fixes from master.
@vcunat vcunat added the 8.has: port to stable A PR already has a backport to the stable release. label Jul 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants