Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildEnv: Don't warn about collisions if contents match when ignoreCollisions=true. #60365

Closed

Conversation

ambrop72
Copy link
Contributor

This prevents giving many false-positive collision warnings to NixOS users and makes it easier to see real collisions that should be addressed.

Looking at the history, @aszlig at one time committed a version that behaved like this but then immediately changed it to how it currently works, that is it warns for every collision without checking contents when ignoreCollision=true (as it is for the NixOS system path). While the resulting environment is indeed the same, I think it is worth doing the content check so that users are warned only about real collisions.

This reduced the number of collision warnings on my desktop system from 388 to 56.

Things done

I tested this with the NixOS system path in a hacky way so that it only affected that (the change as-is causes a mass rebuild because lots of things make use of buildEnv).

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ambrop72 ambrop72 force-pushed the no-false-positive-collision-warnings branch from 8fc35c9 to 3a7c93e Compare April 28, 2019 19:39
@ambrop72
Copy link
Contributor Author

I changed that line to
if (!$checkCollisionContents || !checkCollision($oldTarget, $target)) {
and then refactored that section to be (hopefully) nicer, and should be completely equivalent.

…llisions=true.

This prevents giving many false-positive collision warnings to NixOS users and makes it easier to find real collisions that should be addressed.

Contents are checked only if checkCollisionContents is true, which is the case for the NixOS system-path by default. If
checkCollisionContents is false, there is still a warning for every collision.
@ambrop72 ambrop72 force-pushed the no-false-positive-collision-warnings branch from 3a7c93e to 8734c75 Compare April 28, 2019 21:50
Copy link
Member

@FRidh FRidh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should warn, but differently. See #41144.

@ambrop72
Copy link
Contributor Author

ambrop72 commented May 4, 2019

I think we should warn, but differently. See #41144.

With the current state of this PR, the behavior is very logical and consistent: checkCollisionContents determines whether an identical-content situation is deemed to be a collision, and ignoreCollisions determines whether to just warn or fail on collisions. So with checkCollisionContents=true and ignoreCollisions=true, in case of identical-content, we don't consider it a collision and therefore don't even warn.

The number of identical-content collisions in real NixOS configurations is so large that users are spammed with hundreds of collision errors, and real collisions are obscured (a different message doesn't help, once you get hundreds of warnings). So in the current state of NixOS, I can only see warnings about identical-content collisions as something that developers are interested in, not users. So if we have a feature to show this, I think it should be behind a flag and not the default (until the majority of identical-content collisions are resolved).

@FRidh
Copy link
Member

FRidh commented May 5, 2019

With the current state of this PR, the behavior is very logical and consistent: checkCollisionContents determines whether an identical-content situation is deemed to be a collision, and ignoreCollisions determines whether to just warn or fail on collisions. So with checkCollisionContents=true and ignoreCollisions=true, in case of identical-content, we don't consider it a collision and therefore don't even warn.

That is indeed logical and consistent.

The number of identical-content collisions in real NixOS configurations is so large that users are spammed with hundreds of collision errors, and real collisions are obscured (a different message doesn't help, once you get hundreds of warnings). So in the current state of NixOS, I can only see warnings about identical-content collisions as something that developers are interested in, not users. So if we have a feature to show this, I think it should be behind a flag and not the default (until the majority of identical-content collisions are resolved).

Collisions of identical files is not good either, but at least the implications aren't as bad. I agree that users preferably shouldn't see things that don't really impact them, but at the same time, hiding these type of collisions will also mean that the developers are also less likely to notice. Should we then have NixOS tests to test these don't occur? That obviously raises the question what packages to include.

In my opinion a collision, regardless of whether the file is the same or not, is a bug, although I can imagine there are cases these are hard to prevent (Python 2 namespace packages e.g.).

@ambrop72
Copy link
Contributor Author

ambrop72 commented May 5, 2019

How about this: by default we don't warn for every identical-content collision, but if there were any, at the end we print a single message saying something like "Note: there were N collisions with identical file content, if you want to see them set environment.warnForIdenticalContentCollisions=true".

Should we then have NixOS tests to test these don't occur? That obviously raises the question what packages to include.

We could make buildEnv produce a secondary output that includes a file with collision information (e.g. a file with all collision messages) and tests could check that. This would be better than invoking buildEnv with ignoreCollisions=false because the test can make more informative errors/warnings (not sure what infrastructure for collecting such info is available).

Copy link
Member

@FRidh FRidh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure exactly what to do here, but I won't block this change.

@ambrop72
Copy link
Contributor Author

ambrop72 commented May 5, 2019

Ok then I suggest we merge this, I anyway have more changes planned for buildEnv and I will implement more warning control soon.

@ambrop72
Copy link
Contributor Author

ambrop72 commented May 6, 2019

There are no objections, what is the hold-up? I will address the concern from @FRidh about letting one seeing all collisions in a follow-up PR, which will not be as simple as this one and will likely have some refactoring to the collision detection code to give better messages in general.

@ambrop72
Copy link
Contributor Author

ambrop72 commented Jul 12, 2019

Closing due to disagreements. I was planning to make other improvements to buildEnv but it looks there's general tendency to leave that thing as is.

@ambrop72 ambrop72 closed this Jul 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants