Bazel: clear markers completely #65374
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Turns out markers are non-deterministic after all and even our patching still doesn't solve this problem completely. For example (tensorflow deps, this is a complete diff so actual dependencies don't differ): 30509c30509 < bc527ff00916b15caee38793bca8f294c748df4a256de55c5199281be0489e73 result/@bazel_skylib.marker --- > 4e0303e815c78df1e43d4b88dfe65e73046e0c6157fb10aa9a4e8b910113cd9c result/@bazel_skylib.marker 31045c31045 < fa13d04b2316214c3b4008b52546c2d5b633e006f6f019d597bb3f9745bacf7b result/@bazel_toolchains.marker --- > b36174bf5535e5157801b6de30c35ee03a03fe57766306393c3d65dd65cbebf4 result/@bazel_toolchains.marker 31144c31144 < b0ce4a3ac29ac22528336dd3a54b5b7af9ecc43bef2a2630713c1981a5cbbb51 result/@build_bazel_rules_swift.marker --- > 7492528068ec4f8e7ace2ecf8f933ec4e1b2235bd7426ce6f70177919f1cd05e result/@build_bazel_rules_swift.marker 36245c36245 < be2993536a8233d63251b664caf35b1e7cd57d194ab2a39a293876c232d6bbd0 result/@io_bazel_rules_closure.marker --- > b6655cc3f2c78525e5a724d8a4e93b1e7f09f1e09fc817d231109e7f39103e88 result/@io_bazel_rules_closure.marker 36329c36329 < 087bc674c9509dfe157400d111db4a13eeb45fc76aeccd490cee9aad6771ecad result/@io_bazel_rules_docker.marker --- > f920ec07315ec71e800b05cd22b2a341c0a80807c6e335ee81739b13c532b422 result/@io_bazel_rules_docker.marker 79544d79543 < 85893a05a817036c61f6cd9f8247757baa1654f473c494ce4fc5253c2bbd2790 result/@platforms.marker And here's an example of differences: $ cat result-a/@bazel_skylib.marker 7dc7472d37424ba5ec6a5532765bc911 $MANAGED cat result-b/@bazel_skylib.marker a8f3f577798201157128e8e9934c4705 $MANAGED Instead of trying to patch these markers further we now completely clear them. Nix hacks for ignoring markers are restored and expanded so that we don't even attempt to parse the marker.
abbradar
force-pushed
the
bazel-new-hacks
branch
from
5a652b3
to
d433f9c
Compare
uri-canva
reviewed
Jul 25, 2019
| - markerPath.delete(); | ||
| - return null; | ||
| - } | ||
| + return new Fingerprint().addString(content).digestAndReset(); |
There was a problem hiding this comment.
At this point rather than clearing out the markers and reading the empty files why not hardcode an empty string and delete the markers completely?
There was a problem hiding this comment.
The idea is to check that marker exists at all; this should catch dependencies which weren't downloaded at all, I think.
timokau
approved these changes
Jul 26, 2019
Profpatsch
reviewed
Jul 29, 2019
10 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
Turns out markers are non-deterministic after all and even our patching still
doesn't solve this problem completely. For example (tensorflow deps, this is a
complete diff so actual dependencies don't differ):
And here's an example of differences:
Instead of trying to patch these markers further we now completely clear them.
Nix hacks for ignoring markers are restored and expanded so that we don't even
attempt to parse the marker.
Things done
sandboxinnix.confon non-NixOS)nix-shell -p nix-review --run "nix-review wip"./result/bin/)nix path-info -Sbefore and after)