linux_5_2, linux_5_3: fixing nondeterminism #69222
Conversation
ofborg
bot
added
10.rebuild-darwin: 0
|
Regarding testing the result: I successfully booted into a locally built |
|
Nice work! It would probably be worth it to submit this upstream, I'd think, in the long run; |
|
@thoughtpolice: Alright, I can give it a try. Sounds like a sensible thing to do. |
d-goldin
force-pushed
the
fix_linux_5_determinism
branch
from
790a8a8 to
f420f66
Compare
|
Gave it another run with 5.3, looks good! |
|
I submitted a slightly modified patch covering the case of undefined timestamp for kernel builds not desiring reproducibility and got it a bit more in line with the kernel style guides; https://lkml.org/lkml/2019/9/22/69 I updated the patch in this PR to reflect the patch submitted upstream - while for the nixos case we wouldn't really care much for the case of timestamp being undefined, maybe there is some merit in keeping it more in line until/if it lands in mainline. If reviewers are happy, I can squash them into one. Kept separately for clarity. |
|
I think we can wait for a few days to get the kernel maintainers feedback on this. Apart from that the changes looks good to me. |
In 5.2 kernel a new mechanism was introduced which embeds the kernel headers in the kernel image and exposes them in procfs for simplified use by userland tools. It was introduced in torvalds/linux@43d8ce9 and later modified a bit in torvalds/linux@f7b101d The archive containing the header files had nondeterminism through the header files metadata - specifically `mtime`, but I also decided to normalize some other aspects just in case. In our default setup we currently compile this as a module, so to expose the headers to test the functionality `kheaders` needs to be loaded. See https://lkml.org/lkml/2019/10/4/1036 and https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=2cc99c9cdc8fde5e92e34f9655829449cebd3e00 I commented out the documentation part of the patch to make it cleanly apply to 5.2 and 5.3, see remark in the patch itself.
d-goldin
force-pushed
the
fix_linux_5_determinism
branch
from
90882a8 to
bdccffa
Compare
|
@Mic92: A slightly modified version of the patch has been accepted into the kbuild maintainer's fork. It was modified to include an update to the reproducible builds doc (and a typo fix). I pulled the updated patch into this PR but commented out the documentation hunk. |
ofborg
bot
added
10.rebuild-linux: 501+
10.rebuild-linux: 501-1000
and removed
10.rebuild-linux: 101-500
labels
|
Fantastic! Let's go ahead and merge this. I think moving to |
|
@GrahamcOfBorg build linux_5_2 linux_5_3 |
Motivation for this change
Addresses: #69100
In 5.2 kernel a new mechanism was introduced which embeds the kernel
headers in the kernel image and exposes them in procfs for simplified
use by userland tools.
It was introduced in
torvalds/linux@43d8ce9
and later modified a bit in
torvalds/linux@f7b101d
The archive containing the header files had nondeterminism through the
header files metadata - specifically
mtime, but I also decided tonormalize some other aspects just in case.
In our default setup we currently compile this as a module, so to expose
the headers to test the functionality
kheadersneeds to be loaded.Without this patch this particular nondeterminism was trivially reproducible locally. I ran a few compilations of 5.2 and 5.3 and the local difference is resolved with this.
Things done
sandboxinnix.confon non-NixOS)nix-shell -p nix-review --run "nix-review wip"./result/bin/)nix path-info -Sbefore and after)Notify maintainers
cc @grahamc @thoughtpolice