autogen: fix build reproducibility issues #66552
Conversation
| echo ! tar cvf ${tag}.${sfx} ${tag} | ||
| -tar cvf - ${tag} | $gz > ${top_builddir}/autoopts/${tag}.${sfx} | ||
| +tar --sort=name --mtime=0 cvf - ${tag} \ | ||
| + | $gz > ${top_builddir}/autoopts/${tag}.${sfx} |
There was a problem hiding this comment.
rbopts=""
if test ! -z "$SOURCE_DATE_EPOCH"
then
f=$(tar --help|grep -q sort=) # backticks are inconvenient here
test ! -z "$f" && {
rbopts="--sort=name --format=gnu --clamp-mtime"
rbopts="$rbopts --mtime @$SOURCE_DATE_EPOCH"
}
fi
tar cvf - $rbopts ${tag} | \
$gz > ${top_builddir}/autoopts/${tag}.${sfx}
|
That patch will not work. It assumes tar is GNU tar. This patch requires a
configure time test to ensure that that is true. I'd recommend a shell
function that wraps the "tar" invocation. The contents of that function
would have to depend on finding a GNU tar.
…On Tue, Aug 13, 2019 at 12:50 AM Jörg Thalheim ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In pkgs/development/tools/misc/autogen/libopts-r13y.patch
<#66552 (comment)>:
> @@ -0,0 +1,13 @@
+--- a/pkg/libopts/mklibsrc.sh 2019-08-13 06:43:38.670875866 +0200
++++ b/pkg/libopts/mklibsrc.sh 2019-08-13 06:42:08.046283976 +0200
+@@ -113,7 +113,8 @@
+ cd ..
+ echo ! cd `pwd`
+ echo ! tar cvf ${tag}.${sfx} ${tag}
+-tar cvf - ${tag} | $gz > ${top_builddir}/autoopts/${tag}.${sfx}
++tar --sort=name --mtime=0 cvf - ${tag} \
++ | $gz > ${top_builddir}/autoopts/${tag}.${sfx}
cc @andir <https://github.com/andir>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#66552?email_source=notifications&email_token=AAJPOOBBBHDVJE4VA3P6HUDQEJRTZA5CNFSM4ILHC4KKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCBLJ4OY#pullrequestreview-274112059>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAJPOOG6CCJZGP3JHPEJKKDQEJRTZANCNFSM4ILHC4KA>
.
--
- Bruce
|
|
It may be a good idea to use a larger value for the timeout, to ensure builds are not failing due to the timeout even on slower platforms. Debian used 78: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794892. We can probably import the bash code from https://salsa.debian.org/debian/autogen/-/blob/master/pkg/libopts/mklibsrc.sh#L130-140 for the tar issue. |
|
This issue is *FIVE* years old. I've retired since then, by the way. I'll
add a configure option that will allow you to fiddle the timeout, all I
have to do is figure out how the configure process changed over the years
so I can once again configure and build from the source repo. Right now,
configure has gotten so clever that I cannot get it working.
…On Thu, Feb 27, 2020 at 8:22 AM Benno Fünfstück ***@***.***> wrote:
It may be a good idea to use a larger value for the timeout, to ensure
builds are not failing due to the timeout even on slower platforms. Debian
used 78: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794892.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#66552?email_source=notifications&email_token=AAJPOOGUWDFFAUZE3GUXULTRE7SCPA5CNFSM4ILHC4KKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENE6YSQ#issuecomment-592047178>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJPOOHE45SN43SXGEHMB5LRE7SCPANCNFSM4ILHC4KA>
.
--
- Bruce
|
|
Hello, I'm a bot and I thank you in the name of the community for your contributions. Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human. If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do: If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list. If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past. If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments. Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel. |
stale
bot
added
the
2.status: stale
|
Still relevant according to r13y.com. Right now the only comments on this PR I don't really see as blockers (the timeout issue is theoretical, and the tar vs. GNU tar problem doesn't apply in the Nix build environment). |
stale
bot
removed
the
2.status: stale
|
I think it would be great to fix this reproducibility issue. Am I correct that the tar issue was fixed in autogen 5.18.14 so that patch is no longer needed? |
|
Wow. i totally forgot about this PR exiting and already having dived into this. @zimbatm, @aszlig and I had some time looking at that today and came up with #102280 but never thought someone else might have done the work already. Sorry @delroth! I think we can close this now event thought you really deserve some credit for the work here. |
|
Is the libopts patch still needed? We only have seen the first issue in practice but it doesn't mean that the second one isn't valid. |
Motivation for this change
https://r13y.com/diff/2d1d9ebcbadce93e8dc711b4519442c992003b68ab1ad8445a425ea7dd7bf1e2-dd2c04513f0896c22e745c5368ff35cf0bdd607c121e9821958676305d59617a.html
Note: patch not sent upstream -- I see "GNU", I don't bother. I have no interest in signing the GNU CLA or figuring out the most likely convoluted process involved in submitting a 2 lines patch ¯\_(ツ)_/¯ @brkorb fyi
Things done
sandboxinnix.confon non-NixOS)nix-shell -p nix-review --run "nix-review wip"./result/bin/)nix path-info -Sbefore and after)Notify maintainers
cc @grahamc (let me know if you'd rather not be cc'd on these -- since you maintain r13y.com I assume you have an interest in these changes)