This replaces #78781

An (untested) alternative might be this patch:

--- doc/mkdefsinc.c	2017-08-28 12:22:54.000000000 +0200
+++ doc/mkdefsinc.c.new	2020-05-15 22:54:13.974749616 +0200
@@ -109,7 +109,7 @@
 
   for (; (file = *files); files++)
     {
-      if (!*file || !strcmp (file, ".") || !strcmp (file, ".."))
+      if (!*file || !strcmp (file, ".") || !strcmp (file, "..") || !strcmp (file, "defsincdate"))
         continue;
       if (stat (file, &sb))
         {

@emilazy Same here. (built on NixOS stable).

I also put this upstream for discussion: https://dev.gnupg.org/T4947

With some more investigation, this actually caused by

sha256-XHquyKgPcDEANtboM/rSyY2pCdYigI+eyHoxeFUmMJw=

for

result -> /nix/store/6xy2vh6z6ssajs0bw6dhxwm676y6dsq1-gnupg-2.2.19

Reproduced sha256-XHquyKgPcDEANtboM/rSyY2pCdYigI+eyHoxeFUmMJw= with e0c1fadbda592381856ce9a7714eea27f9fbc23e.

Reproduced sha256-XHquyKgPcDEANtboM/rSyY2pCdYigI+eyHoxeFUmMJw= with e0c1fad.

Which is correct and doesn't help. :) We got both fooled and simply computed the hash of the symlink. Quick confirmation:

../foo1:
lrwxrwxrwx 1 xxx users 6 18. Mai 20:33 result -> ./test
-rw-r--r-- 1 xxx users 4 18. Mai 20:33 test

../foo2:
lrwxrwxrwx 1 xxx users 6 18. Mai 20:34 result -> ./test
-rw-r--r-- 1 xxx users 4 18. Mai 20:33 test

[/tmp/foo2]$ nix hash-path result
sha256-I/i001eIYWHT088PoPGOW0yXbAbEjLz3rxWwBrbWBmo=

[/tmp/foo2]$ cd ../foo1nix hash-path /nix/store/6xy2vh6z6ssajs0bw6dhxwm676y6dsq1-gnupg-2.2.19
sha256-i7fvywV7zXgifoIgnOnn0PH38LBWjj3X+Xkzvt7+kxM=

[/tmp/foo1]$ nix hash-path result
sha256-I/i001eIYWHT088PoPGOW0yXbAbEjLz3rxWwBrbWBmo=

[/tmp/foo1]$ cat test ../foo2/test
foo
bar

So what actually needs verifying is

nix hash-path /nix/store/6xy2vh6z6ssajs0bw6dhxwm676y6dsq1-gnupg-2.2.19
sha256-i7fvywV7zXgifoIgnOnn0PH38LBWjj3X+Xkzvt7+kxM=

which I can confirm for two builds on different days and on different machines.
@emilazy Can you confirm, too?

Gah, I specifically worried about whether that might be the case but forgot to check...

Thankfully, the hash of the actual tree seems to match as well:

emily@renko ~/s/nixpkgs (gnupg-make-reproducible)> realpath result
/nix/store/6xy2vh6z6ssajs0bw6dhxwm676y6dsq1-gnupg-2.2.19
emily@renko ~/s/nixpkgs (gnupg-make-reproducible)> nix hash-path (realpath result)
sha256-i7fvywV7zXgifoIgnOnn0PH38LBWjj3X+Xkzvt7+kxM=

@GrahamcOfBorg eval
(rebased to current tip of staging)

@fpletz @peti Do you think this can be merged?
Upstream has added a patch (see #87908 (comment)) that now uses SOURCE_DATE_EPOCH instead of simply blanking out the timestamp then building from the tarball and patching the documentation, while this PR preserves the timestamp by touching the file with the timestamp as extracted from the tarball. (This prevents having epoch instead of a more realistic time/date in the docs.)
When both are present (e.g. after a version update in nixpkgs), the method in this PR should take precedence unless evaluation of the corresponding make target(s) is forced.

My preferrence is to go with the solution used by upstream, to be honest.

@peti This would require that we set SOURCE_DATE_EPOCH to a value derived from the source.

Currently (nixpkgs/master) we get:

$ head -n 3 /nix/store/rak06773gwr9mcycz8p7mnd71zwj1fzj-gnupg-2.2.19/share/info/gnupg.info
This is gnupg.info, produced by makeinfo version 6.5 from gnupg.texi.

This is the 'The GNU Privacy Guard Manual' (version 2.2.19, May 2020).

Using the upstream patch without further adjustment would result into something like

$ head -n 3 /nix/store/???-gnupg-2.2.19/share/info/gnupg.info
This is gnupg.info, produced by makeinfo version 6.5 from gnupg.texi.

This is the 'The GNU Privacy Guard Manual' (version 2.2.19, January 1970).

So my preference is to either keep the simple touch on the timestamp file included with the sourceode to prevent rebuilding it or extracting its value into SOURCE_DATE_EPOCH in a preBuild hook.

edit to keep context: we patch a path in the source for the info file, which then triggers the make-rule for updating the timestamp file, leaking the build time into the info pages.

ping @peti

/marvin opt-in

Hi! I'm an experimental bot. My goal is to guide this PR through its stages, hopefully ending with a merge. You can read up on the usage here.

/status needs_reviewer

@wamserma Trying to follow the discussion here, and I feel like I'm missing something based on your latest recommendation; it seems to me that you're suggesting relying on touch (ie. impure -- right?) to ensure that the date is recent, rather than reproducible. Is that correct, or am I misunderstanding?

@srhb Is that correct, or am I misunderstanding?

Slight misunderstanding. You are right about reproducibility in general, but we have a special setting here: There is a timestamp for the source in the file that is touched. This timestamp is delivered in the source tarball.
The nix derivation patches the sources of the documentation, which in turn makes make (correctly) believe the timestamp in the file is outdated, since the file holding the timestamp is older than the latest timestamp of the documentation sources.
So this timestamp is regenerated. But this timestamp is also used to print the release date in the source documentation, so it should neither be the current system time or epoch or some other arbitrary value. Touching the timestamp file avoids its rebuild and everything is fine.
Based on these observations upstream started to respect SOURCE_DATE_EPOCH, but in our case we would have to extract the desired timestamp first, then keep it in SOURCE_DATE_EPOCH. Simply touching the timestamp file is working fine and the least invasive option to choose.
Everything else about the build was already reproducible afaik. Only having the build time show up in the documentation was left to fix.

To sum up (and answer you question). While the touch is impure, we don't care about the timestamp of the touched file w.r.t. to the build outputs, but about protecting the file's content from being touched by make.

@wamserma Thank you for clarifying. I still have some questions -- please bear with me. 😄

As I understand your elaboration, upstream has patched the build such that SOURCE_DATE_EPOCH will be respected. You say:

Based on these observations upstream started to respect SOURCE_DATE_EPOCH, but in our case we would have to extract the desired timestamp first, then keep it in SOURCE_DATE_EPOCH.

This "extracting of the desired timestamp" already happens automatically in the standard builder, through the _updateSourceDateEpochFromSourceRoot setup hook in the postUnpack phase. This leads me to believe that this PR is no longer necessary, given that our patches do not interfere with that hook, as patching happens later than postUnpack. Please correct me if I'm (stil) wrong. 😄

Unfortunately, it seems the upstream patch is broken, as it complains that mkdefsinc: bad date 'OURCE_DATE_EPOCH' -- which seems like a trivial issue upstream.

If my understanding is correct, then I propose we fix this upstream instead.

fwiw,

    sed -i 's/$SOURCE_DATE_EPOCH/''${SOURCE_DATE_EPOCH}/' doc/Makefile.am
    sed -i 's/$SOURCE_DATE_EPOCH/''${SOURCE_DATE_EPOCH}/' doc/Makefile.in

... appears to do the trick with the recently released 2.2.21, which includes the source date patch, but again, please correct me if I'm wrong. :)

fwiw,

    sed -i 's/$SOURCE_DATE_EPOCH/''${SOURCE_DATE_EPOCH}/' doc/Makefile.am
    sed -i 's/$SOURCE_DATE_EPOCH/''${SOURCE_DATE_EPOCH}/' doc/Makefile.in

... appears to do the trick with the recently released 2.2.21, which includes the source date patch, but again, please correct me if I'm wrong. :)

I cannot confirm.

diff /nix/store/zk0rmj21ch7zgqxyjz7221dry3dcqv1v-gnupg-2.2.21/share/info/gnupg.info /nix/store/hbz49kavh7a8l6j0id90sn6220y10cii-gnupg-2.2.21/share/info/gnupg.info
3c3
< This is the 'The GNU Privacy Guard Manual' (version 2.2.21, April 2020).
---
> This is the 'The GNU Privacy Guard Manual' (version 2.2.21, July 2020).

zk0... is 2.2.21 with the touch method
hbz... applies you suggestion in postPatch

@wamserma I don't think "July" is wrong here -- the unpacked source indeed has sources that are from July, and that's what the build hook picks up.

Superseded by #93976

For completeness: Debian has patched this issue quite a while ago, but somehow I never came across their patch. (They simply remove the defsincdate update from the Makefile). The Debian patch is referenced in #93937.