Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make __DATE__/__TIME__ deterministic when NIX_ENFORCE_PURITY=1 #51510

Closed
wants to merge 1 commit into from
Closed

Make __DATE__/__TIME__ deterministic when NIX_ENFORCE_PURITY=1 #51510

wants to merge 1 commit into from

Conversation

rht
Copy link
Member

@rht rht commented Dec 4, 2018

Extracted from 3dba999.

See #2281. This is the gcc stuff in the list made by @Ekleog in #2281 (comment), excluding the PGO commit.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@rht rht requested a review from Ericson2314 as a code owner December 4, 2018 14:41
Mic92
Mic92 reviewed Dec 4, 2018
View reviewed changes
pkgs/build-support/cc-wrapper/cc-wrapper.sh
# When enforcing purity, pretend gcc can't find the current date and
# time
if [[ "${NIX_ENFORCE_PURITY:-}" = 1 ]]; then
extraAfter=(-D__DATE__=\"???-??-????\"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean?

Suggested change
extraAfter=(-D__DATE__=\"???-??-????\"
extraAfter=(-D__DATE__=\"??-??-????\"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know what debian is doing here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that there has been a patch to gcc (since 2016) to allow using SOURCE_DATE_EPOCH https://wiki.debian.org/ReproducibleBuilds/TimestampsFromCPPMacros.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some packages (e.g. pycrypto) do already have SOURCE_DATE_EPOCH defined, though it would be preferable to do this globally.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is

nixpkgs/pkgs/stdenv/generic/setup.sh

Lines 244 to 249 in 33b9aa4

# Set a fallback default value for SOURCE_DATE_EPOCH, used by some
# build tools to provide a deterministic substitute for the "current"
# time. Note that 1 = 1970-01-01 00:00:01. We don't use 0 because it
# confuses some applications.
export SOURCE_DATE_EPOCH
: ${SOURCE_DATE_EPOCH:=1}
, but not used by all packages built with gcc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But why we need these macros then, when the environment variable is already exported?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a packages ignores environment variables, we are likely detecting this because cc wrapper will also not find libraries and headers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose since the SOURCE_DATE_EPOCH is only supported for gcc >=7.x, but not so for earlier versions, and so I have to modify it to be enabled only for early gcc's.

Mic92
Mic92 reviewed Dec 4, 2018
View reviewed changes
pkgs/build-support/cc-wrapper/cc-wrapper.sh
if [[ "${NIX_ENFORCE_PURITY:-}" = 1 ]]; then
extraAfter=(-D__DATE__=\"???-??-????\"
-D__TIME__=\"??:??:??\"
-Wno-builtin-macro-redefined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mhm. Can this hit us somewhere else?

@edolstra
Copy link
Member

edolstra commented Dec 4, 2018

I don't think setting an invalid date is the right thing to do. We should use SOURCE_DATE_EPOCH (which stdenv already sets anyway, see pkgs/build-support/setup-hooks/set-source-date-epoch-to-latest.sh) instead.

@rht
Copy link
Member Author

rht commented Dec 4, 2018

SOURCE_DATE_EPOCH applies only for gcc >= 7, what about the early versions?

@rht
Copy link
Member Author

rht commented Dec 4, 2018

Wait, the early versions have already been patched as well, so this fix is obsolete:

pkgs/development/compilers/gcc/5/default.nix
56:      [ ../use-source-date-epoch.patch ]

pkgs/development/compilers/gcc/4.9/default.nix
58:      [ ../use-source-date-epoch.patch ../parallel-bconfig.patch ./parallel-strsignal.patch ]

pkgs/development/compilers/gcc/6/default.nix
56:      [ ../use-source-date-epoch.patch ]

@rht rht closed this Dec 4, 2018
@rht rht deleted the deterministic-build branch December 4, 2018 17:20
@Ekleog Ekleog mentioned this pull request Dec 5, 2018
Closed
@Ekleog
Copy link
Member

Ekleog commented Dec 5, 2018

@rht I have updated my comment to reflect your investigation in this PR, thank you :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mic92 Mic92 Mic92 left review comments

@Ericson2314 Ericson2314 Awaiting requested review from Ericson2314 Ericson2314 is a code owner

Assignees
No one assigned
Labels
10.rebuild-darwin: 501+ 10.rebuild-darwin-stdenv 10.rebuild-linux: 501+ 10.rebuild-linux-stdenv
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@rht @edolstra @Ekleog @Mic92 @GrahamcOfBorg