Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[20.09] Staging next #102662

Merged
merged 40 commits into from Nov 16, 2020
Merged

[20.09] Staging next #102662

merged 40 commits into from Nov 16, 2020

Conversation

roberth
Copy link
Member

@roberth roberth commented Nov 3, 2020

https://hydra.nixos.org/jobset/nixpkgs/staging-20.09

Motivation for this change
Things done

before merge

  • check hydra
  • check ofborg

primeos and others added 12 commits October 28, 2020 10:28
SECURITY ISSUE: Attempted to make RSA PKCS#1v1.5 decryption more
constant time, to protect against Bleichenbacher vulnerabilities. Due to
limitations imposed by our API, we cannot completely mitigate this
vulnerability and a future release will contain a new API which is
designed to be resilient to these for contexts where it is required.
Credit to Hubert Kario for reporting the issue. CVE-2020-25659

(cherry picked from commit 1083cdd)
[staging-20.09] openldap: add patch to fix unauthenticated nullptr dereference in slapd
Re-enable the Systemd remote support
To stay within hydra limit of 2^31 output size on aarch64-linux
(cherry picked from commit 8904ce2)
A recent addition to the test suite turned out to be sensitive to
DST. The main code is ok. Patch only required to make test succeed.

See https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=4a071afbd056282746a5bc9362e87f579a56402d

(cherry picked from commit 09a59b3ba43e4b68f7cea9c5685b424c83382a6f)
A recent addition to the test suite turned out to be sensitive to
DST. The main code is ok. Patch only required to make test succeed.

See https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=4a071afbd056282746a5bc9362e87f579a56402d

(cherry picked from commit 88f84e5)
This reverts commit f7360dc.

It breaks golang bootstrapping and should be update later.
@roberth
Copy link
Member Author

roberth commented Nov 7, 2020

Blocked on hydra build timeouts, see NixOS/hydra#830

This matches a similar change made to Firefox in #96767.
A lot of code is shared between Firefox and Thunderbird
so it makes sense to increase timeouts for Thunderbird
as well.

(cherry picked from commit 4897591)
roberth and others added 4 commits November 11, 2020 17:22
The default timeout is 10h on Hydra currently, so this timeout
setting is pointless or counterproductive.
This commit seems to have been made in error
#96767 (comment)

This reverts commit 1733d51.
The timezone dumps have switched to a "slim" format since 2020b.
This has broken various packages, including
 - go 1.4 (used for bootstrapping)
 - haskellPackages.tz
 - libical

The "fat" format can still be generated, as this commit shows.
It seems to create files that are *mostly* the slim versions with
some more data attached.

(cherry picked from commit d328ba1)
(cherry picked from commit 8904ce2)
@roberth
Copy link
Member Author

roberth commented Nov 11, 2020

This iteration was stuck on darwin's ghc timing out. I'm starting the next mass-rebuild iteration hoping that hydra will pick up the updated timeout duration because of the new drv.

@roberth
Copy link
Member Author

roberth commented Nov 11, 2020

The new evaluation is https://hydra.nixos.org/eval/1626139
It's quite a mass-rebuild, but only the linux ghc is affected by the shadow rebuild. So I don't think the more lenient timeout for ghc will be picked up by hydra this time either.
I'll intentionally create a ghc 884 rebuild on darwin to see if that causes hydra to pick up the updated timeout for the new drv.

@roberth
Copy link
Member Author

roberth commented Nov 11, 2020

Here's the new evaluation https://hydra.nixos.org/eval/1626149
This will have an increased timeout for ghc on darwin, assuming Hydra picks up such settings when the drv is all-new.
I hate having to intentionally mass-rebuild, but my other options were exhausted.

@vcunat
Copy link
Member

vcunat commented Nov 11, 2020

Yes, I think Hydra will just reuse the job if the output hash is the same as in the previous evaluation. What might work is to now revert that rebuild and re-eval :-)

@roberth
Copy link
Member Author

roberth commented Nov 11, 2020

if the output hash is the same as in the previous evaluation

However, if it looks up by drv globally rather than previous job, it will still be bad.

might

I'm a bit tired of this whole hydra timeout exercise tbh, but I appreciate the input.

@vcunat
Copy link
Member

vcunat commented Nov 11, 2020

I'm hopeful the current one will work. I don't think the change will really cost anything (rebuilding just what failed anyway). IIRC I once ran into this "issue" with a fixed-output derivation (bad download URL or something)...

@roberth
Copy link
Member Author

roberth commented Nov 11, 2020

this "issue" with a fixed-output derivation (bad download URL or something)...

That probably only worked because the output was cached, so the old derivation didn't have to be run after you reverted to it.

@vcunat
Copy link
Member

vcunat commented Nov 11, 2020

No, it was failed so it wasn't cached. And I couldn't make Hydra retry it with fixed URL, because it was rerunning with the old one due to the output hash being the same (and thus thinking it's the same derivation). EDIT: I mean, I most likely wouldn't think of such an issue if I were writing hydra...

@roberth
Copy link
Member Author

roberth commented Nov 11, 2020

Could be. It doesn't save any builds though, so I wouldn't bother.

Most of the builds are because of tzdata and shadow, not the darwin ghc rebuild, which seems to be quite independent. Nonetheless some packages will depend on both, so we're saving some build time by combining these changes.

Also if the darwin-only comment happens to stick around, it's not so bad because it will be made irrelevant by the next release.

I'd say there's not much we can do to improve things now.

@roberth
Copy link
Member Author

roberth commented Nov 15, 2020

Not sure how to compare the jobsets, but I've made an attempt by searching in staging-20.09 and opening the various release jobsets.
Do we have a documented method for comparing these?

Staging stats:

based on eg https://hydra.nixos.org/eval/1626407?filter=x86_64-darwin&compare=1626149&full=#tabs-still-fail
and adding the failure tabs
x86_64-linux: 69 failures
x86_64-darwin: 977 failures
aarch64-linux: still building, haskell works

Release stats:

x86_64-linux: ~160 failures
x86_64-darwin: 921 failures https://hydra.nixos.org/jobset/nixpkgs/nixpkgs-20.09-darwin
aarch64-linux: ~7000 because of broken haskell

Conclusion

Seems like a improvement. Will merge base again to check for rebuilds from release-20.09 changes. If those aren't too many, I think we can merge.
What do you think @vcunat @jonringer ?

@roberth
Copy link
Member Author

roberth commented Nov 15, 2020

Do we squash any commits?
tzdata was applied/reverted/applied. Thunderbird timeout was applied/reverted. postgresql: Fix timetz test failure 2× can be squashed into one.
That seems to be all the cleanup that can be done, if desired.

@vcunat
Copy link
Member

vcunat commented Nov 15, 2020

I guess there's no need to wait for aarch64, as that would probably take several more days, I'm afraid. It sounds OK to me to merge as is (without rewriting history), but I don't mind if you/someone prefer to clean it up a bit instead.

@jonringer
Copy link
Contributor

Not sure how to compare the jobsets, but I've made an attempt by searching in staging-20.09 and opening the various release jobsets.
Do we have a documented method for comparing these?

we don't do this well even on unstable's staging/staging-next

Seems like a improvement. Will merge base again to check for rebuilds from release-20.09 changes. If those aren't too many, I think we can merge.

I would say we should move forward

@jonringer
Copy link
Contributor

Do we squash any commits?
tzdata was applied/reverted/applied. Thunderbird timeout was applied/reverted. postgresql: Fix timetz test failure 2× can be squashed into one.

If it already exists on one of the main NixOS/Nixpkgs branches (master, staging*, staging-next, release-*), I would avoid force-pushing at all. Ideally those commits and reverts should have been better tested before merging

@roberth roberth merged commit 2da313c into release-20.09 Nov 16, 2020
20.09-Staging automation moved this from Current Staging-next to Done Nov 16, 2020
@roberth
Copy link
Member Author

roberth commented Nov 16, 2020

Thanks everyone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet