Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hydra: 2020-02-06 -> 2020-03-{24,27} #83600

Merged
merged 1 commit into from Mar 28, 2020
Merged

Conversation

Ma27
Copy link
Member

@Ma27 Ma27 commented Mar 28, 2020

Motivation for this change

Upgrades Hydra to the latest master/flake branch. To perform this
upgrade, it's needed to do a non-trivial db-migration which provides a
massive performance-improvement[1].

The basic ideas behind multi-step upgrades of services between NixOS versions
have been gathered already[2]. For further context it's recommended to
read this first.

Basically, the following steps are needed:

  • Upgrade to a non-breaking version of Hydra with the db-changes
    (columns are still nullable here). If system.stateVersion is set to
    something older than 20.03, the package will be selected
    automatically, otherwise pkgs.hydra-migration needs to be used.

  • Run hydra-backfill-ids on the server.

  • Deploy either pkgs.hydra-unstable (for Hydra master) or
    pkgs.hydra-flakes (for flakes-support) to activate the optimization.

The steps are also documented in the release-notes and in the module
using warnings.

pkgs.hydra has been removed as latest Hydra doesn't compile with
pkgs.nixStable and to ensure a graceful migration using the newly
introduced packages.

To verify the approach, a simple vm-test has been added which verifies
the migration steps.

[1] NixOS/hydra#711
[2] #82353 (comment)

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@grahamc
Copy link
Member

grahamc commented Mar 28, 2020

shipit!

Upgrades Hydra to the latest master/flake branch. To perform this
upgrade, it's needed to do a non-trivial db-migration which provides a
massive performance-improvement[1].

The basic ideas behind multi-step upgrades of services between NixOS versions
have been gathered already[2]. For further context it's recommended to
read this first.

Basically, the following steps are needed:

* Upgrade to a non-breaking version of Hydra with the db-changes
  (columns are still nullable here). If `system.stateVersion` is set to
  something older than 20.03, the package will be selected
  automatically, otherwise `pkgs.hydra-migration` needs to be used.

* Run `hydra-backfill-ids` on the server.

* Deploy either `pkgs.hydra-unstable` (for Hydra master) or
  `pkgs.hydra-flakes` (for flakes-support) to activate the optimization.

The steps are also documented in the release-notes and in the module
using `warnings`.

`pkgs.hydra` has been removed as latest Hydra doesn't compile with
`pkgs.nixStable` and to ensure a graceful migration using the newly
introduced packages.

To verify the approach, a simple vm-test has been added which verifies
the migration steps.

[1] NixOS/hydra#711
[2] NixOS#82353 (comment)
@grahamc grahamc merged commit 4d226ba into NixOS:master Mar 28, 2020
@Ma27 Ma27 deleted the hydra-two-stage-deploy branch March 28, 2020 23:24
@Ma27
Copy link
Member Author

Ma27 commented Mar 28, 2020

Ah, thanks for merging!

(FYI already working on the backport :))

@Ma27
Copy link
Member Author

Ma27 commented Mar 28, 2020

Backported as dabdd3a.

@Ma27 Ma27 added 8.has: port to stable A PR already has a backport to the stable release. and removed 9.needs: port to stable A PR needs a backport to the stable release. labels Mar 28, 2020
@nlewo
Copy link
Member

nlewo commented Mar 29, 2020

LGTM.
Nice job @Ma27! Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants