New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nixos/nextcloud: fix upgrade path from 19.09 to 20.03 #82353
Conversation
@Ma27 does it make sense to just call the packages by their version number? We could default anything with an old |
Hmm, that's an even better idea 👍 Will update the patches soon :) |
If you do it this way, you won't need to worry about nixpkgs-update, because it makes sure the version number is compatible with the attrpath: https://github.com/ryantm/nixpkgs-update/blob/master/src/Version.hs#L35 |
What about keeping the |
I think @aanderse is on the right track here. After discussing this with @Ma27, we found some edgy use cases we should look out for:
So to sum up, we would suggest do the following:
|
@bachp |
9114710
to
5d9dc12
Compare
ac8bf3e
to
a40b7e2
Compare
@Ma27 quick glance over it and looks fine 👍 |
@worldofpeace @disassembler would you mind taking a look at this? This should IMHO land in 20.03 :) |
It's impossible to move two major-versions forward when upgrading Nextcloud. This is an issue when comming from 19.09 (using Nextcloud 16) and trying to upgrade to 20.03 (using Nextcloud 18 by default). This patch implements the measurements discussed in NixOS#82056 and NixOS#82353 to improve the update process and to circumvent similar issues in the future: * `pkgs.nextcloud` has been removed in favor of versioned attributes (currently `pkgs.nextcloud17` and `pkgs.nextcloud18`). With that approach we can safely backport major-releases in the future to simplify those upgrade-paths and we can select one of the major-releases as default depending on the configuration (helpful to decide whether e.g. `pkgs.nextcloud17` or `pkgs.nextcloud18` should be used on 20.03 and `master` atm). * If `system.stateVersion` is older than `20.03`, `nextcloud17` will be used (which is one major-release behind v16 from 19.09). When using a package older than the latest major-release available (currently v18), the evaluation will cause a warning which describes the issue and suggests next steps. To make those package-selections easier, a new option to define the package to be used for the service (namely `services.nextcloud.package`) was introduced. * If `pkgs.nextcloud` exists (e.g. due to an overlay which was used to provide more recent Nextcloud versions on older NixOS-releases), an evaluation error will be thrown by default: this is to make sure that `services.nextcloud.package` doesn't use an older version by accident after checking the state-version. If `pkgs.nextcloud` is added manually, it needs to be declared explicitly in `services.nextcloud.package`. * The `nixos/nextcloud`-documentation contains a "Maintainer information"-chapter which describes how to roll out new Nextcloud releases and how to deal with old (and probably unsafe) versions. Closes NixOS#82056
a40b7e2
to
702f645
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I approve of the backport to 20.03
Great, thanks a lot for the review! Will merge tomorrow then :) |
It's impossible to move two major-versions forward when upgrading Nextcloud. This is an issue when comming from 19.09 (using Nextcloud 16) and trying to upgrade to 20.03 (using Nextcloud 18 by default). This patch implements the measurements discussed in #82056 and #82353 to improve the update process and to circumvent similar issues in the future: * `pkgs.nextcloud` has been removed in favor of versioned attributes (currently `pkgs.nextcloud17` and `pkgs.nextcloud18`). With that approach we can safely backport major-releases in the future to simplify those upgrade-paths and we can select one of the major-releases as default depending on the configuration (helpful to decide whether e.g. `pkgs.nextcloud17` or `pkgs.nextcloud18` should be used on 20.03 and `master` atm). * If `system.stateVersion` is older than `20.03`, `nextcloud17` will be used (which is one major-release behind v16 from 19.09). When using a package older than the latest major-release available (currently v18), the evaluation will cause a warning which describes the issue and suggests next steps. To make those package-selections easier, a new option to define the package to be used for the service (namely `services.nextcloud.package`) was introduced. * If `pkgs.nextcloud` exists (e.g. due to an overlay which was used to provide more recent Nextcloud versions on older NixOS-releases), an evaluation error will be thrown by default: this is to make sure that `services.nextcloud.package` doesn't use an older version by accident after checking the state-version. If `pkgs.nextcloud` is added manually, it needs to be declared explicitly in `services.nextcloud.package`. * The `nixos/nextcloud`-documentation contains a "Maintainer information"-chapter which describes how to roll out new Nextcloud releases and how to deal with old (and probably unsafe) versions. Closes #82056 (cherry picked from commit 702f645)
Backported to 20.03 as d148bb0. Thanks a lot to everyone involved! |
Upgrades Hydra to the latest master/flake branch. To perform this upgrade, it's needed to do a non-trivial db-migration which provides a massive performance-improvement[1]. The basic ideas behind multi-step upgrades of services between NixOS versions have been gathered already[2]. For further context it's recommended to read this first. Basically, the following steps are needed: * Upgrade to a non-breaking version of Hydra with the db-changes (columns are still nullable here). If `system.stateVersion` is set to something older than 20.03, the package will be selected automatically, otherwise `pkgs.hydra-migration` needs to be used. * Run `hydra-backfill-ids` on the server. * Deploy either `pkgs.hydra-unstable` (for Hydra master) or `pkgs.hydra-flakes` (for flakes-support) to activate the optimization. The steps are also documented in the release-notes and in the module using `warnings`. `pkgs.hydra` has been removed as latest Hydra doesn't compile with `pkgs.nixStable` and to ensure a graceful migration using the newly introduced packages. To verify the approach, a simple vm-test has been added which verifies the migration steps. [1] NixOS/hydra#711 [2] NixOS#82353 (comment)
Upgrades Hydra to the latest master/flake branch. To perform this upgrade, it's needed to do a non-trivial db-migration which provides a massive performance-improvement[1]. The basic ideas behind multi-step upgrades of services between NixOS versions have been gathered already[2]. For further context it's recommended to read this first. Basically, the following steps are needed: * Upgrade to a non-breaking version of Hydra with the db-changes (columns are still nullable here). If `system.stateVersion` is set to something older than 20.03, the package will be selected automatically, otherwise `pkgs.hydra-migration` needs to be used. * Run `hydra-backfill-ids` on the server. * Deploy either `pkgs.hydra-unstable` (for Hydra master) or `pkgs.hydra-flakes` (for flakes-support) to activate the optimization. The steps are also documented in the release-notes and in the module using `warnings`. `pkgs.hydra` has been removed as latest Hydra doesn't compile with `pkgs.nixStable` and to ensure a graceful migration using the newly introduced packages. To verify the approach, a simple vm-test has been added which verifies the migration steps. [1] NixOS/hydra#711 [2] #82353 (comment) (cherry picked from commit bd5324c)
Upgrades Hydra to the latest master/flake branch. To perform this upgrade, it's needed to do a non-trivial db-migration which provides a massive performance-improvement[1]. The basic ideas behind multi-step upgrades of services between NixOS versions have been gathered already[2]. For further context it's recommended to read this first. Basically, the following steps are needed: * Upgrade to a non-breaking version of Hydra with the db-changes (columns are still nullable here). If `system.stateVersion` is set to something older than 20.03, the package will be selected automatically, otherwise `pkgs.hydra-migration` needs to be used. * Run `hydra-backfill-ids` on the server. * Deploy either `pkgs.hydra-unstable` (for Hydra master) or `pkgs.hydra-flakes` (for flakes-support) to activate the optimization. The steps are also documented in the release-notes and in the module using `warnings`. `pkgs.hydra` has been removed as latest Hydra doesn't compile with `pkgs.nixStable` and to ensure a graceful migration using the newly introduced packages. To verify the approach, a simple vm-test has been added which verifies the migration steps. [1] NixOS/hydra#711 [2] NixOS#82353 (comment)
Upgrades Hydra to the latest master/flake branch. To perform this upgrade, it's needed to do a non-trivial db-migration which provides a massive performance-improvement[1]. The basic ideas behind multi-step upgrades of services between NixOS versions have been gathered already[2]. For further context it's recommended to read this first. Basically, the following steps are needed: * Upgrade to a non-breaking version of Hydra with the db-changes (columns are still nullable here). If `system.stateVersion` is set to something older than 20.03, the package will be selected automatically, otherwise `pkgs.hydra-migration` needs to be used. * Run `hydra-backfill-ids` on the server. * Deploy either `pkgs.hydra-unstable` (for Hydra master) or `pkgs.hydra-flakes` (for flakes-support) to activate the optimization. The steps are also documented in the release-notes and in the module using `warnings`. `pkgs.hydra` has been removed as latest Hydra doesn't compile with `pkgs.nixStable` and to ensure a graceful migration using the newly introduced packages. To verify the approach, a simple vm-test has been added which verifies the migration steps. [1] NixOS/hydra#711 [2] NixOS#82353 (comment) (cherry picked from commit bd5324c)
Motivation for this change
As described in #82056, the upgrade path of
nextcloud
from 19.09 to 20.03 is broken as we'd have to move two major versions forward which isn't supported by Nextcloud.This PR introduces versioned attributes for Nextcloud and a few assertions (relying on the
stateVersion
) to help users with upgrading.Things done
sandbox
innix.conf
on non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
./result/bin/
)nix path-info -S
before and after)