Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qt5: 5.12.7 -> 5.14.2 #84232

Merged
14 commits merged into from Jul 31, 2020
Merged

qt5: 5.12.7 -> 5.14.2 #84232

14 commits merged into from Jul 31, 2020

Conversation

ghost
Copy link

@ghost ghost commented Apr 4, 2020

Motivation for this change

Fixes #74355
Fixes #81894

To-Do list:
  • Fix loading of shared libraries as observed in Plasma and Quassel Core (fixed by @DIzFer, thanks!)
  • Update the following broken packages or move them back to Qt 5.12 if no fix is available
    • boomerang
    • chatterino2
    • deepin.* (does not build on master)
    • diffpdf
    • digikam
    • falkon
    • golden-cheetah
    • ibus-engines.mozc
    • lumina.lumina-pdf
    • megasync
    • lxqt.*
    • mpc-qt
    • netease-cloud-music (does not start on master)
    • nheko
    • notepadqq
    • qgroundcontrol
    • qlcplus
    • qtox
    • quassel
    • quaternion
    • rockbox_utility
    • rssguard
    • sdrangel
    • smplayer
    • smtube
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ghost ghost requested a review from ttuegel as a code owner April 4, 2020 03:02
@ghost
Copy link
Author

ghost commented Apr 4, 2020

qt5.qttools fails to build, because:

Error copying /build/qttools-everywhere-src-5.14.2/lib/cmake/Qt5AttributionsScannerTools/Qt5AttributionsScannerToolsConfigVersion.cmake to /nix/store/v52ixjwqg4dqa65qcqfrhd9b9fp8za3a-qttools-5.14.2/lib/cmake/Qt5AttributionsScannerTools/Qt5AttributionsScannerToolsConfigVersion.cmake: Cannot open /build/qttools-everywhere-src-5.14.2/lib/cmake/Qt5AttributionsScannerTools/Qt5AttributionsScannerToolsConfigVersion.cmake for input

But some desktop applications like tdesktop and qutebrowser are working fine.

Copy link
Member

@cole-h cole-h left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Diff LGTM, except for minor nits. No expert in this domain, so I'll defer to someone else (no way will my computer handle building and verifying the ~6000 affected packages lol).

pkgs/development/libraries/qt-5/5.14/default.nix Outdated Show resolved Hide resolved
pkgs/top-level/all-packages.nix Show resolved Hide resolved
@ghost
Copy link
Author

ghost commented Apr 5, 2020

I will run nix-review at some point.
And good news: I was able to fix the qttools build! Will try to rebuild my system with this now.

@ghost
Copy link
Author

ghost commented Apr 5, 2020

Now using this on my everyday system \o/
I had to override quassel src to master because it has some fixes to build with the new Qt. Otherwise it built fine after I fixed the QtAttributionsScanner problem in qttools.

@FRidh FRidh added this to Needs review in Staging Apr 5, 2020
@ghost
Copy link
Author

ghost commented Apr 6, 2020

I've been working on the Plasma 5 NixOS test. I had to update KDE frameworks, plasma5, quassel and grandlee so far to make them build with the new Qt.

Update: Plasma builds and starts fine now, but some of the patches will have to be re-done properly since wallpapers and some shared libraries are not detected properly. Take a look at https://github.com/petabyteboy/nixpkgs/commits/feature/plasma-5-18

@cole-h cole-h mentioned this pull request Apr 7, 2020
23 tasks
@ghost
Copy link
Author

ghost commented Apr 7, 2020

I am unsure what is the best approach to get this new Qt version to NixOS users.

One possible approach is to set the new version as default from the beginning and try to fix as much as possible rightaway. This will probably be the quickest way to get as many applications as possible on the new Qt version, which is by the way required for some of the fixes to work (wayland clipboard will only be better in the applications that are built with the new Qt).
The downside of this will be the enourmous diff size and time required to review it. It might also lead to more user issues in the short-term, because I can't possibly test the functionality of all applications that use Qt.

Another approach would be to leave 5.12 as the default version and gradually move packages to the new version. This has the advantage that we can check every package thoroughly, but we risk that many applications will still use 5.12 in NixOS 20.09.

@ghost ghost changed the title qt5: 5.12.7 -> 5.14.2 WIP: qt5: 5.12.7 -> 5.14.2 Apr 7, 2020
@ghost ghost changed the title WIP: qt5: 5.12.7 -> 5.14.2 [WIP] qt5: 5.12.7 -> 5.14.2 Apr 7, 2020
@ttuegel
Copy link
Member

ttuegel commented Apr 8, 2020

@petabyteboy In the past, we have done Qt 5 upgrades by changing the default version (the qt5 attribute) and downgrading the packages that break. We will ping the maintainers of any downgraded packages, too. This is much less work than fixing all the packages, but gives a better result than trying to gradually upgrade (which won't happen). When Qt 5.12 is eventually EOL'd upstream, we will remove it and mark any packages still using it as broken.

@DIzFer
Copy link
Contributor

DIzFer commented May 6, 2020

Quassel situation isn't very nice: both core and client seem to work when updated to latest git commit, but that's technically not a stable release, so probably wiser to kick it back to Qt5.12 until next release? Not sure how we would be supposed to remember to pull it back to the future once it's officially updated...

Other than that, I'm here because of qutebrowser, but something about QtWebEngine appears not to agree with my machine, so that I can't test. My desktop is at the moment rebuilding without QtWebEngine, which means I'll accidentally test some of #84542 too.

@ghost
Copy link
Author

ghost commented May 6, 2020

Hey, let me see what I can do for you in terms of QtWebEngine and qutebrowser. I will set up a small repository that includes an overlay and a fixed version of nixpkgs, similar to colemickens' nixpkgs-chromium and push the build results to a Cachix cache.

About quassel: The daemon has problems finding it's database library, so even then it doesn't work. I'm just using the client with the latest master commit, but that shouldn't be something we commit to nixpkgs. Instead we should either backport the necessary patches to the last Quassel release or leave Quassel with Qt5.12 until there is a new release.

@DIzFer
Copy link
Contributor

DIzFer commented May 6, 2020

That would be most gracious of you, but nevermind, I've somehow successfully built qutebrowser and webengine without realising... As you can tell from my comment, nix frequently conspires against my expectations. Which in the end means this seems to build for me all right!

Besides a few other apps you don't mention to be broken not being broken, I checked on some others that might affect me:

I'll report with more details about digikam once I finish shaving the required yaks, which don't appear to be few...

@DIzFer
Copy link
Contributor

DIzFer commented May 6, 2020

Apparently a big enough yak and many yaks in a row look very similar. Digikam failed because building libqtav fails. libqtav has a new version github is displaying slightly different, apparently tags != releases? Whatever, but maybe that's why we haven't updated it automatically. In any case, latest libqtav doesn't build either, but HEAD felt like working. Currently 60% along building qtwebkit (dear god why help me not another html engine), and only one more derivation left before digikam's turn.

Additionally, reading through the list, rssguard caught my attention, I read up on it, and out of curiosity I decided to attempt to fix it. Once again, it's one release behind, but latest stable doesn't build either. Besides qmake being bollocks, I don't understand the error in question (make[1]: *** No rule to make target '../../localization/qtbase_uk.qm', needed by 'rcc/qrc_rssguard.o'. Stop.), so I'm stopping here with this one.

@ghost
Copy link
Author

ghost commented May 6, 2020

Thanks for helping with testing. I have rebased on top of master and confirmed the ones you mentioned to be working.

Unfortunately I ran into another issue that is giving me headaches:
When I first tested this Qt update I wasn't aware of the seperate update schedules for Plasma, kde-frameworks and Qt. Because of this and a build failure with Qt 5.14.2, kdeFrameworks 5.56 and Plasma 5.17.5, I tested the new Plasma 5.18.x and Qt 5.14.x in one branch. The Plasma settings application and some other things were not properly working on this branch, but back then I thought this issue was caused by the Plasma update, which it is not. The new Plasma with the old Qt works fine, while the old Plasma with the new Qt also has the issue.
TL;DR: Something with Qt applications loading shared libraries is seriously broken. I have observed this in the Plasma settings application and in Quassel core not being able to find its Postgres database adapter library.

If anyone is in need of prebuilt QtWebEngine binaries for Qt5.14, maybe because they want to use a QtWebEngine-based browser with TLSv1.3 support, this is available here.

I can also confirm the following applications still don't build for me after the rebase:
chatterino2 boomerang digikam quassel diffpdf falkon rssguard

@DIzFer
Copy link
Contributor

DIzFer commented May 6, 2020

Right, the plasma issues make sense. You just prevented me from flooding the plasma update PR with panic and "everything is broken" comments :'D

Short version is, yeah, randomly missing QML modules, no wallpapers, most systemsettings KCMs are broken.

I just contacted the email listed at the github.com/kde organisation, in case that reaches someone that can fix digikam being out of date.

So you go to all this effort to get a more secure QtWebEngine... and I just wanted to have a browser not blind me in every website. Nice priorities, go me :P

@DIzFer
Copy link
Contributor

DIzFer commented May 7, 2020

I have news about digikam: It failed to build! Hooray!

When trying to re-build it with useful logs (nix log appears to be unhelpful this time, possibly because it failed before reaching buildPhase?), well, I was digging out how to update kde-frameworks, so they were now pointing to 5.69 (unstable has 5.68)... so I'm rebuilding the universe once again. [1/95/112] ATM, apparently everything remaining needs kio to be done.

While I'm at it, I'll see if the rest of plasma gets any better with new frameworks. Applications seems to be up-to-date.

@DIzFer
Copy link
Contributor

DIzFer commented May 7, 2020

Something with Qt applications loading shared libraries is seriously broken. I have observed this in the Plasma settings application and in Quassel core not being able to find its Postgres database adapter library.

Wait, are Qt applications failing in general? I've only noticed brokenness with KDE stuffs, which would point to "updating frameworks" being a helpful step. But you're right, quassel doesn't seem to depend on that. Welp :(

@ghost
Copy link
Author

ghost commented May 7, 2020

If you experience the "everything is broken" with the current state of the Plasma update PR, without the Qt update, please flood it with comments. I would expect most these problems to only occur when the Qt update is applied as well.

Multiple failures I have anlayzed were caused by an issue with the outputs of qttools. For example the mpc-qt build fails, because qttools-5.14.2-bin/bin/lupdate is called, but this does not exist. diffpdf fails becuase it tries to call qttools-5.14.2-bin/bin/lrelease, but this does not exist. Both binaries exist in the dev output and did not exist in the bin output of qttools in 5.12.

About the general problems with some applications built agains new Qt: Quassel does actually use quite a few of the k* libraries. I'm not sure if the problem lies in the kdeFrameworks, I haven't seen an application that is broken and doesn't use the k* libraries. I guess updating the frameworks would be worth a try, but I wouldn't count on it fixing the issue.

@DIzFer
Copy link
Contributor

DIzFer commented May 7, 2020

Progress with digikam: cmake failure, with viable logs. Seems like multiple outputs are messing up cmake, and the .cmake in exiv2-dev points to a file found in main exiv2. I know this must be easily fixable, but not how exactly. I'm parking this for now.

Only editing to mention the possibility of that being exactly this issue :/

If you experience the "everything is broken" with the current state of the Plasma update PR, without the Qt update, please flood it with comments. I would expect most these problems to only occur when the Qt update is applied as well.

Yeah, between my tendency to try to do a hundred things at once and the resulting confusion, I'd expect at least some of my issues to be my own fault. I'll try to separate, but at some point these are going to have to learn to live together :P

For now, I confirm Qt 5.14 + Plasma 5.18 + Frameworks 5.69 fails miserably :(

Currently rebuilding Plasma 5.18 without Qt or Frameworks updates.


nix show-derivation nixpkgs.quasselDaemon doesn't seem to list anything from frameworks. The client does, though, so I'd expect that to be the reason frameworks are listed in the package .nix.

Multiple failures I have anlayzed were caused by an issue with the outputs of qttools. For example the mpc-qt build fails, because qttools-5.14.2-bin/bin/lupdate is called, but this does not exist. diffpdf fails becuase it tries to call qttools-5.14.2-bin/bin/lrelease, but this does not exist. Both binaries exist in the dev output and did not exist in the bin output of qttools in 5.12.

The split in qttools looks fun. -bin has lupdate-pro, but lupdate lives in -dev. -dev also has a lupdate-pro, but it actually is a symlink to -bin. Many of the files in qttools-dev/bin are like that, a symlink to -bin/bin. Hopefully this make more sense if one understands what qttools is, but I don't.

@reinhardt
Copy link
Contributor

@Reinhart did you build from this branch without any rebase or change? So commit 8fdd19f?

Yup, exactly. That was the first error I got, before touching anything. Later I fiddled with the versions a bit and got it to build but had trouble with ssl at runtime.

@ghost
Copy link
Author

ghost commented Jul 31, 2020

Unless anyone raises new or existing concerns I will merge this today

@ghost ghost merged commit b1d838e into NixOS:staging Jul 31, 2020
Staging automation moved this from Ready to Done Jul 31, 2020
@jonringer
Copy link
Contributor

nice work everyone :)

@vcunat
Copy link
Member

vcunat commented Aug 11, 2020

Can some of you have a look at the channel-blocking darwin failure? #95199

@callahad callahad mentioned this pull request Sep 25, 2020
10 tasks
callahad added a commit to callahad/nixpkgs that referenced this pull request Sep 26, 2020
This un-breaks libqtav by upgrading it to the current upstream HEAD.
Previously, Qtav was broken by the Qt 5.12 -> 5.14 migration (NixOS#84232).
See commit 819bb63

This also un-pins ffmpeg_3; libqtav works fine with current ffmpeg.

Digikam is the only derivation in nixpkgs that has referenced libqtav, thus
upgrading to a pre-release revision poses minimal risk.
rapenne-s pushed a commit to rapenne-s/nixpkgs that referenced this pull request Oct 6, 2020
This un-breaks libqtav by upgrading it to the current upstream HEAD.
Previously, Qtav was broken by the Qt 5.12 -> 5.14 migration (NixOS#84232).
See commit 819bb63

This also un-pins ffmpeg_3; libqtav works fine with current ffmpeg.

Digikam is the only derivation in nixpkgs that has referenced libqtav, thus
upgrading to a pre-release revision poses minimal risk.
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Done
Development

Successfully merging this pull request may close these issues.

Update shiboken2 (eventually) Qt needs update to 5.14 and more maintainers