Building my system now against this PR.

Plasma and applications that I have installed work fine.

I just tried it (thanks a lot for this work), and I got rid the first errors I use to have (so guess it's quite an improvement already and it would make sense to merge since 20.09 is completely broken right now), but I still have an error with a package mlt that is marked as broken:

Package ‘mlt-6.22.1’ in /nix/store/3xbc4j5k2qlhlw2wy5jbw2qfrhc6daca-source/pkgs/development/libraries/mlt/qt-5.nix:67 is marked as broken, refusing to evaluate.

I didn't explicitely installed that package, so not sure where it comes from. Maybe it's not related at all with this PR, but the fact that the issues comes from qt-5.nix makes me think that this PR could be related.

Thanks a lot!

@tobiasBora do you happen to have synfigstudio in your configuration? Or kdenlive?

kdenlive fix for master in #102364

@FRidh Indeed, I do have kdenlive! I just disabled it, and I'll try the kdenlive fix after. Meanwhile, I got another issue:

Package ‘qtwebkit-5.212.0-alpha4’ in /nix/store/3xbc4j5k2qlhlw2wy5jbw2qfrhc6daca-source/pkgs/development/libraries/qt-5/qtModule.nix:54 is marked as broken, refusing to evaluate.

was able to build largely everything... was about to merge xD

This will make all Plasma users on NixOS 20.09 install two versions of Qt. ☹️

Am I unusual in installing all of kdeApplications? Something is broken via qtwebkit and I'm not sure what

This will make all Plasma users on NixOS 20.09 install two versions of Qt. frowning_face

Yeah, I did specify it would be essential if people could wait to merge because your review is essential ☹️

IIRC, there was already 5.12 and 5.14 likely to be present, or did you mean 5.15 in addition to this?

Because we don't have a qt-applications.nix, we have many packages built only for a specific qt version, declared in all-packages.nix and we don't know for sure if they are using 514 specifically because they really need to, or because they were falsley identified as broken with qt514.

In 78b8ce7 and 22167ae @ttuegel you marked many such packages, while I suspect that they were broken because some of their inputs weren't part of libsForQt5*, but declared in the widest scope. I explained this false identification of the issue and fixed it in b5c6505 .

So, if someone is bothered that they have many qt versions in their closure, they should track down whatever it is that pulls these older qt versions, and see if pinning them to qt514 is justified. Before attempting to build a package with a different libsForQt5*.callPackage, one should use the Nix script with the explained usage in the 1st comment of #101369 (comment) - to make sure that by changing {libsForQt514 -> libsForQt5}.callPackage there are no incompatible inputs suddenly present in the expression.

Even if a package doesn't use inconsistent qt versions and does not build with qt515, one should contact upstream and tell them. Ideally, a link to such an upstream issue should be added to where qt514 is pinned. And you know, if there's a patch or a new version available that fixes that ...

This will make all Plasma users on NixOS 20.09 install two versions of Qt. ☹️

It's better to have more qt versions in the closure then broken packages.

IIRC, there was already 5.12 and 5.14 likely to be present, or did you mean 5.15 in addition to this?

There should only have been Qt 5.14 if you install the KDE PIM suite, which is unavoidable because KDE PIM depends on the newer Qt. The desktop alone only depended on Qt 5.12, but now it depends on both Qt 5.12 and Qt 5.15.

It's better to have more qt versions in the closure then broken packages.

I strongly disagree. If a maintainer is bothered that their package is broken, they should fix the package. If they don't fix the package, then the package is de facto unmaintained, and we hope a user will step up to help maintain it. Either way, there is a strong signal: a broken package. This pull request replaces a strong signal with a weak one: closure size creep.

Checking the closure of my own system, the only application I have that is pulling in Qt 5.14 is ktorrent. The only part that is pulling in 5.12 is qtvirtualkeyboard.

Maybe ktorrent can work with 5.15, I can't recall whether we checked it. Regardless, it functions. Yes, it may pull in a whole extra Qt, but if that is what needed to make it function from our side, then I think that is acceptable. Of course, if, with minor work, we could make it work with 5.15, we should.

I don't know about qtvirtualkeyboard, I hope that could be switched to 5.15.

edit: I was checking only qtbase, not the other libs.

IIRC, there was already 5.12 and 5.14 likely to be present, or did you mean 5.15 in addition to this?

There should only have been Qt 5.14 if you install the KDE PIM suite, which is unavoidable because KDE PIM depends on the newer Qt. The desktop alone only depended on Qt 5.12, but now it depends on both Qt 5.12 and Qt 5.15.

Do you have an example?

It's better to have more qt versions in the closure then broken packages.

I strongly disagree. If a maintainer is bothered that their package is broken, they should fix the package.

The issue is more complicated then that, and I oversimplified it previously. My opinion is that we shouldn't expect every maintainer of such a broken package to comply with the override you forced for all kde attrsets, and that was inherited in the widest scope here which is the override and the inheritance that broke many packages. Let's take as an example okteta which was broken before this PR, i.e at e.g 6928309 :

Here's how it was defined:

It builds, but it was using qtbase 5.15 but other qt libraries from kde attrsets built with qt5.14. So should it also use qt5.14? Just because it's inputs are by default built with qt5.14? What is the casual maintainer of it is expected to do? Do you expect every package maintainer to realize that because of this:

And alike, their package is broken? I consider myself somewhat experienced with Nix and it took me quiet some time to understand what's wrong with everything.

Say the maintainer does understand the above, and they still want to use qt5.15 for their package, do they need to override the libsForQt5.callPackage function with a newScope? Do you truly think that's more sensible then the current situation? (Which I still don't understand what's wrong with it).

Even if all of these maintainers would have understood the reason for their package to break, did we expect them to think "I know that the kde override breaks my package but I don't know why all kde attrsets are stuck with qt5.14 and I won't touch it because it's @ttuegel's land, so I'll put an ugly override in all-packages such as this, so my package will use qt5.15 but still work"? For how long did we plan to pin kde to use qt5.14?

Hopefully next year we can just package the latest non-lts version. So that we have plasma using a more recent version of Qt. Would save a lot of this mess. https://discourse.nixos.org/t/what-should-stable-nixos-prioritize/9646/56

IIRC, there was already 5.12 and 5.14 likely to be present, or did you mean 5.15 in addition to this?

There should only have been Qt 5.14 if you install the KDE PIM suite, which is unavoidable because KDE PIM depends on the newer Qt. The desktop alone only depended on Qt 5.12, but now it depends on both Qt 5.12 and Qt 5.15.

Do you have an example?

Do I have an example of... the default Plasma install depending on Qt 5.12 and Qt 5.15? Yes: Dolphin and Konsole are part of the default installation and are now built with Qt 5.15.

Let's take as an example okteta...

It builds, but it was using qtbase 5.15 but other qt libraries from kde attrsets built with qt5.14. So should it also use qt5.14? Just because it's inputs are by default built with qt5.14? What is the casual maintainer of it is expected to do?

I introduced libsForQt5.callPackage so that maintainers don't have to worry about this. Every library is built with every available version of Qt. If a package gets all its Qt dependencies through this mechanism, then the situation you're describing won't happen unless there is a bug, such as a dependency that does not follow the instructions in the manual for adding Qt libraries to Nixpkgs. I expect bugs to be reported so that they can be fixed. I certainly don't want the obscure override you describing.

In this specific case, I expect Okteta to be built with Qt 5.15. The dependencies which are incorrectly pinned to Qt 5.14 should be unpinned. If the maintainer isn't comfortable un-pinning the dependencies themselves, I hope they will open a bug report about the incorrectly pinned libraries and tag the maintainers.

Hopefully next year we can just package the latest non-lts version. So that we have plasma using a more recent version of Qt.

I hope so, too, but that will require that Plasma's maintainers keep up with the latest Qt, which they have not done so far.

In this specific case, I expect Okteta to be built with Qt 5.15. The dependencies which are incorrectly pinned to Qt 5.14 should be unpinned. If the maintainer isn't comfortable un-pinning the dependencies themselves, I hope they will open a bug report about the incorrectly pinned libraries and tag the maintainers.

The issue I opened regarding unpinning deps is #100707 . The deps which were supposed to be unpinned are all of kde attrsets, and I had no answer to the question why were they pinned in the 1st place (the commits in #97242 didn't explain why) and testing has proved that the changes here were appropriate and that kde does support qt515.

Do you have an example?

Do I have an example of... the default Plasma install depending on Qt 5.12 and Qt 5.15? Yes: Dolphin and Konsole are part of the default installation and are now built with Qt 5.15.

I don't understand. So is there a problem now or not?

$ nix-store --query --references $(nix-build --no-out-link -A konsole -A dolphin) | grep qt
/nix/store/dw26fwjr4lgwig2daikfagrpw4w2x8y5-qtbase-5.15.0
/nix/store/0vp7fwmxxympnc4nlan55z8z14q7r5x3-qtdeclarative-5.15.0-bin
/nix/store/anw198g8cb87b2bkhbb5i8vwf9cjjqja-qtwayland-5.15.0-bin
/nix/store/gqw7v161mvb99y6r42xbdy8d8gj6kpn4-qtbase-5.15.0-bin
/nix/store/hdwnp8phf0d6nkkai16nmkxm0ksxigc8-qtsvg-5.15.0-bin
/nix/store/l1bbai6n58zlyqj8i8z2m3kk40z1bmvx-qttools-5.15.0-bin
/nix/store/m6xv8b2g9sm5h0b3141n9r6y0ybf1dln-qtquickcontrols-5.15.0
/nix/store/vh6r2n93w9nppvxm6z785a1560jvqi36-qtquickcontrols2-5.15.0-bin
/nix/store/mcjffi20a1b69mpkzmkh8x0sw5rjh6ip-qtmultimedia-5.15.0-bin
/nix/store/j2hfnvw6hdfjsiazkhb1fcs8ri1nnw16-qtxmlpatterns-5.15.0

... such as a dependency that does not follow the instructions in the manual for adding Qt libraries to Nixpkgs.

You are ignoring the fact that some people in the past have added packages which they considered back then as applications and they used libsForQt5.callPackage in all-psckages (as explained in the manual, ex 15.12), and only afterwards someone else used them as inputs. Maybe the person that used the application as an input should have been smart enough to realize that they are using an input which is not part of libsForQt5, and that may cause incompatibilities. But again, even if one would have thought about it, they should have understand that it might cause incompatibilities only if in the future someone might pin that "application" (which is also an input for the newer package) to a certain qt version - and this is what was done to many packages with reversed dependencies in #97242 .

An example of the "phenomena" above is kile: Which used konsole as an input while konsole is probably considered by many as an "application", but konsole was pinned to qt5.14. Specifically in this case, kile was also pinned to qt5.14, so there were no incompatibilities there I think, but it was impossible to make it use 5.15 unless it's konsole (and maybe more inputs) was unpinned as well.

@doronbehar @FRidh

This MR somehow pulled in a change to aliases.nix from commit 0f1be68 without any of the other changes. The result is that ant-dracula-theme is now broken on 20.09:

$ nix-build -A ant-dracula-theme '<nixpkgs>'
error: Alias ant-dracula-theme is still in all-packages.nix

The aliases.nix file states we should use dracula-theme instead of ant-dracula-theme yet that package doesn't exist in 20.09. I think we need to either fix aliases.nix or backport #99704.

I don't understand. So is there a problem now or not?

Yes, there is a problem, but I admit it is not the problem I originally thought. Here is the real problem: Plasma packages have been changed to use Qt 5.15!

$ nix-store --query --references $(nix-build --no-out-link -A plasma5.kwin) | grep 'qt'
/nix/store/brj4907l6gqc9ybm87xfq8xjvwy49zal-qtbase-5.15.0
/nix/store/f053wk2nnd2jbj7qry406vlsw25kxjng-qtx11extras-5.15.0
/nix/store/3df5ckc81mv483h135001vamd3b2isjg-qtbase-5.15.0-bin
/nix/store/vna7la11hsfk7ngi6wcml292iv5aq3yj-qtdeclarative-5.15.0
/nix/store/821x5f1mbkhivj3lhm8knlvyfb9zmw3d-qttools-5.15.0-bin
/nix/store/r0q303084y8yrm0fp7cs0i4zhd26d9d5-qtquickcontrols-5.15.0
/nix/store/rdjpx7ihfhy57wzfl1wlald22d48srcm-qtdeclarative-5.15.0-bin
/nix/store/cjhdg9chljbsk97kh2f6ixqgzj919gwj-qtsvg-5.15.0
/nix/store/xsrcic7gw9qapchip8slwvy5wnbrsnih-qtsvg-5.15.0-bin
/nix/store/ya0v5fwbbwflpyaf1rx19fdfpk4ghrvs-qtwayland-5.15.0-bin
/nix/store/1jvp0jp59y3fma32d5qk2nbfyiwm5ani-qtgraphicaleffects-5.15.0
/nix/store/6aal08ag6xsz8nyk4cavayhbrlcv6nj6-qtquickcontrols2-5.15.0-bin
/nix/store/5ajwvq9hk623xhdm4q5snzjq1j9v70fr-qtwebengine-5.15.0-bin
/nix/store/v27nwfy8rwcscw1a9pxwwp6bqs4ay916-qtmultimedia-5.15.0-bin
/nix/store/7crdwnqrc6pcs6a05lw3arnj8vid33ld-qtvirtualkeyboard-5.15.0
/nix/store/gsgyizlna2yqylhp06f3qayg5w5kk0j4-qtwebchannel-5.15.0-bin
/nix/store/rvxhn4d0gr95cpfd6ad4brj631r7smhi-qtlocation-5.15.0-bin
/nix/store/a96lh8zrfzdgs8cy7741szi7b1am4g2j-qtsensors-5.15.0-bin
/nix/store/zlmyaa31ihyxcfqihrf186zk5d0dq24a-breeze-qt5-5.18.5
/nix/store/dnvjmrlmyjzz47442fiylhg8yic2pcwx-breeze-qt5-5.18.5-bin
/nix/store/icmfnhxwmvvd10saz257wkxyxj7xhnpn-qtxmlpatterns-5.15.0

Plasma 5.18 does not officially support Qt 5.15. The packages can build with Qt 5.15, but upstream does not test them with that version. My own testing revealed a lot of glitches. We intentionally released NixOS 20.09 with Plasma 5.18 built with Qt 5.12, because my view was that Plasma 5.18 built with Qt 5.15 was not stable. It would seem that somewhere along the way, my judgement was overridden by a higher power, who did not deign to make themselves known.

EDIT: Oh, I guess (contrary to the title) this pull request also makes Plasma use Qt 5.15? 😮

You are ignoring the fact that some people in the past have added packages which they considered back then as applications and they used libsForQt5.callPackage in all-psckages (as explained in the manual, ex 15.12), and only afterwards someone else used them as inputs.

I am not ignoring that. People add packages all the time that are broken in subtle ways; broken in ways that we don't realize until later. I add packages sometimes that are broken in subtle ways. I expect that to happen. We should find ways to make it easier for people to do the right thing. We should fix the broken packages as we find them. I think people want to package things the right way, and I think they look for examples in Nixpkgs when they do it, so we should not make more broken packages (bad examples) for them to follow. In my view, this does not make good examples. However, in light of the above changes (moving everything to Qt 5.15) then my comments in this paragraph are irrelevant and may safely be ignored. 🤷