Build all derivations on Linux, unless specified otherwise #19990
Conversation
|
@domenkozar, thanks for your PR! By analyzing the history of the files in this pull request, we identified @edolstra, @rbvermaa and @chaoflow to be potential reviewers. |
|
Of what exactly does this change the behavior? The Hydra building the release? |
|
You wanted to say «all except those marked as broken», right? |
| @@ -85,7 +85,7 @@ rec { | |||
| packagePlatforms = mapAttrs (name: value: | |||
| let res = builtins.tryEval ( | |||
| if isDerivation value then | |||
| value.meta.hydraPlatforms or (value.meta.platforms or []) | |||
| value.meta.hydraPlatforms or (value.meta.platforms or [ platforms.linux ]) | |||
There was a problem hiding this comment.
Isn't platforms.linux already a list? Why the extra [] around?
domenkozar
force-pushed
the
buildtheworld
branch
from
e9d7ce3 to
098cfeb
Compare
|
@7c6f434c correct. |
|
Any idea how many that would be? Maybe we could restrict them just to |
|
Doesn't this just mean "unless overridden to only be on other platforms"? I reacted negatively to the title but the code looks more reasonable. |
vcunat
changed the title
|
Yes, it does. |
|
@vcunat yeah |
|
Yeah, now we have some security mass rebuilds. |
|
@domenkozar: why do you think a separate jobset is needed? This can only add new builds, not break anything, right? |
|
@vcunat this is going to find packages I've already been hunting down that no longer have their sources available online. I would expect some breakage |
|
Perhaps, but not breakage of anything that works now, so it shouldn't hurt anyone. And we're do have a few hundred failures already, so some more (temporary) red lines shouldn't make the result significantly more confusing. |
|
Looks like @7c6f434c's feedback was resolved. I'd say we merge this and deal with the newly exposed brokenness. |
domenkozar
force-pushed
the
buildtheworld
branch
from
098cfeb to
5e2f1be
Compare
domenkozar
force-pushed
the
buildtheworld
branch
from
5e2f1be to
5c750e4
Compare
|
Additional 862 builds: http://hydra.nixos.org/eval/1308378?full=1&compare=trunk#tabs-new |
|
Is it expected that so many Haskell packages fail on Darwin? Are we looking to fix the failures or drop some of these packages, especially if they have not been updated in a long time? |
|
Darwin failures have nothing to do with this PR :) |
|
Tim Steinbach <notifications@github.com> writes:
Are we looking to fix the failures or drop some of these packages,
especially if they have not been updated in a long time?
I'd suggest we're looking to reduce problems and failures by way of
either dropping unsupported packages, or fixing them, whatever is more
appropriate.
|
|
862 new jobs but > 14k queued jobs. It would be nicer to base this on a more recent master to avoid that. (It would also actually prepare binaries useful after the merge.) |
|
@vcunat the parent commit is from yesterday. The queue was long since it just needs to be processed. I'm merging this, there are 95 build failures but I see no reason to block this change. |
|
Oh, I'm sorry, I should've checked the parent. The first-pass processing tends to be relatively fast and this felt rather slow. There seems unfortunately no indication how much of queued jobs is in the first-pass stage... |
Or they will be tried to be built on x86_64 since #19990.
This was added in <NixOS#19990> to ensure that packages that previously weren’t building at all would get at least one platform’s worth of coverage. However, since most `runCommand` calls don’t explicitly set any `meta` information, this meant that things like `tests.makeWrapper` were only being run for `x86_64-linux`. Since most trivial builder calls should work on all platforms, it doesn’t make much sense for them to be restricted in this way, and we shouldn’t unnecessarily penalize other platforms by treating an empty `meta.platforms` inconsistently with `check-meta`’s maximally‐permissive interpretation. Actual packages that only work on a subset of platforms should, of course, set `meta.platforms` explicitly.
This was added in <NixOS#19990> to ensure that packages that previously weren’t building at all would get at least one platform’s worth of coverage. However, since most `runCommand` calls don’t explicitly set any `meta` information, this meant that things like `tests.makeWrapper` were only being run for `x86_64-linux`. Since most trivial builder calls should work on all platforms, it doesn’t make much sense for them to be restricted in this way, and we shouldn’t unnecessarily penalize other platforms by treating an empty `meta.platforms` inconsistently with `check-meta`’s maximally‐permissive interpretation. Actual packages that only work on a subset of platforms should, of course, set `meta.platforms` explicitly.
This was added in <NixOS#19990> to ensure that packages that previously weren’t building at all would get at least one platform’s worth of coverage. However, since most `runCommand` calls don’t explicitly set any `meta` information, this meant that things like `tests.makeWrapper` were only being run for `x86_64-linux`. Since most trivial builder calls should work on all platforms, it doesn’t make much sense for them to be restricted in this way, and we shouldn’t unnecessarily penalize other platforms by treating an empty `meta.platforms` inconsistently with `check-meta`’s maximally‐permissive interpretation. Actual packages that only work on a subset of platforms should, of course, set `meta.platforms` explicitly.
This was added in <NixOS#19990> to ensure that packages that previously weren’t building at all would get at least one platform’s worth of coverage. However, since most `runCommand` calls don’t explicitly set any `meta` information, this meant that things like `tests.makeWrapper` were only being run for `x86_64-linux`. Since most trivial builder calls should work on all platforms, it doesn’t make much sense for them to be restricted in this way, and we shouldn’t unnecessarily penalize other platforms by treating an empty `meta.platforms` inconsistently with `check-meta`’s maximally‐permissive interpretation. Actual packages that only work on a subset of platforms should, of course, set `meta.platforms` explicitly.
This was added in <NixOS#19990> to ensure that packages that previously weren’t building at all would get at least one platform’s worth of coverage. However, since most `runCommand` calls don’t explicitly set any `meta` information, this meant that things like `tests.makeWrapper` were only being run for `x86_64-linux`. Since most trivial builder calls should work on all platforms, it doesn’t make much sense for them to be restricted in this way, and we shouldn’t unnecessarily penalize other platforms by treating an empty `meta.platforms` inconsistently with `check-meta`’s maximally‐permissive interpretation. Actual packages that only work on a subset of platforms should, of course, set `meta.platforms` explicitly.
This was added in <NixOS#19990> to ensure that packages that previously weren’t building at all would get at least one platform’s worth of coverage. However, since most `runCommand` calls don’t explicitly set any `meta` information, this meant that things like `tests.makeWrapper` were only being run for `x86_64-linux`. Since most trivial builder calls should work on all platforms, it doesn’t make much sense for them to be restricted in this way, and we shouldn’t unnecessarily penalize other platforms by treating an empty `meta.platforms` inconsistently with `check-meta`’s maximally‐permissive interpretation. Actual packages that only work on a subset of platforms should, of course, set `meta.platforms` explicitly.
This was added in <NixOS#19990> to ensure that packages that previously weren’t building at all would get at least one platform’s worth of coverage. However, since most `runCommand` calls don’t explicitly set any `meta` information, this meant that things like `tests.makeWrapper` were only being run for `x86_64-linux`. Since most trivial builder calls should work on all platforms, it doesn’t make much sense for them to be restricted in this way, and we shouldn’t unnecessarily penalize other platforms by treating an empty `meta.platforms` inconsistently with `check-meta`’s maximally‐permissive interpretation. Actual packages that only work on a subset of platforms should, of course, set `meta.platforms` explicitly.
We have packages dating back to 2010 that never get built or tested. In order to prevent that in the future, I propose we always build packages at least on Linux platform.
This will increase failed builds, but we should go through them and see what's feasible to keep.
If there won't be strong reasons not to do this, I'll make a separate hydra jobset to get the first impression.
PS: Haskell, Python and other will still not get build due to overrides in
pkgs/top-level/release.nixPS2: Broken derivation will still not build