Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0046] Platform Support Tiers #46

Merged
merged 21 commits into from Jan 14, 2020

Conversation

7c6f434c
Copy link
Member

@7c6f434c 7c6f434c commented Apr 28, 2019

I hope to have some meta-discussion here so that #31 gets a language to discuss its specific merits.

render link

@Ericson2314
Copy link
Member

Could we have an "embedded" level for and https://github.com/NixOS/nixpkgs/blob/master/lib/systems/examples.nix#L102-L113 NixOS/nixpkgs#59285, this is where very few if any packages are expected to work, but we do provide toolchains for users' convenience. This, as the lowest level support by far, should be an auto-accept for just about anything the compiler supports.

@7c6f434c
Copy link
Member Author

Indeed.

* OS kernel
* C compiler
* C library
* NixOS/non-NixOS global layout, in case of Linux with glibc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some things (OpenGL, for example) where level of support for Nix-on-non-NixOS-Linux is lower than on NixOS.

@globin globin changed the title An RFC on platform support tiers [RFC 0046] Platform Support Tiers Apr 28, 2019
@Ericson2314
Copy link
Member

Thanks for adding embedded!

@matthewbauer
Copy link
Member

matthewbauer commented Apr 29, 2019

My opinion is we need to maintain an official list of what works and what doesn't. We can change this as time goes on, but we should codify and document what we support. For what exists now, my list of tiers would be:

Tier 1

All packages are expected to work.

Available in binary cache + tier 2 + tier 3

  • x86_64-linux
  • x86_64-darwin
  • aarch64-linux

Tier 2

Most packages are expected to work.

Bootstrap tools are available + tier 3.

  • armv{6,7,8}*-linux
  • mipsel-linux
  • i686-linux
  • powerpc64le-linux

Tier 3

Some packages are expected to work.

Cross compilation support is available. Toolchains are built in cross-trunk via release-cross.nix.

  • aarch64-none
  • avr
  • arm-none
  • i686-mingw32
  • i686-none
  • x86_64-none
  • x86_64-mingw32
  • wasm-wasi
  • powerpc-none
  • powerpcle-none
  • riscv32-linux
  • riscv64-linux
  • sparc-linux
  • sparc64-linux
  • alpha-linux
  • a few more

Tier 4

No packages are expected to work.

Broken or no support is provided, but may be possible.

  • aarch64-darwin
  • i686-darwin

ABIs

ABIs are a different concern, but still worth addressing. They should map to one tier.

  • gnu{,eabi,eabihf} - tier 1
  • musl{,eabi,eabihf} - tier 2
  • android{,eabi,eabihf} - tier 3
  • mingw32 - tier 3
  • apple-ios - tier 4

Toolchains

  • GCC+Binutils - tier 1
  • Clang+ld64 - tier 1
  • Clang+binutils - tier 2
  • Clang+LLVM - tier 3
  • GCC+LLVM - tier 4

@7c6f434c
Copy link
Member Author

@matthewbauer Thanks for a much better enumeration of platforms! I will try to incorporate most of it into the RFC draft. Do you want to be a coauthor of the RFC? Do you want branch access?

I would say that an honest description of what exists now would have x86_64-darwin and aarch64-linux as tier 1.5 — we aim to have everything working, but significantly more breakages happen there than on x86_64-linux.

I wonder what is our static build support? It should be considered an ABI variant with just gnu-static and musl-static as tier 3 ABIs?

@vcunat
Copy link
Member

vcunat commented Apr 30, 2019

I suspect that "all packages [with matching meta.platforms]" might be a little too hard to fulfill, though it depends on how hard we take the "expected to work" part and/or whether we tweak the exact meaning of meta.platforms.

In any case I expect it will be better to start with this simpler style – and only eventually consider upgrading the approach, say, to (conceptual) mapping: (package, platform) -> (level of support, maintainers). I think it's quite common that (particular) maintainers only really care about some platforms, even though the package often kind-of works on more of them. In general there can be tension when one platform "slows down" another one in some way. There's a bit more discussion also in the thread linked by GitHub just above my post.

@matthewbauer
Copy link
Member

@matthewbauer Thanks for a much better enumeration of platforms! I will try to incorporate most of it into the RFC draft. Do you want to be a coauthor of the RFC? Do you want branch access?

Yeah I can coauthor!

I would say that an honest description of what exists now would have x86_64-darwin and aarch64-linux as tier 1.5 — we aim to have everything working, but significantly more breakages happen there than on x86_64-linux.

Yeah I agree that x86_64-linux is the gold standard. But there's no reason we can't support aarch64-linux or x86_64-darwin equally. They both are built fully by Hydra.

I wonder what is our static build support? It should be considered an ABI variant with just gnu-static and musl-static as tier 3 ABIs?

I would consider the static builds to be an overlay - not a different platform. Support is definitely low since we don't have any Hydra jobs for it. Tier 3 for Musl probably makes the most since. My understanding is support is much better for static linking in Musl than Glibc.

@FRidh
Copy link
Member

FRidh commented Apr 30, 2019

But there's no reason we can't support aarch64-linux or x86_64-darwin equally. They both are built fully by Hydra.

Not every developer has access to those platforms nor interest to support them.

(package, platform) -> (level of support, maintainers) might indeed be too fine-grained for now, however I think we should already consider something like https://nixos.wiki/wiki/User:Profpatsch#nixpkgs_support_matrix
(Was referred to in NixOS/nixpkgs#60005)

@vcunat
Copy link
Member

vcunat commented Apr 30, 2019

For aarch64-linux there's the community machine and @GrahamcOfBorg, so not too hard if one is interested; macOS is probably hard for those who don't have it. In any case, the worst thing is human time/effort.

@7c6f434c
Copy link
Member Author

@FRidh Then I look at the list of questions and list of platform dimensions in this RFC and find the linked support matrix a bit too much of a simplification…

@matthewbauer I do want to describe the current/near future state — I do not see Darwin rising from tier-1.5 to tier-1 in the next 3 months.

Overlay or whatever, but static builds are a type of compilation target in my opinion.

I tried to merge the list of build types, and I moved things down to reflect the current state. I hope I am not too wrong.

Gave you push access (apparently I can only do it for the entire fork, but whatever), included as a coauthor in the next revision.

@Mic92 Mic92 added status: open for nominations Open for shepherding team nominations and removed status: new labels May 2, 2019
@nixos-discourse
Copy link

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/rfc-45-and-rfc-46-are-open-for-nominations/2821/1

@CrystalGamma
Copy link

CrystalGamma commented May 2, 2019

Would it make sense to track separate "support levels" (which describe how much is expected to work) and "acceptance levels" (which describe how much platform-specific code – be it configuration conditionals, patches or entire packages – will be accepted).

The levels should then be defined such that, given enough development activity, a "support level" corresponding to the "acceptance level" can reasonably be reached with the amount of platform-specific changes permitted by the "acceptance level".

That would allow us to differentiate between platforms that are mature (support ≥ acceptance) or in the process of being developed (support < acceptance) without having x.5 tiers. Though perhaps shorthands like "tier 3+" for "support level 3, acceptance level 2" would still be useful.

Regardless of whether this makes sense, one thing that should be clarified is what actually constitutes a "platform-specific fix": NixOS/nixpkgs#45474 (which originally started this discussion) was about disabling the intel Mesa driver on non-x86, so that Mesa can be built on ppc64le; is that a ppc64le-specific fix? I would argue that it's actually an x86-specific change (even though it served the purpose of fixing the build on other platforms), considering that the intel driver does not work (or at least didn't at the time of that PR, haven't checked since) on other architectures.

@7c6f434c
Copy link
Member Author

7c6f434c commented May 2, 2019

@CrystalGamma Well, the only thing I am reasonably confident, is that we should consider at least the questions listed in the general introduction. But there are too many questions to track separately… In some sense, there are three key dimensions: support, acceptance, and how many packages are used are known to work. (Even x86_64-linux is not perfect in that last metric).

I guess if we think in terms of per-direction tiers, acceptance is weaker than both support and coverage, and support can either come as recognition of coverage or because of coordinated effort of enthusiasts donating build cycles etc.

I tried to word that carefully: I think that disabling intel driver on non-x86 is a cleanup of incorrect assumptions that only ARM and x86 exist, and a problem on low-acceptance platform is enough to replace the wrong assumption with a better one.

Maybe a good test is «do we expect the exact same fix to help on currently unsupported platforms?» (be it RISC-V instead of POWER or BSD libc instead of musl). Should we add this into RFC?

@Ericson2314
Copy link
Member

do we expect the exact same fix to help on currently unsupported platforms

That's a good question, because the answer is usually "yes"!

@7c6f434c
Copy link
Member Author

7c6f434c commented May 2, 2019

@Ericson2314 there is a bit of selection bias — the fixes applied to non-compiler-toolchain packages to fix minority platforms might have to be generic cleanups to stay in the mainline…

@7c6f434c
Copy link
Member Author

7c6f434c commented May 3, 2019

I guess another interesting question is whether we want to define a place for the support table to live, if we want the low-acceptance additions to be cheap and only increases in acceptance to require RFCs…

@matthewbauer
Copy link
Member

I guess another interesting question is whether we want to define a place for the support table to live, if we want the low-acceptance additions to be cheap and only increases in acceptance to require RFCs…

I think it's definitely important to document what's built on Hydra + what we have bootstrap tools for.

@7c6f434c
Copy link
Member Author

7c6f434c commented May 3, 2019

I agree, and maybe this RFC should include the initial list, but where do we want to maintain the updates to the list?

@matthewbauer
Copy link
Member

Perhaps here: https://nixos.org/nixpkgs/

Although that's not so visible...

@erictapen
Copy link
Member

The RFC mentions an appendix to the Nixpkgs manual:

If this RFC is accepted, the following changes will happen:

  • The definition of support tiers and a list of platforms with the corresponding support tiers is added as an appendix to the Nixpkgs manual. The list of platforms is further maintained as a part of the Nixpkgs manual. The platform list in the manual will be initially based on the list in the appendix of the present RFC.

Did this already happen? If not is there a way to help with it?

@Mic92
Copy link
Member

Mic92 commented Aug 4, 2020

I don't think so. I don't think anyone would mine if you start a draft pr for the manual.

@7c6f434c
Copy link
Member Author

7c6f434c commented Aug 5, 2020 via email

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/21-05-retrospective/13426/3

@L-as
Copy link
Member

L-as commented Sep 3, 2021

Is there a list of the platforms and their tiers other than this RFC?

@rjzak
Copy link

rjzak commented Oct 10, 2022

@CrystalGamma how's the powerpc64le-linux support looking currently? I can't get things to build on my Power9 system.

@sternenseemann
Copy link
Member

how's the powerpc64le-linux support looking currently? I can't get things to build on my Power9 system.

There has been work on this platform recently by mainly @amjoseph-nixpkgs which is available in the unstable nixpkgs/nixos channels. There is no formal support / support requirement at the moment. You can report any specific issues you are encountering over at the nixpkgs issue tracker.

@ghost
Copy link

ghost commented Oct 10, 2022

how's the powerpc64le-linux support looking currently? I can't get things to build on my Power9 system.

There has been work on this platform recently by mainly @amjoseph-nixpkgs

By cherry-picking the PRs below (and their dependent PRs) you can cross-compile a NixOS install image for powerpc64le-linux or mips64el-linux from x86-linux:

The powerpc64le-linux image even boots. I haven't tried the mip64el-linux image yet.

In the long term, it would be nice if we could add an (absolutely not channel-blocking in any way, shape or form) Hydra job for these install images. That way people can try NixOS on these machines without having to trust binaries from anywhere other than Hydra.

That's the limit of my ambitions for these platforms. I don't think it makes sense to build anything more than the installer on Hydra. It would be nice if we had a name for this level of functionality; something like "enabled but unsupported".

@rjzak
Copy link

rjzak commented Oct 10, 2022

I haven't cross-compiling from x86_64 since I have a ppc64le machine I use as my desktop, and Void Linux PPC has a package for Nix, which seems to work (except for getting things to compile due to the stack checking method).

@ghost
Copy link

ghost commented Oct 10, 2022

You can native-compile the installer too.

@asymmetric
Copy link
Contributor

asymmetric commented Jul 25, 2023

Is there a reason why the @NixOS/aarch64-maintainers team was never created?

@zimbatm
Copy link
Member

zimbatm commented Jul 25, 2023

Probably because it wasn't clear who should do it. Let me know who should be on the team and I will create it.

@asymmetric
Copy link
Contributor

Any takers?

@Atemu
Copy link
Member

Atemu commented Jul 31, 2023

Well, at least we know why it wasn't created now.

@kip93
Copy link

kip93 commented Aug 1, 2023

Probably because it wasn't clear who should do it. Let me know who should be on the team and I will create it.

Hello!

Heard about this on matrix. Don't have much experience working on nixpkgs itself but I have been using nix for many years now, mostly for aarch64 systems and doing a lot of cross compiling, so maybe I can be of some help here? If so, I'd be more than happy to join and help.

@JulienMalka
Copy link
Member

Any takers?

I have a aarch64 macbook running NixOS and just ordered an ampere dev kit, I would be interested if this team was created to be part of it :)

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-08-03-documentation-team-meeting-notes-69/31263/1

@delroth
Copy link

delroth commented Aug 5, 2023

Probably because it wasn't clear who should do it. Let me know who should be on the team and I will create it.

I use NixOS on 3 different AArch64 servers, happy to help with aarch64-specific issues.

EDIT: cross-posted your request on #nixos-on-arm:nixos.org since I suspect your lack of volunteers is mostly because nobody is subscribed to this PR thread.

@aforemny
Copy link

Any takers?

I am using NixOS on aarch64 machines (laptop, chromebook) and would be interested in becoming a platform maintainer. Could you please add me to the team?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet