This sounds great—both the design proper and the follow-up proposals, which mirrored exactly my thoughts as I read the detailed design.

The modularity problem is interesting, and I'd like to come back to it, but the core benefit here is enough I'm happy to punt for now.

Also fwiw I learned the other day there is some extra stuff to pass for all cmake derivations with cmakeFlags for cross compilation. This would also make doing that more elegant.

@Ericson2314 the cross-compilation use case sounds like it could make a good example to add to the RFC, do you mind expanding on that?

I do have some vague ideas on the modularity aspect, but agreed on wanting to punt that for now.

Sure https://github.com/NixOS/nixpkgs/pull/25218/files#r113298669

BTW https://github.com/NixOS/nixpkgs/blob/master/lib/composable-derivation.nix is some ancient thing for configureFlags.

If the implementation of this is rock solid, I am in favor. If it is implemented OKish, I am not on board. I have concerns about eval in particular. Merely suggesting eval seems to suggest a lack of general shell scripting experience.

So, the general idea of having special support for CMake is reasonable, although not strictly needed, but the devil will be in the details. I would consider all existing instances of eval to be candidates for being a bug. I.e., just because someone used eval, doesn't mean we should continue to do it.

@aneeshusa: In order to move this forward the @NixOS/rfc-steering-committee would like to open this up for shepherd nominations in case you are still interested in completing this and think it generally is ready for further discussion? (see the new RFC workflow defined in #36 and in the updated README)

@globin I'm still interested in this, yes! I was just looking at the associated nixpkgs PR yesterday and left a comment with a number of improvements to the approach, many based on new features of Nix 2 and which fill in some of the TODOs here.

I'd like to get a chance to update this RFC (likely this weekend) before opening it up for a ton of debate. Not 100% sure of the process but seems reasonable to open this up for shepherd nomiations in the meantime.

@aneeshusa we will meet next Thursday for the shepherd nominations.
It would be great if you can update your status until then.

I think it would be fine to open this up for nominations then right away, since the general idea is clear, fleshing it out with the shepherds is the idea anyway!

=> Open for nominations

I'd also hereby nominate @Ericson2314, @matthewbauer as shepherds.

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/rfc-13-open-for-shepherd-nominations-ergonomic-cmakeflags/1907/1

I nominate @Ericson2314

Oops guess @globin beat me to it 😂

i can think of 2 ways to implement this
first one is just __structuredAttrs, but if that is globally on, it can break things
and if its conditionally on, how does the stdenv know when to do it? now the stdenv has cmake specific things

second idea is to have nix level setup hooks
so the stdenv will map over every buildInput, and if it contains a special function, run that as-if it was passed to overrideAttrs
then simply adding cmake to the inputs will allow cmake to magically transform the attrs back into a flat list

@cleverca22 nix-level setup hooks was how @aneeshusa originally did it. There is also a third way which is a buildCMakePackage. I think I learn towards that as getting nix-level setup hooks to compose sanely is some work.

I agree with you. Something in the lines of mkCMakeDerivation makes more sense than adding support for very generic names like generator in the already featureful mkDerivation. pkgs.cmake.mkDerivation is also a good option.

We should probably standardize buildFooPackage vs mkFooDerivation too. Odd we mainly have the former.

I'm strongly in favor of doing this as part of a migration to structured attributes. That way, the implementation doesn't require any special handling at evaluation time (like a mkCMakeDerivation) and can be handled in the cmake setup hook just as it is today.

@edolstra that's fine, but in general getting setup hooks to commute / play nice is already somewhat non trivial. I don't think setup hooks are a sustainable model with or without structured attributes.

Per meeting of the @NixOS/rfc-steering-committee: Shepherding Team is @edolstra as leader, @Ericson2314 and @matthewbauer as long as they both agree otherwise we will follow up with further candidates.

EDIT: Also shepherds, please be reminded that it could make sense to have a chat on IRC or by videoconferece, to discuss your opinions and with the author to move this forward in an orderly and constructive way!

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/shepherds-chosen-for-rfc-13-17-and-39/1964/1

I accept!

I agree with @edolstra that structured attributes would be the best way to handle this. A couple questions as I haven't gotten the chance to play with them:

• What are the downsides to turning them on, other than requiring Nix 2? @cleverca22 you mentioned it can break things - can you expand?
• Based on the answers to that question, what would the timeline to turn structured attributes on globally look like? Have we as a community started planning a path forward for that?

One idea is to enable structured attrs conditionally. One option would be checking for cmakeArgs directly in mkDerivation, which is tight coupling as @cleverca22 points out. A better idea I had was to enable structured attrs if there are any fields which are non-derivation attrsets - this would fail evaluation without structured attrs so it's safe to do (can't break existing code), isn't cmake specific, and presumably lets us start using structured attributes today.

That being said, if there's good reasons to wait on structured attrs entirely, I would love to use one of the other mechanisms as a fallback so we can take advantage of the better interface in the meantime, and clean up the implementation later.

About the other ideas:

1. adding directly to mkDerivation: My PR used to do this. I'm not a fan of this, hence why I changed my PR over to nix-level setup hooks.
2. nix-level setup hooks: I currently have these implemented (prototype stage) in the accompanying PR. These are a bit icky (I don't think there would be a need for these once we have structured attrs), but they seem like the least-bad alternative. Still on the fence about if these are worth doing with the prospect of a structured attrs future. One question: @Ericson2314, why would we need composability of these? I would expect any of these to only touch attributes they own/consume themselves and wasn't planning on supporting composability.
3. buildCMakePackage - I don't want to make cmake special here, I really like our existing setup hooks user experience where you simply have to add cmake as a native build input and I don't want to degrade that. That also breaks composability if every tool that needs special code to run has a separate function.

Apologies for the slow reply - once I get some feedback on these points I'll try to add these alternatives to the RFC, hopefully this weekend.

Also, I think the status: open for nominations label is out of date - can that please be updated?

this would fail evaluation without structured attrs so it's safe to do (can't break existing code), isn't cmake specific, and presumably lets us start using structured attributes today.

Actually today richer data is often just stripped away. Also arrays in particular become strings, which the setup script bash much work around.

why would we need composability of these? I would expect any of these to only touch attributes they own/consume themselves and wasn't planning on supporting composability.

What happens if cmake and say meson or build2 are in the same derivation? Which one gets to do the build or configure phases? Building relies on one such build tool being primary and delegating to the others, which setup hooks aren't really qualified to figure out.

Yeah I can help shepherding this RFC.

@aneeshusa @Ericson2314 @matthewbauer Are you interested in having a chat (e.g. on Hangouts or IRC) to discuss how to move forward with this IRC? Maybe some time next week?

@edolstra Yes, that would be okay with me.

Thanks for the ping. This week is a bit busy for me, but definitely happy to chat sometime next week - IRC sounds good.

@aneeshusa @Ericson2314 @matthewbauer How about next week (April 15-19)? I'm relatively unconstrained wrt time.

https://www.when2meet.com/?7756595-quZ4l OK scheduling take 2!

@edolstra @matthewbauer @aneeshusa Let's....try again? Would something be better than www.when2meet.com to coordinate a time?

Unfortunately, looks like I don't have as much time to contribute to Nix/Nixpkgs as I thought :( I'm going to step down from this RFC and close this PR. However, I'm still very interested in this feature (I still run an updated version of original branch locally) and would love for a new champion with more time to pick this up! Process-wise, I imagine you would make a new PR from your own fork but with the same RFC number? Not sure.

My most recent thoughts here: Per my previous comment I looked into conditionally turning on structured attrs, but it ended up causing infinite recursion errors. I took another look at the Nix manual and learned that structured attrs mode also writes out a .attrs.sh file that the existing bash builder should be able to use with no or hopefully a minimum of modifcation. So, what I imagine the next steps here would be:

• turn on structured attrs unconditionally in nixpkgs, using the .attrs.sh to avoid having to rewrite all of stdenv
• teach the cmake setup-hook to take advantage of the .attrs.json data, either with jq or shell-ing out to e.g. Python

Thanks @aneeshusa! Yeah, I think switching to structured attrs globally is the way to go, though obviously a rather big operation. But then we get ergonomic cmakeFlags almost for free.

@edolstra there's a number of changes that would be nice for a "stdenv 2.0", and I'm curious what you think as to how they should land (piecemeal?, one big leap?). Should there be an RFC for this?

Due to Nix writing out.attrs.sh, I would hope that switching to structured attrs would be a small change that shouldn't require large changes to stdenv or be visible to consumers of stdenv - definitely something that I'd rather land incrementally than wait to bundle into a "stdenv 2.0". If this isn't possible for some reason, I'd want to know more about why not and figure out ways to make it possible to land incrementally.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/structured-cmakeflags-attrs/6261/1