What do I use as an attribute name when I want to do both "d /tmp ..." and "Z /tmp ..."? If I can't use the path as the attribute name then there isn't any sane merging and you're no further ahead than you were with a list.

ping @arianvp @doronbehar @flokli @infinisil

What do I use as an attribute name when I want to do both "d /tmp ..." and "Z /tmp ..."? If I can't use the path as the attribute name then there isn't any sane merging and you're no further ahead than you were with a list.

Good point, never used it with 2 times the same path.
What about using the submodule or list of the submodule? Then we only have a list of we need the path more then once.

Or you can use /tmp_2 as attribute name and set tmp_2.path = "/tmp"

What do I use as an attribute name when I want to do both "d /tmp ..." and "Z /tmp ..."? If I can't use the path as the attribute name then there isn't any sane merging and you're no further ahead than you were with a list.

Good point, never used it with 2 times the same path.
What about using the submodule or list of the submodule? Then we only have a list of we need the path more then once.

I use some tmpfiles rules more then once and for some paths I use different as well. This idea has been bothering me so much that I now suffer from this warning when I nixos-rebuild switch:

/etc/tmpfiles.d/00-nixos.conf:71: Duplicate line for path "/var/lib/mpd/playlists", ignoring.

Generally, the motivation should be to make sure anyone, including services between each other, are capable of overriding rules for certain paths.

Or you can use /tmp_2 as attribute name and set tmp_2.path = "/tmp"

To answer @aanderse's question, I think we should split the whole attribute set to types, i.e the example should be:

"d" = {
  "/var/lib/my/path" = {
    mode = " ..";
    user = "...";
    group = "....";
    # ..
  };
"a" = {
  "/var/lib/my/path" = {
     # ...
  };

@doronbehar Sounds good as well, will try to implement it in the following days

Or you can use /tmp_2 as attribute name and set tmp_2.path = "/tmp"

This invalidates the benefit of merging/overwriting.

As it currently stands I think this PR makes tmpfiles more difficult to work with than the current implementation. If I consider all the modules that use tmpfiles and how the defaults often aren't sufficient - this PR doesn't address the overwriting/merging problem people face. Additionally to that you should consider how many tmpfiles rules are added via packages now, so those can't be merged in either as they are textual values taken from upstream.

In summary, to reiterate, attribute sets most often provide value because of merging/overwriting and I would expect merging/overwriting to be the main motivation for rewriting this option - this PR makes makes overwriting more confusing to the end user.

@aanderse
what do you think of the format @doronbehar suggested?

Something along those lines is definitely an improvement 👍 I'll point out that there is still the issue that we have systemd.tmpfiles.packages which inserts raw text into the end result - not conducive to proper merging/overwriting.

I look forward to hearing some opinions of other people, some of whom I know who have already thought about this in more detail than I have.

I'll point out that there is still the issue that we have systemd.tmpfiles.packages which inserts raw text into the end result - not conducive to proper merging/overwriting.

What if during activation, we'd read the packages' tmpfiles rules and use jq to merge our rules to the packages'?

Yes, the insert new text, but in a different file. IIRC those get merged by systemd, following the filename of the files.

Changed it to the other format.

What do we do about the merging with tmpfiles.packages? Is the merging from systemd enough for that?

So should I convert every nixos module now, is there something left to discuss, or should the change be a separate pr?

I'd wait and hear more opinions.

Seems like there are no more opinions?

Honestly I'm still not that keen on this. I'd really like to hear from at least one of @arianvp, @flokli, or @infinisil before anyone consider merging this.

For one, I think the interface should be

{
  systemd.tmpfiles.paths = {
    "/tmp".d = {
      mode = "1777";
      age = "10d";
    };
}

This corresponds to "declaring a set of rules for a file", which makes much more sense than the other way around (d."/tmp"), which would be "Declaring a set of files for a rule".

But now I wonder: Can't we create a more descriptive API for this? And could we perhaps create an abstraction that doesn't rely on tmpfiles?

How about something like

{
  pathOperations = {
    "/tmp" = {
      type = "directory";
      setMode = "1777";
      cleanEvery = "10d";
    };
    "/var/lib/foo" = {
      type = "directory";
      setUserRecursive = "foo";
      # Only run when /var/lib/foo isn't owned by foo
      condition.user.foo = false;
    };
    "/home/paul" = {
      setUser = "paul";
      copyFrom = ./home-skeleton;
      condition.exists = false;
  };
}

I think there's a lot of potential to make a much better user interface.

For one, I think the interface should be

{
  systemd.tmpfiles.paths = {
    "/tmp".d = {
      mode = "1777";
      age = "10d";
    };
}

@infinisil this interface also seemed to me the most straight forward at first, but I think, it's not easy (or maybe not even possible?) to make it override-able. Consider what would happen if a service file declares:

d /var/lib/mpd/music - mpd syncthing - -

And then a user would want to add:

a /var/lib/mpd/music/ - - - - group:syncthing:rwx,other::rx
a /var/lib/mpd/music/ - - - - default:group:syncthing:rwx,default:other::rx

Hence since the directory's path could be used more then once in .tmpfiles files, it shouldn't be the sole "key" which declares a rule for a certain directory, I think.

That could probably be done by specifying the merging policy in the argument option. So that both arguments specified are joined together.

Nope, that's the nice thing about the module system, it allows you to specify e.g. attributes multiple times, and it merges the values according to the specified type. E.g. this would work:

{
  imports = [{
    systemd.tmpfiles.paths."/tmp".d.mode = "1777";
  }];

  systemd.tmpfiles.paths."/tmp".d.age = "10d";
}

OK maybe I was wrong, let's take another example:

Say a module imported defines:

systemd.tmpfiles.paths = {
  "/var/lib/mpd/music" = {
    d = {
      mode = "2775";
      user = "mpd";
      group = "syncthing";
    };
  };
};

And you want:

d /var/lib/mpd/music 2775 mpd syncthing
a /var/lib/mpd/music/ - - - - group:syncthing:rwx,other::rx
a /var/lib/mpd/music/ - - - - default:group:syncthing:rwx,default:other::rx

So you write:

systemd.tmpfiles.paths = {
  "/var/lib/mpd/music" = {
    a = {
      options = [ "group:syncthing:rwx" "other::rx" ];
    };
  };
};

Will it merge eventually into this?:

systemd.tmpfiles.paths = {
  "/var/lib/mpd/music" = {
    a = {
      options = [ "group:syncthing:rwx" "other::rx" ];
    };
    d = {
      mode = "2775";
      user = "mpd";
      group = "syncthing";
    };
  };
};

Or will it evaluate to this?:

systemd.tmpfiles.paths = {
  "/var/lib/mpd/music" = {
    a = {
      options = [ "group:syncthing:rwx" "other::rx" ];
    };
  };
};

To my humble experience, it seems like the later will be the final result, since the "/var/lib/mpd/music" attribute is overridden, and hence it should replace the previous value of "/var/lib/mpd/music". But, I don't know enough about the merging policies available in the module system to dictate whether the 1st result could be evaluated.

We discussed about this a bit during the NixOS systemd meeting.

Merging these is non-trivial, is somewhat hard to get right, and would need even more documentation and testing.

Given we're slowly deprecating usage of tmpfiles in favor of StateDirectory= etc., I propose keeping the existing structure, and closing this.

Ok with me as I currently don't have time to work on it.

NixOS systems meetings? Can you invite me to them? Could be interesting for me and my work homed

Alright, I'll close it them.

Yeah, most of this discussion is happening in #nixos-systemd :-)

Given we're slowly deprecating usage of tmpfiles in favor of StateDirectory= etc., I propose keeping the existing structure, and closing this.

In every distro, the level of control one has with tmpfiles compared to that of StateDirectory= is not comparable. On NixOS, the level of control one could have with structured attributes of tmpfiles is tremendously better compared to array tmpfiles or StateDirectory. Hence I'd very much like this to get merged in an agreeable form as I'm using tmpfiles heavily and I can't imagine StateDirectory could replace my usage, while allowing me not to not duplicate what a StateDirectory= does in a service, disable it and replace that with an appropriate tmpfiles rule.

Perhaps if StateDirectory will become more widely adopted in the future, we could reconsider - as less tmpfiles rules will need to be rewritten with attributs.

@flokli theres my broken irc setup again. Sometime I have to host a own matrix irc bridge, so I can join.

@doronbehar maybe write me an exact interface you would like (5 rules for example or so) than I can see if I find the time in my holidays.

The problem with tmpfiles is that even the way lines in tmpfiles.d merge isn't super-well defined upstream.

Also, tmpfiles run "imperatively" once during system boot (sysinit.target), and every time a nixos config is activated. This doesn't account for things mounted at a later time, and the dependencies required to be satisfied before that.

{State,Runtime,…}Directory= however run immediately while starting up the service using it. They're constrained to certain paths, sure, but is that what rules them out? Can you give some examples of the things you can't do with them, but where systemd.tmpfiles.rules isn't sufficient, and you need the composability of what this PR introduced?

I would say it creates a benefit of working with /tmp for example. I think there is some default for /tmp, but you could override it to give it a own btrfs device or so

Can you give some examples of the things you can't do with them, but where systemd.tmpfiles.rules isn't sufficient, and you need the composability of what this PR introduced?

The current implementation of systemd.tmpfiles.rules is almost sufficient, but personally, I experience a conflict with a nixos service I enable + some extra rules I write for the same directories the service enables. I gave more details of my setup in #97702 (review) and the problem with it. The current version of it is in:

https://gitlab.com/doronbehar/nixos-configs/blob/master/mpd.nix#L67-79

With this, plus services.mpd.enable = true; I get a warning from systemd on every activation:

/etc/tmpfiles.d/00-nixos.conf:71: Duplicate line for path "/var/lib/mpd/playlists", ignoring.

Luckily the line that is ignored is the line that gets written because of the nixpkgs' mpd service, and not my own, so I get the behavior I want, but that's not ideal IMO.

In general, what (upstream) tmpfiles can do that StateDirectory and friends can't is to set advanced permissions on the directories. I need these special permissions so that syncthing and mpd which are on part of the same unix group on my machine and they share directories among each other. I also come and go in these directories manually and I need the permissions there to have the acl attributes I want. These are also directories that should survive boots as they have precious data like my music library etc.

You can just drop the d line from your own config, which is the duplicate that systemd-tmpfiles complains about.

The a ones are additive, and should be applied after the d, even though tmpfilles.d(5) isn't super clear about this, only about them happening in the same fixed order:

       If multiple files specify the same path, the entry in the file with the
       lexicographically earliest name will be applied (note that lines suppressed due to the "!"  are filtered before application, meaning that if an early line carries the exclamation mark and is suppressed because of
       that, a later line matching in path will be applied). All other conflicting entries will be logged as errors. When two lines are prefix path and suffix path of each other, then the prefix line is always created
       first, the suffix later (and if removal applies to the line, the order is reversed: the suffix is removed first, the prefix later). Lines that take globs are applied after those accepting no globs. If multiple
       operations shall be applied on the same file (such as ACL, xattr, file attribute adjustments), these are always done in the same fixed order. Except for those cases, the files/directories are processed in the order
       they are listed.

Maybe I should open a PR to replace the usage of tmpfiles in mpd 🤔

You can just drop the d line from your own config, which is the duplicate that systemd-tmpfiles complains about.

I can't because I want different permissions there - 2770 instead of -.

Maybe I should open a PR to replace the usage of tmpfiles in mpd thinking

And what would you do?

Well... I was going to file a PR for mpd but ended up wasting a ton of time debugging the NixOS test before realizing it is broken in master 🙄 Anyone want to take a look at that?

Ran fine for me with:

nix build -f. -L nixosTests.mpd

~/nixpkgs (master)> git rev-parse HEAD
15a07be25037d7d13ad561fd4b0339850c4d120b

~/nixpkgs (master)> nix-build nixos/tests/mpd.nix
...
client: must fail: /nix/store/gyn0mq4f13bk31labqss7w0j9malfm3s-mpc-0.33/bin/mpc --wait -h serverPulseAudio status
serverPulseAudio # [   12.650466] refused connection: IN=eth1 OUT= MAC=52:54:00:12:01:03:52:54:00:12:01:01:08:00 SRC=192.168.1.1 DST=192.168.1.3 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=63023 DF PROTO=TCP SPT=46206 DPT=6600 WINDOW=64240 RES=0x00 SYN URGP=0
serverPulseAudio # [   13.659952] refused connection: IN=eth1 OUT= MAC=52:54:00:12:01:03:52:54:00:12:01:01:08:00 SRC=192.168.1.1 DST=192.168.1.3 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=63024 DF PROTO=TCP SPT=46206 DPT=6600 WINDOW=64240 RES=0x00 SYN URGP=0
serverPulseAudio # [   15.708050] refused connection: IN=eth1 OUT= MAC=52:54:00:12:01:03:52:54:00:12:01:01:08:00 SRC=192.168.1.1 DST=192.168.1.3 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=63025 DF PROTO=TCP SPT=46206 DPT=6600 WINDOW=64240 RES=0x00 SYN URGP=0
serverALSA # [   24.272787] nscd[837]: 837 checking for monitored file `/etc/netgroup': No such file or directory
serverPulseAudio # [   19.740156] refused connection: IN=eth1 OUT= MAC=52:54:00:12:01:03:52:54:00:12:01:01:08:00 SRC=192.168.1.1 DST=192.168.1.3 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=63026 DF PROTO=TCP SPT=46206 DPT=6600 WINDOW=64240 RES=0x00 SYN URGP=0
serverPulseAudio # [   24.342456] nscd[829]: 829 checking for monitored file `/etc/netgroup': No such file or directory
serverPulseAudio # [   28.124514] refused connection: IN=eth1 OUT= MAC=52:54:00:12:01:03:52:54:00:12:01:01:08:00 SRC=192.168.1.1 DST=192.168.1.3 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=63027 DF PROTO=TCP SPT=46206 DPT=6600 WINDOW=64240 RES=0x00 SYN URGP=0
client # [   24.376075] nscd[809]: 809 checking for monitored file `/etc/netgroup': No such file or directory
client # MPD error: Timeout while connecting
(30.01 seconds)
(48.98 seconds)
test script finished in 49.01s
cleaning up
killing client (pid 49)
killing serverALSA (pid 9)
killing serverPulseAudio (pid 29)
(0.00 seconds)
/nix/store/0hih2zc6ham7rd1f11rmspgz50yir301-vm-test-run-mpd

🤷‍♂️

Our commands are equivallent and we are 4 commits apart, and I ran mine with --rebuild now and it succeeded again.

Whatever the reason maybe, what did you want to change in the mpd module?

Whatever the reason maybe, what did you want to change in the mpd module?

I would remove the usage of tmpfiles from the module.

For a number of reasons it is too difficult to make valid assumptions about a systems filesystem layout, networked mounts, ACLs, and other unknown unknowns. This makes using tmpfiles, which only run on system activation, or worse yet sporadically on a timer, too difficult to reasonably support from an OS point of view. Much like how we only provision system users and groups when the sysadmin doesn't specify any custom values, in most situations we should only provision state directories when the sysadmin doesn't specify any custom values.

I'll likely file a PR to do this sometime soon... something like this.

See #106697 for PR.