New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nixos/gitolite: customize .gitolite.rc declaratively #29295
Conversation
@pvgoran, thanks for your PR! By analyzing the history of the files in this pull request, we identified @thoughtpolice, @abbradar and @bjornfor to be potential reviewers. |
I'm somewhat confused about the desired commit topic format. https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md says I should use I'd love to see some clarification to this. :) |
I suggested/added the "nixos/<thing>" to CONTRIBUTING.md some weeks ago. I didn't know that information was also in the manual :-/ The manual should be updated. |
Thank you! |
@@ -49,6 +62,17 @@ in | |||
''; | |||
}; | |||
|
|||
extraGitoliteRc = mkOption { | |||
type = types.str; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it makes sense that extraGitoliteRc can be set in multiple modules, better change from types.str to types.lines so that multiple definitision can be merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, I think it's possible for it to make sense. If anything, it can make implementation of the upcoming enableGitAnnex
option a bit shorter. However, I can't think why end users would want to separate services.gitolite
configuration into multiple modules.
Taking this in consideration, what would you advise? Should I change the type?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it works with types.lines I think we should do it. You never know when someone would like to modularize their configuration. (For instance, I have my samba smb.conf defined in two modules.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I'll make the change.
Cool! I didn't know that it was possible to append things to .gitolite.rc like that. So far I've been maintaining mine manually. I know very little perl. Does this method allow customizing/overriding everything in .gitolite.rc? As in, here are no drawbacks by this "append" style of manipulating the config file? Having multiple
sections in the file is OK etc.? |
@@ -71,7 +95,6 @@ in | |||
systemd.services."gitolite-init" = { | |||
description = "Gitolite initialization"; | |||
wantedBy = [ "multi-user.target" ]; | |||
unitConfig.RequiresMountsFor = cfg.dataDir; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing this line looks like a bad conflict resolution.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right. I didn't even notice it. :(
@@ -7,6 +7,19 @@ let | |||
# Use writeTextDir to not leak Nix store hash into file name | |||
pubkeyFile = (pkgs.writeTextDir "gitolite-admin.pub" cfg.adminPubkey) + "/gitolite-admin.pub"; | |||
hooks = lib.concatMapStrings (hook: "${hook} ") cfg.commonHooks; | |||
orNull = cond: val: if cond then val else null; | |||
rcAppendFile = | |||
orNull (cfg.extraGitoliteRc != "") (pkgs.writeText "gitolite-rc-append" rcAppendText); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMHO, I think it'd be easier to read if using plain "if/then/else" instead of having the "orNull" function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, it would make the line too long, or require splitting the if-then-else construct. I'll see if it can be rewritten elegantly.
@@ -81,6 +104,12 @@ in | |||
script = '' | |||
cd ${cfg.dataDir} | |||
mkdir -p .gitolite/logs | |||
>.gitolite.rc gitolite print-default-rc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't know IO redirection could be done at the start of the line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to work in the middle of the line as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mind changing the IO redirect to the end of the line? My guess is that having it at end-of-line is more common and people reading the code won't have to stop and think about what the code does if it looks more familiar. (At least that's how I feel.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, placing the redirects at the beginning creates prominent similarity between the commands that write to the file, therefore visually "grouping" them. This was my motivation for placing redirects like this. However, a predictable, well-known way of doing things is probably more valuable.
So with this PR, .gitolite.rc will be fully declarative. But still the file will be writeable, only to be overwritten on every service start. Could we instead make .gitolite.rc a symlink to /nix/store/...-gitolite.rc? |
It should be possible to customize everything, but it would require some Perl knowledge. (For example, removing items from a list is a bit more complex than adding them.)
Well, if the structure of the configuration defined in the (default)
Yes. They are just expressions; the last expression in the file determines its return value, which should be non-zero.
I don't know a proper way of implementing it. Can we run |
Sounds good! You can make a gitolite.rc file in the Nix store like this:
If gitolite works with .gitolite.rc being a symlink to this nix store file, I think that's preferable. It gives a much stronger signal that the file is managed declaratively. |
Perhaps we should think about the upgrade path too. For users already having a .gitolite.rc (like me), it's not so nice to just blow it away. How about this. Assuming the symlink to nix store thing works, we can have the gitolite service check the type of the .gitolite.rc file. If it's a plain file, abort the service with message like """Since NixOS 17.09, .gitolite.rc is managed declaratively and customizations must be done with the services.gitolite.extraGitoliteRc option. You currently have a .gitolite.rc file that is not managed by NixOS. Please take any customizations you may have in .gitolite.rc and move them to the new extraGitoliteRc option. Then remove ${datadir}/.gitolite.rc to allow NixOS to manage the file for you.""" If the .gitolite.rc file is a symlink, we check if it points to the /nix/store. If it does, find then we replace it with a symlink to the new file. If it's a symlink but not pointing to /nix/store, abort with the same message as above. (Or print a warning instead of aborting?) Thoughts? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to see attempt to use .gitolite.rc from nix store to prevent accidental user changes being overwritten. Also, consider upgrade path for users already having an imperatively configured .gitolite.rc file. (Possible solution outlined in comment.)
So, Also, what makes it possible to create
I assume it does. At least, gitolite's documentation mentions this as a possible approach (see below).
This looks good to me. I don't think the script should abort if a "legacy" Also, users may have valid reasons not to manage |
Correct.
It seems gitolite just needs a place to store log files. I would be surprised if the files will be needed afterwards. I found out this by trial and error. I didn't find flag to disable logging to file.
Correct. That directory will be created in the temporary build directory of the derivation (typically /tmp/nix-build-* something, IIRC).
You can prefix messages with a number in angle brackets, indicating severity level.
Good idea. |
What is the proper procedure for changing the pull request? Should I just add another commit to the branch, or replace/amend the commit and force-push the branch? |
Amend and force push. |
13625d9
to
d536eb4
Compare
I think I addressed everything with the new commit. Here are (somewhat long) list of concerns I have with relation to this code:
I can use the right arrow key to see the remainder of overlong lines, but it's very hard to read the log if I have to scroll the screen left and right for each line I read. For now, I limited the width of message lines to ~50 characters so that they would fit in my terminal, but this is obviously non-optimal: (1) this way, the message takes 11 lines which seems too much, (2) users will have different terminal width, for some 50-character lines will be too long, for some they will be too short. Unless I'm missing something and there is a better way to access systemd logs, it looks like I can't format the warning message in a way that would be convenient for most users. Do you have any recommendation of what to do here? |
I think it's fine.
Example NixOS assert:
Perhaps reduce the size of the message. Maybe instead of having a large message, simply point users to "man configuration.nix" and the "manageGitoliteRc" option? |
Great! This is exactly what I was looking for. (Provided that it works in this case.)
This may work. I'll think about it. I'll update the PR once more. |
type = types.lines; | ||
default = ""; | ||
example = literalExample '' | ||
push( @{$RC{ENABLE}}, 'Kindergarten' ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind adding this line to the example?
$RC{UMASK} = 0027;
(To help out complete Perl dummies like me.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Also added some more helpful examples.
d536eb4
to
d10aafb
Compare
d10aafb
to
d37d49e
Compare
Now everything seems to be in order. I also updated the commit/PR topic and description. Not sure if I should have added backticks to the commit description (github doesn't interpret them), but it's a small thing. |
( [[ -L "$GITOLITE_RC" ]] && [[ "$(readlink "$GITOLITE_RC")" =~ ^/nix/store/ ]] ) | ||
then | ||
'' + rcSetupScriptIfDefaultFileOrStoreSymlink + | ||
'' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, this mixing of Nix and shell looks more complicate than necessary, I think. There is at least one if/else in Nix+sh that cancel each other out, if you take another look.
I think using this style when expanding nix values inside strings is more readable:
''hey, a nix value ${var} in the middle of the text''
IMHO, the added complexity of checking if .gitolite.rc is unchanged is not worth it. Personally, I'd say "given that manageGitoliteRc is set, is .gitolite.rc a symlink to nix store? If yes, update file, else complain."
If you make some of these changes, all of "rcDir, rcDirScript, rcAppendFile, rcAppendText" can become just "rcFile":
rcFile = runCommand "gitolite.rc" {}
''
export HOME=gitolite-home
mkdir -p gitolite-home/.gitolite/logs
${gitolite}/bin/gitolite print-default-rc > "$out"
cat << EOF >> "$out"
${cfg.extraGitoliteRc}
EOF
''
Love the documentation!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, this mixing of Nix and shell looks more complicate than necessary, I think. There is at least one if/else in Nix+sh that cancel each other out, if you take another look.
Could you elaborate? I don't see it. Nix and shell conditions operate on different data.
I think using this style when expanding nix values inside strings is more readable ...
It is surely more readable, but if I put ${rcSetupScriptIfDefaultFileOrStoreSymlink}
on a separate line inside a multi-line string, it will introduce an extra empty line after the contents of the variable, which doesn't look pretty. One could argue that readability of the source is more important that readability of the generated code, though...
IMHO, the added complexity of checking if .gitolite.rc is unchanged is not worth it.
I'm pretty sure it's important. The thing it, gitolite setup
creates a default .gitolite.rc
file if it doesn't exist. So all existing users of gitolite will already have it. And we don't want to bug them with warnings if they didn't actually make any customizations to the file, or make them wonder why their declarative configuration doesn't work, in case they add it and don't notice the warning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all of "rcDir, rcDirScript, rcAppendFile, rcAppendText" can become just "rcFile":
Embedding ${cfg.extraGitoliteRc}
into the shell script creates a danger of misinterpretation of its contents by the shell (if it has a lone EOF
line, for some reason). My approach doesn't have this problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you elaborate? I don't see it. Nix and shell conditions operate on different data.
Perhaps I described it wrong. A concrete example of what I'm thinking of is replacing
'' + rcSetupScriptIfDefaultFileOrStoreSymlink +
(which has a long name and a if/else in its implementation)
with
${lib.optionalString cfg.manageGitoliteRc ''
echo "warning message" # or echo "${warningMessage}" if you don't want multi-line shell code here in the .nix file
''}
In other words, I find some of the abstractions in this code too abstract. (But yes, that does probably add a blank line in case the condition is false.)
I'm pretty sure it's important. The thing it, gitolite setup creates a default .gitolite.rc file if it doesn't exist. So all existing users of gitolite will already have it. And we don't want to bug them with warnings if they didn't actually make any customizations to the file, or make them wonder why their declarative configuration doesn't work, in case they add it and don't notice the warning.
Good point. Though I figured now that there is a manageGitoliteRc option the warning message could be turned into an error, and that users perhaps better be made aware of the change from imperative to declarative config from the start. Even if they didn't make a change to the file. But OK, I get your reasoning.
Embedding ${cfg.extraGitoliteRc} into the shell script creates a danger of misinterpretation of its contents by the shell (if it has a lone EOF line, for some reason). My approach doesn't have this problem.
You can use echo '${foo}' > file
if you don't like using heredoc. (You can also use a more special string than EOF. THIS_ENDS_THE_FILE for example?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In other words, I find some of the abstractions in this code too abstract. (But yes, that does probably add a blank line in case the condition is false.)
While not ideal, I find that the approach (using named shell script fragments) is the best way (known to me) to convey the intent of the code. Embedding Nix if-then-else inside shell's if-then-else would look messy and confusing. (And we can't use optionalString
instead of if-then-else, because shell syntax doesn't allow us to leave then- or else- branch empty.)
... the warning message could be turned into an error ...
Does this mean "exit with error", or just "write journal message with loglevel 3 instead of 4"?
You can use echo '${foo}' > file if you don't like using heredoc.
I found a way to properly escape it:
optionalString (cfg.extraGitoliteRc != "") ''
echo ${escapeShellArg ''
# Added by NixOS:
${removeSuffix "\n" cfg.extraGitoliteRc}
# per perl rules, this should be the last line in such a file:
1;
''} >>"$out/gitolite.rc"
'';
Does this look better? It's a bit too heavy with nested code-inside-strings, but it's somewhat shorter, and doesn't use additional variables.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean "exit with error", or just "write journal message with loglevel 3 instead of 4"?
Both :-)
[...] Does this look better?
IDK. I think both look kind of verbose. But I'm okay with either. (I would probably have written printf -- '%s' '${cfg.extraGitoliteRc}' >> "$out"
.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean "exit with error", or just "write journal message with loglevel 3 instead of 4"?
Both :-)
I don't think we can do the former. It would break existing installations that have customized .gitolite.rc
, I suppose this should not happen except in truly exceptional circumstances, right?
I don't mind raising the loglevel to provide better visibility, though. Also, maybe it's worth adding a note to the release information for 17.09, to make visibility even better? (I have no idea how this works, though. What is good enough to be added to release notes, how to do it, etc.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we can do the former. It would break existing installations that have customized .gitolite.rc,
My thinking is that either we error out if .gitolite.rc is customized when upgrading to NixOS 17.09, or the manageGitoliteRc option would have to be turned off by default.
Hm, defaulting manageGitoliteRc to false is an easy way to not cause breakage...
Release notes for NixOS 17.09 is in nixos/doc/manual/release-notes/rl-1709.xml.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, defaulting manageGitoliteRc to false is an easy way to not cause breakage...
This will not fulfill your original intent of making users "aware of the change from imperative to declarative config from the start". :) And it feels less "natural" for NixOS.
Leaving manageGitoliteRc
defaulting to true would at least issue the warning.
Release notes for NixOS 17.09 is in nixos/doc/manual/release-notes/rl-1709.xml.
Do I just prepare a PR for them as well? Should it be a separate PR, or it needs to be included in this one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will not fulfill your original intent of making users "aware of the change from imperative to declarative config from the start". :)
Well, there is less need to be aware of something if it doesn't happen :-)
And it feels less "natural" for NixOS.
Agreed. But since it is a breaking change, we could let the transition go over one release cycle instead of introducing breakage right away. So for NixOS 17.09 the manageGitoliteRc option (=false) is introduced (and mentioned in the release notes). Then for NixOS 18.03 the default can be changed.
Do I just prepare a PR for them as well? Should it be a separate PR, or it needs to be included in this one?
A separate commit in this PR is preferred.
878c7c6
to
b569983
Compare
For now, I got rid of extra variables for |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make manageGitoliteRc default to false (option default can be changed in next release, but for now be gentle). Update text that is affected by that change. Adding release note entry is optional if doing it this way, since there is no breaking change.
UPDATE: Oops, I meant "Make manageGitoliteRc default to FALSE" obviously.
@bjornfor Sorry, I still don't understand the upgrade strategy you have in mind.
|
Reasoning:
No, I don't think that's a good idea. Possible exception: if there was a way to differentiate between a config having the NixOS default AND explicitly setting the option to true/false in own config. I.e., if we could gently "push" users to make a choice. But I don't think that's possible. |
My strong opinion is that we can't do changes that will break users' installations "by default" (without them doing anything other than upgrading), no matter how far in the future. (Unless it can't be avoided, or we're talking about something really prominent and important for everyone, like
This is possible (unless I misunderstand you here): I can make the With this, we can have a broader array of behaviour in case there are custom modifications. For example, if In case of no custom modifications, the An alternative variant would be to |
I just noticed I had written the exact opposite of what I meant, in the review status. Based on the context I obviously meant to suggest that manageGitoliteRc is set to FALSE by default, so that we DON'T break existing users setup. (Sorry for the confusion.) Good point about the tristate option. I'd prefer to just make a minimum implementation now for NixOS 17.09. That is, boolean manageGitoliteRc defaulting to false. Deciding on upgrade path can be done later (if at all needed). What do you think? |
In fact, if |
This makes sense. So, if you don't think that the currently implemented approach ( |
Yes, please remove manageGitoliterc option for now. |
b569983
to
02833e0
Compare
Done. |
@pvgoran: Tested and ready to go? If so, then I'll add it to master and backport to release-17.09. |
Thanks for writing such good documentation in the NixOS option! |
Yes, and I just tested it again. |
@pvgoran: Git author is "Someone someone@localhost". Sure you don't want something else here? |
Or maybe someone else? (I'll show myself out...) |
Add the `extraGitoliteRc` option to customize the `.gitolite.rc` configuration file declaratively. Resolves NixOS#29249.
02833e0
to
d6e496a
Compare
Ok, I resolved my identity crisis with a new commit. :) |
Add the
extraGitoliteRc
option to customize the.gitolite.rc
configuration file declaratively.Resolves #29249.
Things done
build-use-sandbox
innix.conf
on non-NixOS)nix-shell -p nox --run "nox-review wip"
./result/bin/
)