Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the TTL for disk cache configurable #2036

Merged
merged 5 commits into from Apr 6, 2018

Conversation

AmineChikhaoui
Copy link
Member

This should allow modifying the disk cache TTL or completely disabling it to force another binary cache query.

Copy link
Member

@shlevy shlevy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a section in the manual about these options? Also, if possible some clarification in the help text on what "negative" and "positive" mean might help.

@shlevy shlevy self-assigned this Apr 2, 2018
@shlevy shlevy added the triaged label Apr 2, 2018
@AmineChikhaoui
Copy link
Member Author

@shlevy yeah sure

@AmineChikhaoui
Copy link
Member Author

@shlevy done let me know if anything should be modified/added.

@AmineChikhaoui
Copy link
Member Author

Well setting --option positive-disk-cache-ttl 0 in all deployment scripts is probably not a good idea as you'll be querying every single path in the closure (also much more load on cloudfront side) but it's more of a convenience config for when someone needs to just disable the disk cache for pulling one store path or at least set a smaller TTL value without removing the whole binary-cache*.sqlite db to get around an issue. It also doesn't make much sense to me in general to have this config hard-coded.


<listitem>

<para>The TTL in seconds for negative lookups. If a store path is queried from a substituer but
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=> substituter.


<listitem>

<para>The TTL in seconds for positive lookups. If a store path is queried from a substituer, the result of
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=> substituter.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops.

<para>The TTL in seconds for positive lookups. If a store path is queried from a substituer, the result of
the query will be cached in the local disk cache database including some of the NAR metadata. Setting a TTL
for positive lookups can be useful in case of builds that aren't reproducible, in which case having a more
frequent cache invalidation would prevent hash mismatch issues.</para>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`Setting a TTL..." implies that there is no TTL by default, which is not the case. Would be better to say something like "The defautl TTL is bla bla. Setting a shorter TTL can be useful in case...".

BTW this scenario may not be completely clear. I assume this is when the binary cache has frequent garbage collections (in which case it doesn't really matter whether the builds are reproducible).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edolstra yeah that's also another case where this is useful, I'll mention that instead.

@@ -313,6 +313,14 @@ public:
Setting<Strings> trustedUsers{this, {"root"}, "trusted-users",
"Which users or groups are trusted to ask the daemon to do unsafe things."};

Setting<unsigned int> ttlNegativeDiskCache{this, 3600, "negative-disk-cache-ttl",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing: "disk cache" is a bit ambiguous as there can be many disk cache (e.g. nix search also has a disk cache). Maybe it should be called narinfo cache (e.g. narinfo-cache-negative-ttl).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good

disk cache lookup for example by doing:
  nix copy --from <binary-cahe> <store-path> --option \
  positive-disk-cache-ttl 0

Issues: NixOS#1885 NixOS#2035
@AmineChikhaoui
Copy link
Member Author

@edolstra options name updated.

@edolstra edolstra merged commit e10a7ec into NixOS:master Apr 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants