Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix-optimise: do not run in container #48098

Merged
merged 1 commit into from Oct 10, 2018
Merged

Conversation

peterhoeg
Copy link
Member

Motivation for this change

There is no point in running nix-optimise inside a nixos-container. It should be on the host instead.

Been running with this in production for months - this is part of "let's upstream to avoid having to carry a ton of patches around which make upgrading painful".

Cc: @volth

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@FRidh
Copy link
Member

FRidh commented Oct 9, 2018

Indeed, I think we should remove the service and use auto-optimise-store instead.

@peterhoeg
Copy link
Member Author

Do we have control over when auto-optimise-store runs? If not, I would say we keep the scheduled option in.

@FRidh
Copy link
Member

FRidh commented Oct 9, 2018

From the manual:

If set to true, Nix automatically detects files in the store that have identical contents, and replaces them with hard links to a single copy. This saves disk space. If set to false (the default), you can still run nix-store --optimise to get rid of duplicate files.

I am pretty sure the optimisation is done when creating a store path.

@vcunat
Copy link
Member

vcunat commented Oct 9, 2018

My operational practice is to auto-optimize iff the store is on a SSD. (and no other optimize)

On a rotating drive the savings tend not to be worth the price, regardless of doing the dedup online or offline, and on an SSD the price of auto-optimize seems negligible.

@vcunat
Copy link
Member

vcunat commented Oct 9, 2018

The overhead I'm (sometimes) interested in isn't in CPU but in disk IO. I think the constant queries into the huge /nix/store/.links/ directory was the cause of that major slow-down I experienced. (One probe per new file in nix store, and unlink and link op.) EDIT: this may depend on the FS used, as having so many files inside a single directory isn't a norm; stratifying it might help (think .git/objects/).

@peterhoeg
Copy link
Member Author

This particular PR has no negative effects and the discussion concerning disabling scheduled optimisations looks like it needs to be had separately from this.

Does anybody have any specific concerns about this change? If not, we should go ahead and merge this and we can always rip out the scheduled option at a later stage.

@FRidh
Copy link
Member

FRidh commented Oct 10, 2018

Right, go ahead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants