nix-gitignore: Optimise performance #106172
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adisbladis
force-pushed
the
nix-gitignore-perf
branch
from
6d44714
to
cbdf537
Compare
There are a few operations in this library that naively runs on every iteration while they could be cached. For a simple test repository with a small number of files and ~1000 gitignore patterns this brings memory usage down from ~233M to ~157M and wall time from 2.6s down to 0.78s. This should scale similarly with the number of files in a repository.
adisbladis
force-pushed
the
nix-gitignore-perf
branch
from
cbdf537
to
ef3ed45
Compare
|
The equivalent PR was merged upstream, merging here too. |
|
This appears to break negative patterns in some way. With this commit there are no files included where as in the parent commit the files are included as expected. src = pkgs.nix-gitignore.gitignoreSource [
"*"
"!Gemfile*"
] ./.;Since this is just a performance optimization should we revert now and debug later? |
vcunat
referenced
this pull request
Jan 4, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are a few operations in this library that naively runs on every
iteration while they could be cached.
For a simple test repository with a small number of files and ~1000
gitignore patterns this brings memory usage down from ~233M to ~161M
and wall time from 2.6s down to 0.78s.
This should scale similarly with the number of files in a repository.
Motivation for this change
Things done
sandboxinnix.confon non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"./result/bin/)nix path-info -Sbefore and after)cc @siers @andir