Sort directory entries after they are returned by readdir #2784
Conversation
readdir does not guarantee a consistent order between calls and one some file systems this will result in non deterministic store paths for the same collections of files.
|
Do you have a specific case where this caused a problem? For example we already sort directory entries in NAR files to ensure a canonical representation. |
|
Hmmm - Where is that code? I didn't spot any ordering when I was tracing the code paths from nix-store --add, but I definitely may have missed something. A while ago I was using nix to generate flat os disk images with programs embedded and was frustrated by reruns appearing to not be cached correctly. I was eyeballing the nix code for unrelated reasons and realized that this may have been the root cause (I was on a btrfs file system then). I couldn't reproduce changing hashes via nix-store on ext4, but ext4 actually has consistent readdir semantics due to it's index per folder. |
|
Tracing the write path though, I definitely see the code running into so this PR may be redundant with something else that I'm missing. |
|
Ah - the map iteration at is guaranteed to be in sorted order per the c++ standard. |
The readdir syscall does not guarantee a consistent order between calls and on some
file systems this will result in non deterministic store paths for the
same collections of files.