nixos/xdg/mime: disable fdatasync when building the XDG MIME database #62076
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Back in 2013, update-mime-database started using fdatasync() to write out its changes after processing each file in /share/mime, with the reasoning that a corrupted database from an interruption midway would be problematic for applications[1]. Unfortunately, this caused a significant regression in the time required to run update-mime-database: commonly from under a second to half a minute or more. This delay affects the time required to build system-path on NixOS, when xdg.mime.enable is true (the default). For example, on one of my systems system-path builds in ~48 seconds, 45 of which are update-mime-database. This makes rapidly building new system configurations not fun. This commit disables the calls to fdatasync(). update-mime-database checks an environment variable, PKGSYSTEM_ENABLE_FSYNC, to determine whether it should sync, and we can set this to false. system-path already only has whatever filesystem commit guarantees that the Nix builder provides. Furthermore, there is no risk of a failed MIME database update messing up existing packages, because this is Nix. (This issue was also reported at and discussed by Debian, Red Hat, and Gentoo at least.) [1] https://bugs.freedesktop.org/show_bug.cgi?id=70366
infinisil
approved these changes
May 27, 2019
|
Well, faster drive certainly helped, and I expect it will still make a noticeable difference after this PR :-) (say some SSD vs. 5400 rpm.) |
|
Yeah, all HDDs here. Thanks folks!
…On Mon, 27 May 2019, at 11:45, Vladimír Čunát wrote:
Well, faster drive certainly helped, and I expect it will still make a noticeable difference after this PR :-) (say some SSD vs. 5400 rpm.)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#62076?email_source=notifications&email_token=AAGASTN7FGYUOWYLCTAG6PTPXQT2TA5CNFSM4HPV5V5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWKLBRY#issuecomment-496283847>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGASTJBVROEMTF374VPK3TPXQT2TANCNFSM4HPV5V5A>.
|
|
Benchmark: For my configuration, adding a new package to |
|
A note, for posterity: one possible alternative would be to |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
Make system-path builds much faster!
Back in 2013, update-mime-database started using fdatasync() to write out
its changes after processing each file in /share/mime, with the reasoning
that a corrupted database from an interruption midway would be
problematic for applications[1]. Unfortunately, this caused a
significant regression in the time required to run update-mime-database:
commonly from under a second to half a minute or more.
This delay affects the time required to build system-path on NixOS, when
xdg.mime.enable is true (the default). For example, on one of my systems
system-path builds in ~48 seconds, 45 of which are update-mime-database.
This makes rapidly building new system configurations not fun.
This commit disables the calls to fdatasync(). update-mime-database
checks an environment variable, PKGSYSTEM_ENABLE_FSYNC, to determine
whether it should sync, and we can set this to false. system-path
already only has whatever filesystem commit guarantees that the Nix
builder provides. Furthermore, there is no risk of a failed MIME
database update messing up existing packages, because this is Nix.
(This issue was also reported at and discussed by Debian, Red Hat, and
Gentoo at least. I tried to find discussion here on Github but didn't see
anything.)
[1] https://bugs.freedesktop.org/show_bug.cgi?id=70366
Things done
sandboxinnix.confon non-NixOS)installerandxdg-desktop-portaltests.nix-shell -p nix-review --run "nix-review wip"./result/bin/)nix path-info -Sbefore and after)