Add config option for flatfs no-sync #1963
Conversation
|
oh yeah, I was gonna do that and totally forgot about it. |
|
It appears that fsync is no longer the main bottleneck (particularly with no-insertnodeatpath and no-chunk-channel): Edit: the plot below is misleading, this is because the sync flag in |
|
@rht thats interesting. I don't see any harm in having the option around for future testing though. |
| @@ -39,7 +39,7 @@ func openDefaultDatastore(r *FSRepo) (repo.Datastore, error) { | |||
| // including "/" from datastore.Key and 2 bytes from multihash. To | |||
| // reach a uniform 256-way split, we need approximately 4 bytes of | |||
| // prefix. | |||
| blocksDS, err := flatfs.New(path.Join(r.path, flatfsDirectory), 4) | |||
| blocksDS, err := flatfs.New(path.Join(r.path, flatfsDirectory), 4, !r.config.Datastore.NoSync) | |||
There was a problem hiding this comment.
let's do it like this, to avoid hard to read code:
syncfs := !r.config.Datastore.NoSync
blocksDS, err := flatfs.New(path.Join(r.path, flatfsDirectory), 4, syncfs)
|
I still haven't figured out the cause, but wish to know. One plausible explanation is because 1. each hash update of the root (and all the subroots) happens as many times as the number of folders+files it(they) contain(s), 2. each fsync is more expensive than an lstat (in this case |
| } | ||
|
|
||
| var _ datastore.Datastore = (*Datastore)(nil) | ||
|
|
||
| func New(path string, prefixLen int) (*Datastore, error) { | ||
| func New(path string, prefixLen int, sync bool) (*Datastore, error) { |
There was a problem hiding this comment.
@whyrusleeping sync doesn't get passed into Datastore initialization. Ref: #1964 (comment)
License: MIT Signed-off-by: rht <rhtbot@gmail.com>
|
This updated |
|
this LGTM |
|
merged in #1985 |
No description provided.