Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise IO::Buffered for reads 4096 <= size < 8192 #3416

Merged
merged 1 commit into from
Oct 13, 2016

Conversation

RX14
Copy link
Member

@RX14 RX14 commented Oct 12, 2016

When count is more than half the buffer size, an unbuffered read is likely
to be happening in more than half of the read calls, so it's unlikely to be
worth the extra buffer copy.

Benchmarks show this to be true:

before

2048   7.74  (± 0.46%)  1.24× slower
4096   7.39  (± 0.45%)  1.30× slower
6144   7.52  (± 0.57%)  1.28× slower
8192   9.59  (± 0.79%)       fastest

after

2048   7.73  (± 0.64%)  1.24× slower
4096   8.11  (± 0.76%)  1.18× slower
6144   8.61  (± 0.66%)  1.11× slower
8192    9.6  (± 0.89%)       fastest

Sizes 2048 and 8192 have the same behaviour before and after this commit,
so we see similar speeds. However, we see a 10% speedup for 4096 and 6144
bytes as they call unbuffered_read less often.

Benchmark code:

require "benchmark"

macro benchmark(size)
  b.report("{{size}}") do
    in = File.open("1gb", "r")
    buffer = uninitialized UInt8[{{size}}]
    while in.read(buffer.to_slice) > 0
    end
    in.close
  end
end

Benchmark.ips do |b|
  benchmark(2048)
  benchmark(4096)
  benchmark(6144)
  benchmark(8192)
end

@RX14 RX14 force-pushed the optimise-buffered-io branch from fb5a62e to a520004 Compare October 12, 2016 15:56
@RX14 RX14 changed the title Optimise IO::Buffered for reads 2048 <= size < 8192 Optimise IO::Buffered for reads 4096 <= size < 8192 Oct 12, 2016
@RX14 RX14 force-pushed the optimise-buffered-io branch from a520004 to d23ecd9 Compare October 12, 2016 16:02
@RX14
Copy link
Member Author

RX14 commented Oct 12, 2016

Looks like I broke the compiler in this travis build.

@asterite
Copy link
Member

@RX14 Ah, no, it was me in a middle commit that I already changed. It's a separate bug but it's minor so I'll fix it later. If you rebase against master it should be green in travis.

Unverified

The committer email address is not verified.
When count is more than half the buffer size, an unbuffered read is likely
to be happening in more than half of the read calls, so it's unlikely to be
worth the extra buffer copy.

Benchmarks show this to be true:

before
2048   7.74  (± 0.46%)  1.24× slower
4096   7.39  (± 0.45%)  1.30× slower
6144   7.52  (± 0.57%)  1.28× slower
8192   9.59  (± 0.79%)       fastest

after
2048   7.73  (± 0.64%)  1.24× slower
4096   8.11  (± 0.76%)  1.18× slower
6144   8.61  (± 0.66%)  1.11× slower
8192    9.6  (± 0.89%)       fastest

Sizes 2048 and 8192 have the same behaviour before and after this commit,
so we see similar speeds. However, we see a 10% speedup for 4096 and 6144
bytes as they call unbuffered_read less often.

Benchmark code:
```
require "benchmark"

macro benchmark(size)
  b.report("{{size}}") do
    # Read a 1 gigabyte file
    in = File.open("1gb", "r")
    buffer = uninitialized UInt8[{{size}}]
    while in.read(buffer.to_slice) > 0
    end
    in.close
  end
end

Benchmark.ips do |b|
  benchmark(2048)
  benchmark(4096)
  benchmark(6144)
  benchmark(8192)
end
```
@RX14 RX14 force-pushed the optimise-buffered-io branch from d23ecd9 to b1f40f4 Compare October 12, 2016 18:46
@RX14
Copy link
Member Author

RX14 commented Oct 12, 2016

Rebased!

@asterite
Copy link
Member

@RX14 Looks good, thank you!!

@asterite asterite merged commit bb208bc into crystal-lang:master Oct 13, 2016
@asterite asterite added this to the 0.20.0 milestone Oct 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants