-
-
Notifications
You must be signed in to change notification settings - Fork 925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IO.copy_stream doesn't seem to limit the chunk size #4701
Comments
IO.copy_stream on JRuby doesn't seem to limit the chunk size, it always reads the entire content at once. jruby/jruby#4701 Since we don't care about what is the size of yielded chunks, we just test whether the yielded chunks sum up to the entire content. While here we also bring back HTTP::Request::Body#each returning an Enumerator, so that tests can be simpler.
IO.copy_stream on JRuby doesn't seem to limit the chunk size, it always reads the entire content at once. jruby/jruby#4701 Since we don't care about what is the size of yielded chunks, we just test whether the yielded chunks sum up to the entire content. While here we also bring back HTTP::Request::Body#each returning an Enumerator, so that tests can be simpler.
If you're going from a file to a file we use the builtin Java implementation of FileChannel.transfer, which will do whatever's efficient on that platform. For non-file copying, we do it manually via a loop, and this is likely where we're not choosing a reasonable block size. I'll have a look. |
This snippet shows that source = StringIO.new("foo")
destination = File::NULL
def source.read(*args)
p args
super
end
IO.copy_stream(source, destination) # output on JRuby
[nil] I think this is problematic because the reason people use MRI's # output on MRI
[16384, ""]
[16384, "foo"] To provide background, I'm a maintainer of two libraries which handle file uploads and use |
Fixed along with #4842. |
Awesome, thank you! |
Environment
Expected Behavior
When I use
IO.copy_stream
on MRI, it limits the chunk size to 16KB.Actual Behavior
On JRuby
IO.copy_stream
appears not to limit the chunk size, it just always reads the entire content at once, because the output of the above script isI tried with a
StringIO
up to 10MB of size, and the behaviour is still the same, all 10MB of content is read at once. The behaviour is the same if I change the source IO into aFile
object.The text was updated successfully, but these errors were encountered: