Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add IO::Buffered#rewind_buffer #3287

Closed
wants to merge 1 commit into from

Conversation

kostya
Copy link
Contributor

@kostya kostya commented Sep 10, 2016

sometimes needed to read something less than buffer from io(socket), parse something, and send this io to some other class in chain.

@bcardiff
Copy link
Member

But it is kind of a leak of abstraction. You don't have a way to control were the buffer is left. Am i missig something?

If some sort of seek is supported doing the math with the buffer, even if the underlying io is forward only i would +1 this.

@kostya
Copy link
Contributor Author

kostya commented Sep 10, 2016

readed buffer always here, it is in_buffer, with rewind_buffer @in_buffer_rem just pointing on it head again.
may be if we can set buffer size, would be nice, also

@kostya
Copy link
Contributor Author

kostya commented Sep 10, 2016

or may be better even rewind to requested size. for example read 3 bytes, and shift 3 bytes back

@asterite
Copy link
Member

@kostya Can you provide some real code where you'd use this? I find it confusing too. What if you want to peek 3 bytes but the first byte is the last one in the buffer? You'll re-fill the buffer but then you won't be able to rewind.

In any case, it would maybe be useful to peek the IO::Buffered's buffer for some optimizations. For example when we parse a CSV from an IO we need to read char by char until we find a comma. Meanwhile we need to put these chars in a MemoryIO and when we find a comma we build a string out of it. If we knew the IO is buffered we could search the comma in the IO's buffer, and if it's there we could get a string right out from the buffer, without needing to append chars to an intermediate buffer. This is why parsing JSON/CSV from a String is usually much faster than parsing it from an IO. Though we'd have to think how to provide this optimization for all IOs that buffer data (currently IO::Buffered and MemoryIO)

@kostya
Copy link
Contributor Author

kostya commented Sep 11, 2016

if you request return 3 bytes, but buffer switched, should be exception. Btw, this is only for manual usage, when you known what you do, what the buffer size, and where you position in parsing.

@kostya
Copy link
Contributor Author

kostya commented Sep 11, 2016

this is actually enough for me:

module IO::Buffered
  def rewind_buffer(size : Int32)
    new_buf = @in_buffer_rem.to_unsafe - size
    new_size = @in_buffer_rem.size + size
    if new_buf >= in_buffer
      @in_buffer_rem = Slice.new(new_buf, new_size)
    else
      raise "invalid shift"
    end
  end
end

@kostya kostya closed this Sep 11, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants