Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IO::MultiWriter #3404

Merged
merged 3 commits into from
Oct 15, 2016
Merged

Conversation

RX14
Copy link
Member

@RX14 RX14 commented Oct 9, 2016

Useful for writing log output to multiple locations (stdout + file) or any sort of parallel processing of IO data.

return if @closed
@closed = true

@writers.each { |writer| writer.close } if sync_close?
Copy link
Contributor

@yxhuvud yxhuvud Oct 9, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can close raise(*)? If so, does that need to be handled?

(*) It is not obvious from reading the code. Flush calls unbuffered write at least in some IO variants that are mixins.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it needs to be handled. Errors from write calls bubble up to the caller in #write.

@RX14
Copy link
Member Author

RX14 commented Oct 9, 2016

Maybe we should attempt to write to the remaining writers if one call fails? I'm not entirely sure it's beneficial at all, but it should be considered.


@writers.each { |writer| writer.write(slice) }
rescue ex
close
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Closing the IO on error seemed a good idea at the time. This comes back to the question on what we should do if one of the IOs raises. Should we attempt to recover or just write off the IO as failed.

Is there realistically any reason why an IO would raise if it wouldn't raise on successive calls? Most IO code is written not to handle exceptions from IO objects, and to simply reconnect if such an event happens.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a reason for closing: if one of the write calls fails, then one of the underlying IOs may have written more than the rest. This means that the IOs are no longer consistent, and so we should fail all subsequent calls so that the two IOs don't go out of sync.

Unfortunately this isn't desired behaviour for logging because you don't want stdout to stop when the disk is full, so maybe it should be configurable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I'll avoid closing the IO. In other IOs if a write fails the IO doesn't get automatically closed, so I don't think this should be done for this particular IO.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@RX14 RX14 force-pushed the feature/IO-MultiWriter branch 2 times, most recently from ce48008 to 8bbb175 Compare October 12, 2016 14:24
property? sync_close
getter? closed = false

@writers : Array(IO)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be removed as it will be correctly inferred from the constructor below

def write(slice : Slice(UInt8))
check_open

@writers.each { |writer| writer.write(slice) }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be @writers.each &.write(slice), but you can leave it like this if you prefer it. Same applies to close

@RX14 RX14 force-pushed the feature/IO-MultiWriter branch from 8bbb175 to 7260fec Compare October 14, 2016 20:59
@RX14
Copy link
Member Author

RX14 commented Oct 14, 2016

Updated!

@asterite
Copy link
Member

@RX14 Thank you for this! ❤️

@asterite asterite merged commit bad73b0 into crystal-lang:master Oct 15, 2016
@asterite asterite added this to the 0.20.0 milestone Oct 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants