Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add event loop Channel abstraction #1173

Merged
merged 3 commits into from Jul 2, 2018
Merged

add event loop Channel abstraction #1173

merged 3 commits into from Jul 2, 2018

Conversation

andrewrk
Copy link
Member

This is akin to channels in Go, except:

  • implemented in userland
  • they are lock-free and thread-safe
  • they integrate with the userland event loop

The self hosted compiler is changed to use a channel for events,
and made to stay alive, watching files and performing builds when
things change, however the main.zig file exits after 1 build.

Note that nothing is actually built yet, it just parses the input
and then declares that the build succeeded.

Next items to do:

  • add windows and macos support for std.event.Loop
  • improve the event loop stop() operation
  • make the event loop multiplex coroutines onto kernel threads
  • watch source file for updates, and provide AST diffs
    (at least list the top level declaration changes)
  • top level declaration analysis

@binary132
Copy link

binary132 commented Jun 30, 2018

Can we discuss whether there should be a coroutine <=> thread mapping? I think it should be explicit and user-controlled. Otherwise performance guarantees can be hard to establish (consider NUMA cache optimization.)

Also, I suggest being a little careful with the Go analogy since Go channels are built on mutex locking, and are typically used as a synchronization device. I'm not that familiar with lock-free queues, but wouldn't they be non-blocking? I.e., no "await" or blocking "select" or "for _ = range x" or whatnot.

@andrewrk
Copy link
Member Author

The coroutines and thread mapping is entirely in userland and will be an option that std.event.Loop provides.

The implementation is lock free but it integrates with coroutines, and the get() and put() functions of Channel are async. So you define the channel buffer size (possibly zero) and that many put() awaits will resolve immediately. The next put() await will resume when the next get() completes.

So it still acts like a synchronization device, with async/await.

This is akin to channels in Go, except:
 * implemented in userland
 * they are lock-free and thread-safe
 * they integrate with the userland event loop

The self hosted compiler is changed to use a channel for events,
and made to stay alive, watching files and performing builds when
things change, however the main.zig file exits after 1 build.

Note that nothing is actually built yet, it just parses the input
and then declares that the build succeeded.

Next items to do:
 * add windows and macos support for std.event.Loop
 * improve the event loop stop() operation
 * make the event loop multiplex coroutines onto kernel threads
 * watch source file for updates, and provide AST diffs
   (at least list the top level declaration changes)
 * top level declaration analysis
@ghost
Copy link

ghost commented Jul 2, 2018

Exiting news !

@andrewrk andrewrk merged commit 22b7312 into master Jul 2, 2018
@andrewrk andrewrk deleted the event-loop-channel branch July 2, 2018 20:01
@isaachier
Copy link
Contributor

FWIW, I think this broke the self-hosted build on my Mac.

$ ./bin/zig build --build-file ../build.zig
/Users/isaachier/proj/zig/zig/src-self-hosted/module.zig:270:62: error: expected type 'BuildError', found '@typeOf(Module_addRootSrc).ReturnType.ErrorSet'
                await (async self.events.put(Event{ .Error = err }) catch unreachable);
                                                             ^
/Users/isaachier/proj/zig/zig/src-self-hosted/module.zig:262:48: note: called from here
        _ = try async<self.a()> self.buildAsync();
                                               ^
/Users/isaachier/proj/zig/zig/build/lib/zig/std/os/path.zig:1158:43: note: 'error.NotSupported' not a member of destination error set
                    posix.ENOTSUP => error.NotSupported,
                                          ^
The following command exited with error code 1: ...

@andrewrk
Copy link
Member Author

andrewrk commented Jul 2, 2018

can you try this?

--- a/src-self-hosted/module.zig
+++ b/src-self-hosted/module.zig
@@ -115,6 +115,7 @@ pub const Module = struct {
         InputOutput,
         NoStdHandles,
         Overflow,
+        NotSupported,
     };
 
     pub const Event = union(enum) {

I'll clean this up soon.

@isaachier
Copy link
Contributor

That did the trick!

andrewrk added a commit that referenced this pull request Jul 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants