Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thread support #5549

Closed
wants to merge 56 commits into from
Closed

Thread support #5549

wants to merge 56 commits into from

Conversation

bararchy
Copy link
Contributor

@bararchy bararchy commented Jan 6, 2018

Following discussion in Gitter with @RX14 and @asterite, we discussed having the unfinished work merged into master using a spawn_pool or other macro to call a spawn with multi-thread support.

We also mentioned that a slower solution is better then no solution at all, and having this added to the lib might be the first step in having proper multi-thread support

waj and others added 30 commits September 30, 2016 13:59
…ntext switches. Set GC stackbottom only during garbage collection.
…ecuted after a context switch. Added some temporary logging.
…tored within the thread. Removed the extra "reschedule" fiber.
The scheduler now chooses the next thread randomly.
Added persistent event to libevent to avoid the loop to exit.
Also add the scheduler to the @@ALL pool, regardless of whether it executes the
EventLoop or not.
Before this, we were checking if a channel was ready and performing the
operation without locking, so race conditions could happen if some other
thread write/read from the channel between both steps.

Now we synchronize both operations under the channel's mutex.
Alternative operations for send/received were provided
(send_immediate/receive_immediate) to avoid lock reentrancy issues.
Remove duplicated code between send/receive and send_immediate/receive_immediate.
- Add a mutex to protect access to `@readers` and `@writers` in
  `IO::FileDescriptor`
- Install read/write event immediately if after a successful read/write there
  are still fibers pending on the fd. Ie. don't use the fiber callback in this
  case since it can be overwritten later by the next blocking operation (eg.
  channel IO)
- Add logging in various classes participating in scheduling and concurrent
  exectution
Forking the process as it was previously done is no longer supported (and is not
expected to be in the near future) and parallelization using threads is not yet
possible because we are not using LLVM in a thread-safe way.
This is to avoid re-registering the same reader twice. Apparently thread-local
storage is reused after a thread is terminated, and since the reader is in such
address space it was being registered twice, forming a loop in the brlock
readers list.
Every time a runnable is added to the runnables queue in a scheduler, increment
a global atomic stamp to signal other spinning threads, even after they have
attempted to steal jobs from all existing schedulers. This avoids the rare race
condition where all threads go to the sleeping state and there are still
runnables in some schedulers.
ggiraldez and others added 26 commits February 16, 2017 15:41
Avoids creating more than one SignalChildHandler if invoked simultaneously from
different threads.
- Reify the channel used for synchronization of stream copy fibers in the
  invoking fiber instead of waiting to lazily initialize it in each fiber
- Make sure the SIGCHLD handler is installed before performing the fork, to
  avoid losing the signal
Introduce a new class `FiberTicket` to represent a fiber willing to send/receive
in a channel. The channels needs to checkout the fiber from the ticket before
enqueueing it in the scheduler. It also has a lock which allows for safe usage
in a select operation.
Passing -Dsingle_thread will simply not start new worker threads.
This allows for multiple callbacks to be executed after a Fiber is switched.
Running the event loop in the main thread means background threads won't
receive IO notifications if the main thread is stuck computing.
Decouple events queue from schedulers, to avoid initializing a dummy
scheduler for the event loop thread (which enqueues events only to be
stolen by other threads but never executes anything else)
Use instead of ThreadLocal variables which don't play well with the GC and they
are not supported in all platforms. Thanks @asterite, @RX14 and @ysbaddaden!
We are using the GC to allocate memory in LibXML, but it keeps a per-thread
global structure with pointers to dynamically allocated objects, which is
allocated using a regular malloc(). Push this structure to the GC's roots list
such that the GC knows about these pointers and doesn't free them on collection.
This should aid in debugging memory corruption problems, missing GC roots, etc.
For yet unknown reasons, this spec crashes when compiling with the GC debug
functions
@bararchy bararchy closed this Jan 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants