This can't be merged in because it doesn't yet deal with GC lifetimes of objects. We need to figure that out before we can offer this API.

I imagined as much – creating a second context caused all sorts of crazy errors. What needs to be done to figure the GC lifetimes out? Does something need to change in the Rubiunus core, or is it the way I am handling things in this API implementation?

I believe he means we need to define an API for releasing and acquiring handles to objects we want to use outside of the vm

Ramsey Nasser reply@reply.github.com wrote:

I imagined as much – creating a second context caused all sorts of crazy errors. What needs to be done to figure the GC lifetimes out? Does something need to change in the Rubiunus core, or is it the way I am handling things in this API implementation?

Reply to this email directly or view it on GitHub:
#1099 (comment)

Is this on the roadmap? Is there any way I can contribute to this effort? I'm super interested in this functionality.

As @evanphx said, the api for the lifetime of objects either referred to outside of or bound into the VM needs to be decided or prototyped. Also who is in charge of them needs to be figured out. http://donttreadonme.co.uk/rubinius-irc/rubinius.log.20110623.html is the conversation we had a while ago discussing it.

I mentioned this briefly on IRC, but I would recommend looking at JNI for the more mechanical parts of the API (runtime lifecycle, object lifecycle). I also see no reason this couldn't be developed as a generic API for embedding any Ruby impl. The functions you have so far are all easily supportable outside of rbx, and your mechanism of passing around a context is exactly what JNI does with JNIEnv.

Hopefully we can collaborate on a single API, rather than having yet another C API that only works well on one impl.

I agree completely. The lack of a robust embedding API is one of Ruby's biggest shortcomings. For a language so adept at implementing DSLs, it's a real shame that those DSLs can't be used to script an application!

I'm very happy to collaborate on the design of a generic API and an implementation for Rubinius. My fellowship at Eyebeam allows me to work on it pretty much full time. I would love to hear from you all about how to move this forward.

I think the best way to proceed would be to collaborate on defining a header that has no direct connection with rbx, but which defines the basic functions we would want for embedding. We can look at implementing that in JRuby in parallel and help shake out issues.

It would probably also be worth your time to read the JNI (Java Native Interface) API set, which does a very good job of allowing JVMs to be embedded in native applications (and vice versa) while keeping the VM internals well-isolated. That pattern would advise our common Ruby embedding (and eventually, common C API replacement) very well.

Sounds great. I started a gist with a an initial sketch of the header. It is based on the header used in this pull request, which is in turn based on this work by @gaffo. I just removed any Rubinius/C++ dependent code.

I'll read up on the JNI. I've also been looking at the Python and Lua embedding APIs for how they handle the dynamic features of those languages.

@nasser, thanks for picking this up and running with it. A single embedding API for Rubinius and JRuby will be absolutely fantastic.

Thanks!

@headius do you have any links handy for the JVM / JNI api's just to make the collaboration easier? Putting them in this ticket would make things easier for anyone coming in late.

@gaffo The JNI APIs are documented here: http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/jniTOC.html

You'll want to read over the list of functions, which should give a good idea of where a Ruby embedding API should go. Especially interesting are the functions for managing the lifecycle of a given object reference, which is the biggest pain for JRuby/Rubinius C extensions right now (we have to assume "handles" live forever...or for a very long time).

I should also point out Wayne Meissner's xrbapi: https://github.com/wmeissner/xrbapi

It is rather C++ oriented, but the general principals behind it are the same as JNI: pass an opaque VM context around, control access to object references, use functions for all object-internal data manipulation.

This pull request passes (merged d53c858 into 151932a).

#3446 will cause issues on Mac when embedded dynamically.

@nasser Thanks for your work on this. I'm still interested in making Rubinius embeddable, but there's a substantial amount of work that needs to be done initially and it's not very high priority due to very little demand.

I'm going to close this because it will need to be rewritten anyway and it hasn't seen any activity in a long time.

Understood!

@nasser for a bit more context, please see this comment from a related ticket. These two concepts (embedding and C++ API) are closely related and will likely end up under the same milestone.