Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

correct __send__ routing of native methods when it has overloads #5067

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kares
Copy link
Member

@kares kares commented Feb 28, 2018

suppose a method is implemented in block-less form in native code e.g.

static civil(ThreadContext context, IRubyObject self)
static civil(ThreadContext context, IRubyObject self, IRubyObject[] args)

doing a Date.civil will properly route to civil(context, self)
while a Date.send(:civil) ends up being routed through args case

managed to hit 2 issues while implementing over-loaded native method matching, the above case is simplified, the full case (date.rb -> RubyDate.java soon to land in a PR) looks as follows :

    @JRubyMethod(name = "civil", alias = "new", meta = true)
    public static RubyDate civil(ThreadContext context, IRubyObject self) {

    }

    @JRubyMethod(name = "civil", alias = "new", meta = true)
    public static RubyDate civil(ThreadContext context, IRubyObject self, IRubyObject year) {

    }

    @JRubyMethod(name = "civil", alias = "new", meta = true)
    public static RubyDate civil(ThreadContext context, IRubyObject self, IRubyObject year, IRubyObject month) {
        return new RubyDate(context.runtime, (RubyClass) self, civilImpl(context, year, month));
    }

    @JRubyMethod(name = "civil", alias = "new", meta = true)
    public static RubyDate civil(ThreadContext context, IRubyObject self, IRubyObject year, IRubyObject month, IRubyObject mday) {

    }

    @JRubyMethod(name = "civil", alias = "new", meta = true, optional = 4) // 4 args case
    public static RubyDate civil(ThreadContext context, IRubyObject self, IRubyObject[] args) {
        // IRubyObject year, IRubyObject month, IRubyObject mday, IRubyObject start

        // TODO interpreter needs a ThreeOperandArgNoBlockCallInstr otherwise routes 3 args here
        if (args.length == 3) return civil(context, self, args[0], args[1], args[2]);
  }
  1. as mentioned in the TODO above compared to JRuby 1.7.x
    9K does no longer route the (arg1, arg2, arg3) case as before
    (so far this turned out to be non-critical as we only hit it with AR-JDBC and adjusted accordingly)
    still it was quite annoying and ugly - not sure how much time it needs to have a proper IR instr

  2. case addressed here is that Date.send(:civil) will not take the same path as Date.civil
    due (native) send having the (ThreadContext, ..., Block) version signature implemented
    the send(:civil) version ends up calling civil(context, IRubyObject self, IRubyObject[] args)

... this is something I would really like to address as its even more annoying than 1.
while I do not like the additional if+check this is the simplest change-set I can come up with
could move the check down to generated invokers (would need to generate Block forms routing block-less)
... or also implement non-block versions of Kernel.send and BasicObject.__send__ not sure if that will work (the populator/binding if I recall right can not handle this) but if working that should route properly

related invoker, so far, looks like this :

    public IRubyObject call(ThreadContext var1, IRubyObject var2, RubyModule var3, String var4, IRubyObject var5) {
        return RubyDate.civil(var1, var2, var5);
    }

    public IRubyObject call(ThreadContext var1, IRubyObject var2, RubyModule var3, String var4, IRubyObject var5, IRubyObject var6) {
        return RubyDate.civil(var1, var2, var5, var6);
    }

    public IRubyObject call(ThreadContext var1, IRubyObject var2, RubyModule var3, String var4, IRubyObject var5, IRubyObject var6, IRubyObject var7) {
        return RubyDate.civil(var1, var2, var5, var6, var7);
    }

    public IRubyObject call(ThreadContext var1, IRubyObject var2, RubyModule var3, String var4) {
        return RubyDate.civil(var1, var2);
    }

    public IRubyObject call(ThreadContext var1, IRubyObject var2, RubyModule var3, String var4, IRubyObject[] var5) {
        if (var5.length > 4) {
            Arity.checkArgumentCount(var1.runtime, var5, 0, 4);
        }

        return RubyDate.civil(var1, var2, var5);
    }

... while JavaMethodN (super) routes send() (with a NULL_BLOCK) as :

        @Override
        public IRubyObject call(ThreadContext context, IRubyObject self, RubyModule clazz, String name, Block block) {
            return call(context, self, clazz, name, IRubyObject.NULL_ARRAY);
        }

suppose a method is implemented in block-less form in native code e.g.
```
static civil(ThreadContext context, IRubyObject self)
static civil(ThreadContext context, IRubyObject self, IRubyObject[] args)
```

doing a `Date.civil` will properly route to `civil(context, self)`
while a `Date.send(:civil)` ends up being routed through args case
@kares
Copy link
Member Author

kares commented Feb 28, 2018

obviously (even with the proposed change) still won't work for send-ing throught an *args method ...
... __send__ should really do a switch for its args[] version to always route same count to same place?
currently most over-loaded native methods handle the fallback but its an added complication for ext writers if they need to special account for :send every time they do an overload e.g. (arg1) + (args[])

@enebo
Copy link
Member

enebo commented Feb 28, 2018

@kares I believe anno binding will work with block and non-block signatures. I might be missing something but if we can do that I would prefer it over an if in all block accepting signatures (I guess we also need callsites to properly call those versions too -- is that the issue?). I would like to ultimately have n call paths with little to no actual extra logic to keep methods smaller/simpler for inlining potential.

@kares
Copy link
Member Author

kares commented Feb 28, 2018

@enebo ok - makes sense, callsites are fine and dispatch is all good for the normal method call.
only about that send case - which I will try to expand the overloads to non-block versions as well.
... and see whether anno binder binds them and thus can get rid of the extra logic presented here.

@headius
Copy link
Member

headius commented Mar 1, 2018

9K does no longer route the (arg1, arg2, arg3) case as before

Are you saying that 3-argument calls aren't going to 3-argument overrides now?

case addressed here is that Date.send(:civil) will not take the same path as Date.civil

Yes, I agree this should be addressed. I'll have a look at the patch.

@headius
Copy link
Member

headius commented Mar 1, 2018

Ok thoughts.

  • I don't believe InvocationMethodFactory can generate stubs for methods that define both block and blockless forms. The main limitation is that there's forms of all JavaMethodXXXX with Block or without Block, but none that handles both.
  • I think we need a fix inside JavaMethod to avoid boxing in cases where the provided block is not given. The block versions would do the check you have here (small added overhead to all block calls to non-block methods, not common), and then dispatch to the non-block version or the args-boxing block version. The default non-block versions would just dispatch to the args-boxing version.

I realize as I type this that this introduces a new problem: calling a method with no block that is set up to receive a block will start routing through args boxing. The problem just gets moved to the other side.

Now...I have generally tried to make sure all my args[] methods do check arity; they can be called from Java, after all, and there's no guarantee on the size of the incoming array. So with that in mind I'm wondering if we really should fix this. If we always route 0-3 args to appropriate overloads, and stop checking the args array length, we're going to have a lot of methods that are poisonous to call from Java without the right size array.

But maybe we can compromise...should we bite the bullet and try to expand the number of arities we can directly pass?

@kares
Copy link
Member Author

kares commented Mar 1, 2018

@kares I believe anno binding will work with block and non-block signatures.

confirmed (as @headius) noted we can not have block and block-less form at the same time for binder e.g.

    @JRubyMethod(name = "send", omit = true)
    public static IRubyObject send(ThreadContext context, IRubyObject self, IRubyObject arg0) {
        ...
    }
    @JRubyMethod(name = "send", omit = true)
    public static IRubyObject send(ThreadContext context, IRubyObject self, IRubyObject arg0, Block block) {
        ...
    }

thus exactly :

I don't believe InvocationMethodFactory can generate stubs for methods that define both block and blockless forms. The main limitation is that there's forms of all JavaMethodXXXX with Block or without Block, but none that handles both.

I realize as I type this that this introduces a new problem: calling a method with no block that is set up to receive a block will start routing through args boxing. The problem just gets moved to the other side.

yeah - I looked at JavaMethod first but quickly realized that it already routes non-block to block (if it exists). so while I can easily fix my problem by always having block forms I'd still like to not be run down later again.

If we always route 0-3 args to appropriate overloads, and stop checking the args array length, we're going to have a lot of methods that are poisonous to call from Java without the right size array.

point taken (although polished for core still easy to get 'ugly' bugs in user-exts - send mis-behaving).
the other approach would be to extend invoker generation to route block forms simply ignoring the blocks.
that would match up how methods implemented in Ruby work, and solve it all, maybe?

9K does no longer route the (arg1, arg2, arg3) case as before

Are you saying that 3-argument calls aren't going to 3-argument overrides now?

yep that's the case at least in interpreter as there's only OneOperandArgNoBlockCallInstr and TwoOperandArgNoBlockCallInstr

@enebo
Copy link
Member

enebo commented Mar 1, 2018

So we could fix this but we would need to redo all those types in order to handle both. My question is which is a better long-term outcome? Maybe we do not need to address all issues today but it might be a time to redo those types (or deprecate and make a new hierarchy) if it would be nicer to just have all paths in a single hierarchy. I am just asking the question though :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants