Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mouse hover behavior #5291

Closed
wants to merge 7 commits into from
Closed

Add mouse hover behavior #5291

wants to merge 7 commits into from

Conversation

Bakterija
Copy link
Contributor

@Bakterija Bakterija commented Jul 19, 2017

Hello, i would like to add a mouse hover behavior for kivy widgets.
It uses kivy WeakProxy instances and iteration loops, should be as efficient as it can be in python.

This has been asked before and we have some old closed issues #4690 #3471, and various findable post on google.

Edit: current example app in gist

Edit2: "dispatch_cursor_movement" is disabled by default, you have to set it to "1" in config to test this branch.

@Bakterija Bakterija changed the title Added mouse hover behavior Add mouse hover behavior Jul 19, 2017
@matham
Copy link
Member

matham commented Jul 19, 2017

This feels like it belongs in garden.

@Bakterija
Copy link
Contributor Author

Bakterija commented Jul 19, 2017

@matham you mean that code is not good enough for core or that such keyboard, mouse centered behaviors, widgets don't fit in core?
Kivy on desktop by default is very bare bones right now, people do expect hover effects and various other things in desktop apps that we don't have right now.

Another issue is code duplication, if we add sdl cursors or some other things, hovering will have to be duplicated in each module.

@matham
Copy link
Member

matham commented Jul 19, 2017

I guess it's more that the api seems needlessly complex and doesn't mesh very well with the kivy way of handling events like touches. It's also quite difficult to understand what the code is doing and the reasons for the various things there, which makes it harder to maintain in the long term. Some of the choices seem a bit arbitrary and specific to something that you use but may not apply to a general hover type of behavior. That's why I thought it should be in garden rather than in core.

@tshirtman
Copy link
Member

I understand some people want this feature, but i see a number of issues there, and i don't think this is something we can let in, at least not in the current state, because it's working quite differently from the normal touch dispatching, and introduce a weird concept of hover_height of widgets, probably to have a way to provide a way to emulate the end of dispatching of touch events. Aside of that there are certainly a few technical issues, like the fact that it binds at import time and is imported by default, so create lot of dispatching on desktop for all apps. it would be possible to only bind when the first widget with HoverBehavior is instanciated. On the over hand, you seem to have avoided the pitfalls of creating memory leaks (weak proxy use, and adding/removing automatically from list when parent changes), nice job on that. But i feel the issues are mostly conceptual, hence i think garden being possibly a better place. On the other hand, it's certainly a good idea to have a discussion about how to bring that feature.

@Bakterija
Copy link
Contributor Author

Bakterija commented Jul 19, 2017

@matham the example displays it all. Some kind of height/depth system is mandatory when you want it to work with kivy popups. Hover grab is mostly for sliders, as you can see in example. Otherwise you just bind on hovering.
on_enter, on_leave methods could also be added if people want those, it just seemed redundant to me when everything was already possible by binding on "hovering".

Edit: I would be willing to put in some more time into it to change/improve the code.

@Bakterija
Copy link
Contributor Author

Bakterija commented Jul 19, 2017

@tshirtman touch is not dispatched on mouse movemet, and we don't want to do that for performance reasons.

Edit: You are mistaken about the dispatching, it uses a weak ref list with a loop, this means that mouse_pos is dispatched only once, it also just returns when hover widget list is empty.
About the import, i guess that it would be fine to have an if statement in init that binds on Window.mouse_pos only once.

@tshirtman
Copy link
Member

tshirtman commented Jul 19, 2017

Before the code, it's important to get a clear agreement on the idea, because once we merge a feature, we kind of have to support it, and make future code works well with it.

I know touch is not dispatched on mouse motion, but you are binding on mouse mouvement, and updating the widgets that asked for it, trying to only do that for the top level one, it feels like maintaining the order will be a burden in any complex app, and will look like double accounting, from managing your tree, and the height of your widgets, which can now be contradictory, and cause unexpected behaviors.

I would rather (i think, i have not really tested the idea) have the dispatching follow the same path as the touch info, being managed by the widget base class, for example, a widget would be hovered (by default), if the hover events collides it but not any of its children, and that would cascade. It then would be an integrated part of the way the widget tree behaves. Possible downside could be performances, but on desktop only, and certainly manageable, and it would be easy to disable it by overriding the on_hover_event (or whatever), of widgets that have good reasons not to want to dispatch it to their children.

Anyway, it's just an alternative idea, that i feel would be more in line with kivy's way of working, and produce less incoherence in the API, the point is to discuss how to achieve best this feature.

@Bakterija
Copy link
Contributor Author

@tshirtman that might work well too. Could do a dispatch for Window children first for popup, then on app.root. Only problem would be that higher widgets have to be before lower widgets in children list. I guess that it wouldn't be too hard to work with that, just worse performance.

@matham
Copy link
Member

matham commented Jul 19, 2017

Well, you don't have to do any these things manually. Take a look at how on_touch_down/up does it (https://github.com/kivy/kivy/blob/master/kivy/core/window/__init__.py#L1174). You just dispatch it to the window which automatically takes care of the depth thing.

@tshirtman
Copy link
Member

the higher/lower <=> first/last equivalence is already something kivy users have to deal with, it's sometime a bit painful, (BoxLayout/GridLayout mainly, but maybe we could solve it just by adding direction/options to them, like StackLayout), but that's another topic.

I'm not sure of the performances cost without testing, but i feel it's less of a concern because it's only on desktop that we would do the dispatching (at Window level), and these machines tend to have more processing power and less energy issues than phones/tablets, but i agree it should be mesured, and deciding if dispatching to children happens even if it doesn't collide with the parent (like on_touch_* events do), could be a way to decide if we want to go for performance or if there are good reasons to follow strictly the same idea as touch event management).

@matham
Copy link
Member

matham commented Jul 19, 2017

But it does imply adding a on_hover to window and to Widget.

Perhaps it's better if we add a generic on_percolate_event to Window and Widget (i.e. it percolates down the tree) so it can be reused for other types of events that we want to dispatch to all the widgets. I don't like adding one specifically for just hover. E.g. today we want hover, tomorrow we want to notify a widget when the eye looks on it.

@Bakterija
Copy link
Contributor Author

Bakterija commented Jul 19, 2017

@matham that is one reason why i did it like that. The dispatch would add new methods for Window and all widgets, and make it more complex.

@tshirtman
Copy link
Member

well MotionEvent already has such a system to qualify what kind of motion it is, and we could even decide that hover is just another motion event, but with a different profile, that would be tricky because most kivy apps certainly do not check for the profile of their touches, and it would break all of them.

@matham
Copy link
Member

matham commented Jul 19, 2017

I think motion in this context has a special meaning because kivy translates motion events to have a touch down/move/up. I'm thinking that we just a generic event that can be percolated down the widget tree. Similar to touch. So Window and widgets will get a new on_percolate event and this event could be a hover event or e.g. an indication that the eye currently looks at the object etc.

That would get around the issue that currently we don't check for profiles in on_touch_down/move/up.

@matham
Copy link
Member

matham commented Jul 19, 2017

But if we can think of a way to add a profile in a backwards compat way maybe that's better. Because even a hover has down (start), move, and up (end) concept. But I cannot see how we can do that.

@matham
Copy link
Member

matham commented Jul 19, 2017

On second thought, a touch does have a beginning and end while a hover doesn't because the mouse is always somewhere (similarly for the eye). It is nice for touches to have their own specific down/move/up events to make it easier to handle because it's so basic, but having a generic event that is percolated down the tree should handle other types of events nicely.

So yeah, I'd be in favor of adding a on_percolate to Window and Widget. And it'd percolate the event down the widget tree similarly to how a touch is done. We just have to think of what it's argument is. And it can be a motion event instance with a special profile so at least for this we can re-use the touch event machinery.

@tshirtman
Copy link
Member

tshirtman commented Jul 19, 2017

well, since we have on_touch_* for touch management, i'd argue that having on_hover, with its own specific mechanism (even if it mimics on_touch_*) is ok, i'm not against generic, but do we agree that it would result in something like.

class Stuff(Widget):
    def on_percolate(self, motionevent):
        if "hover" in motionevent.profile:
            # code removed for sake of brievety, and because it got me thinking about… multitouch hovering? grabbing and stuff
        else:
            return super(Stuff, self).on_percolate(motionevent)

being the standard way to manage hovering? Instead of a differenciated on_hover method, but that wouldn't be that much different, just one more test than if we had on_hover.

@matham
Copy link
Member

matham commented Jul 19, 2017

@tshirtman right, but this event will have to be added to Widget and Window so it can percolate down the widget tree. So if we added a on_hover event directly, then tomorrow we'd add a on_eye_track to Window and Widget and the day after a on_lazer_strike ;). As opposed to:

class Stuff(Widget):
    def on_percolate(self, motionevent):
        if not collide:
            return super(Stuff, self).on_percolate(motionevent)
        if "hover" in motionevent.profile:
            # code removed for sake of brievety, and because it got me thinking about… multitouch hovering? grabbing and stuff
        elif 'eye' in motionevent.profile:
            ...
        elif 'lazer' in motionevent.profile:
            ...
        else:
            return super(Stuff, self).on_percolate(motionevent)

@tshirtman
Copy link
Member

Yeah, a similar bit of code would be added to Widget/Window, i was talking about how to override, like it's a common thing for widgets to override on_touch_* methods.

I'm not sure about the name yet, but the idea does sound sound.

Copy link
Contributor

@KeyWeeUsr KeyWeeUsr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except that stuff, just try to launch the example with python example.py -m inspector and press Ctrl+E. Once it's on, you can't revert the step I think. I guess that's either because of an animation or because of a new widget in Window directly. Yep, you just killed the debugging tool and once you use it then even the app itself (~30% CPU on idle 2.4GHz, app slowly starts freezing).

displays available functionality::

from kivy.properties import NumericProperty, ObjectProperty, BooleanProperty
from kivy.uix.properties import HoverBehavior
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo? 👼

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, that was a last minute change.


from kivy.properties import BooleanProperty, NumericProperty, ObjectProperty
from kivy.weakproxy import WeakProxy
from kivy.core.window import Window
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quire sure why this particular line decided to throw an ImportError in docs, but it might be worth looking at it if it'd get to a state where we could merge it.

HoverBehavior.hover_widget_refs = [HoverBehavior._hover_grab_widget]
else:
HoverBehavior.hover_widget_refs = HoverBehavior._hover_widgets
for ref in HoverBehavior.hover_widget_refs:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I think there's too much stuff going on that's bound to mouse_pos which isn't a stable number because:

  • not an integer (I have float tuple, but it's from SDL2, so...)
  • when a user uses touchpad the input touch isn't sane (fat fingers for example, bad touch surface, etc)

therefore if you bind anything to that, it has to be flat. Such thing like nested for loops is quite bad unless the content you iterate over is really really small(or no 60fps). Now imagine someone to add that on each widget. Freeze at best :D

defaults to `False`.
'''

hover_height = NumericProperty(0)
Copy link
Contributor

@KeyWeeUsr KeyWeeUsr Jul 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to me like implementing a Z in Kivy, where all position is 2D which probably isn't the best thing to do, but anyway... maybe it'd be a better idea to put the refs/widgets/whatever into a dictionary (quicker lookup) and use list for storing the order of the widgets instead of "height". Then you could use the order with a behavior similar to Widget.children and just move the widgets' order if necessary. Maybe even better would be to cache the widgets' position to that order list and update it with widgets' on_pos. It'll still be slow(or better said executed too many times), but you wouldn't need to touch the widgets alone, just the list. Then again if you use it with Animation it'll go to hell anyway.

Edit: @tshirtman's idea about MotionEvent is better.

@matham
Copy link
Member

matham commented Jul 19, 2017

We can reuse on_motion. Just allow etype to be None in https://github.com/kivy/kivy/blob/master/kivy/core/window/__init__.py#L1174.

@matham
Copy link
Member

matham commented Jul 19, 2017

@KeyWeeUsr what happened? :D we were talking about changing the whole approach rather then fixing a few little things...

@Bakterija
Copy link
Contributor Author

Bakterija commented Jul 19, 2017

Ok, so, are you suggesting to remove the behavior and integrate it into kivy Widget class and window, or should it work similiar to a plugin maybe ?

A pluginish way to do it would be to have the event be bindable or some attr where you can set, then send it to hover behavior, if it is true. Something like that.

@KeyWeeUsr
Copy link
Contributor

@matham browser forgot how to refresh GitHub content in background I guess :D

@matham
Copy link
Member

matham commented Jul 19, 2017

If we agree to something like using on_motion, then you'd modify Window to dispatch on_motion to window's children tree similar to on_touch_down of window does. Then you'll add the on_motion event to Widget and make it also dispatch its widget tree like on_touch_xxx.

Finally, you'll need to somewhere make mouse_pos dispatch a on_motion with a special profile (hover?). Ideally this should be disabled by default but should be enabled through the config, so that it's dispatched only when wanted. I think that's about it?

@KeyWeeUsr
Copy link
Contributor

Also, don't forget to check with a more complex example or at least run it with the Inspector module. We can't kill one of the few layout/widget debuggers we provide by using a behavior or on_something event. That would be kind of silly if you want an application to scale more than to 2 buttons and maybe something else if you're lucky.

@matham
Copy link
Member

matham commented Jul 19, 2017

Rereading https://github.com/kivy/kivy/blob/master/doc/sources/guide/inputs.rst (you should probably re-read also to understand who we handle touch events), what you should probably do is in on_motion in Window (https://github.com/kivy/kivy/blob/master/kivy/core/window/__init__.py#L1183) if it's not a touch event, if the config is set to allow forwarding, dispatch it through to its children.

So all you need to do is add on_motion to Widget and you're done. You don't event need to make it a special profile. It already has a pos attribute indicating that it is a hover type event.

@Bakterija
Copy link
Contributor Author

Another potential annoyance here is that hover behavior does not respond when multitouch circle is moved, it responds only when circle is added/removed, but i don't see that as an important feature right ow.

@KeyWeeUsr
Copy link
Contributor

@Bakterija if you feel that the "potential annoyance" is a bug in the behavior, it might be a good idea to fix it. Although there were suggestions to move the multitouch simulator to the background and enable the correct desktop right button behavior by default, making differences between the simulator and the multitouch itself isn't a good way to go in any case, nor merging a behavior with a known fault in it.

If this potential annoyance behaves the same on a real display with multitouch support, then it's up to you whether it's a bug or not, however testing it e.g. in an android app or on some desktop with multitouch display couldn't do any harm.

@Bakterija
Copy link
Contributor Author

Bakterija commented Jul 29, 2017

@KeyWeeUsr right now cursor_movement motion is enabled only for mouse input provider. Mouses are usually used on laptops and pcs where usually multitouch emulation is used for testing only, that is why i see it as not very important right now.

Touch input providers don't do any hover at all right now, because touch movement requires a touch to be grabbed by some widget and it doesn't make any sense to have hover effects of collide point widgets when a button is held already.

Edit again: i will look at updating multitouch emulation movement later too.

@Bakterija
Copy link
Contributor Author

The cursor_movement motion profile is meant be be a generic profile that could be used by a eye tracking or other kind of provider, just as matham requested.

@Bakterija
Copy link
Contributor Author

Ok, i was thinking about multitouch again and i think that ignoring movement there is a fine way to do it, it is supposed to simulate a button press that gets released later, the expectation in desktop applications is to do a hover grab on a held button.
Next we need a hover grab for mouse_pos events, adding a hover_grab method in HoverBehavior and putting self in a WeakProxy attr seems like an ok idea.
What do you guys think?

@Bakterija
Copy link
Contributor Author

It seems to me that noone wants it.
I am thinking about deleting the whole branch to get rid of clutter and so i can stop thinking about it.

Do you guys want to keep the branch?

@matham
Copy link
Member

matham commented Sep 3, 2017

Sorry, I haven't been very active recently so didn't follow all the discussion. But I like it. I think it should be marked experimental so we can change stuff without worry too much about backward compat initially.

Also, maybe don't call the event touch in on_motion so it's clear it's not only touch.

Finally, others would also approve this. @tshirtman @tito ?

last = HoverBehavior._last_hover
if last and last != self:
HoverBehavior._last_hover.hovering = False
HoverBehavior._last_hover = WeakProxy(self)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of creating new instance of WeakProxy you can use existing one at self.proxy_ref.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks.

@KeyWeeUsr
Copy link
Contributor

It seems to me that noone wants it.

@Bakterija it's not that no one wants it. Quite the opposite. It's just that the current implementation breaks stuff when active. I found only the Touchtracer after the recent fixes, but that demo is basically a build stone for multitouch testing if you want to quickly test stuff.

I'm not really sure what else could break, but if Touchtracer is broken, I guess a lot can show up later when merged in the current state.

@rnixx
Copy link
Member

rnixx commented Sep 4, 2017

@Bakterija seems that you've chosen a hard nut to crack ;). Stay tuned!

kivy/config.py Outdated
@@ -842,6 +842,9 @@ def name(self, value):
elif version == 18:
Config.setdefault('kivy', 'log_maxfiles', '100')

elif version == 19:
Config.setdefault('kivy', 'dispatch_cursor_movement', '0')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably belongs in the postproc section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, should have done before, is updated now.

@Bakterija
Copy link
Contributor Author

Ok, well, it stays then.

@matham
Copy link
Member

matham commented Sep 5, 2017

@KeyWeeUsr What is the touch tracer issue?

@pythonic64
Copy link
Contributor

@Bakterija good work on this feature!

Looking at the code I found some cases that are not yet handled:

  1. Mouse doesn't move, but widget changes its center (size or pos).
    For this case behavior should implement on_center method, check collision and set hovering attribute.
    An attribute like ButtonBehavior.last_touch, must be used to keep event instance for collision check in on_center method.
  2. HoverBehavior is used in widget inside of RelativeLayout.
    For this case RelativeLayout should override on_motion method and transform event in similar way on_touch_* methods do.
  3. HoverBehavior is not registered in factory_registers.py.
    If registered, it can be used inside kv rule.

Other minor things:

  1. Check in hover.py line 12 seems unnecessary to me.
  2. Line 19 in hover.py, HoverBehavior._last_hover is already dereferenced in last variable, so line 19 should be last.hovering = False

@matham
Copy link
Member

matham commented Sep 7, 2017

@pythonic64

Mouse doesn't move, but widget changes its center (size or pos).

I don't think we need to consider this kind of situation because imagine as the mouse hovers over a widget, now all of a sudden the widgets underneath start moving. This is not a typical UI and would probably be confusing to the user. At least for typical hover usage.

Of course one could design a app which does do this kind of thing, but then they should probably implement their own custom hover behavior. This is more like an example on for common hover usage so it doesn't need to target everything, rather one thing it does well

HoverBehavior is used in widget inside of RelativeLayout.

Unlike touch, the on_motion event is meant as a generic motion event which typical widgets do not specialize. In fact, I'd suggest that we make it very clear in the docs that it is intended for events that need to bubble down the widget tree without know anything about them and could be used e.g. for eye tracking etc. That means you need to check that the event in on_motion is the event you want to process.

Specifically, unlike touch where we make sure on_touch_xxx gets the touch in parent coordinates, we should make it very clear that the mouse motion event comes in window coordinates. So only in the behavior that uses it it needs to convert from window to parent (so, yeah, that does need to be fixed for the HoverBehavior). That means that RelativeLayouts etc. don't need to know anything about on_motion.

Check in hover.py line 12 seems unnecessary to me.

Correct. In fact because on_motion is generic, you cannot know what it means for it to return true other than that it ate the event. E.g. it could be a eye tracking event and not a hover event. So setting hovering to False when it returns True seems incorrect.

@matham
Copy link
Member

matham commented Sep 7, 2017

I would also suggest that you rename hovering to hover.

Finally, you really should try to track down @tito, maybe on IRC, to ask about this before you invest too much more time. This is a bit of a fundamental design change which is different than how we do e.g. touch, so I would like to get their input.

@pythonic64
Copy link
Contributor

@Bakterija @matham

Mouse doesn't move, but widget changes its center (size or pos).

Cases I had in mind are when widget is changing its center due to animation or keyboard input (scrolling with keyboard) or mouse scroll up/down.
That's way I'm suggesting to implement on_center method. It will handle these cases, it's a few lines of code so it doesn't complicate the class and it makes this feature more complete.

HoverBehavior is used in widget inside of RelativeLayout.

Ok, when HoverBehavior receives event it should check if this is a mouse hover event, covert widget pos coords to window coords, check collision and set hovering attribute.

My opinion is if HoverBehavior is implemented as suggested above, user would have to subclass it and it will set hovering attribute correctly in static widgets, animated widgets, view widgets like in RecycleView and it will do that out-of-the-box.

If you suggesting changing hovering then I think hovered would be more appropriate. Names hover and hovering seems like widget itself is hovering over some other widget, but it really is hovered by mouse indicator.

@matham I assume you agree with case 3. and "minor things" 2.?

@Bakterija
Copy link
Contributor Author

Bakterija commented Sep 29, 2017

Thanks for the time and attention, working on it was interesting, but i don't think that this kind of system will be accepted by tito. So i am closing this pr.

@Bakterija Bakterija closed this Sep 29, 2017
@Bakterija Bakterija deleted the hover_behavior branch September 30, 2017 18:56
@pythonic64
Copy link
Contributor

@Bakterija did you asked @tito for opinion on this feature?

I think that this feature is needed for desktop platforms and it can be added as experimental. If you don't have time to work on it, maybe someone else can take over.

@Bakterija
Copy link
Contributor Author

Bakterija commented Oct 2, 2017

@pythonic64 we have the opinion - http://paste.ubuntu.com/25661582/ .

I think that it is important too, but core devs seem to be thinking that it would be adding too much complexity and/or cpu usage.

I do have some time, but i can't finish anything without core dev input.

More:

We talked a lot about this, we have multiple @ tito messages here too, i think that everyone already said what they wanted.

@Bakterija Bakterija restored the hover_behavior branch October 2, 2017 16:22
@Bakterija
Copy link
Contributor Author

Bakterija commented Oct 2, 2017

Ok well, i restored the branch, you might want to look at it again.

@pythonic64
Copy link
Contributor

@Bakterija thanks for posting the conversation between @tito and @matham .

I like what they said about this feature. Maybe as a next step matham can create a new issue in which he can layout all things this feature should support (multi hover, filtering etc) and a general plan of implementation.

I can test and review the work in progress if you or someone decides to work on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Incomplete Issue/PR is incomplete, but it's real
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants