Raspberry Pi OpenGL issue with screenmanager #5581
Comments
|
I have completed some more tests to try to isolate this issue by using smaller images and a simple slideshow test app. In all cases, screenmanager appears to be causing the OpenGL glGetError 0x502 and 0x505 errors. In addition, using To provide these test cases, I have uploaded a copy of the simple screenmanagertest.py and sample images in the link below to a folder containing test files to duplicate the symptoms of this problem. Below is the memory usage while running these tests: |
|
Does anyone support Kivy anymore? I've not received any response here in over a week. Should I just give up? |
Unfortunately nobody gets paid to fix issues or respond to bug reports around here. There are more bugs than resources available to fix them, and the issue backlog spans several years.. Certainly don't give up, but consider that you are not paying for the products development or maintenance. The onus is on individual users (like yourself and me) to participate in development, discussions, contribute code, etc. I don't have an answer for you, as I never used Kivy with RPi. But it seems like you are using SDL2 and running under X, is that correct? As far as I know, this even being possible is a relatively recent development on the RPi side. Did you try the BCM driver without X? I don't know for sure if it works on RPi3 at all, but it seems worth a shot - if nothing else it could further isolate the issue to "SDL2 on RPi3" .. |
|
Thanks bionoid for your comments. I understand the resources are scarce since I spent over a year developing a prototype on Android that failed, which is why I am migrating to RPi3. I was hopeful there might be more developers for Linux to help but apparently not. As for the SDL2, X, and BCM drivers, this is beyond my comprehension as to how these are installed or used. I am also interested in learning alternative solutions to participate in development, discussions, contribute code, etc. Any suggestions are appreciated. |
|
First things first... Does this happen with fewer images? Like two? If that doesn't produce an error, you are probably just out of memory..
Kivy comes with a window provider specifically for the Raspberry GPU, source link (this originates back with RPi2). I referred to it as bcm, that's maybe not technically correct, broadcom makes the SoC, "videocore" is maybe better... anyway, X-windows is the graphical user interface on Linux. Quoting from your log above:
It detects SDL2 and the Raspberry GPU, but you don't show the part of log that says which window provider is actually being used. I am guessing you are using it from within the X graphical user interface? |
|
Thanks again bionoid for your help. I've been busy building and testing to learn more about these different packages, which is why I'm late responding. I also got some advice from pmpp on the kivy forum about using gl4es which I also tested. I also created a boot image of the Kivy-Pie build but I get the same results when testing screenmanager (simple app like mine that exhibits similar symptoms). My tests are using two jpg images and I've attached the code, console log, and images used in the slideshow (that need to be located in /home/pi/.masterpics/pix folder). The Kivy-Pie build uses only the FluxBox windows manager, so I believe Kivy is running under X on that build but still generates the 'glGetError 0x0502' and 'FBO Initialization failed: Incomplete attachment' errors that you see in this log. I also see the 'glGetError 0x0505' out-of-memory error immediately after the FBO init failure. As for gl4es, I installed that on the NOOB build but did not see any differences in the logs (both NOOBS and Kivy-Pie). So, I don't know what to do next with this. I asked on the kivy forum for help regarding this but did not get an answer. I can try again. So, I thought I'd share where I am and ask if you can help me better understand how drivers are made to work. You mentioned videocore driver but I'm not sure if it is already installed or not and how to install it if not. I found videocoreiv but not sure what to do with it. Sorry, I may need to take time to study more about the Raspberry history to figure all this out. Any suggestions from you or anyone is greatly appreciated. |
|
Unfortunately I don't know anything about OpenGL or Raspberry, so I' don't have much suggestions beyond what I said in previous comment. The videocore driver is what Kivy uses if you do not run it under X, which is what I proposed you try for comparison. On RPi2 it was not possible to run under X as far as I know, so the setup you are using may not have been tested as widely |
|
And to be clear, as far as I know, you do not need to install or do anything special to use the videocore, short of having it compile successfully and not run it under X. As long as this is present when you are building, I think you are set: |
|
It appears that all the GL “drivers” are installed, even videocore, per the line below from the log: “OpenGL renderer <“VideoCore IV HW”>“ Should I ask someone in the kivy dev org if he/she has any suggestions for me to research and fix? This is my last attempt to use Kivy for my product. I had hoped Linux would be successful. |
Well it is the name of the hardware; what you need to check is the reported window provider, I think it should be
Kivy maintainers are reading here on the issue tracker, but most probably do not use RPi3 (or even own one to try to reproduce). You can try posting on the kivy-users mailing list, or asking on freenode, maybe someone there has ideas. |
|
Thanks again for your help. Can you point me in the direction to do what you suggest below? I’m not sure about how to configure fluxbox or which Kivy driver to use. The window provider is egl_rpi and the text provider is sdl2. “So under fluxbox, you are not using Kivy's included driver. You should at the very least try it, if nothing else to confirm that it has the same problem.” I am also posting on the OpenGL forum and hoping what you’ve provided will help me from their side. Thanks again! |
|
Honestly I have no clue, my best guess is the link I gave you in first comment - enable console mode. From there it should just be compiling and running kivy as far as I know |
|
Thank you again bionoid. I built an image of Kivy-Pie and used my stripped down code to test without starting FluxBox. I received the same results as the NOOB version which I also tested without the Desktop GUI (only boots to console mode). The reason I refererenced the #5348 issue is because it says “undefined symbol: glGetFramebufferAttachmentParameteriv” and that looks similar to the error message I’m getting below (and also above in my original post). These errors continue to occur until the GPU runs out of memory and crashes the test app. FBO Initialization failed: Incomplete attachment (36054) Today, I’m going to try to enable Kivy GL Debug environment variable to see if I can get more details during my tests. I have also been advised to revert back to an earlier version of Cython because it has caused similar results on other OSs. |
|
I have now tried multiple builds without XWindows, two versions of cython, and a kivy rebuilt using gl4es drivers. I continue to get the following error message when running a simple screenmanager test app. Below the FBO error are logs from memory consumption and Kivy OpenGL Debug when running the same app. FBO Initialization failed: Incomplete attachment (36054) 16GB-Deb-screenmanagertest-memory-results.zip From Dark Photon on the OpenGL for RPi, he said the following:The GPU memory consumption report you got while your "GPU out of memory" problem was occurring definitely details clearly what under-the-hood is consuming all that GPU memory. 480MB consumed with only 5.4MB free certainly confirms there's a problem! Of the GPU memory consumers listed, it looks like the biggest fish are:
Below is a link to my code that I ran to generate the OpenGL debug log above. I would like to offer to help anyone with this problem to find a solution. Maybe someone might volunteer if I offered a FREE Raspberry Pi 3 to debug this problem? I'm open to any suggestions and recommendations at this point. |
|
Actually, looking a bit further into the log you have attached in your first post, to me it looks like all of that is caused by a missing API related to shaders. It says: Furthermore, the docs state (emphasis mine):
And now my next (completely uninformed) question would be: is Kivy able to hand over shaders in source form, or are binaries a requirement? Edit: If my hunch is right and looking at the source (kivy/graphics/shader.pyx) these get built automatically by Kivy, even in the current master branch (haven't checked older versions): property vs:
'''Vertex shader source code.
If you set a new vertex shader code source, it will be automatically
compiled and will replace the current vertex shader.
'''
def __get__(self):
return self.vert_src
def __set__(self, object source):
if source is None:
source = default_vs
source = source.replace('$HEADER$', header_vs)
self.vert_src = source
self.build_vertex()Note the last line. My guess is you'd need a version of this that:
In short: you'd probably need to program a workaround yourself (only if my guess is correct, though). Edit 2: Actually, I'm seeing some stuff about shader-source in there, but don't have the complete picture yet. It's kind of pointless for me anyway, I can't test it. |
|
@frankgould |
|
Thanks for all your help with this dolang. I was able to get QEMU to run a small img build but not others, like KivyPie. The version I was able to emulate is from the link pasted below. https://github.com/ccrisan/motioneyeos I'm going to do more testing to learn how to get the console to display. This version had very few controls. I tried the minimum build from Raspian but it failed to load the kernel. I think I need to upgrade my RAM from 4GB to 8GB. |
|
Sorry I missed your kivy forum chat yesterday afternoon and thanks very much for your time and efforts. I'm not in any rush so whatever you can do it is truly appreciated. I've also contacted the KivyPie forum and am trying to get them to help. Maybe we should exchange email addresses when we're not online at the same time? |
|
Are you using any shader transitions in your code? Shader transitions don't work reliably with sdl backend. Fortunately this can be fixed with the right display drivers and opengl backends. I think I had the same problem a while ago, please post the first 10 lines from kivy output where opengl driver and backend is shown, maybe I can help you. |
|
Thanks pro90 for helping out with this. I started to paste the start of the log, like you requested, but when I found the attached log, I thought the whole run log might be helpful that includes the startup. At the end, you'll see where I inserted what happened, which was the whole computer locked up and had to be power cycled to reboot. We're assuming a memory leak. KivyPie-screenmanagertest-multi-pics-log.txt As for am I using shader transitions, I can only assume so but not directly. I'm using kivy screenmanager to animate the slide transitions. I assume that code uses shaders. I would also appreciate any education you can offer or website that helps me understand how these drivers and modules work together. I'm not sure how they interact between each other and I understand some are wrappers for other core graphics modules, like OpenGL. I was also suggested to use gl4es because that's what the gamers use on RPi. I haven't been able to confirm that this is the right solution. Thanks again! |
|
as far as im concerned, you're not using real hardware open gl. Your opengl renderer should be "Gallium 0.4 on VC4 V3D 2.1". First of all, activate the Broadcom OpenGL drivers: sudo raspi-config -> Advanced Options -> "GL (Full KMS) OpenGL desktop driver with full KMS". You should also adjust the shared graphics card memory to at least 64MB. For this to work, you should must use a desktop environment, I suggest PIXEL. You also need packages libgl1-mesa-dri and xcompmgr from apt-get. I suggest starting out from a vanilla rpi image. Let me know if these steps work! |
|
Greetings all watching this thread. I have spent many hours researching and testing environments to establish the right OS build to isolate this bug that eventually consumes all graphic memory and crashes the Kivy app. Once I determined all Raspberry Pi linux builds exhibit this same problem that did NOT occur on Oracle VM Mint Linus (No glGetError messages), I began adding debug messages in shader.pyx that allowed me to isolate where certain actions causing a glGetError 0x502 (Invalid Operation) that I had suspected caused the eventual 0x505 out of memory errors. What I found was a “uniform” named “t” that passed a zero value (i.e. 0) immediately before an opacity named uniform was uploaded. Below is what I found in the logs. [DEBUG ] [Shader ] uploading uniform t (loc=2, value=0) That prompted me to change shader.pyx to ignore these invalid “t0” uniforms and the app stopped generating the glGetError 0x502 errors; however, that did not stop the “memory leak” because eventually, like in 10 minutes, the app crashed with the 0x505 and then 0x506 (invalid frame buffer operation - same FBO error in Kivy log entries) error messages. I think this is also why shader.pyx reports 'Exception: Shader didnt link, check info log' because there is no memory to link. So, now I’ve collected several run logs that show memory usage during the period when 0x505 error messages are displaying on the console. Below are from the results while running 'vcdbg reloc' when these console 0x505 error messages are displayed (see run log named vcdbg reloc - Fresh Boot18Mar-crashingwithGC.txt for full snapshot). 9.8M free memory in 9 free block(s) Of the many entries (like the two below with parenthetical values removed), these two appear to be the majority memory consumption candidates. The amount used varies throughout the log. [ 336] 0x375bf740: used 6.3M 'Texture blob' Then, in one section, I found the following error messages. 0x2c570da0: corrupt trailer (space -256 != 1572961) I'm not sure what to do next other than read the GL runtime debug log to see if I can find what is going on in OpenGL and how to fix it in Kivy or I can try to figure out what screenmanager is doing when it issues the OpenGL commands (or whatever uniform upload is getting). My problem is not knowing how these graphic layers operate under Kivy and which might be suspect. Below is a recap of other suggestions from forums:
Things to try going forward:
You can view the Kivy boot console to see what starts at app launch. Any other suggestions will be greatly appreciated by anyone and everyone. |
|
This is not much (as most of the questions are far outside of my expertise), but maybe it'll help?
there are some transitions that are not shader-based, it might be the reason. (
this might be a nothing, but i'm surprised this is indicated "0" not "0.0" here, the value should certainly be a float, and passing an int might cause the "shader didn't link" issue, though that should be resolved as soon as the transition progresses any further towards 1.0. |
|
Thanks tshirtman for your response. I ran some more tests to confirm what you posted and learned that the ones you reported as not shader-based (i.e. SwapTransition) do show up in the KIVY_GL_DEBUG log and also exhibited the glGetError 0x505 messages. It just took a longer amount of time. I've attached a copy of the log while using SwapTransition and Kivy debug logging to show what's happening in the shader uniform upload. I've also attached a FadeTransition log showing the Vector and Fragment Shaders are compiled during launch then Fragment Shader is compiled again later, after starting the application main loop. As for the uniform t value being a float, I changed it's value in shader.pyx and here's what the log reported below. So, I'm assuming for this 0x502 error caused by t value=0, I should just skip its upload like I coded it to do. I haven't seen a correlation in the memory leak consumption, so I'm guessing it's not related. [ERROR ] [Shader ] converting uniform t with value=0 I really appreciate any other suggestions or ideas. Any little bit helps. SwapTransition-using-shader-debuglog.txt |
|
@frankgould Could you try to pull master and see if you still have the issue or not? If possible, try to follow https://gist.github.com/tito/1174b86ce126fd1ac11afbc77750d9ac - This was written this morning and i found a leak in Fbo/renderbuffers. Fix is commited into master, this script may help you to find more in your software? |
|
After reading the log, in my point of vue, all came from the number of image loaded. At some point, the way to wrote your app will just consume memory. I mean, if you have only 3 images, it works no? Kivy/Python cannot handle memory the same as you. I guess you would need to rewrite an algo to keep only 3 "big" pictures in memory "current, next, previous for example" (even keep the current image loaded, and always release the previous one), and release all the previous one. This way you are in control, and less likely to hit this limitation. If you use "Screen", you could use event like on_pre_enter to load the image, on_leave to release it |
|
Thank you very much Tito for your help with this issue. I ran the screenmanagertest.py slideshow test app over night and memory held constantly with no leaks or errors. It was a relief to see. I also wanted to report that I added a couple lines of code in the Shader.pyx module to fix a uniform t call with a value of zero that generated glGetError 0x502 messages. Below is the git diff of my change that removed the glGetError 0x502 errors. diff --git a/kivy/graphics/shader.pyx b/kivy/graphics/shader.pyx
index 53286893..67e8eab7 100644
--- a/kivy/graphics/shader.pyx
+++ b/kivy/graphics/shader.pyx
@@ -265,6 +265,9 @@ cdef class Shader:
if loc == -1:
#Logger.debug('Shader: -> ignored')
return 0
+ if name == 't' and value == 0:
+ #Logger.debug('Shader: -> t:0 ignored')
+ return 0
#Logger.debug('Shader: -> (gl:%d) %s' % (glGetError(), str(value)))
if val_type is Matrix:As for the test code you commented about, this was just some sample code I found online to save me from stripping down my huge code to test screenmanager. My code handles the screens like you suggested. I will be working on migrating my slideshow app today to confirm it is operational as well. I should also mention that your suggestion to use apitrace did not operate in the RPi environment and I have attached a copy of the launch console log for review. I got the same results with dolang's and my test apps but didn't know what to do to fix this. I had to install apitrace twice on this RPi because both installs failed to load a gltrace.conf file to run. I found one posted online (also attached) and did not change it. I am also going to post a comment on the following two github issues to let them know that there is a new context.pyx version to pull. Please let me know if anyone has any suggestions regarding this post going forward. Thanks again for your time and assistance with this matter. |
|
Nah, the issue about name == "t" and value 0 is a nop. Having an error is not a big deal, but yes we should avoid it. It's a matter of initialization of the shader prior the set of the value, maybe some rework is needed, but doing a such patch actually break anybody else that have a shader with t uniform and value 0 :) Again, thoses errors could be avoided, but it's "ok" right now as it has no impact. |
ghost
mentioned this issue
|
I know it's a closed issue but I'll throw a post on here as well. I get the dreaded "ERROR - Shader: GL error 1281" crash as well as "Uncaught exception (FBO Initialization failed)" on a Pi3B with Kivy 1.10.1 with It seems to be related to the existence of Over the last several months of development (which adds more and more graphics), I've had to bump the Lastly, I'll say that Kivy's caching settings could in theory drop some of the textures, for example, out of GPU memory from time to time. |
Versions
Trace-startup-GLerror.txt
Description
I am migrating an operational kivy app using the same resources now on Raspberry Pi that is failing to display images in screen manager. I am trying to find a solution that other open trouble report appear to be regarding the same issues (see below). I asked on the kivy forum but was told to use KivyPie, which I did but got the same results. My configuration is NOOBS, manually installed Kivy for Raspian Jessie, on a Raspberry Pi 3. I have also imported opencv to manipulate images.
[Raspberry Pi 3] - ScreenManager and the 12 transitions #4662
Raspberry Pi 3 "Shader didnt link, check info log." #4542
In the log after starting my app, the following sections appear regarding OpenGL and Shader (see attached 'python main.py -c kivy:log_level:trace' log). It appears these are working and setup.py reports the library files are in the /opt/vc/lib/ folder.
[INFO ] [GL ] Using the "OpenGL ES 2" graphics system
[DEBUG ] [GL ] glShaderBinary is not available
[INFO ] [GL ] Backend used
[INFO ] [GL ] OpenGL version <OpenGL ES 2.0>
[INFO ] [GL ] OpenGL vendor
[INFO ] [GL ] OpenGL renderer
[INFO ] [GL ] OpenGL parsed version: 2, 0
[INFO ] [GL ] Shading version <OpenGL ES GLSL ES 1.00>
[INFO ] [GL ] Texture max size <2048>
[INFO ] [GL ] Texture max units <8>
[DEBUG ] [Shader ] Fragment compiled successfully
[DEBUG ] [Shader ] Vertex compiled successfully
After my app starts displaying images through screenmanager, the following error appears in the same log (this error means invalid operation):
glGetError 0x502
After 7 of these invalid operation error messages, the following error appears (glGetError 0x505 means out-of-memory):
Exception: FBO Initialization failed: Incomplete attachment (36054)
Exception Exception: Exception('FBO Initialization failed: Incomplete attachment (36054)',) in 'kivy.graphics.fbo.Fbo.raise_exception' ignored
glGetError 0x505
After 3 of these Exeption messages, my app crashes with the following end of process regarding Fbo and Shader:
File "/home/pi/kivy/kivy/uix/screenmanager.py", line 508, in add_screen
self.fbo_out = self.make_screen_fbo(self.screen_out)
File "/home/pi/kivy/kivy/uix/screenmanager.py", line 472, in make_screen_fbo
fbo = Fbo(size=screen.size, with_stencilbuffer=True)
File "kivy/graphics/fbo.pyx", line 152, in kivy.graphics.fbo.Fbo.init
File "kivy/graphics/instructions.pyx", line 753, in kivy.graphics.instructions.RenderContext.init
File "kivy/graphics/shader.pyx", line 184, in kivy.graphics.shader.Shader.init
File "kivy/graphics/shader.pyx", line 695, in kivy.graphics.shader.Shader.vs.set
File "kivy/graphics/shader.pyx", line 555, in kivy.graphics.shader.Shader.build_vertex
File "kivy/graphics/shader.pyx", line 585, in kivy.graphics.shader.Shader.link_program
Exception: Shader didnt link, check info log.
I've searched anywhere I can to find any clues to what to do but below are the attempts to find a solution, unsuccessfully.
I have updated the rpi gpu memory as displayed below:
/boot/config.txt: gpu_mem=512M
I have run memory reports for GPU and it does not appear to be a memory problem, until the final 0x505 error. I have successfully tested glxinfo and glxgears to confirm the OpenGL software is operational.
I get following response when running setup.py:
Detected Cython version 0.27.3
Using this graphics system: OpenGL ES 2
GStreamer found via pkg-config
SDL2 found via pkg-config
Found brcmEGL and brcmGLES library filesfor rpi platform at /opt/vc/lib/
SDL2: found SDL header at /usr/include/SDL2/SDL.h
SDL2: found SDL_mixer header at /usr/include/SDL2/SDL_mixer.h
SDL2: found SDL_ttf header at /usr/include/SDL2/SDL_ttf.h
SDL2: found SDL_image header at /usr/include/SDL2/SDL_image.h
Please let me know what to do to solve this error and app crashing.
The text was updated successfully, but these errors were encountered: