Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uncontrolled recursion in bsp.cpp may cause stack overflow #320

Open
ghoss opened this issue Apr 4, 2018 · 7 comments
Open

Uncontrolled recursion in bsp.cpp may cause stack overflow #320

ghoss opened this issue Apr 4, 2018 · 7 comments

Comments

@ghoss
Copy link
Contributor

ghoss commented Apr 4, 2018

SolveSpace version: 3.0-a49a7384
Operating system: Mac OS X 10.9.5

Models with complex triangle meshes may fail to load, and/or fail to export to STL due to a stack overflow caused by recursive mutual calls of InsertConvex-InsertConvexHow, InsertHow-InsertOrCreate-Insert, InsertOrCreateEdge-InsertEdge and others in bsp.cpp. The level of recursion is not monitored or restricted in the code, leading to the possibility of a stack overflow. In addition, memory allocation by Alloc/AllocTemporary in bsp.cpp is not controlled either, as there is no exception handling for out-of-memory conditions, nor are there tests for null pointers returned by Alloc prior to pointer assignments. Both scenarios will lead to sudden application crashes when attempting to load or export a drawing.

An easy way to reproduce this is to draw two partially intersecting spheres (each created with a 180° arc and a lathe operation), and make the second sphere a "difference" of the first. This boolean operation will fail (why, would be the topic for another issue), but will be generated correctly if NURBS are forced to a triangle mesh – provided that the display chord tolerance is not too small. However, if the export chord tolerance is small enough and an export to STL is attempted, SolveSpace will crash.

Expected behavior

I would suggest implementing exception handling and/or tests for failed memory allocation in bsp.cpp. As for the recursive stack overflow issue, it might be advisable to refactor the recursive mutual functions calls as iterative loops with a local parameter stack instead of recursion, as this could be more easily monitored.

@whitequark
Copy link
Contributor

Yes, this is a known issue.

@ghoss
Copy link
Contributor Author

ghoss commented Apr 5, 2018

I did a proof-of-concept refactoring of the InsertConvex/InsertConvexHow combo, which replaces recursion with a queued approach and therefore mitigates the stack overflow problem to the heap, where it would be easier to detect and handle. This one is able to load my quick-and-dirty sphere models which crash the original code. The other functions are a bit harder to straighten out, but as said it's a start. See the experimental bsp.cpp .

@whitequark
Copy link
Contributor

I think @Evil-Spirit had a patch for this already too, I'll need to look around in the old repository. That said, thanks anyway, this might come in handy!

@Evil-Spirit
Copy link
Collaborator

@whitequark, I have just optimized size of struct. I think @ghoss's patch can solve this problem in better way

@phkahler
Copy link
Member

I combined the @ghoss changes with current master in my "issue320" branch. It compiles but crashes when I force things to triangle mesh - with models that normally work fine. This is hard to debug, as I don't know the original code, nor did I make the changes.

Here is the branch: https://github.com/phkahler/solvespace/tree/issue320

and the commit: phkahler@09c71f2

@phkahler
Copy link
Member

The attached file can be used to replicate the problem with sphere differences. You just need to switch either of the Lathe groups to triangle mesh and then lower the chord tolerance to something like 0.01 - it will use a bunch of memory and crash.

320_death_star.zip

@gnbl
Copy link

gnbl commented Feb 2, 2022

Just had this issue with 3.0~61cc28f8 with and without OpenMP on Windows 10 when a NURBS helix difference failed and needed "force to mesh". Uses about 1.5 GB RAM at this point and as NURBS, the slvs file with 500 kB becomes close to 10 MB as mesh.
Reducing the (export) mesh settings (chord tolerance & piecewise linear segments) helps reduce the risk of running out of memory.
A warning beforehand would be nice :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants