Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exported STEP files do not include the colours #452

Closed
nabijaczleweli opened this issue Jul 29, 2019 · 30 comments · Fixed by #763 or #767
Closed

Exported STEP files do not include the colours #452

nabijaczleweli opened this issue Jul 29, 2019 · 30 comments · Fixed by #763 or #767
Labels
enhancement STEP Issues relating to STEP file read/write

Comments

@nabijaczleweli
Copy link
Contributor

nabijaczleweli commented Jul 29, 2019

System information

SolveSpace version: 3.0~5df53fc5

Operating system: Windows 10

Expected behavior

Models exported into STEP, then imported into KiCad should retain the original colouring specified in the model.

Actual behavior

Having disabled forcing surfaces to mesh and exported this file into the STEP format, yielding this file, and pointing Pcbnew at it yielded the following 3D render:

render with single-tone grey model

Additional information

I don't know if this is a Solvespace bug, a KiCad bug, or, indeed, a bug at all, because I don't understand the ecosystem, so I'm opening this issue after being nudged to, but apologies in advance if this is out of scope.

@whitequark
Copy link
Contributor

I don't think the current STEP exporter exports colors at all.

@nabijaczleweli
Copy link
Contributor Author

So, FWIW, I tried to add an implementation p.a. to some STEP files I found in KiCad and stole off the internet, and landed at the following diff, which by all accounts I could find and understand should work, but Pcbnew's 3D viewer segfaults when it tries to load it and ABViewer 14 shows naught.

diff --git a/src/exportstep.cpp b/src/exportstep.cpp
index 970c925..26ffe72 100644
--- a/src/exportstep.cpp
+++ b/src/exportstep.cpp
@@ -278,6 +278,39 @@ void StepFileWriter::ExportSurface(SSurface *ss, SBezierList *sbl) {
 
         id++;
         listOfLoops.Clear();
+
+        fprintf(f, "#%d=PRESENTATION_LAYER_ASSIGNMENT('','',(#%d));\n", id, id + 2);
+        id++;
+        fprintf(f, "#%d=MECHANICAL_DESIGN_GEOMETRIC_PRESENTATION_REPRESENTATION('',(#%d),#168);\n", id, id + 1);
+        id++;
+        fprintf(f, "#%d=STYLED_ITEM('',(#%d),#%d);\n", id, id + 1, advFaceId);
+        id++;
+        fprintf(f, "#%d=PRESENTATION_STYLE_ASSIGNMENT((#%d));\n", id, id + 1);
+        id++;
+        fprintf(f, "#%d=SURFACE_STYLE_USAGE(.BOTH.,(#%d));\n", id, id + 1);
+        id++;
+        fprintf(f, "#%d=SURFACE_SIDE_STYLE('',(#%d));\n", id, id + 1);
+        id++;
+        fprintf(f, "#%d=SURFACE_STYLE_FILL_AREA(#%d);\n", id, id + 1);
+        id++;
+        fprintf(f, "#%d=FILL_AREA_STYLE('',(#%d));\n", id, id + 1);
+        id++;
+        fprintf(f, "#%d=FILL_AREA_STYLE_COLOUR('',(#%d));\n", id, id + 1);
+        id++;
+        fprintf(f, "#%d=COLOUR_RGB('',%f,%f,%f);\n", id, ss->color.redF(), ss->color.greenF(), ss->color.blueF());
+        id++;
+        fprintf(f, "\n");
+
+        // (opt?) #679 = PRESENTATION_LAYER_ASSIGNMENT (  '', '', ( #681 ) ) ;
+        // #680 = MECHANICAL_DESIGN_GEOMETRIC_PRESENTATION_REPRESENTATION (  '', ( #681 ), #{168} ) ;
+        // #681 = STYLED_ITEM('color',(#682),#{advFaceId});
+        // #682 = PRESENTATION_STYLE_ASSIGNMENT((#683));
+        // #683 = SURFACE_STYLE_USAGE(.BOTH.,#684);
+        // #684 = SURFACE_SIDE_STYLE('',(#685));
+        // #685 = SURFACE_STYLE_FILL_AREA(#686);
+        // #686 = FILL_AREA_STYLE('',(#687));
+        // #687 = FILL_AREA_STYLE_COLOUR('',#673);
+        // #673 = COLOUR_RGB('',0.824000000954,0.819999992847,0.78100001812);
     }
     sblss.Clear();
     spxyz.Clear();

@whitequark
Copy link
Contributor

Yeah... no ideas here. I think STEP is an awful format, and we shouldn't be emitting it manually anyhow, but there are few good libraries either.

@nabijaczleweli
Copy link
Contributor Author

nabijaczleweli commented Jul 30, 2019

whoops, turns out it's supposed to be #%d=SURFACE_STYLE_USAGE(.BOTH.,#%d);

With that fixed, both programs load the model, but with 0 difference to the original, nor've I managed to arrive at anything by fudging some ID refs to other roughly-applicable ones.

And this is where I'm out of ideas, because every file is just an inconsistent indistinguishable identifier soup, and no viewer as much as acknowledges the changes, and the documentation is absolute dogshit :)

@whitequark
Copy link
Contributor

Unfortunately your guess or Google search is as good as mine here, I've never looked at STEP in this level of detail.

@nabijaczleweli
Copy link
Contributor Author

nabijaczleweli commented Jul 30, 2019

imho that's a very good decision on your part

Also saving the link to the docs here for later/reference.

@phkahler
Copy link
Member

@nabijaczleweli This may be a dumb question, but did you verify that SolveSpace actually wrote valid color values to the file with that patch?

@ruevs
Copy link
Member

ruevs commented Aug 1, 2019

Wow! The people that wrote the Step format are either pure geniuses or evil geniuses or both :-)

I somehow clicked to the SI units definitions...
https://www.steptools.com/stds/stp_aim/html/t_dimensions_for_si_unit.html
and now I think the Step format may be Turing complete! Let's rewrite Solvespace in Step!

@nabijaczleweli
Copy link
Contributor Author

nabijaczleweli commented Aug 1, 2019

@phkahler Writing out COLOUR_RGB('',%f,%f,%f) should be correct, but I did also verify by mass-replacing all instances with COLOUR_RGB('',0.9,0.0,0.0), COLOUR_RGB('',0.9,0.1,0.1), COLOUR_RGB('',1,0,0), and COLOUR_RGB('',0.148000001907,0.144999995828,0.144999995828) (stolen from TO-92.step), just to make sure, to no effect.

@phkahler
Copy link
Member

Another resource for STEP files is cax-if.org
In particular the document on Model Styling and Organization under recommended practices.

It says it is customary to include a global style for a model, and any components or surfaces are free to over ride that style. It seems SolveSpace doesn't specify any style at all in STEP. It might be worth it to try that first and assign a colour at the top level before trying to do so at the surface level.

@phkahler
Copy link
Member

phkahler commented Aug 31, 2019

@nabijaczleweli What is the simplest way to view a STEP file with colors? I installed FreeCAD, made a box in SolveSpace, exported step, imported in FreeCAD, colored it, saved as STEP, and have been trying to get SolveSpace to output valid color information based on the FreeCAD output with no luck (testing by opening in FreeCAD). Then I decided to open the STEP file I save from FreeCAD and it doesn't show any color (its in the step though). So now I'm not sure how to even validate the step output with color information.

Should I just install KiCad? Do I Have to open a component design or something in order to even load a step file there? Is there any other FLOSS viewer out there that my distro (Fedora) probably supports out of the box?

@nabijaczleweli
Copy link
Contributor Author

I can't recommend anything less… intrusive?/more dev-friendly than KiCad because it's the only software I use that uses 3D models, so your google search is as good as mine, sorry.

However, to open a model in KiCad, you'd open it, open the "PCB layout editor", add a footprint, save the project so you don't have to do it again, press E over the footprint, and, in the 3D settings tab, "Edit Filename" to your model's, and hope it doesn't crash.

@phkahler
Copy link
Member

phkahler commented Sep 2, 2019

No luck.

I pushed changes to my "color" branch that mimic FreeCAD color export, but it doesn't work in KiCAD (not colors shown). It appends per-surface color information at the end of the STEP file using all the same chained elements. They point to the correct ADVANCED_FACE IDs, or at least the IDs with those strings in them. So that part is consistent with FreeCAD color output.

The fact that this doesn't work suggests something else is wrong with the file, but I have no idea what.

Note: FreeCAD does not seem to import the colors that it exports but KiCAD/pcbnew does.

@nabijaczleweli
Copy link
Contributor Author

Sounds like we're on the same page, then, sadly

@phkahler
Copy link
Member

phkahler commented Sep 3, 2019

I'm seriously considering a new STEP exporter that uses the same entity types as FreeCAD. This would probably need to be an option so we don't break compatibility with the existing STEP export. It wouldn't be significantly better, just different and more compatible with KiCad for colours.

@whitequark any objection to subclassing the StepFileWriter ?

@jwesthues SolveSpace exports using this:
`
#416=CLOSED_SHELL('',(#235,#271,#307,#343,#379,#415));
#417=MANIFOLD_SOLID_BREP('brep',#416);
#418=ADVANCED_BREP_SHAPE_REPRESENTATION('',(#417,#70),#50);

#419=SHAPE_REPRESENTATION_RELATIONSHIP($,$,#169,#418);

`
When I round trip through FreeCAD I get no entities that indicate any kind of solid. lot's of SHELL_BASED_SURFACE_MODEL items. There is also CLOSED_SHELL vs OPEN_SHELL that happens in translation. This makes me wonder if STEP has different representations for solids vs planar surfaces. Maybe something is lost in the translation through FreeCAD, or if it wasn't defined fully by SolveSpace? I guess i'm looking for specific hazards you may be aware of with this or any additional understanding of step you may have.

@whitequark
Copy link
Contributor

@phkahler I've long been thinking about migrating from the handwritten STEP exporter (which is very ad-hoc and doesn't work so well) to STEPcode, primarily so that we can get STEP import but better STEP export is also a good reason to do it. Would you mind taking a look?

@phkahler
Copy link
Member

phkahler commented Sep 3, 2019

@whitequark If I read the description correctly, STEPcode read in a schema and ouputs C++ code for reading and writing STEP files conforming to that schema. The have a number of schema files there - one of them is 1.6MB. What I haven't seen is the resulting file read/writer which is ultimately what we'd want. Using something like that is probably a good idea, but I feel like it's beyond what I'm willing to take on right now. I'm not even completely sure why I want to solve this color issue, I just do :-)

@whitequark
Copy link
Contributor

whitequark commented Sep 3, 2019

So I'm mostly hesitant to have two options for "STEP export (broken)" and "STEP export (KiCAD compatible, probably broken in different way we don't know yet)". And it would require some annoying restructure of all platform code to even have those two options, I think, because we currently use the file extension as a key for formats.

@phkahler
Copy link
Member

phkahler commented Sep 3, 2019

It's not that bad. OpenCASCADE is used in KiCAD, FreeCAD, and Gmsh and all of those will be able to read both exported files. FreeCAD even has STEP and STEP with colours. The big question is use with other (commercial) software we don't even have access to.

Your concern about the platform code seems more significant and brushes against another idea - why not just use GKT (or other) on all platforms? With a commitment to a toolkit we could do things like switch to SVG icons on the toolbar and scalable text.

@whitequark
Copy link
Contributor

whitequark commented Sep 3, 2019

Your concern about the platform code seems more significant and brushes against another idea - why not just use GTK (or other) on all platforms? With a commitment to a toolkit we could do things like switch to SVG icons on the toolbar and scalable text.

This has been discussed to death elsewhere on this issue tracker. In short: GTK doesn't really work (it provides a rather bad experience on macOS and Windows); Qt might work and reduce the amount of supported GUI toolkits a bit; we would still need the platform portability layer to support the Emscripten port, and that means drawing the text window ourselves.

As for the SVG icons, these toolkits all cache them as rasters at specific resolutions anyway, so we can already do it if someone would redraw the icons as SVG.

@whitequark
Copy link
Contributor

It's not that bad. OpenCASCADE is used in KiCAD, FreeCAD, and Gmsh and all of those will be able to read both exported files. FreeCAD even has STEP and STEP with colours.

Then I think we should have that as the only option. Fusion360 is free so we might want to also check with that, and it's going to be the #1 proprietary option.

@jwesthues
Copy link
Member

STEP is a huge, sprawling standard, and everyone seems to implement a slightly different narrow subset. I wrote the existing exporter by closely following a sample exported from Rhino, with limited understanding of what was where or why.

So Rhino should always be able to import our current STEP, and I suspect that the tools mentioned above will have trouble with STEP exported from Rhino; but that of course doesn't help with anyone's practical problem. Maybe a library would be helpful here, though I'm not sure how much of the implementation differences that would actually abstract. You could also repeat my cargo-cult approach, just starting from a file (with as few surfaces as possible!) known to import correctly into whatever tools you care about.

@phkahler phkahler added the STEP Issues relating to STEP file read/write label Mar 30, 2020
ruevs added a commit to ruevs/solvespace that referenced this issue Oct 21, 2020
The implementation may be sub-optimal, since the colour and alpha is
defined for each NURBS surface instead of on group level, but the STEP
export currently does not represent group structure at all and I am not
familiar with the format in order to change this.

Fixes: solvespace#452
@ruevs
Copy link
Member

ruevs commented Oct 21, 2020

I implemented it:
JensenAlphaSolveSpace
JensenAlphaKisters

@ruevs
Copy link
Member

ruevs commented Oct 21, 2020

@nabijaczleweli your model includes colors now :-)
SOT110-1

@nabijaczleweli
Copy link
Contributor Author

well, that's quite pleasing

phkahler pushed a commit that referenced this issue Oct 21, 2020
The implementation may be sub-optimal, since the colour and alpha is
defined for each NURBS surface instead of on group level, but the STEP
export currently does not represent group structure at all and I am not
familiar with the format in order to change this.

Fixes: #452
@phkahler
Copy link
Member

@ruevs I tried exporting with the update and KiCAD still doesn't get the colors.

ruevs added a commit to ruevs/solvespace that referenced this issue Oct 22, 2020
Make the color export work in KiCAD and Horison EDA which do not support
transparency.

Fixes: solvespace#452
solvespace#763
@ruevs
Copy link
Member

ruevs commented Oct 22, 2020

Horison EDA did not work either - it seems they do not support transparency. This is why I left the commented lines 290-299 in. With them it works. I fixed it and merged my pull request to master.

@phkahler If it still does not work in KiCAD please reopen the issue - merging the pull request closed it.

My suspicion that this may be a problem came from this issue in OpenCascade.

@phkahler phkahler reopened this Oct 23, 2020
@phkahler
Copy link
Member

Doesn't work in KiCAD yet. All I can say is STEP is complicated.

@ruevs
Copy link
Member

ruevs commented Oct 23, 2020

@phkahler OK KiCAD... 1.2GB later... KiCAD (5.1.7) works for me.
The yellow triangle with the cylinder is a SolveSpace exported model.
KiCADWorks
KiCADWorks

And here is a board with this advanced new component:
KiCADWorksBoardRender

@phkahler
Copy link
Member

Well then I guess it works! ;-) I'm glad we have something for our KiCAD OSS friends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement STEP Issues relating to STEP file read/write
Projects
None yet
5 participants