Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CSV Import issue with floats (same comma both used as separator and decimal point). #494

Closed
enjoy-digital opened this issue Jun 3, 2021 · 3 comments
Closed

CSV Import issue with floats (same comma both used as separator and decimal point). #494

enjoy-digital opened this issue Jun 3, 2021 · 3 comments
Labels
bug Something isn't working driver Hardware drivers
Milestone
v0.1

Comments

@enjoy-digital
Copy link

Hi,

first thanks for this awesome project! I wanted to explore it for a long time but haven't had the chance to do so before now.

While playing with the CSV import, I had difficulties importing float values and then noticed that comma , was both used as the CSV separator and float decimal point which then prevents import from working correctly with floats. By adapting the code to use a different separator I got the CSV import to work with floats.

So the decimal point should probably be changed to . or the CSV separator changed to something else.
@azonenberg
Copy link
Collaborator

Did your generated data use commas as a decimal separator? If so, that's a data source problem and your files are malformed. If the code is expecting commas as decimal separators then we've got locale problems.

CSV by definition uses commas as field separators, and they're not legal in field content.

@enjoy-digital
Copy link
Author

I was initially generating the CSV using , as CSV separator and . as decimal point, here is a minimal repro:

CSV file:

Time, ADC0
0.0000000000, 2.500000
0.0000000242, 2.500000
0.0000000485, 3.500000
0.0000000727, 3.500000

With some debug added to MockOscilloscope.cpp:

diff --git a/scopehal/MockOscilloscope.cpp b/scopehal/MockOscilloscope.cpp
index 5b89729..ca99421 100644
--- a/scopehal/MockOscilloscope.cpp
+++ b/scopehal/MockOscilloscope.cpp
@@ -566,6 +566,7 @@ bool MockOscilloscope::LoadCSV(const string& path)
                        {
                                float f;
                                sscanf(tmp.c_str(), "%f", &f);
+                               printf("%f\n", f);
                                row.push_back(f);
 
                                if(line[i] == '\0')

We get the following when importing the CSV:

0,000000
0,000000
2,000000
0,000000
2,000000
0,000000
3,000000
0,000000
3,000000

@azonenberg
Copy link
Collaborator

Yeah, looks like the issue is printf/scanf not using . as the decimal separator here. Will look into it.

@azonenberg azonenberg added bug Something isn't working driver Hardware drivers labels Jun 4, 2021
@azonenberg azonenberg added this to the v0.1 milestone Jun 4, 2021
@d235j d235j mentioned this issue Jan 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working driver Hardware drivers
Projects
None yet
Milestone
v0.1
Development

No branches or pull requests
2 participants
@azonenberg @enjoy-digital