Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement parsing to a bit vector in C++ #5

Open
HackerFoo opened this issue Nov 16, 2020 · 4 comments
Open

Implement parsing to a bit vector in C++ #5

HackerFoo opened this issue Nov 16, 2020 · 4 comments

Comments

@HackerFoo
Copy link

Bitstream generation with xc-fasm uses a significant amount of time, most of which is spent parsing FASM text with textX.

Given that the FASM format is fairly simple, and the input is in the megabytes, this could be much faster if written in a low level language such as C++.

While the FASM format is well established at this point, it would be nice to keep the same Python interface to allow easy integration and filtering of the FASM input and bit vector output.

Plan to use C++ within Python

There are a number of ways to interface Python and C: cython, Boost Python, ctypes, and many more, several of which are compared in this article.

I have chose ctypes for the following reasons:

  • The interface should be very simple (filename or string in, byte vector out), so there is no need to model C++ objects.
  • ctypes comes with Python, so there is no extra dependencies required.

I have written some test code that approximates how this will work: https://github.com/HackerFoo/cpyo

So then, kernel.cpp will be replaced with code that converts FASM input to a bit vector, reading from prjxray as needed, and I will use ctypes to call it from xc-fasm in place of fasm_assembler from prjxray.

@litghost
Copy link
Contributor

Given that your own data shows that 99% of the time is spent in textx, why would you convert the fasm_assembler stuff to C++? The vast majority of the gain is from avoiding textx for the parsing tree, not in the python used to assemble the bit vector?

@HackerFoo
Copy link
Author

The assembler isn't very large or complex (201 lines for fasm_assembler.py, and 128 for bitstream.py, about half of which is comments), and doing only parsing in C++ would require passing a lot more data across the FFI than necessary.

@litghost
Copy link
Contributor

You also need to manage reading segbits, and pathing to segbit data.

@HackerFoo
Copy link
Author

@litghost @mithro I have created a document here with more details, which I will keep updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants