Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow PLY to generate parsing/lexing tables. #63

Merged
merged 1 commit into from Nov 24, 2019

Conversation

FLHerne
Copy link
Contributor

@FLHerne FLHerne commented Nov 21, 2019

...and modify setup.py to create them at build time.

On my laptop, pre-generated tables reduce the time spent in
NMLParser.__init__() from 2.2s to 0.2s, halving the total runtime
of nmlc for small input files.

PLY will attempt to regenerate these automatically at runtime if the
grammar has changed. In the case where they're system-installed and not
writeable, the grammar is also system-installed and not writeable. :P

@glx22
Copy link
Contributor

glx22 commented Nov 21, 2019

A small change is required for pyinstaller. I made the changes locally and can push them to your branch if it's ok.

@FLHerne
Copy link
Contributor Author

FLHerne commented Nov 21, 2019

@glx22 Yes, please go ahead. :-)

@andythenorth
Copy link
Contributor

andythenorth commented Nov 21, 2019

I tested this with Iron Horse and --verbosity=4 flag. Python is 3.5.

Results:

  • Init parser: consistently 0.0s with this PR, 0.4-0.5s in master
  • Parsing: less consistent results, likely due to other system load. With this PR 16.4-17.3s. Master is 17.0-20.0s.

@glx22
Copy link
Contributor

glx22 commented Nov 22, 2019

Hmm pip install -e . or python setup.py develop fails with the setup.py changes if ply is not yet installed. Without the changes the dependencies are installed and everything is fine.

Copy link
Contributor

@glx22 glx22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The suggested changes fix the develop issue

setup.py Outdated Show resolved Hide resolved
setup.py Show resolved Hide resolved
LordAro
LordAro previously approved these changes Nov 23, 2019
...and modify setup.py to create them at build time.

On my laptop, pre-generated tables reduce the time spent in
 `NMLParser.__init__()` from 2.2s to 0.2s, halving the total runtime
 of nmlc for small input files.

PLY will attempt to regenerate these automatically at runtime if the
 grammar has changed. In the case where they're system-installed and not
 writeable, the grammar is also system-installed and not writeable. :P

This patch includes changes suggested by glx22.
@LordAro LordAro merged commit 09e8bf8 into OpenTTD:master Nov 24, 2019
@FLHerne FLHerne deleted the gen-ply-tabfiles branch April 2, 2021 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants