Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add XML::Reader ParserOptions Support #6332

Merged
merged 1 commit into from Jul 6, 2018
Merged

Add XML::Reader ParserOptions Support #6332

merged 1 commit into from Jul 6, 2018

Conversation

felixbuenemann
Copy link
Contributor

This adds support for passing XML::ParserOptions flags to XML::Reader, similar to XML.parse.

Passing flags like XML::ParserOptions::NOBLANKS can give huge memory savings and speedups compared to the default parser options.

@felixbuenemann
Copy link
Contributor Author

felixbuenemann commented Jul 4, 2018 •
edited

Here's a small performance example:

  • Parsing a 4 GB pretty printed and gzipped XML using XML.parse(io) takes 84 seconds and 30 GB of heap (and requires GC_INITIAL_HEAP_SIZE=32G).
  • Parsing the XML using XML.parse(io, NOBLANKS | NOENT) takes 54 seconds and 17 GB of heap.
  • Parsing a 4 GB pretty printed and gzipped XML using XML::Reader.new(io) takes 85 seconds and 82 MB of heap.
  • Parsing the XML using XML::Reader.new(io, NOBLANKS | NOENT) takes 55 seconds and 50 MB of heap.

@felixbuenemann
Add XML::Reader ParserOptions Support
1578626 
This adds support for passing `XML::ParserOptions` flags to
`XML::Reader`, similar to `XML.parse`.

Passing flags like `XML::ParserOptions::NOBLANKS` can give huge memory
savings and speedups compared to the default parser options.
@sdogruyol sdogruyol merged commit e153291 into crystal-lang:master Jul 6, 2018
@sdogruyol sdogruyol added this to the Next milestone Jul 6, 2018
@felixbuenemann felixbuenemann deleted the xml-reader-parser-options branch July 6, 2018 12:09
@RX14 RX14 modified the milestones: Next, 0.26.0 Jul 30, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sdogruyol sdogruyol sdogruyol approved these changes

@RX14 RX14 RX14 approved these changes

Assignees
No one assigned
Labels
kind:feature topic:stdlib
Projects
None yet
Milestone
0.26.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@felixbuenemann @sdogruyol @RX14