New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump Gemini PostgreSQL JDBC driver to match the other projects #2626
Conversation
@zloster Thanks for this! Something in the changes between 9.4.1200 and 9.4.1208 has had a significant impact on performance for our database tests as observed in Preview 3 for Round 14: https://www.techempower.com/benchmarks/previews/round14/r14p2-vs-r14p3.html#gemini-postgres.db It initially struck me as implausible that a micro version bump would yield such a performance boost, but looking at the Postgres JDBC driver changelog reveals several performance-oriented changes since micro version 1200: https://jdbc.postgresql.org/documentation/changelog.html#version_9.4.1208 I am going to keep an eye on this since it still instinctively feels unlikely. I look forward to the next results to see if the observed performance in this new preview are reproduced/consistent. |
@bhauer That's indeed an impressive boost.
It's common misunderstanding that the PgJDBC releases are "micro". The recently changed radically their versioning to tackle this. The latest driver is version The DB tests are taxing the DB drivers very hard - there are a lot of very simple queries passing. So even minor improvements in the statement processing path should be quite visible in the results. Gemini seems not to have any major bottlenecks so the results should be sensitive to improvements in the DB drivers. My PR #1937 mentioned 15% throughput increase and decrease in latencies but this was measured in a local Vagrant environment. In this environment the major bottleneck is the CPU emulation speed and it seems that if one could reliably improve a test result there with 10% than on real (or not that much restricted) hardware this will be multiplied. Seems like a new approach to the performance tuning :) For example check the
There are three frameworks that are Java-based, have a mature code implementation and have tests for both MySQL and PostgreSQL - Servlet, Undertow and Dropwizard. You could measure their results with changed PgJDBC driver for comparison. Also note that with these frameworks the results with PostgreSQL DB are consistently better than the MySQL with very few exceptions (Servlet's Fortune test comes to my mind). And this wasn't the case with Another observation - several rounds ago MySQL was beating PostgreSQL, but then PostgreSQL was And another thing (I promise to be the last): I have to make PR to revert the batch processing for the Cheers and thank you for this great project |
Interesting. Thank you for the links and additional background. Indeed, having looked at the changelog, I realized that these were not "micro" versions in the traditional sense! :) Looking forward to 42.1.0.
Yes! The database tests are intended to be a brutal exercise of the database drivers and ORM (where applicable). You're right that it stands to reason that tuning in the driver will show up as a significant gain in these tests, assuming that in so doing you don't reach a different bottleneck in the framework itself. Nevertheless, the delta in this particular test was a big surprise. The Postgres JDBC driver folks deserve a lot of credit for tuning the drivers up so well!
Indeed. Also a very impressive bump from what appears a single library update. Incidentally, I fixed the linked diff file to include the named anchors. We've also fixed the diffs for Preview 3 to correctly show the "20-iteration" versions of the multi-query and updates tests.
This would be terrific. Certainly the momentum in Postgres has shown no sign of slowing; each new build brings a lot of exciting improvements.
This is a good question. We restart the database services between each test implementation so that Framework A will not affect Framework B, but we do not restart database services between each concurrency level. So if a database deadlocks at a low concurrency level, 0-RPS measurements would be expected at the subsequent higher concurrency levels. For the time being, we don't have any countermeasures in place for that, and restarting after each concurrency level would likely add a lot more wall-time to the full run.
And thank you for your massive contributions and insights! Your participation is greatly appreciated. |
That's perfectly fine. The change in the diff HTML report makes the problem visible if it happens. The previous version was giving impression that everything is OK. |
@zloster , is 10-15% based on your own measurements? What I mean is if current pgjdbc logging approach is bad, we need to figure out the conditions that trigger high overhead. |
@vlsi No, it is not my measurement.
The second commit is reverting from 42.0.0 to 9.4.1212. This implementation is relatively new in TBF and the maintainer is quite active. Note that the project implementation is having some weird results during the Round 14 previews (preview 4 is the current one). So I can't be sure what is the reason to revert the driver version. I'll try to get some test results myself this weekend to check for difference between 42.0.0 and 9.4.1212 and report back in the mailing list. |
@zloster , did you had a chance to review the benchmark? |
@vlsi I've replied on the pgsql-jdbc mailing list a long time ago: https://www.postgresql.org/message-id/d5d39fffac03727934f262c793899321%40edno.moe |
The other Java projects that are using
JDBC
are onpostgresql-9.4.1208.jar
. So it will be good andGemini
to use it as well.