{"id":2717,"date":"2012-06-21T22:11:45","date_gmt":"2012-06-21T21:11:45","guid":{"rendered":"http:\/\/www.gamlor.info\/wordpress\/?p=2717"},"modified":"2021-07-03T13:31:49","modified_gmt":"2021-07-03T12:31:49","slug":"some-performance-numbers-with-akka-async-sql","status":"publish","type":"post","link":"https:\/\/www.gamlor.info\/wordpress\/2012\/06\/some-performance-numbers-with-akka-async-sql\/","title":{"rendered":"Some Performance Numbers With Akka Async SQL"},"content":{"rendered":"<p>A while back I wrote about my small <a href=\"https:\/\/www.gamlor.info\/wordpress\/2012\/05\/async-sql-and-akka\/\">Async SQL library for Akka and its asynchronous MySQL driver<\/a>. Besides a few tiny fixes I&#8217;ve added nothing new recently.<br \/>\nHowever I want to share a few number from my performance experiments.<\/p>\n<div id=\"attachment_2735\" style=\"width: 310px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/adbcj-vs-jdbc.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2735\" src=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/adbcj-vs-jdbc-300x147.png\" alt=\"Racing Stuff\" title=\"adbcj-vs-jdbc\" class=\"size-medium wp-image-2735\" width=\"300\" height=\"147\" srcset=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/adbcj-vs-jdbc-300x147.png 300w, https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/adbcj-vs-jdbc.png 811w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-2735\" class=\"wp-caption-text\">Racing Stuff<\/p><\/div>\n<h2>Good Benchmarking, Science for itself<\/h2>\n<p>Good benchmarking is a small science for itself. Creating a balanced benchmark, ensure equal conditions for multiple runs, cranking out usable numbers, do proper analysis etc. I didn&#8217;t do that. I just wrote some code, ran that stuff and compared the numbers.<br \/>\nIn the end it heavily depends on your scenario anyway. For example when you run trivial queries against a local database, then JDBC will probably win, since it only blocks a thread very briefly. If you add web-like latencies to the database communication, then my ADBCJ solution has an huge advantage.<\/p>\n<h2>The Benchmark Scenario<\/h2>\n<p>This is basically the benchmark scenario which I measured. It simulates a web like application.<\/p>\n<ul>\n<li>A request comes in<\/li>\n<li>We query for the most important data. Also we query for some less important data. The more important data is returned asynchronously to the client first, while the other queries are still running. This should simulate a page, which first shows the important stuff and loads the rest in the back ground, like many websites do.<\/li>\n<li>When the first part of the result arrives at the client, another request is sent, which updates some data<\/li>\n<li>When all queries, updates and operations complete, we&#8217;re done. We measure the time between issuing the request and completing everything<\/li>\n<\/ul>\n<p>For the plain JDBC implementation I used the <a href=\"http:\/\/doc.akka.io\/docs\/akka\/2.0.2\/scala\/futures.html\">Akka future API<\/a>, to run that work in the background and continue with the other operations. The ADBCJ implementation relies on its asynchronous API.<br \/>\nFor these number I ran the database locally, so there is no huge round trip overhead. Only the queries need a bit of time to complete, from sub milliseconds up the around 6 milliseconds.<\/p>\n<h2>Result, JDBC vs ADBCJ<\/h2>\n<p>Well ADBCJ is as expected faster for this task. Mainly because it can send new queries, while processing the in streaming result of previous queries. Maybe the JDBC implementation also ends up exhausting the thread pool with blocked threads, but I didn&#8217;t check it if that&#8217;s the case:<\/p>\n<table width=\"40%\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\">\n<tbody>\n<tr>\n<td><\/td>\n<td>JDBC<\/td>\n<td>ADBCJ<\/td>\n<\/tr>\n<tr>\n<td>Min<\/td>\n<td>36.00ms<\/td>\n<td>20.00ms<\/td>\n<\/tr>\n<tr>\n<td>Median<\/td>\n<td>47.00ms<\/td>\n<td>32.00ms<\/td>\n<\/tr>\n<tr>\n<td>Mean<\/td>\n<td>66.26ms<\/td>\n<td>41.25ms<\/td>\n<\/tr>\n<tr>\n<td>Max.<\/td>\n<td>222.00ms<\/td>\n<td>155.00ms<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Here&#8217;s also a box plot of the data. There is quite a high variance in the data, and a few latency spikes:<\/p>\n<div id=\"attachment_2724\" style=\"width: 310px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/JDBC-vs-ADBJC.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2724\" src=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/JDBC-vs-ADBJC-300x225.png\" alt=\"JDBC vs ADBCJ\" title=\"JDBC vs ADBCJ\" class=\"size-medium wp-image-2724\" width=\"300\" height=\"225\" srcset=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/JDBC-vs-ADBJC-300x225.png 300w, https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/JDBC-vs-ADBJC.png 800w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-2724\" class=\"wp-caption-text\">JDBC vs ADBCJ<\/p><\/div>\n<h2>JDBC with Connection Pool, ADBCJ via JDBC bridge<\/h2>\n<p>Now this is a little unfair, since you usually use a connection pool for JDBC. And we didn&#8217;t check out the ADBCJ via JDBC fallback driver. So I ran the benchmark as well with the <a href=\"http:\/\/jolbox.com\/\">BoneCP connection pool<\/a> and ADBCJ via JDBC.<br \/>\nGuess what, JDBC with a connection pool reacts the fastest. This means that a connection pool has a huge effect in this benchmark, since it opens and closes database connection often. The conclusion is that ADBCJ desperately needs its own connect pool implementation, to handle similar scenarios.<br \/>\nAnother interesting fact is that ADBCJ via JDBC performs the worst here. It\u2019s clear that it is probably a bit slower the JDBC version, because I manually split the JDBC calls reasonably across threads. The ADBCJ to JDBC bridge has no insight how to smartly dispatch the blocking calls. However the huge difference between JDBC and ADBCJ via JDBC is surprising. This could indicate that there are some implementation issues present in the ADBCJ to JDBC bridge.<\/p>\n<table width=\"60%\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\">\n<tbody>\n<tr>\n<td><\/td>\n<td>JDBC with Pool<\/td>\n<td>ADBCJ<\/td>\n<td>JDBC<\/td>\n<td>ADBCJ via JDBC<\/td>\n<\/tr>\n<tr>\n<td>Min<\/td>\n<td>17.00ms<\/td>\n<td>20.00ms<\/td>\n<td>36.00ms<\/td>\n<td>40.00ms<\/td>\n<\/tr>\n<tr>\n<td>Median<\/td>\n<td>30.00ms<\/td>\n<td>32.00ms<\/td>\n<td>47.00ms<\/td>\n<td>72.00ms<\/td>\n<\/tr>\n<tr>\n<td>Mean<\/td>\n<td>41.37ms<\/td>\n<td>41.25ms<\/td>\n<td>66.26ms<\/td>\n<td>85.35ms<\/td>\n<\/tr>\n<tr>\n<td>Max.<\/td>\n<td>156.00ms<\/td>\n<td>155.00ms<\/td>\n<td>222.00ms<\/td>\n<td>197.00ms<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Again a box plot of this data. Again high variance and quite a few spikes:<\/p>\n<div id=\"attachment_2727\" style=\"width: 310px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/withJDBCPool.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2727\" src=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/withJDBCPool-300x225.png\" alt=\"JDBC Connection Pool, ADBCJ via JDBC\" title=\"withJDBCPool\" class=\"size-medium wp-image-2727\" width=\"300\" height=\"225\" srcset=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/withJDBCPool-300x225.png 300w, https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/withJDBCPool.png 800w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-2727\" class=\"wp-caption-text\">JDBC Connection Pool, ADBCJ via JDBC<\/p><\/div>\n<h2>High Load Performance<\/h2>\n<p>Well, I wanted to see if the picture doesn&#8217;t change if the load is much higher. So I cranked the executed request up, so that the machine is fairly busy. The database is still running locally. The picture doesn&#8217;t change much, maybe there is a tendency that the performance advantage of ADBCJ gets bigger. But I didn&#8217;t to more scaling testing to verify that.<br \/>\nAlso the ADBCJ to JDBC bridge crashed, because it just issued way to many connections. Well, as mentioned, the ADBCJ to JDBC bridge is more testing facility anyway.<\/p>\n<table width=\"40%\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\">\n<tbody>\n<tr>\n<td><\/td>\n<td>JDBC<\/td>\n<td>ADBCJ<\/td>\n<\/tr>\n<tr>\n<td>Min<\/td>\n<td>41.00ms<\/td>\n<td>20.00ms<\/td>\n<\/tr>\n<tr>\n<td>Median<\/td>\n<td>142.00ms<\/td>\n<td>59.50ms<\/td>\n<\/tr>\n<tr>\n<td>Mean<\/td>\n<td>160.22ms<\/td>\n<td>77.43ms<\/td>\n<\/tr>\n<tr>\n<td>Max.<\/td>\n<td>604.00ms<\/td>\n<td>298.00ms<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>And again in a box plot:<\/p>\n<div id=\"attachment_2729\" style=\"width: 310px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/HighLoad-JDBC-vs-ADBCJ.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2729\" src=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/HighLoad-JDBC-vs-ADBCJ-300x225.png\" alt=\"Higher Load: JDBC vs ADBCJ\" title=\"HighLoad-JDBC-vs-ADBCJ\" class=\"size-medium wp-image-2729\" width=\"300\" height=\"225\" srcset=\"https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/HighLoad-JDBC-vs-ADBCJ-300x225.png 300w, https:\/\/www.gamlor.info\/wordpress\/wp-content\/uploads\/2012\/06\/HighLoad-JDBC-vs-ADBCJ.png 800w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-2729\" class=\"wp-caption-text\">Higher Load: JDBC vs ADBCJ<\/p><\/div>\n<h2>Conclusion<\/h2>\n<p>Well, don&#8217;t draw to many conclusion from these numbers. We certainly can see that ADBCJ can be faster for some scenarios. We also can clearly  see, that the missing connection pool for ADBCJ is a huge performance penalty for many applications.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A while back I wrote about my small Async SQL library for Akka and its asynchronous MySQL driver. Besides a few tiny fixes I&#8217;ve added nothing new recently. However I&hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[268,15,187],"tags":[245,162,295,226],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts\/2717"}],"collection":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/comments?post=2717"}],"version-history":[{"count":28,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts\/2717\/revisions"}],"predecessor-version":[{"id":3889,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts\/2717\/revisions\/3889"}],"wp:attachment":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/media?parent=2717"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/categories?post=2717"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/tags?post=2717"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}