More Real World Performance Data

With the new release of JRuby, I’decided to rerun my LogWatchR benchmark for ruby 1.8.5p12, ruby 1.9.0, and JRuby 0.9.9. Neither XRuby nor rubinius can run the YAML library from the Standard Library yet, so neither of them will be included in these benchmarks yet.

In running this set of tests, I ran into some anomolies with last weeks data. I’ve figured it out now, and need to make a quick apology. The 1.8.5 results last week were run with my laptop on battery power (and thus stepped down), while the 1.9 tests were run while it was connected to regular power. This was a bigger cause of the variation in times than improvements in 1.9. I’m reporting both sets of data this week, but will be excluding battery driven runs starting with the next report.

There was a note on programming.reddit.com asking me to publish LogWatchR so that other people could repeat my benchmarking. I will be releasing LogWatchR at some point (hopefully soon), but I’d rather see people run their own applications for each of the implementations. If it won’t work now, work with the implmentors—in every case I’ve seen, they’re happy to help and interested in your results. So, please, give your application a try and let us know how it looks!

Without further ado then, here are the results:

On Battery Power
Ruby Implementation	Time
ruby 1.8.5 (2006-12-25 patchlevel 12) [i686-linux]	31.0 +/ 3.1
ruby 1.9.0 (2007-04-18 patchlevel 0) [i686-linux]	31.1 +/ 3.27
ruby 1.8.5 (2007-04-23 rev 3539) [i386-jruby0.9.9]	67.5 +/ 7.15
On Regular Power
ruby 1.8.5 (2006-12-25 patchlevel 12) [i686-linux]	12.0 +/ 1.2
ruby 1.9.0 (2007-04-18 patchlevel 0) [i686-linux]	12.4 +/ 1.8
ruby 1.8.5 (2007-04-23 rev 3539) [i386-jruby0.9.9]	33.3 +/ 6.83

Update!

The JRuby guys pointed me in the right direction to squeeze some extra improvment out of their platform. Instead of running jruby logwatcher.rb < 20_minute_log, I can run JAVA_OPTS=”-Djruby.objectspace.enabled=false -Djruby.jit.enabled=true”; jruby -C logwatcher.rb < 20_minute_log and take advantage of their compiler and JIT work. Following their advice I got the following results (on regular power):

Ruby Implementation	Time
ruby 1.8.5 (2007-04-23 rev 3539) [i386-jruby0.9.9]	26.7 +/ 3.79

These results are about a 20% improvement. That’s a good argument for really knowing your platform. JRuby still isn’t competing with Ruby’s speed, but it handles 20 minutes worth of log entries in about 27 seconds so I’d have to say it’s fast enough for what I’m doing. (And, I have this sneaking suspicion that the JRuby team has some more optimization work in their plans.)

If you enjoyed this article on profiling, you might also want to look at these:

• Some Real World Performance Notes
• JRuby 1.0.1 Real World Performance Notes
• JRuby 1.1 Performance Numbers