Some Real World Performance notes on Ruby 1.8 and 1.9

Update:

The performance numbers for Ruby 1.8.5 (below) are incorrect due to a mistake on my part. For updated in formation, please see my newer performance post.

p>One of the knocks against Antonio's Ruby Performance Shootout is that it uses a synthetic set of tests (the one's Ko1 wrote to exercise the parts of YARV he'd already done some optimization on). In an effort to get a 'more real' set of numbers (at least for me), I decided to use my LogWatchR tool to do some benchmarking.

LogWatchR is a program to watch syslog (we aggregate several hundred boxes' syslog entries to a pair of servers at my $JOB). I wrote it to use catalogs of good and bad entries (with dependencies, number of occurence thresholds, etc.) written in YAML and stored in external files, so that even a Junior Sys Admin could add new catalog entries pretty easily. To keep myself honest, I have a 20 minute sample of log entries and a simple pair of catalogs I use:

• 73607 log entries in the sample
• 1 entry in the good patterns catalog
• 5 entries in the bad patterns catalog

I can use this to make sure LogWatchR is performant, and works as expected.

In order to time LogWatchR, I ran the 20 minute sample through the program 5 times on each implementation of Ruby. Here are the results I got:

version	running time (in seconds)
ruby 1.8.5 (2006-12-25 patchlevel 12)	34.829 +/- .287
ruby 1.9.0 (2007-04-18 patchlevel 0)	12.967 +/- .095

At this point, all I can say is I'm really impressed with the work that the 1.9 team has done to speed Ruby up. Anytime you can get a better than 2.5x performance boost, you've gotta be doing something right.

JRuby 0.9.8 and rubinius don't work yet. They both have YAML related failures. I've talked with Evan about rubinius, and he says he's hoping that YAML will be working soon. I've also sent in a bug report to Ola, and hope that we'll see a fix in 0.9.9.

I plan on rerunning this benchmark periodically, so hopefully we'll see JRuby and rubinius included in the results soon.

If you enjoyed this article on profiling, you might also want to look at these:

• More Real World Performance Notes
• JRuby 1.0.1 Real World Performance Notes
• JRuby 1.1 Performance Numbers