Tuesday, January 08, 2008

JRuby 1.1RC1 Real World Performance Update

With the release of JRuby 1.1RC1 I’ve run a new set of LogWatchR performance tests. This time, I’ve only run a set of 1.0 and 1.1 versions of JRuby. If you’re really interested in seeing more about 1.8 and 1.9 performance, you can always go look at my older post on the topic.

As a bit of clarification up front, this test measures the execution time of a simple minded log analysis tool I wrote/use against a 20 minute sample of syslog output. Contrary to the common assumption, this is not an IO bound process (see here for further details). I’m running this test with the following flags through 1.1b1:

JAVA_OPTS="-Djruby.objectspace.enabled=false \
  -Djruby.jit.enabled=true -server" 
jruby -C

Version 1.1RC1 says that jruby.jit.enabled=true is no longer valid, so I’m usingthis instead:

JAVA_OPTS="-Djruby.objectspace.enabled=false \
  -Djruby.compile.mode=JIT -server" 
jruby -C
Build Avg Execution
Time (in secs) 		Std Dev
Jruby 1.0.2 21.03 0.69
Jruby 1.0.3 21.22 0.39
Jruby 1.1b1 18.8 0.95
Jruby 1.1RC1 15.41 0.22

It looks like the JRuby team is continuing to do great things with performance. This build is a 27% improvement over the 1.0.2 release and an 18% improvement over the first 1.1 beta.

I’m not a JRuby or Java performance geek, so if anyone on the team would like to lend me some performance enhancing clues I’d be happy rerun the tests.

UPDATE: Based on Charles comments below, I've now tried this with:

  • +C, which compiles every method
  • -C, which turns off compilation
  • the default compilation (no C switch at all) which will compile any method called more than 20 times
Since I'm making fairly small runs, any compilation is a pretty big hit to my run times (e.g., +C takes 25 seconds to complete on average). Maybe this weekend will be a good time to try some much longer runs.

4 comments:

  1. Charles Oliver Nutter1/8/08, 2:44 PM

    A few things:

    1. -C actually disables compilation now... "minus Compilation". Just use the default setting (no compile flags).
    2. ObjectSpace is off by default, so no need to specify that property.

    There's other settings, but they may not be relevant to such a short run.

    ReplyDelete
  2. ronen1/8/08, 2:57 PM

    Looks pretty cool, i wonder how much improvement will see with the JRuby compiler :)

    ReplyDelete
  3. gnupate1/8/08, 3:18 PM

    Charles,
    thanks for letting me know, I'll rerun this tonight with out the extra flags and post an update.

    ReplyDelete
  4. Unknown1/11/08, 8:17 PM

    I'm wondering what tweaking some of the generation and gc parameters might do for performance. I was thinking of something like:

    JAVA_OPTS="-Xms=8m -Xmx16m -XX:+UseConcMarkSweepGC -XX:NewSize=8m -XX:-Djruby.objectspace.enabled=false
    -Djruby.compile.mode=JIT -client"

    I was also thinking that for this benchmark, the client JIT would probably give better performance since it JITs code much sooner than the server JIT.

    ReplyDelete