Monday, July 31, 2006

OSCon Take-aways

I'm just starting my first work week after OSCon, and as I ride the bus in to work, I'm trying to put some order to my thoughts after the conference. There were a grundle of great talks and tutorials, any number of important hallway (or lunch, or whatever) conversations, a bunch of interesting blog posts from other attendees, and (of course) FOSCon. Beyond the things I expected to learn (like more about DSLs in Neal Ford's talk), I took away a number of themes that will play out in my activities over the next several months (and some future blog posts). Here's my list:

Ruby Community

  • The Ruby community is both really big and really helpful — even more than I thought.
  • The Rails community really is a part of the Ruby community — despite my fears to the contrary.
  • Starting & Supporting local/regional Ruby communities is key to the ongoing success of the greater Ruby community.

Conferences

  • It's the people/social setting that makes a conference really great.
  • A wider range of opportunities at the conference is important (to a point).
  • Organizing conferences is hard — thanks to ORA & Ruby Central for their work on OSCon, RubyConf, and RailsConf.

Publishing

  • Publishers can be leading or lagging indicators in a community.
  • Communities ned to communicate with publishers to drive the creation of the content we want/need.

Personal

  • I have to try harder with my own LOTY (Language Of The Year) — I want to hit Javascript, C, Haskell, and Smalltalk over the next several years.
  • I know more about Ruby than I thought I did, but less about Ruby than I want to.
  • More people read this blog than I though — I need to write more worthwhile stuff and less drivel.

An Early Announcement

Just a quick post to keep myself honest. I've heard that telling people about a goal you've set, and committing to keep them up to date is one of the best ways to keep yourself on target for that goal. I've tried it with some other things, and it's seemed to help. So without further ado, my announcement.

I'm going to try to lose fifteen pounds — oops, wrong goal. I'm writing a book. I've signed a contract with the Pragmatic Programmers and have just started writing. I'm not ready to start tossing around titles or topics yet, other than that it's about Ruby. Let me get a bigger chunk written, then I'll be able to tell you more.

I'm hoping to write about 50 pages each month, and will drop a progress report here on my blog every Friday. If I can keep to this schedule, I should be able to make a bigger announcement in the fall.

Your job, should you choose to accept it, is to gently pester me about my progress. Feel free to drop the occasional email, or to make comments here. (Oh, and if you want to bug me about the weight thing, that probably wouldn't hurt either.)

Saturday, July 29, 2006

OSCon, It's the all about the people

I loved being at OSCon and catching up with people from my past and meeting new folks. Tuesday night, I invited a bunch of folks for an impromptu dinner between the last session and the start of the evening activities. We ended up with: Andy Hunt, Andy Lester, Dave Thomas, David Black, Glenn Vandenburg, Jim Weirich, John Labovitz, Lennon Day-Reynolds, Mark Conway, Mike Clark, Pat Eyler, Phil Tomson, and Audrey Eschright. (Sadly, Avi Bryant, Ingy Dot Net, and Stas Bekman couldn't make it.) There were a lot of great conversations floating around an outdoor table at the Produce Row Cafe. Where else could you gather a group like that?

What's up with RWB?

I didn't have any luck finding a new owner for RWB, but it turned out that something even better was about to happen. There were a lot of things I'd wanted to do with RWB to make it really useful. I just didn't have the time to sit down and get them done. Since no one was interested in taking it over, I've been trying to make the decision between letting it die (not my favorite option) or finding time to get stuff done on it (not really pleasant either).

I'd basically decided to cut back on a couple of other things so that I could push RWB ahead slowly, when Zed announced RFuzz. there seemed to be a lot of overlap, so I dropped Zed a line seeing if he were interested in harvesting the good parts (ideas mostly) from RWB for use in RFuzz.

This merger turned out to align quite well with what Zed was hoping to do with RFuzz anyway, so RWB's future is set — it has none. RFuzz will be the project to use if you want to do capacity, load, or fuzz testing of web applications with Ruby.

I hope to contribute to RFuzz (even if only a little bit). Even more though, I want to see RFuzz run with the ideasI had for RWB and become the kind of web server/webproxy testing tool the community really needs. Once again, I'm left saying: "Go Zed!"

RWB is dead! Long live RFuzz!

Thursday, July 27, 2006

RubyInline: Going a bit Further

Last time around, I wrote about RubyInline and profiling, and raised a couple of questions. This time around I'm going to talk about RubyInline and benchmark in a (probably vain) attempt to answer the existing questions, while raising new ones. There were basically three questions that I'd like to handle:

  • What should we call this stuff?
  • How does it work?
  • How do I use Ruby objects in my C? (Yes, I'm conflating two questions here.)

The simple answer to the first is that yes, we're really just writing Ruby methods (albeit in C). To really understand why and how that is, we need to look at what RubyInline actually does.

RubyInline does a couple of things in the process of working (if I get any of this wrong, hopefully Eric Hodel or zenspider will jump in and correct me). The first time it's run, it copies the C source into a temporary directory (~/.ruby_inline) and tweaks it a bit to make it Ruby-aware. It then compiles the C to a shared library (a .so in my case) which is linked into the running Ruby script, and any methods are made available for calling.

If RubyInline is run again against the same script, it checks to see if the shared library is newer than the original source file and only recompiles if the source is newer than the object file. This helps cut the compile and link latency that would be introduced otherwise.

The idea of Ruby methods written in C might seem foreign, but you actually use them all the time. Array#uniq is just a bunch of C, but you'd never know it without digging around in the Ruby source (not recommended for the faint of heart). RubyInline just makes the process of writing Ruby in C a bit easier (well, a Lot easier).

Let's take a look at this in practice. Since my last post raised some questions about using Ruby objects (Fixnums and Arrays) from C, so I borrowed some code from example2.rb in the source, and worked from there. Here's the code:


#!/usr/bin/ruby -w 

require 'rubygems'
require 'inline'
require 'benchmark'

class Array

  # build the Array#average method in C using Ruby Objects
  inline do |builder|
    builder.c_raw "
      static VALUE average(int argc, VALUE *argv, VALUE self) {
        double result;
        long  i, len;
        VALUE *arr = RARRAY(self)->ptr;
        len = RARRAY(self)->len;
        
        for(i=0; i<len; i++) {
          result += NUM2DBL(arr[i]);
        }
        return rb_float_new(result/(double)len);
      }
    "
  end

  # build the Array#ravg method in Ruby
  def ravg
    Float(self.inject {|sum, elem| sum += elem }) / Float(self.length)
  end

end

# build a good sized loop over a big array
max_loop = (ARGV.shift || 20).to_i
max_size = (ARGV.shift || 1_000_000).to_i
a = (1..max_size).to_a

# benchmark the C versus the Ruby versions
Benchmark.bmbm(10) do |x|
  x.report("C")    { for i in 1..max_loop; a.average; end   }
  x.report("Ruby") { for i in 1..max_loop; a.ravg;    end   }
end

Other than showing the use of Ruby Array objects from C, the only interesting thing is the benchmarking code (the last block in the file). Even this is pretty simple stuff though, it reads something like this:

Run the bmbm method (which runs a benchmarking rehearsal and benchmarking test), with the report headers left justified in a 10 character block. The first blob to benchmark is given the header 'C', and runs a loop averaging the big array once each time through the loop. The second blob will be labeled 'Ruby' and will benchmark the ruby average method.

The output of running this script looks like this:


$ ruby inline_array_benchmark.rb
Rehearsal ---------------------------------------------
C          27.790000   0.110000  27.900000 ( 29.643141)
Ruby      115.280000   0.280000 115.560000 (119.626154)
---------------------------------- total: 143.460000sec

                user     system      total        real
C          27.680000   0.030000  27.710000 ( 28.382200)
Ruby      116.040000   0.340000 116.380000 (120.795489)

It's interesting to note that the rehearsal takes a bit longer for the C version. It's tempting to say that's because it compiled the C code, but that's not really the cause. The compile time takes so little time, it's lost in the noise.

Hopefully this helps answer some of the questions that have come up. If not, feel free to keep asking — I'll keep trying to answer them.