Thursday, March 24, 2011

Ruby and Protocol Buffers, Take One

At work, we're moving from XML to protocol buffers.  While we're mostly a Java shop, the operations/sysadmin team I'm on does a lot of Ruby. I was interested in how we might use the same technology for some of our stuff. After a bit of looking, I found two libraries that looked mature enough to investigate:

ruby-protobuf, by MATSUYAMA Kengo (@macks_jp), was straightforward to install and use.  It has a good online tutorial and the redme has all I needed to get started.
ruby-protocol-buffers, by Brian Palmer was also easy to install and use.  It seems a bit lacking in the online documentation, but does have some examples to follow.  (If Brian's name rings a bell, it might be because I interviewed him some time ago about winning a programming contest sponsored by Mozy's former incarnation.)
I started out with a very simple proto file:

package bench;

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;
}

I compiled this with rprotoc for ruby-protobuf and with ruby-protoc for ruby-protocol-buffers. This generated the following (which I edited lightly).  For ruby-protof:

### Generated by rprotoc. DO NOT EDIT!
### <proto file: bench.proto>
# package bench;
#
# message Person {
#   required string name = 1;
#   required int32 id = 2;
#   optional string email = 3;
# }
require 'protobuf/message/message'
require 'protobuf/message/enum'
require 'protobuf/message/service'
require 'protobuf/message/extend'

module Bench1
  class Person1 < ::Protobuf::Message
    defined_in __FILE__
    required :string, :name, 1
    required :int32, :id, 2
    optional :string, :email, 3
  end
end
for ruby-protocol-buffer
#!/usr/bin/env ruby
# Generated by the protocol buffer compiler. DO NOT EDIT!

require 'protocol_buffers'

# Reload support
Object.__send__(:remove_const, :Bench2) if defined?(Bench2)

module Bench2
  # forward declarations
  class Person2 < ::ProtocolBuffers::Message; end

  class Person2 < ::ProtocolBuffers::Message
    required :string, :name, 1
    required :int32, :id, 2
    optional :string, :email, 3

    gen_methods! # new fields ignored after this point
  end
end

Then I pulled out the statistical benchmarking I wrote about a while ago (Since no one else has taken the bait, maybe I should bundle up a gem for that.).  Instead of quoting the whole thing at you, here are the pertinent loops.  For ruby-protobuf:

msg = Bench1::Person1.new(:name => idx.to_s,
                          :id => idx,
                          :email => idx.to_s)
msg_str = msg.serialize_to_string
msg == msg.parse_from_string(msg_str)

for ruby-protocol-buffer:

msg = Bench2::Person2.new(:name => idx.to_s,
                          :id => idx,
                          :email => idx.to_s)
 msg == Bench2::Person2.parse(msg.to_s)
And here are the results:

$ rvm 1.9.2
$ ruby -v
ruby 1.9.2p0 (2010-08-18 revision 29036) [i686-linux]
$ time ruby ProtoBufBench
testing ruby-protobuff against ruby-protocol-buffer
The deviation in the deltas was 0.021731
The mean delta was 0.198301
max = 0.241761921640842 :: min = 0.154839326146633
ruby-protocol-buffer was better

real 1m50.672s
user 1m50.599s
sys 0m0.092s

$ rvm 1.8.7
$ ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-linux]
$ time ruby ProtoBufBench
testing ruby-protobuff against ruby-protocol-buffer
The deviation in the deltas was 0.009414
The mean delta was -2.205984
max = -2.18715483485056 :: min = -2.22481263341116
There's no statistical difference

real 3m8.131s
user 3m7.996s
sys 0m0.056s

I didn't try compiling the c extension for ruby-protocol-buffers, and I haven't tried any more involved .proto files yet.  I'll work on those in the next couple of days and post results as I see them.

Wednesday, March 23, 2011

Review - Eloquent Ruby

The system management/administration team that I work on is starting to do more scripting and tool building.  That means bringing a bunch of people up to speed on Ruby.  We're using a combination of the Pickaxe Book and pair programming/mentoring to help bootstrap people.  So far it's been working pretty well.
Watching everyone else reading and learning made me want to get in on the action.  Fortunately, there's a new book out from Russ Olsen (@russolsen) — Eloquent Ruby.  I had the opportunity to interview Russ (look for it to show up soon) about his book, and Addison-Wesley was kind enough to send me a copy of it.
Eloquent Ruby is primarily aimed at people coming to Ruby from other languages.  It aims to explore and explain the idioms common to our community, and I think it does a great job of it.
It will serve current rubyists well too.  I learned several things from it as I read, and cemented other concepts as well.  Some of the book's explanations have crept their way into discussions about Ruby here at work.  It's good stuff.
Section One, on basics, is a rich source of little gems that will help you use Ruby's built in classes more effectively.  Section Two, on modules and classes, helps you build better classes of your own.  Section Three, which covers meta-programming, dives into an oft discussed but underused side of Ruby to push your Ruby-fu to the limit.  Section Four covers a variety of things that either don't fit into the other sections or build on concepts from them.
With this book, Russ has hit another one out of the park.  Go grab a copy for yourself.
If you're interested inDesign Patterns in Ruby  you can read my review here.  You can also read a previous interview with Russ.

Wednesday, September 08, 2010

Reading List Update 9/8/2010


The recent news that GDB now supports D makes The D Programming Language jump up a notch or two on my reading list.
I've finished 52 Loaves: One Man's Relentless Pursuit of Truth, Meaning, and a Perfect Crust, it was a fun read.  I really identified with his trip to the French monastery. It seemed like a great climax to his year, with the perfect denouement as he came home to bake his final loaves.
With a chance to get involved with a local restaurant group (not behind the counter though), the food books are still winning out in my what to read next decisions.

Tuesday, August 31, 2010

My Reading List on 8/31/2010


Thanks to Prentice Hall and Addison-Weseley giving me three new books, my reading list has bulked back up.  Here's what I'm working through at the moment:
What are you reading? Why?

Wednesday, August 25, 2010

Ruby|Web Interview with Pat Maddox


Ok, if I'm going to post about GoGaRuCo today, I should also spend some time on Ruby|Web, the latest regional conference from Mike Moore (@blowmage) and friends — truth in advertising, I'm a volunteer on the board for Ruby|Web, so I might be a bit biased.
Just so my biases don't show too much, I asked Pat Maddox(@patmaddox) to answer a few questions for me.  Of course, he's a speaker at Ruby|Web, so the spin is probably still there.
Regardless of our mind-set, this looks like it's going to be an awesome conference.  The only problem is that you need to register before Sep 3rd.  You don't have much time ... maybe you should go register first, then come back and read what Pat has to say.

Ruby|Web is a new name in the regional Ruby conference space.  What drew you to it?
Pat I love coding in Ruby, and I think the web is a great platform to develop for.  I'm eager to spend a few days rubbing elbows with like-minded people.  And between the over all theme, the fantastic organizers running the show, and Snowbird (!!), Ruby|Web jumped to the top of my list.  Plus I don't know anything about HTML 5 or CSS 3 and I need to get on that :)
Seaside is interesting technology, how did you discover it?
Pat Seaside is a fascinating and fun technology!!  I came across it a few years ago, not long after I got into Rails.  Over the years I had a couple of false starts with it...it's a bit opaque at first because the development environment is so different from anything I'm used to.  And it's only fairly recently with Pharo that it's become easier to get started, because it's such a clean environment geared towards development.  Also the documentation for both Pharo and Seaside are getting really good.  There are free books on each at pharobyexample.org and book.seaside.st/book.
Okay as for what's so interesting to me about Seaside... it's 50% the framework and 50% the Pharo environment. Seaside itself represents a step forward in web development similar to how Rails did.  Rails takes care of a lot of the plumbing for you - you don't have to parse query params, set up response headers, manage the session (unless you want to of course).  Seaside does all that of course but also manages application state for you.  So you don't have to worry about putting stuff into a database, then pulling it back out and operating on it.  I can't do it justice in a few sentences, but that's why I'll be showing lots of examples at the conference! :)  At any rate, that same feeling you get when you code Rails for the first time and see how much easier things are, you get that same feeling with Seaside.  It's not a replacement for Rails by any means - Rails definitely has a sweet spot, particularly when it comes to RESTful websites and interoperability with the unix ecosystem - but for the things that Seaside is strong at (which for me so far has been complex and/or configurable workflows), it runs circles around everything else.
The other thing I'm loving about Seaside development is Pharo, an open-source smalltalk environment.  Smalltalk is a great language, and Pharo has great tools that allow you to discover everything in the system.  Honestly it makes RubyMine or etags look plain silly.  The best bit is that nearly everything in Pharo is implemented in smalltalk, including all of the tools.  So if you want to see the mechanics of a refactoring tool, and even build your own, it's trivial to do so, because it's just smalltalk code.
Wow this answer got long.  I could go on all day about this stuff.  Gonna stop now.
What other smalltalk tools/ideas do you think Rubyists should be looking at?
Pat Let's see...I'd love to see Rubyists take the ideas from the Pharo IDE and build some really snazzy development environment for Ruby.  The next killer Ruby app, I think, is going to be a development environment that uses the runtime structure of objects to do all of its magic, rather than just statically analyzing source code.  Even just having portable refactoring tools would be awesome.
I'm also really excited about Maglev, whenever that becomes available for daily use.  It is incredibly liberating to write actual OO code, and so I think my style of coding Rails will change completely once Maglev enters the field.  fingers crossed
Which Ruby|Web presentations are you most looking forward to?

Pat In order of them being listed on the sessions page...
  • BJ Clark's (@robotdeathsquad) talk on HTML / CSS / Javascript.  He told me a few months back when he planned this talk that he thought, "if I were to school Pat on HTML / CSS / Javascript, what would I say?"  He and I have worked together for years and he gets frustrated with my lack of understanding of those things.  So basically this talk is geared specifically to people like me, hardcore backend developers with "div-itis" and who typically use inline javascript and CSS.  I'm looking forward to getting schooled.
  • Alistair Cockburn's (@TotherAlistair) samurai talk.  His talk summary means absolutely nothing to me (on purpose, I'm sure) but he's always a trip to watch speak, and I'm glad to see him get more exposure in the Ruby community.
  • Evan Light's (@elight) iOS talk - Evan is a diverse developer and entrepreneur.  Really excited to learn from his experiences.
  • Joe O'Brien's (@objo) communication talk.  For starters, Joe is one of my favorite people in the Ruby community.  Again, he's one of those folks that combines technical expertise with good business sense and a warm heart.  I think folks attending this conference are going to have more of the entrepreneurial spirit than most, so his talk will be particularly important and insightful for us.
  • Dirt Simple Datamining by Matthew Thorley (@padwasabimasala) - because really, who doesn't love datamining??
It's clearly shaping up to be a rocking conference!!!