New JRuby Interview (part 1)

Don't miss the second part of this interview. This was too long to keep in one piece.

---

We met Charles and Thomas in a previous interview, Ola would you take a moment to introduce yourself?

Ola: I'm a 24 years old systems developer working at Karolinska Institutet in Stockholm, Sweden. I have been coding Java since about '96, but I began my coding carrier with C (in the Scandinavian demo scene). I'm an out-of-the-closet Lisp-lover who tends to find Ruby more pragmatic for most tasks. I got interested in JRuby in the fall of 2005, but didn't begin to pitch in until springtime, when I finally got some free time to do it in. My vision about programming is a platform where you can use the right tools for a task, whatever the tool or task may be. Programmer time is too valuable to emulate closures in Java, hack objects in Perl, search for macros in Ruby or Recursion in classic Fortran.

And how did you discover Ruby?

Ola: I don't remember exactly when or where, but I do know it was when I read Pragmatic Programmer. After that I realized how fed up I was with Java and ASP (which was the major technologies at my workplace at that point). I found the first edition Pick-Axe online, and the rest is history.

Okay, this one's for all of you. How much do you use Ruby in your day to day work and hacking? (How much of that is actually done with JRuby?)

Ola: At this point, after much lobbying, my work is maybe 40% Ruby and the rest Java. I'm trying to increase that figure of course. But on the other hand, I do a fair amount of hacking at my free time, and I tend to prefer Ruby for most things except when I'm working on some of my pet projects involved with getting better languages to the JVM.

My use for JRuby at this moment is mostly confined to making my life as a Java dev easier. I have a jirb session open constantly and try out different paths through the Java API. Interestingly, the one part I most often find myself trying is Java's Regular Expressions library.

I am planning on getting the first Rant/Jake-tasks out the door soon (Rake tasks for Java development), and when I do I'm planning on switching wholesale to JRuby and Rake for my builds.

Thomas: This question may be worth asking again in a few months since I just started working at Sun. At my previous job, I did not get much chance to use JRuby. I did plenty of Java and occasionally I would write a script here or there in JRuby to access/exercise that Java code. I also created small scripts to access databases using JDBC. I also have not had many opportunities to do pure Ruby programming. Between a day job and JRuby hacking after work I did not have much time for pure Ruby development (other than simple data manipulation scripts here and there). I am hoping that changes more now that my day job is JRuby.

Charles: It's a sad truth about working on the JRuby project that up to now we've not really had much chance to write Ruby code. Working deep in the bowels of the interpreter keeps us pretty far removed from day-to-day Ruby work, and until coming to Sun I was a straight-up Java EE architect. I had started to sneak Ruby into the system, however, handling a Bugzilla/CVS/ClearQuest integration layer and starting to write administrative and service management scripts. I'd also been advocating Ruby and Rails to the folks on my team with a little success.

I expect that the Sun move is going to give us more time to focus on Ruby-land tools and code. In particular, there's a lot of work to be done to integrate Ruby apps — including Rails — into a Java world. There are plugins, scripts, and generators to be written, integration scenarios to be tested, and Java-land tools to "Rubify". Now that we can concentrate on JRuby full-time, there should be hours left over to climb outside of the interpreter internals more.

Tom, I'll make a note ... we'll have to try to work in another interview in early 2007. Let's get back to JRuby today. Charles has written that you're starting into some performance work on JRuby. How's it going? What's the process you're using? How are you measuring change?

Charles: Performance work has been a constant throughout my time on the JRuby project. Ruby itself is not known for its blazing speed, and since JRuby was originally just a port of the C code, it suffered from that implementation's issues as well as a number of new ones introduced during the translation. It scored at the bottom of the pack on almost all JVM scripting-language benchmarks and was visibly slow in interactive mode. On top of this, Ruby is a fairly complicated language to implement, and we have never had a bytecode compiler.

In past months, however, we've taken a more radical approach to improving performance. Where the original focus was on correctness above all else, we're now starting to make performance a top priority. As it stands today, JRuby is more correct than it ever has been, and most Ruby scripts now "just run" without modification. Rails is working better every day and odd interpreter failures are fewer and farther between. Our confidence in JRuby working correctly has reached a point where we can start making larger changes to the core interpreter; because we've built a substantial library of tests for core interpreter features, we're in better shape to refactor for performance. Where we used to try to mirror the C code exactly, we now have a much better understanding of what the C code *intends* to do. Because Java code is easily refactored and redesigned, we can take that intent and create more Java-friendly code. Each little step helps performance a bit more, and the current trunk code is already 25% faster than the previous release...while correctness continues to improve.

We're also starting to look at the compiler issue more closely. There are many different levels of compilation we could attempt, and we're going to pull in resources and help from Sun to give us a better idea how to tackle the problem. I've also worked on and off with a partial compiler that's shown 50-75% performance improvements. There's a lot more work to do, but there's now a lot more time in which to do it.

Ola: As Charles reported, progress is good. Really good, actually. I'm tackling performance from my side, by creating extensions in Java instead of Ruby (YAML and ZLib, for example). I'm aiming to continue down this road, taking a look at the Java-integration and see if that can be made faster by porting it to Ruby. I have also taken a sharp look at different parts of the parser (and especially the lexer), to see if we can speed that part of the process up. As of now, there are probably two ways ahead; either switch to lexing bytes instead of chars, or port the lexer into an auto-generated one. I have done a few tests with JFlex and the performance gains seems substantial, but not as good as just lexing on bytes. I usually measure performance pretty ad-hoc. I first find the a specific use case that exercises whatever I want to improve performance on, and then make small changes and see what gives. If the test case is Ruby, I prefer to use the benchmark library, otherwise I write a test case in Java with System.currentTimeMillis() or just use the UNIX time-command.

Thomas: You have probably seen several entries on Charles blog about various performance patches + experiments. All in all JRuby keeps speeding up. We know we still have more room to speed things up because we know we are creating too many stacks/lists during execution. Some amount of this data being managed dynamically represent static relationships. Scoping relationships are one example. Another area for exploitation is our method dispatch call chain. Years of ideas has made it longer and more complex than it should be. We also use reflection for dispatch which adds more time yet. Going over these areas will speed things up quite a bit. All of this is ignoring Charles compiler work which is pretty exciting in itself.

Up until the last 4-5 months the focus has mostly been correctness over performance. We found it more important to have a Ruby implementation capable of running real software than some screaming fast interpreter that is incapable of running anything. Had we been the original project authors of JRuby (none of the original authors are currently part of the project) maybe this goal would have been different. In any case we are where we are and we are now focusing more aggressively on performance.

Your answers make me wonder how much of your performance work might be be portable back into the C Ruby implementation.

Charles: I truly believe that most of the changes and optimizations we're making could be done to C Ruby as well...but only to a certain extent. Much of what we're changing will alter the way methods are invoked, the way variable scope is managed, and the way we utilize the JVM's systems for object, thread, and IO management. While we're keeping the Ruby side of things as correct as possible (and really trying to continue improving correctness as we go) we're mercilessly refactoring and breaking apart subsystems in the JRuby world. This is a luxury we have...since almost all code that runs on JRuby is part of JRuby proper or wholely isolated from it, we can basically rewrite and redesign much of JRuby's internals without breaking third-party code in hard-to-fix ways. Where Matz and others have emphasized keeping C extensions working and interpreter internals consistent, we have no such restriction. So although these optimizations could certainly be applied to Ruby proper, it would likely be at the cost of backward compatibility.

We also have one other advantage in the Java world that Ruby lacks: Java's extensive refactoring capabilities. In the Java world, if we want to rename a method or extract an interface, we can do so at the touch of a button. Such refactoring techniques have allowed us to pull out whole subsystems and reimplement them without breaking any other code in the process. Most Java IDEs provide support for continuous compilation, and many keep a dynamic in-memory model of all project code. Naysayers call the large size and complexity of an Eclipse or a NetBeans bloat, but they fail to realize the twisted thrill to be had from renaming a method with thousands of invocations or migrating an entire subsystem from direct calls to a factory/interface pattern. None of this is easily possible in the C world. Ruby doesn't have an enormous codebase — it's certainly not more than one or two people could understand completely — but the level of footwork required for even simple refactorings is prohibitive. When you combine this with the fact that many or most C extensions call core (potentially internal) Ruby APIs directly, you can start to understand why Matz intends for Ruby 2.0 to break as much as possible. It's simply much more difficult to evolve a C-based project over time, especially one as easy to extend and with as little isolation as Ruby.

(Tom and Ola...chime in if I'm saying anything that's incorrect)

Thomas: I suspect that the C team could use some stuff we are doing. I am not sure I would spend that time right now. Once we are closer to 1.0 status I think our internals will be quite a bit simpler and this may be a good time to look at how we are doing things. At a minimum, it may generate a fresh set of ideas for them. The sharing of ideas is always good for creating a better piece of software.

On the other hand we delegate as much as we can to the JVM. So, for example, we do not need to worry about garbage collection at all. All the memory management stuff C programmers need to manage is no work for us. Our designs may take features like this for granted and by extension may not be easily ported to C.

Ola: Most of the changes we are doing to the implementation makes better user of Javas strong points. This means that changes from our side won't necessarily be good from a MRI [Matz's Ruby Interpreter] point. I do hope that the YARV people will pay attention when we start looking at compilation in earnest. The most important points for all implementations will actually be our drive to get some kind of functional spec for Ruby. For example, the Marshalling code could probably be cleaned up if we just had a spec of it. And we aim to create one, if we can.

Okay Charles, since you brought up refactoring tools — you and Thomas, are supposed to be looking at programmers tools (which most people read as NetBeans). What do you think has been holding back refactoring tools for Ruby? More importantly, what can/will you be doing about it? (Ola, I'd love to hear your thoughts on this too.)

Charles: Refactoring a dynamic-typed language is inherently hard for reasons that are pretty easy to understand. If you can't look at a piece of code and know exactly what type a given variable is, an IDE won't be able to either. Even if you can do so with some difficulty, IDEs are out of luck. This makes the most interesting and useful refactorings monumentally more difficult to implement. If you can't know what type a variable is or what type of object it contains, you can't tell until runtime what operations it actually supports. Ruby's fluid typing and support for metaprogramming complicates things further: sometimes even the list of operations on a known type will change during a given run. The more dynamic and fluid the type system, the more difficult it is to write ahead-of-time refactoring and analysis tools. There's research going on and creative potential solutions in the making, but there's still a long way to go.

I like to think of dynlang refactoring as being in the same class of problems as AI for the game of Go. In Go, the spatial relationships on the board and the broad array of possibilities make long-term strategy much more difficult. For a computer, this causes no end of problems. In general, the best game AIs are the ones that can brute-force a problem by trying more combinations than a human. In Go, where there's such variation, that method quickly becomes intractable. Where humans can infer future results from the current position and get a "feel" for where a game is progressing, computers can only make use of hard, tangible information about the game. When we develop in dynamic languages, we're much better than the computer at feeling our way through the code, inferring types based on parameter names or methods we've seen called, or reading backwards to get a more concrete feel for incoming object types. Perhaps it's why dynlang programming starts to feel so right...it's putting trust and control back in our hands.

You might say that dyntyped languages trust the programmer to do the right thing where static typed languages force the programmer to do the right thing. That trust opens up a world of possibilities, but it also takes some responsibility out of the computer's hands — namely, responsibility for exactly that information that's needed to refactor.

Thomas: I have not had much personal experience with the refactoring issues of Ruby so my thoughts should be taken with a grain of salt. For a more weighty discussion I recommend talking to Jason Morrison or Mirko Stocker. They both have been working towards type-inference/refactoring.

One philosophical barrier I see is that computer scientists like 100% solutions. Refactoring in Ruby is probably always going to be a 90% solution. It's open definitions and detecting callers,etc will only be accurate if you are actually running the code. Which is something you really don't want your IDE/editor to do. I think being only a 90% problem makes it a less desirable or more frustrating problem to tackle.

We have worked well with the RDT team in the past and we hope to continue in that vein. In fact, we really want to help support all IDE/Tools makers in any way that makes sense. Originally, for RDT we added better positioning information in our AST, so that constructs could be highlighted/folded. Next we plan on merging the refactoring changes that Mirko, Thomas, and Lukas did during this summer.

Ola: Refactoring is a great subject, and I really believe Ruby could do incredibly much. I believe this since Ruby isn't really more dynamic than Smalltalk, and the best Refactoring tools have traditionally been developed for Smalltalk. The Refactoring browser was a great tool that I would really see someone port to Ruby. The big difference between Smalltalk and Ruby lies in the fact that Ruby code is file-based, while Smalltalk always has been memory-based. This is a caveat, but not a big one. If I had the time I would love to write an implementation of it for Ruby. (And of course, this would be for Emacs in the first place. =)

Charles: Ola's point about Smalltalk has a lot of weight as well. Smalltalk's refactoring browsers are fully-capable refactoring environments, and they did refactoring before it was even fashionable to do so. However they line up better with Ruby's interactive shell IRB. IRB is an essential tool for any Ruby programmer in the same way that Smalltalk's browser was an essential tool for Smalltalkers. IRB provides a "live" environment where you can query objects, call methods, and inspect types. This could certainly be extended out to an IDE scale, but there's a new class of problems when you go that direction. You need to ensure that you're not causing side-effects as you traverse code. You need to know when metaprogramming bits are coming into play. Perhaps most importantly, you need to be able to roundtrip an in-memory picture of the system back to code. These features are all absent from Ruby and IRB today, so they represent challenges to building a full-featured refactoring Ruby browser. Even then, Ruby's ability to rewrite itself through a wide variety of eval calls makes refactoring with 100% confidence nearly impossible. I believe that some combination of static analysis and online refactoring will be necessary to sufficiently solve the refactoring question.

Are you guys looking at any tools (like FlawFinder) to verify code correctness? MRI is part of the coverity open source scanning (although it doesn't seem to be gaining much traction among developers). Are there other kinds of automated testing, verification, whatever from the Java world that you're using to make JRuby better, faster, stronger?

Ola: Well, this isn't really my domain, but I do know that we have a large amount of test-cases written with a mix of miniunit for Ruby and JUnit for Java. We also use Emma to check test coverage.

Charles: I don't really know what other tools out there might help us, so we're open to suggestions. We've got a lot of known issues to clean up in the code before we start running tools, however; we know there's a lot of cruft and we're actively scrubbing right now.

Thomas: We also want to set up cruise control. Right now cruise control has some issues running though a web proxy (or something like that) that we need to work through. After that is settled we will have a nice generated report and a set of tests to help perform various tasks.

Has Sun's hire of Charles and Thomas (and the publicity around it) increased interest in (and contributions to) JRuby around the 'Net?

Thomas: The hiring sparked a lot of blogging around the net. Some of this translated to new people poking their head on the mailing list. I would speculate that this announcement also helped JRuby/Ruby gain more visibility in the larger Java community. I think most Java developers have heard of Ruby and especially Ruby on Rails. This announcement may influence them to do more than wonder what the big deal is.

I also hope that beyond JRuby/Ruby interest itself, the announcement has also sparked interest in additional languages on the JVM in general. I think Sun hiring us is a strong message that Sun has an interest in fostering additional languages on the JVM. Java is not a one size fits all language and other languages exist that can solve some problems more elegantly than Java.

Charles: I've read just about every blog post that's floated by since the hiring, and they're almost all overwhelmingly positive. Many people commented that they didn't know JRuby existed or had thought it wasn't under development anymore, but that they were very excited to see a major corporation throw some weight behind Ruby. The impact of the hiring was felt throughout the relevant parts of the blogosphere.

Two weeks later I delivered a presentation at RailsConf Europe called "JRuby on Rails", and by then just about everyone knew about the hiring and offered congratulations and best wishes for the future of JRuby. Almost all initial concerns folks had have been put to rest: we're not being shuffled off to NetBeans or other work immediately, we're not going to create JRuby 1.0 Special Edition 4 Release 8 that's incompatible with MRI, and we're not planning to push Sun toward Ruby to the exclusion of all other dynlangs. People are now getting past the little initial FUD and really seeing the vast potential of both Ruby and Rails running in the JVM alongside existing Java code. Contributions have started to tick upwards as we funnel people toward the mailing lists and bug repository, and I expect them to continue climbing as more folks get involved.

Are you seeing interest from internal Sun resources?

Charles: Folks from Sun have been extremely interested in helping, to such a point that we've had to say "we'll get back to you" in a number of cases. Individual contributors keep coming out of the woodwork and most of the major Java engineering teams have stepped up to offer whatever help they can. In a sense we just got lucky; Tom and I are co-leads of "just another Java interpreter" but that interpreter happens to be for the most buzz-worthy language today at exactly the time Sun has decided to make good on their "multilingual JVM" promises. Many folks internal to Sun have known for years that dynlangs are the wave of the future, and there's a lot of excitement internally about the future of Ruby and other languages on the JVM. Now those folks are finally seeing serious progress on not-invented-here language support, and everyone's got ideas on how to put Ruby to work. I'm very pleased at the response; I expected some foot-dragging and sour grapes, but the level of internal excitement has exceeded my expectations.

Thomas: Even at JavaOne, this spring, we had several Sun employees approach us to say how much they like Ruby. Since getting hired we have talked to even more Sun people who like Ruby. So internally at Sun, there is plenty of interest in JRuby/Ruby. We even have offers of help. I think this help will be a huge positive over time versus going it alone (e.g. Not being hired by Sun).

How much effort have you put in to making JRuby work with other Java implementations (IBM's, Kaffe (is this one even still viable), GCJ, etc.)? How will Sun's involvement help or hinder this?

Ola: Well, we try to avoid using Sun-specific libraries, and the code is straight Java 1.4, so in theory all 1.4-compliant JRE's should be able to run it. From what I've heard, GCJ seems to work fine but I haven't tried it.

Charles: We've stuck to Java 1.4 and written plain old Java code for all our changes. JRuby was originally written in 2001, so there's some old-time lineage there as well. Others have tried it out under GCJ and had no major issues, and Tom and I now uses Java 5 almost exclusively for development. The ongoing FUD about Java code not being forward-compatible, really cross-platform, or portable to non-Sun JVMs is really a bunch of nonsense. In the ten years I've worked with Java, it's been trivial to update to a new release or to run across all platforms and implementations. Because we don't do anything to specifically tie JRuby to one platform, it should work fine almost anywhere. That said, we don't go out of our way to test it under every Java VM under the sun, so we're looking to the community for that.

What plans do you have for JRuby over the next six months?

Thomas: It is difficult to know how much progress we will make, but I will make some guesses. We will be running Rails decently soon. Our Marshalling is subpar right now and we still have some smaller bugs. On top of just running Rails we want to help add value to Rails in a J2EE environment. This could mean running many Rails instances and sharing an app servers datasource to the database. Many interesting options.

Java integration is another area that will see some changes. For example, today you cannot extend a Java class from within Ruby and have Java see the methods that are overridden in Ruby. Kresten Krab Thorup has a patch that will help solve this missing feature in JRuby. Recently, we started discussing the semantic gap we have in regards to Java interfaces. Currently, we can only implement a single interface. We plan on changing the syntax to support implementing multiple interfaces.

People have been asking for more work on the AST so projects like RDT can expand their efforts.

Performance of course!

Charles:

It's hard to say how full-time work will affect schedules, but I think it's safe to say you'll see a JRuby 1.0 in the next six months, but we haven't started defining what "1.0" should actually be. We're probably almost there on correctness but we want performance to be greatly improved. We want to claim "full support" for Rails and we want solid integration with Java EE and other frameworks. There may also be a renewed investigation of Ruby grammar and parser work, both to mitigate our own parser woes and to provide a better understanding of Ruby syntax for community projects at large. I'll guarantee at least one major milestone before Javapolis in December, but it's hard to say what milestone it will be. By JavaOne next year we should have JRuby in really good shape.

Ola: Well, Charles and Tom are in charge, of course. But I have my specific needs and wants. I see that right now is the time to start getting JRuby-libraries and applications working. I see Mongrel as a big milestone and I'm also planning on taking charge of the Rant/Jake-ideas. AR-JDBC needs some love too, but that isn't as desperate right now. In 6 months, I would like to see JRuby 1.0, with 30-40% better performance compared to 0.9.0, complete YAML as Java extension, usable Rant-tasks for Java development, Mongrel working "fast enough", and the ability to work with Rails in development mode without too much pain. I would also like the start-up time to be about 100% faster.

Technorati tags: Ruby JRuby interviews

Several (Small) YDN Ruby Missteps

First things first. I hope nobody thinks I'm unhappy that Yahoo has started a Ruby-centric dvelopers network. On the contrary, I'm excited and happy to see it and think there's room for YDN Ruby to do a lot of good for the larger Ruby community. I am sorry to see that they've made some mistakes along the way though. I hope that by pointing them out here, I can help effect change there (and help others avoid the same mistakes).

Now, down to business. I was browssing the main Ruby resource page at Yahoo, and noticed some problems:

• Under "Useful Resources", there's no pointer to the ruby-talk or rails mailing lists (or the newsgroups and fora to which they're linked). There's also no link to any of the many excellent blogs or blog aggregators (like RubyCorner) that serve the Ruby community.
• Under "Learning Ruby", they have a link to the 1st edition of the pick-axe without a mention that it's dated or that the 2nd edition is available from the Pragmatic Programmers. They also left out www.ruby-doc.org off the list.
• Perhaps the worst problem is that they don't have a convenient, well-marked mechanism for providing feedback about the page. I'd love to suggest fixes for any of the points above, but there just no obvious way to do it, so I'm stuck ranting about it here. Working together to build a community is hard enough, but trying to provide a community to users is even harder.

Communities, Ghettos, Walls, and Bridges

The recent creation and announcement of the Yahoo Developers Network for Ruby has been generating a bit of discussion. I think it's another good thing for the Ruby community — to a degree.

Unfortunately (there's always an unfortunately, isn't there?), I think there's a stumbling block in the way. Along with the web page featuring a small, but useful, collection of tutorials and links, Yahoo has created the ydn-ruby mailing list. This is where they went wrong.

Branding is important to corporations and to large projects, but in this case, Yahoo's desire to brand their Ruby efforts, they've created a division in the community. It might not be extremely visible yet, but it's there, and it will likely grow. Today, there was a message about Sun's recent JRuby move. In addition to being wrong about when Sun hired Charles and Thomas, it was a message sent to the wrong place. Other information will be duplicated on both the ydn-ruby and the ruby-talk lists (and not alway correctly). It won't be long before parallel threads diverge, and the aggregate amount of information becomes harder to organize and make use of. This might seem like a small thing, but imagine the developer who comes to Ruby through ydn — what reason will he have to go look at ruby-talk and what will he miss out on because he doesn't?

Yahoo is, in effect, building a ghetto for themselves and their users. It's going to be difficult to keep this from being a bad thing, but it's worth the effort. What really needs to happen is a few people with respected voices need to take the time to poing ydn-ruby users at ruby-talk everytime non-ydn specific traffic is posted to the list — break down the walls and get traffic where it really belongs. Of course, getting publicity off of ydn-ruby and onto ruby-talk is important too. Someone should help ensure that major announcements get corssposted to ruby-talk. Maybe someone could start doing a weekly summary (once ydn traffic gets to be big enough to warrant it. There are a lot of opportunities here, they'll all take work, but I don't think there's another way.

A much better approach would have been to piggy back on the existing ruby-talk mailing list. When there is enough ydn specific traffic to warrant it, then think about splitting off to a separate list. By that time, you'll have attracted Rubyists to ydn, and established a relationship between ydn developers and ruby-talk. It's a much better solution all around. It's too late now to do that for ydn, but hopefully the next big company to start a Ruby movement of some sort won't follow in these particular footsteps.

Hashes, Defaults, and Objects

I just read Ruby Hashes of Arbitrary Depth by Duane Johnson (another UtahValley.rb/BYU-RUG Rubyist), and thought it was worth pointing out. The funny part (to me) was that he and I had spent a bit of time last night scratching our heads over the problem that he describes.

One thing I wish he'd mentioned in his post is an important fact that slipped both our minds and made the problem a bit more mysterious. Let me show you what I mean:

irb(main):001:0> foo = Hash.new({})
=> {}
irb(main):002:0> foo[:bar][:baz] = "wokka"
=> "wokka"
irb(main):003:0> foo
=> {}
irb(main):004:0> foo[:bar]
=> {:baz=>"wokka"}
irb(main):005:0> foo["what's this"]
=> {:baz=>"wokka"}

The thing we forgot, was that when you set a default value for a Hash, you're setting it to a specific object. So foo[:bar][:baz] = "wokka" isn't really creating a Hash called :bar with a key-value pair of :baz => "wokka". It's adding :baz => "wokka" to the default object. You can do this though:

 irb(main):006:0> foo[:pinky] = "blinky"
=> "blinky"
irb(main):007:0> foo
=> {:pinky=>"blinky"}
irb(main):008:0> foo[:possum]
=> {:baz=>"wokka"}
irb(main):009:0> quit

Any key value pairs you explictly create will be there just like you expect.

Just something to remember.

A Post With Legs

A bit ago, I posted about Ola Bini's blog (where, incidentally, he's posted another winner — YAML needs schema). This post (mine, not Ola's) caught the eye of Richard Dale, who posted Prolog as a Ruby DSL after following my links to Ola's post about Ruby and LISP.

Richard's posts lays out a quick plan to write a Prolog implementation as a Ruby DSL. I haven't played with Prolog in a long time, but having a Ruby implementation would be a great reason to look at it again.

Author Interviews: Jacob Harris

The recent best-selling success of the 'Pickaxe' Ruby guide and Agile Web Development With Rails has been a jolting wake-up call to the relatively dormant technical publishing industry. Although the Pragmatic Programmers were first from the gate, every other publisher is now catching up fast with new books (or revised old gems like The Ruby Way) and even brief digital shortcuts distributed only online. More than introducing Ruby to the world, this movement is creating a new wave of technical authors. Jacob Harris is the author of one of Addison-Wesley's first new Ruby titles - "Rubyisms in Rails" - released as a digital shortcut in September. A longtime infrastructure developer, Jacob ran into Ruby through an early version of Rails and liked what he saw. He started writing a blog, read computer books voraciously, became involved in the NYC Ruby Users Group, and hacked Ruby as a relief from his day job coding C++. Recently, he left his longtime employer to start working at New York Times Digital, where he hopes to shake things up at the Gray Lady.

---

How did you discover Ruby?

Jacob: As I said before, I ran into Ruby through an early version of Rails (I think it was 0.4 or something). I wasn't just looking at languages however, I decided I needed to play catch-up on a whole bunch of trends I had missed in the last 8 years (e.g., design patterns, dynamic languages, agile development, etc.). Then, as now, I found the best way to keep up with the zeitgeist was to add a bunch of feeds to my aggregator and soak it all up. This is how I stumbled into the Pragmatic Programmers and Chad Fowler particularly, whose book My Job Went To India (which I got to test read) was a profound inspiration to reshape my career and learn all the new programming trends I had missed. So I began to dabble with Rails and Ruby. I also started attending NYC.rb and hanging out in the #rubyonrails IRC channel (I've since moved on to certain select other channels). Today I've made many friends in the Ruby community from all over the globe; it's true what they say, the Ruby community truly is the warmest and friendliest technical societies I've ever encountered.

What role does Ruby play in your day to day work?

Jacob: Well, I've only just started at my new job, so it's a bit early for me to be making declarations. But I'm sure there will be a place for Ruby here. Rails is a promising web framework that would be good for both internal and external applications. But Ruby in itself has insanely great potential as an "enterprise" glue (where you have many disparate databases and applications to tie together). It's also good to see big consultancies like Thoughtworks raising awareness of Ruby as large-scale development language.

Still, I've always considered myself more of a Ruby dilettante than a hardcore hacker. I am humbled to be in the same company as Hal Fulton here! My previous job was coding in C++, so I had to squeeze all my Ruby hacking into my spare time. I'm just now coming into using Ruby in my real work, and it's an awesome feeling.

How did you come to write a book about Ruby?

Jacob: Last March, I decided it would be good to practice giving a presentation because I've never given a talk, so I signed up to give a talk to NYC.rb. Initially, it was just going to be a talk on some particular tips for people new to Ruby from PHP or Java. But as I began to cast about for examples of distinctive Rubyisms like blocks or metaprogramming, I found I could find good example code within Rails itself -- all of it silently operating just beneath Rails' frontend. All of which made for some excellent real-world examples of what Ruby can do.

Anyhow, it was a fun little talk, very well-received, especially when I posted it on my blog with readers around the globe (I seem to have particular fans in Taiwan). I had already met Debra [my editor at Addison-Wesley] through NYC.rb and I had been reviewing book proposals for her. When I heard in May she was looking for writers in a new series of shortcuts on Ruby, I thought it would be fun to pitch my proposal. Especially since I felt I could turn it around for her to sell in time for Railsconf in June. I was rather naive though, in that the whole process of writing-editing-revising-reviewing took me three months longer!

What other libraries have you peered into and found great code to look at?

Jacob: Hmm, this is a good question. I really like what Eric Hodel has been doing at Robot Coop. He built all these geocoding API stubs on top of a REST client he wrote (dev.robotcoop.com/Libraries/). Anything _why writes is amazing to read as well. Jim Weirich's XML Builder is also really cool, and I should probably look at Ruse as well. In any event, I'm just starting this process myself and I'm sure readers might be able to suggest some other wonderful living examples of Ruby code to read. I suppose we could even create a website where people could write about code like critics wax rhapsodical over movies or music.

I seem to recall there used to be a Computer Science course way back in the early days of Unix where the professor just printed the source for BSD and taught off the source, with yellowing pages of printouts being passed from student to student. I don't remember more than that (I'm not that old), but it's a lovely idea and kinda what I'm talking about. You don't learn how to code by reading documentation, you learn it by reading source. It's like a remote mentorship or rather a fellowship of those who really understand.

What sets your book apart, why should people buy it?

Jacob: My book is targeted to those coders who have reached a particular level of Rails experience. They've created applications and are maybe even lucky enough to be hacking Rails in their job. But they still have yet to become fluent in Rails and write code that resembles some sort of Ruby-inflected Java or PHP more than natural Ruby. Furthermore, they might have the nagging suspicion that Rails is just too "magical" and want to learn more. This book draws aside the curtain a little to explain how things work and illustrate how Ruby is coded in a real-world situation.

David Black did a wonderful book [(Ruby for Rails)]) on the same subject, but I think there is an advantage to my book's brevity as well; it's a very easy book to pick up and pass along and I think it provides enough insight and inspiration to give readers a new way of looking at Ruby.

What was the most rewarding part of writing your book?

Jacob: Computer books are boring. It's just all too easy to fall back on pages of function arguments or code snippets beating your readers' brains into submission. So for me, the most rewarding part was figuring out how to make it interesting. Here, I hatched a silly conceit of the book being a road trip. Which then let me formulate smaller metaphors of iterators as highways, DSLs as road signs, and other things that hopefully made it as fun to read as it was to write. I really hope the personal tone comes through, and it's been rewarding when people write me to say they liked it and it really helped them make sense of how things work.

What was the biggest challenge in writing it?

Jacob: Finding the time to write is really the hardest challenge. Writing turns out to be very much like coding; some days you're in the flow state cranking out pages of text, and other days you can barely manage a few sentences. This is particularly agonizing when you can only schedule a bit of writing time here and there. You'd think a 54-page book would be a walk in the park, but it still involves several iterations of writing-edits-revisions. It is an excellent preview of the process of writing a full book, and I highly recommend it if you ever think you want to write a book yourself. The other main challenge is keeping things brief. I think the initial draft was actually about 70 pages, but I spent a significant amount of time cutting it down to 2/3rds the size. Given that these ebooks sometimes seem to be sold by the page count, I'm glad my publisher never put any pressures on me to make it a certain length or pad it out.

What did you learn from writing it?

Jacob: Mainly, I realized how little I knew Ruby and Rails. I had dabbled around with Rails, but like many developers, I had been content to treat its whole interior workings as a black box. And when I started to look more at Rails' innards, I realized I didn't know a lot of the cool stuff in Ruby they were using: things like metaprogramming, runtime evaluation, domain specific languages, etc. Socrates once said "True knowledge is knowing you know nothing," which meant I was the wisest person in the world. In any event, I think that actually helped, since I was targeting it at newcomers to Ruby and I realized I was a newcomer myself. Which I guess is another point: know your audience. I don't think Socrates said that, it's too _creative writing 101_ for him to say. But it's true. And don't be afraid to be personal; I've noticed that saying _I_ a lot makes it a lot easier to write; I only hope it makes it good to read!

You mentioned that you've been reading voraciously. What five books (Ruby or not -- shoot, technical or not) do you think Ruby and Rails developers should be reading today?

Test Driven Development by Dave Astels (transform your process)

Refactoring by Martin Fowler (transform your code)

My Job Went To India by Chad Fowler (transform your career)

Patterns of Enterprise Application Architecture by Martin Fowler (where ActiveRecord came from)

The Art of Project Management by Scott Berkun (understand your PM coworkers)

Bonus pick: Practices of an Agile Developer, Venkat Subramaniam and Andy Hunt

But of course, there are a lot other cool books that didn't quite make the cut. Some of which are worth it just because they moved my thinking in a particular direction; some because they are very well-written; even some that completely and utterly useless for my job but transmogrify how I think (eg, the Wizard book).

In any event, I'd recommend that technical books are less than 50% of your total reading. You can learn more from a good history book or even a well-crafted novel than 10 technical books sometimes, and you'll be a better person for it as well. We are more than mere hidebound coders. But I'll get off that soapbox for now.

Before you hop off, what are the last couple of non-technical books you've read that you really liked?

Jacob: I'm currently enjoying The Amazing Adventures of Kavalier & Clay. This year, I also have really liked Richard Powers' novel "Gain", the history "Rip It Up And Start Again: A History of Postpunk", and Jessica Abel's graphic novel "La Perdida". Finally, the book "Maps of the Imagination: The Writer as Cartographer" has been a recent inspiration. Thanks!

Yeah, I guess that's way more than a couple. Let's call it a Baker's Couple I guess.

What are your favorite five libraries for Ruby?

Jacob: Okay, I'll be brief here: Builder for making XML generation easy; FeedTools for taking a stab at Atom/RSS support; Rubyful Soup for more fun XML; ZenTest (and RSpec as a bonus!) because I really, really, _really_ should be testing more; and Camping for keeping it fun.

What do you think is next for Ruby?

Jacob: This is an interesting question. Depending on who you talk to, Ruby seems to be at a crossroads between being a small "hobby" language and the next big thing after Java. I think we will see growing interest in Ruby within the corporate world, although this is freaking the hell out of some people especially in the Rails community. But I honestly think Dave Thomas has it mostly right; Rails has to work with legacy databases or other _enterprise_ environments if it wants to be used in places (like the Times for instance) where you can't just start from scratch; but I sympathize with DHH at the same time, there needs to be some pushback to keep it from getting too bloated and compromised. Still, I think Ruby beyond Rails could really be helped by more professional attention to fill in areas where Ruby's libraries are weak. So, Sam Ruby at IBM has been doing some serious work to support Atom, and I think Ruby could use some outside help to get the same level of fast and fully-featured XML support you find with Java or Python. I've also seen some interesting arguments over whether Unicode has a place in the heart of Ruby. One thing I like about Ruby though is that it's so easy to extend even core libraries that you don't have the same pressures to turn the core language into some bloated Enterprise Edition version; people can add what they need on their own.

But the real next thing for Ruby is the performance problem. Yeah, Joel Spolsky is being a twit here, but Ruby _is_ slower than you want sometimes. Even die-hard Rubyists sometimes have admitted a bit of dismay at how long it's taking for YARV to arrive. Sure, I could write C extensions for slow parts, but most of us went to Ruby to get away from C! For the sake of an example, it does irk me somewhat there are like 4 different opcode caching options for PHP at this point for instance. I'd like to see something similar for Ruby. JRuby seems like a promising initiative too, now that Sun is supporting it (another positive example of corporate involvement). I don't doubt that YARV will ultimately be great, but I'd like to see what other cool things happen. Don't get me wrong: Ruby is amazing, but super fast amazing Ruby would be incredible!

What's next for you?

Jacob: It'll be interesting. I can't really say a lot, but the Times is really poised to do some great things in the years ahead, and I will be part of it. So, hopefully you'll be hearing more of me soon. See you at Rubyconf!

Technorati tags: Ruby interviews

Good Reads: Ola Bini

I've been reading Ola Bini's blog recently, and I'm really impressed with what I'm seeing there. Over the last three days, he's managed four solid posts, including two that really stood out to me.

MetaProgramming Refactoring is a great piece on the refactorings that could be cataloged around Metaprogramming. He uses Extract Code Template as an extended example. It's a good read, and a better idea. I'd love to see this get some traction — no matter what mjd thinks of patterns.

Ola also wrote about the power of Ruby in The limits of power: What Lisp can do but Ruby can't. In this case he jumps off from Why Ruby is an acceptable LISP, and talks about how LISP's syntaxlessness (how's that for a word?) enables extra capabilities in a couple of edge cases.

Take a few minutes and go check out his blog. You won't be disappointed.

Legion Improving

I've been doing a bit of tinkering with Legion and it's already showing improvements. I've also found my first victim, err, volunteer to help get more tests running. Best of all though, Legion has found a couple of issues that can be directly addressed (one of them already has).

Improvements first: I've added another library (Ruport) to the test suite, and added the Ruby 1.9 CVS tree to implementations. The new test results look like this:

ruby ran 730 tests with 2452 assertions in 17.178 seconds.
There were 0 failures and 23 errors.
The average time per assert was 0.007 seconds.

ruby-1.9 ran 188 tests with 819 assertions in 4.549 seconds.
There were 1 failures and 7 errors.
The average time per assert was 0.006 seconds.

ruby-yarv ran 596 tests with 1425 assertions in 12.055 seconds.
There were 3 failures and 146 errors.
The average time per assert was 0.008 seconds.

jruby ran 578 tests with 1987 assertions in 96.666 seconds.
There were 29 failures and 16 errors.
The average time per assert was 0.049 seconds.

Now I need to work on some tools to drill down from this. (I can do so by hand, but I'd like to automate it.) If you've got ideas about what kinds of reports you'd like to see, let me know.

I got an email from Ben Bleything offering to help. If anyone else is interested, I'd like to get the conversation going this coming week (18-24 Sep). Drop me an email, or leave me a comment.

While I was adding Ruport, Legion identified a (really minor) problem. Gregory Brown has already fixed it.

I've fixed this in trunk and stable, revision 211. Legion is already helping out! :)

w00t! It also looks like Enumerator still needs to be built into JRuby. I'm not sure if it was on the list yet, but I've reported it to the developers mailing list to make sure.

I still have a lot of cleanup to go to get this really useful, but it's getting there.

Author Interview: David Black

Just before RailsConf Europe, I interviewed David Black, the author of "Ruby For Rails", which is rapidly becoming one of the books on Ruby that everyone recommends. Read on to get a better feel for David, and buy a copy of his book if you haven't already.

---

What role does Ruby play in your day to day work?

David: At this point, I'm doing Ruby and Rails consulting and training full-time. So Ruby is front and center. Then again, since I'm a consultant, work isn't always "day to day" at all! But I've got conference planning and writing projects and other Ruby-related things to fill in the gaps.

How did you come to write a book about Ruby?

David: Manning Publications contacted me in July, 2005, with a general question: did I, as a Ruby expert, think the time was right for publication of Ruby books? And, if so, what did I think was a good idea for such a book, and who should write it?

I took the opportunity to say: yes, it's a good time to publish on Ruby; and the book should be a Ruby-language guide for Rails developers; and I should write it!

The publisher agreed, and within a few weeks I had a contract.

What sets your book apart, why should people buy it?

David: What sets it apart is the combination of subject matter and emphasis: it's about Ruby, the language, but it presents Ruby in such a way as to be of maximum use to "the Rails generation," the people who are using Ruby mainly because of Rails.

After the book appeared, people started saying that it was actually of potential value to all Ruby programmers, not just those involved with Rails. It's gratifying to hear that. The scope of the book is not narrow: it doesn't include the whole Ruby language, but it includes a lot, and I tried very hard to provide lucid, deep explanations of Ruby. That seems to have paid off. Many people have told me that they felt they really understood Ruby for the first time after reading my book.

So I'd recommend it, first, to anyone using Rails who actually wants to know what they're doing (i.e., what their code means) and who wants to know how to do more (i.e., tap into the power of Ruby to raise the level of their Rails development). And, second, I'd encourage anyone interested in the general study of Ruby to have a look at the book.

What was the most rewarding part of writing your book?

David: It was very rewarding just to have the opportunity to sit down and write, in one place, so much about Ruby. I've always participated on mailing lists, chat rooms, conferences, and users groups; and I've always made a point of trying to help people understand Ruby, as well as expanding my own understanding of it. But that kind of communication, by its nature, is a bit scattershot. What I enjoyed most about writing the book was the process of pouring so much explanatory and expository text into one project, knowing that it would be easily available for people to read in one place.

What was the biggest challenge in writing it?

David: The biggest challenge was refining the concept of the target audience. Rails attracts people from lots of different backgrounds: programmers, Web designers, database administrators, project and team managers... and while they all play different roles, they all have a stake in Rails projects and they all want to understand what Rails actually does. It's not easy to write a technical book whose audience has the potential to be that wide-ranging.

When the going got tough, target-audience-wise, I would just remind myself that this is a book about Ruby. Yes, the book is optimized for Rails users. But it's _about_ Ruby. I came to understand that I didn't really have to worry about why a given reader wanted to learn about Ruby. My job was to explain Ruby.

That perspective helped me keep focused whenever the issue of the target audience started to seem daunting.

What did you learn from writing it?

David: I learned a lot about Ruby and Rails. I've never been a mind-dump kind of writer. For me, writing is dynamic and performative: it happens in the moment, as you go along. In part, that means that exactly how I'm going to express something, or even deciding what I'm going to express, isn't known in detail until I start writing. It also means that what I write is not just a subset of what I already know. As I work, I'm testing things, rethinking things, reading up on things, and therefore learning things. So I learned a lot about the subjects I was writing on, during the writing process. I also learned a lot about the book production process. This wasn't my first single-authored book, but the first one was in another field and much more academic. (_Law in Film: Resonance and Representation_, in case anyone's curious!) Of course, writing any given book is much like writing any other book, in many respects. But different sectors of the publishing world are very different from each other, too; and it was interesting to experience the technical publishing world close up over a period of time.

Any more books on the horizon?

David: Not at the moment, but stay tuned! I've been very busy with my change of career for most of this year, having left the academic world and set out on my own as a consultant and trainer. So my writing projects have been shorter. But I've got more books in me, I have no doubt.

What else do you have up your sleeve?

David: Lots of training. It's sort of ironic to have resigned from a tenured professorship and now be making most of my living from teaching Ruby and Rails! But I'm enjoying it, and working on plans to keep it going and growing.

I've also got a grab-bag of activities up my sleeve, if that isn't too mixed a metaphor. I'm in touch with a variety of people about a variety of small publication projects, learning materials, and other possible ventures. And in my capacity as co-director of Ruby Central, I'm working on several conferences, as well as our Regional Ruby Conference grant program and other initiatives.

I must say that, all in all, the process of shifting into full-time Ruby activity has been, so far, a very pleasant and rewarding one!

Technorati tags: Ruby interviews

Tests, thy name is Legion

It's no secret that I think we in the Ruby community can learn a lot for the communities around other programming languages. The Perl community is huge, mature, and a good source of ideas. One of the ideas that I'm trying to steal from them right now is the Phalanx Project — a testing tool for new releases of Perl.

Instead of building a large suite of unit tests, Phalanx is taking the top 100 Perl modules from CPAN and using their test suites to exercise Perl. This has several benefits. First, they can see how new versions of Perl perform in 'real world' settings. Second they can watch changes in correctness (nd performance) over time. Third, the testing wizards in the Phalanx project can help module owners improve their own testing.

I'm working on a project I call Legion, with the intent of doing much the same thing. So far, I'm using four libraries from RubyForge with a total of 4062 assertions. I'm running the tests against the latest releases of Ruby, JRuby, and YARV. As I collect data from these tests, I can feed it back into each Ruby implementation project to help them build a better Ruby.

I also want to feed the data back into the projects who's test suites I'm using. If I can identify gaps in their test coverage, I'll certainly work at building tests to close those gaps. Perhaps we'll even find some bugs through more strenuous testing.

There are a couple of things Legion isn't doing yet. With some help, I think they'll be easy to implement:

• More platforms — currently, I'm only testing on x86 Linux. I'm planning on adding PPC OS X shortly. I'd love to add x86 OpenSolaris, Win32, and any other OSes people are willing to run Legion on (as soon as it's ready for public consumption).
• Better reporting — I'm still trying to figure out how much information to record, what to report on, and where to make those reports. I need feedback (especially from Ruby implementers) to make sure I'm providing a tool that's useful.
• More libraries — I'd like to add more libraries to the test suite. Right now, they need to be pure Ruby (i.e., no C), and need to have a non-Rake method of running all their tests. I may move to Rake to automate the testing, but then will need to write a Legion specific task for each library in the suite.
• More Ruby implementations — as soo as I have things a bit more stable, I'm going to add CVS/SVN versions of Ruby (both 1.8 and 1.9 trees), JRuby, and YARV. I'f like to add Cardinal, Metaruby, and other implementations as soon as they're ready.

If you're interested in helping with any of these, please let me know.

I've already collected some data (and posted it to the dev lists for YARV and JRuby). I'd like to share the highlights here. As I can coordinate with the library maintainers and the developers, I'll open up the reporting a bit more. For now though, here's a high level view of the "big three" Ruby implementations according to Legion:

Ruby  
  total time:     115.094538 seconds
  time/assertion:    .02819  seconds
    total asserts:  4082
    total failures:   29
    total errors:     27
  
YARV
  total time:      12.012201 seconds
  time/assertion:    .00842  seconds
    total asserts:  1425
    total failures:    3
    total errors:    146

JRuby:
  total time:      97.978000 seconds
  time/assertion:    .04996  seconds
    total asserts:  1961
    total failures:   29
    total errors:     17

I noticed three things on a simple look at the tests:

• JRuby seems more compliant with Ruby than YARV right now.
• Ruby is nearly 2x faster than JRuby right now.
• YARV is ~3x faster than Ruby.

I expect to see these change over time. For example, JRuby is just starting a push for optimization now that the developer feel like they're close to being functionally complete (see Charles Nutter's blog posts, Performance: Block Variables Breakdown, Performance: Inlining Strings, and Nibbling Away at Performance).

JRuby: Thoughts from the Ruby.Net Camp

Dr. Wayne Kelly of the Ruby.NET camp, was kind enough to answer a couple of questions about the recent Sun move with JRuby for me. Here's what he said:

---

How do you think Sun's hiring of Charles Nutter and Thomas Enebo to work on Jruby will affect Ruby as a language?

Wayne: Quality alternative implementations of Ruby will hopefully change the mindset that the Ruby language is defined by a particular implementation of that language.

This will hopefully lead to a more formal specification of the semantics of the language which will ultimately benefit developers. I believe Matz likes to say that "Ruby does what you expect it to do". That's an admirable goal, but it should be backed up with a document that states exactly what that is.

Making Ruby available on new platforms such as the JVM and .NET will also enable it to be used in scenarios where it wasn't previously applicable and will allow those communities to leverage one another's resources.

Your implementation of Ruby.NET?

Wayne: In implementing Ruby on .NET we have virtually the same technical challenges that JRuby faces. I have had discussions with Charles in the past and expect to continue doing so. The more time he has to work on JRuby - the greater his insight that we will be able to leverage.

Ruby Hacker Interview: Kevin Tew

Kevin Tew is a software architect and freelance consultant. He recieved a master and bachelor degrees in computer science from Brigham Young University. Kevin's primary interests are programming language theory, large web systems, and virtualization. He has also done academic research in parallel processing, distributed systems, and phylogenetics.

---

How did you find your way to Ruby?

Kevin: Over the past couple of years I've heard about Ruby but never took more than a cursory look at it. Little over a year ago, I left full time employment to finish my Master Degree in Computer Science. While at Brigham Young University, I met Devlin Daley the current BYU Ruby Users Group president. Being the great proponent of Ruby that he is, he invited me to the BYU RUG and Utah RUG meetings. Since then I've met a bunch of very friendly and helpful Ruby Hackers.

What role does Ruby play in your day to day programming?

Kevin: Ruby definitely has its spot in my tool belt. I have a keen interest in programming language theory (PLT) so I know and use a wide variety of languages.

I'm still exploring Ruby. I use it for a lot of short quick scripts in the same way that I use Perl. I've started to use Ruby in place of Python for medium sized, more architected scripts.

I'm not the type to religiously argue that language A is better than language B. I like what I see in Ruby and hope that it will continue to evolve.

I believe that programming languages should empower the programmer to make greater and greater abstractions. Languages successfully evolve when they continue to introduce better abstractions while minimizing increases in complexity.

How did you get interested in Parrot/Cardinal?

Kevin: My interests are wide spread. I have a lot of background in systems and systems programming. I'm a big fan of Virtualization because it's another abstraction that increases my power and productivity as a technologist. Currently there are two big areas of Virtualization work as I see it. Processor and physical machine virtualization such as VMWare and language virtualization at the compiler/runtime level. I have interests in both areas as both have importance. Parrot is the most mature project attempting to build a virtual machine architecture for dynamic languages.

Parrot aims to be the JVM for dynamic languages. I also am very excited about Perl6. Parrot seemed like the right place for a newbie like me to get involved. One of the best ways to throughly learn a new language is to implement it. So I'm trying to do that for Ruby on Parrot.

So, can you give us an example of what cardinal can do at this point?

Kevin: Not a lot. It can do if, unless, while, until, it can throw a very basic exception and catch it, it can do execute a simple puts such as puts "a" or put 10, it can support functions without any arguments

parrot/languages/cardinal/t/ has the basic test suite that cardinal passes. It fails the BEGIN{}, END{}, class, and function call with args tests.

The suite is very minimal at this point in time

I realize that it's really premature to ask this, but how does cardinal compare to Ruby in terms of speed (for the things it does)?

Kevin: See Great Language Shootout:

• Ruby vs. Parrot
• Parrot vs. Perl
• Parrot vs. Python

Note, that this is parrot not cardinal. Much of parrot is still not optimized for speed. Work is primarily focused on functionality at this point.

Cardinal will run slower than the parrot benchmarks because of its focus on functionality not speed, but the promises for the future are evident.

Are you going to try to get the Cardinal Rubyforge site updated?

Kevin: Haven't thought a lot about that. Cardinal definitely needs a web site and a discussion area. If the current Rubyforge and past cardinal maintainers concur, I see no reason not to host Cardinal's web presence there. Cardinal source is currently hosted inside the parrot subversion repository. Given the pace of change in parrot and integration concerns, it is the right place for cardinal source for the foreseeable future.

What hurdles have you had to deal with in working on Parrot and Cardinal?

Kevin: Lack of formalism and standardization. Ruby could really use a formal grammar standard and a hierarchical test suite. Theses things are hard to do when you have a relatively young language and are volunteer supported. The Perl community is much more mature in years than Ruby and they are just now implementing these ideas. That said, investing time early can save a lot of headache in the future.

The Parrot community has been very accepting and positive about Ruby on Parrot. Parrot is a fast moving target. It is much more stable and feature complete than it has been in the past, but is still developing in a couple key areas. Namespace support has recently been nailed down and exception support is being worked on currently. Parrot has a functional class based object system, which is sufficient for the present. A major revision of the object system is Parrot's last major hurtle from a language implementor perspective. There are still a lot of other issues that parrot needs to address, but Parrot is finally coming to age.

If someone wanted to jump in and help with Cardinal, how should they go about it?

Kevin: Anyone and everyone is welcome to contribute to Cardinal in any way they can. We need code, tests, docs etc. Here are some places people can help:

• Writing basic tests. This is an easy place to start. We need a huge battery of tests that exercise the most basic parts of Ruby.
• Implementing core classes and modules such as Float, Fixnum, Array, Hash, String, Enumerable, etc. This requires C programming and learning a little about parrot internals.
• Implementing language features such as case, classes, modules, method_missing, etc.
• Writing test harness that compare cardinal output with Matz's Ruby.
• Writing test that verify cardinal's ability to parse and execute the Ruby standard library.

Writing tests is great way to ease into a project like Cardinal. However the more daring individuals who want to jump in and code are especially welcome.

I'm willing to mentor and help anyone that is interested in cardinal. Distance isn't a problem for me. I'd be happy to help anyone over Skype, SubEthaEdit, Gobby, IRC, screen, instant messenger, etc.

What's your take on the other Ruby reimplementation projects?

Kevin: I'm very impressed. There are obviously some very skilled and intelligent people working on both JRuby, Ruby.Net, and MetaRuby. I'm anxious to meet other Ruby implementors and learn from them. Each platform seems to have it's struggles. In many ways I'm envious of the mature platforms JRuby and Ruby.Net have to build on top of. Parrot doesn't have that maturity. I'm very curious to see the work that both Sun and Microsoft are doing to add better support for dynamic languages. In contract to the JVM and .NET Parrot's primary target is dynamic languages, while static languages take secondary position. This gives parrot advantages such as native supports for continuations. I think all Ruby implementors have advantages and disadvantages presented to them by their platform. I believe tighter cooperation would benefit all involved. Unfortunately time resources are always lacking, especially in volunteer supported projects.

How can you (the various projects) help each other?

Kevin:

• Standard Grammar Definitions and documentation for both LR and LL parsers.
• A set of standard documents like Perl6's Synopsis.
• A hierarchical test suite that has increasing levels of conformance for language re-implementors:
  • a very simple level for minimal interpreters or very early interpreters
  • additional levels where more complex language features are tested
• A minimal test engine, MetaRuby's BFTS already has an example of this.
• A sanity test suite, like Perl6 has. A sanity test suite tests only the minimal functionality needed to run the minimal test engine mentioned above. The sanity test suite should be executable using the minimal test engine.

What other projects are you working on?

Kevin: Presently, I'm writing a simple Rails application. I'm also doing some hacking to figure out how the internals of MythTV can be made to work with IPTV. (I'm trying to understand MythTV internals.) I'm looking at future XPath 2.0 functionality in Scheme.

What's next for Ruby?

Kevin: I don't know, I consider myself a newbie, but here are some things:

• Better and cleaner support for higher order functions (i.e., Procs )
• I think continuations are important and deserve more attention.
• A virtual machine. Native Thread support. Get native thread support and build whatever other abstractions you want on top. Not the other way around. This is a must for multiple processor and multiple core support.
• Software Transactional Support as a alternative to locking.

What's next for Cardinal?

Kevin: Support for functions and very basic class support.

What's next for Kevin?

Kevin:

• A short rest from School.
• Finish off backlogged TODO lists.
• Looking for opportunities to collaborate with motivated individuals on interesting projects.

Thanks for taking some time to talk with me about Parrot, Cardinal, and Ruby.

Kevin: Thanks for taking the time to show interest in cardinal.

Technorati tags: Ruby interviews

RedGreen and rspec

I keep telling myself that I need to do something with RedGreen but to me, it really doesn't itch all that much — and so, I procrastinate. Every once in a while, I read a blog post somewhere about a cool RedGreen trick that someone has done. Then, I think "Wow, I really need to integrate that." or, maybe, "Hmm, maybe they'd like to take RedGreen over." It never seems to work out though.

This morning, I noticed RedGreen over on the rspec mailing list. It turns out that Aslak Hellesoy has added RedGreen functionality to rspec. (You can read his email and the resultant thread starting here.) Pretty cool stuff. If you're using RSpec, this is a great chance to add some color to your tests. Just spec -c your_test and away you go.

Ruby, JRuby, and Java over at Ablog

I do a little bit of blogging over at Ablog, and I try to be a Ruby advocate in that role. This morning I read Cory Foy's post Why Sun’s hiring of the JRuby developers isn’t good for Ruby. It's a provocative title, but I'm not sure it's right. Cory and I discussed it and he posted my recommended alternative: 'Why Sun’s hiring of the JRuby developers isn’t just good for Ruby'. (By the way, Cory also posted this article at his home blog: here — unfortunately that spawned two sets of comments.

Cory's point was that this is a great thing for the JVM, and he's right. I happen to think it's also a great thing for Ruby. But you know what, we're probably both right.

Author Interview: Leonard Richardson

How did you discover Ruby?

Leonard: I'd known about Ruby for many years but only started seriously using it in 2005. My first serious Ruby project was Rubyful Soup (a port of my Python HTML parser), and my first professional project was the Cookbook.

At the time I was working in a Java-oriented job, tired of Java, and writing articles about Python web frameworks, so I probably would have entered Ruby through Rails were it not for the Cookbook.

What role does Ruby play in your day to day work?

I'm working on a book now that's not about Ruby, but that uses Ruby as the implementation language, the way _Design Patterns_ uses C++. I'm also doing miscellaneous Ruby consulting work.

How did you come to write a book about Ruby?

Leonard: That's a good question because I came into this project something of a Ruby newbie, certainly not anyone's top choice for a name on a Ruby book. I think I'd built up a good reputation from my work on the Wrox book _Beginning Python_, and when my agent suggested me, O'Reilly listened. It probably didn't hurt that all the big Ruby names were busy working on their own books. :)

As for coming into tech writing in general, I owe it to a former co-worker of mine, Tony Steidler-Dennison.

What sets your book apart, why should people buy it?

Leonard: I think it's got better breadth than other books (as you'd hope from a cookbook), but also good depth. I tried to make the recipes solve not just the specific problem stated in the title, but also a conceptual cluster of related problems. My idea was that we can't have a recipe for every possible problem you might encounter, but we can group similar solutions together to increase the chance that you'll find what you need.

Of course, some problems are really open-ended and some libraries have a huge number of features. A recipe can only be a few pages long, so for some recipes, all we could write was "solve the problem with this library, here's basically how to use it, good luck". Those recipes contribute more to the breadth than the depth.

My other goal was to write a book that you can use as a Ruby tutorial if you already know two or three programming languages. Most programming tutorials are written as though you've never written a variable assignment before, and it's kind of patronizing by the time you're learning your third or tenth programming language. My plan is that that you can start up an irb session, work the examples in the first ten chapters, and then you know Ruby. As far as I know no one has actually tried this, but I'm really interested in hearing whether or not it works.

What was the most rewarding part of writing your book?

Leonard: Ever since I bought my first O'Reilly animal book ten years ago (it was probably Shishir Gundavaram's CGI Programming on the World Wide Web) I've wanted to have my name on one; I feel like I'm part of an old tradition. Probably not as important a tradition as it was back when O'Reilly was the only publisher putting out books for hackers, but I'm still proud of it.

But really the most rewarding part is that people are using the book. I wrote it to be used, I use it all the time, and when other people use it I'm happy.

What was the biggest challenge in writing it?

Leonard: Coming to terms with my own foolishness. I'd write a huge chunk of code and a reviewer would point out that you could do it in one line, or that there was a gem for it. And that's the stuff that got caught. My favorite recipe in the whole book (14.20, "A Real-World HTTP Client") has a bug in it. It'll be fixed in the next printing, and I think there are actually relatively few bugs in the book because I could automatically test most of the code, but it's still aggravating.

What did you learn from writing it?

Leonard: I learned a lot about Ruby internals, from diving into the libraries and the C code to see how things really work. I learned a huge amount about Ruby in general, since as I mentioned earlier I started out almost a newbie. I think the best way to learn something is by teaching. The next best way is by doing, which is why the Cookbook focuses so heavily on trying out concepts for yourself in an irbsession.

The book was also a good general education in fields I'd never really looked at before: SSL, GUIs, distributed programming... I also enjoyed going through the RAA and the gems on Rubyforge, looking for interesting software to write about.

What are your favorite five libraries for Ruby?

Leonard: Hard to pick just five, but I'll showcase some lesser-known libraries that I think deserve attention:

• hpricotby _why, which makes me think I should just pack up Rubyful Soup.
• Starfish, a really simple distributed programming library that Lucas wrote.
• char-encodings deserves more attention. By which I mean, people should work on it as a way to improve Ruby's internationalization support, and yet I shouldn't have to do any work on it.
• Ferret is a Ruby port of Lucene, the best Java library ever. It lets you do full-text search on structured data.
• Finally, ActiveResource isn't a real library yet, and everybody knows about it, but it's going to be awesome.

What do you think is next for Ruby?

Leonard: I'm looking forward to more libraries that use Ruby's idioms to radically simplify entire domains. I think this is where dynamic languages like Ruby and Python show their power: Rails, ActiveRecord, ActiveResource, gserver, DRb, Starfish, Twisted, PyGame, etc. These libraries tackle a problem that's been around for years, and succeed by hiding a huge amount of the work and/or changing the way you think about the problem.

Many Ruby libraries (I'm thinking right now of the GUI libraries and RMagick) are just simple bindings to C libraries. Working with them feels like writing C code except without the performance benefits. Ferret is awesome but, because it's a Java port, you need to instantiate five different classes just to blow your nose.

It's great that Ruby has access to the best of other programming languages, but I think a binding or a port should be a signal to enterprising hackers to radically rethink the whole thing. For instance, just off the top of my head it seems like Ferret could support a simple ActiveRecord-like interface with no loss of power. (There I go, putting work on someone else's plate again.)

What's next for you?

Leonard: As I mentioned, I'm working on another book, which will hopefully be out early next year. I'm also writing science fiction, which might or might not ever be published. Financially, writing science fiction is to writing technical books as writing technical books is to having a "real" programming job.

RubyConf*MI, a Retrospective

In the aftermath of a successful RubyConf*MI, I wanted to check back in with some of the organizers to get their feedback on how it went, and what they learned. After taking a short break to recover from all their hard work Craig Demyanovich, Zach Dennis, Brandon Keepers, and Mark Van Holstyn were kind enough to answer a few questions for me. (You can read my pre-conference interview with Zach, Mark, Craig, and Brandon Keepers here, and a pair of conference wrap-up posts are here and here.

---

What do you think was the biggest success of RubyConf*MI?

Craig: Some people said that the attraction of the conference was simply that it existed. Imagine it like this: "I couldn't attend RailsConf in July, and I won't be able to attend RubyConf in October. Bummer. Hey, wait! What's this? Another Ruby conference? Only $20? I don't have anything going on the 26th! I'm there!" That we could provide something great, especially for those who, for whatever reasons, couldn't attend the bigger conferences was our biggest success.

Zach: The interaction, communication and sharing of ideas. It is very encouraging and motivating when you get people people together, and those people talk and listen to one another.

Mark: I think the conference as a whole was great. I know I had a lot of fun. I think the biggest success was the ability to meet and talk with others. I enjoyed meeting new people, especially at Grand Rapids Brewing Company afterwards.

Brandon: People came, they enjoyed themselves, and hopefully they left knowing more than they came with.

What was your biggest surprise?

Zach: That people wanted coffee. I am not a coffee drinker, so I never thought twice about having it at the conference. Next year we will have lots of coffee!

Mark: I was surprised how many people we were able to attract, especially those who traveled many hours to get there. I am glad that we were able to spark so much interest and hope to make that grow next year.

Craig: I must say, I was also surprised a few days before the conference to read on the Ruby mailing list a post wherein a RubyConf*MI ticket was offered because someone couldn't make the trip from, I think, Minnesota. I thought it was great that someone would plan to come from that far and offer his ticket when he couldn't make it.

What do you plan on changing for next year?

Craig:Well, some people seem to like a cup of coffee in the morning; we should have some to offer. ;-) Also, I'd like to be able to offer a few prizes to the attendees. Beyond that, I'd have to have a look at the detailed feedback we collected, which I haven't yet been able to do.

Brandon: Next year, we want to step it up a notch all around: better preparation, better marketing, better communication, a bigger variety of topics, better refreshments.

Zach: Personally, I would like to do a two day conference where one day focuses on speakers and presentations and another day focuses on more of hands on training lab. I haven't talked to the everyone else on this yet, but I think that will make the conference more productive for those who attend both days.

There will definitely be more beverage choices, that is for sure!

One thing I don't want to change is keeping costs as low as possible for those who attend, to help encourage and promote the growth of ruby and the interaction between students, hobbyists and professionals.

In hindsight, what do you wish you'd done better in your prep work?

Brandon: I wish we had done a better job of building community leading up to the conference.

Craig: We should've started a bit earlier. However, the idea for the conference was only born sometime in the Spring, if I recall correctly. Given the short time to make it happen, I'd say we did well. Another idea: we should've asked for more advice from people who have planned and executed a conference. We've all been to various conferences. We had some idea what it would take. Doing it, though, was something else.

Zach: Overall things went pretty smooth. I know that we'll try to get more of the correspondence out of the way up front rather then trying to write announcement emails and brochure-like marketing papers as they're needed. Not that we were at Kinko's the day before the conference finalizing and printing things, it was more like 6:30pm Friday. ;)

Any advice for would be regional conference planners out there?

Zach: Have a great time planning your conference, keep an open mind, and get feedback from as many people as possible as early as possible.

Mark: Start planning early, get many peoples opinions, and don't try to do everything yourself! It is very rewarding to be able to bring such a wonderful day to so many people.

Craig: Start early! Many things will take longer than you think. Also, keep it simple. We had many grand ideas for the first RubyConf*MI; I'm so glad we didn't try to implement them all!

Brandon: Zach, Mark, & Craig are right on: start early, get lots of people involved, and have fun. The only thing that I would add is you should have someone that can be the "task master". Most of our planning was by committee, and while that is helpful and necessary, there are times when things just need to get done.

JRuby, what's in it for us?

A lot of words are being passed around on the ruby-talk mailing list about Sun's move with JRuby. Recently, the talk has turned to "What's in it for us?". Beginning (more or less) with James Moore's post, which started out like this:

So I have to ask - is JRuby a good thing for the Ruby community, as opposed to the Java community? To me - admittedly, someone without a huge depth of Ruby experience - it's not an obvious good idea.

James went on to talk about the JVM vs C, but I think he missed the bigger picture. Because of his focus on the JBM, most of the replies clustered in that space as well.

I'd like to refocus the conversation. Here are five benefits I see for the whole Ruby community, and three possible problems.

The Good Stuff

Increased visibility for Ruby — With a big name getting involved in Ruby (and not just Ruby on Rails), there's even more opportunity to push back on the FUD that keeps popping up. It also creates a space to discuss Ruby with co-workers or peers that might not have existed before.

Work on a Ruby specification — One of the problems that many of the Ruby re-implementations face is that there's no formal specification of Ruby. Charles Nutter has been trying to work on this for a while. Now that he's being paid to work on JRuby, I think that this will accelerate.

Ruby, the core library, and the standard library testing — Charles has also been trying to drive for a common testing suite for Ruby, and all the libraries that ship with it. Given a common suite, all the implementations will benefit. Something like this could even grow into some rough performance testing.

Ruby documentation — Much of the documentation written for JRuby should roll right into Ruby. Given that 'bad docs' is one of the most common complaints about Ruby, this could be huge!

Ruby on more platforms — We've seen problems getting Ruby on AIX. If JRuby provides a parallel implementation that's easier to get running there (or on other platforms), it can help speed adoption on those platforms. (E.g., I have three AIX boxes that are slowing the spread of Ruby at my day job.)

Corporate sponsorship of Ruby — Now that Sun has jumped in visibly (and Microsoft in a less visible way), there's some home that other corporations will also step in to help push Ruby (or JRuby, cardinal, etc.) development.

And The (Potential) Bad Stuff

Corporate influence on (J)Ruby — Any time a corporation steps into an Open Source community, there's some fear that they'll exert some undo (and malignant) influence. In the past, this has mostly proved not to be a problem. Having strong developers at the core of the project seems to be the key mechanism to ensure that it doesn't happen. I feel comfortable that Charles and Thomas will be that stron core.

Divergence of implmentations — What happens when the JRuby behaves differently from Ruby? I've already found one (very minor) case where this has happened. The randomizer for JRuby and Ruby >= 1.8.4 give different numbers when they're provided the same seed with srand. (Actually Ruby < 1.8.4 also behaves differently than either of the others, so this isn't an issue unique to JRuby).

Bloat — Perhaps the biggest complaint about Java is that it's become tremendously bloated. Since this has happened under Sun's watchful eye, I think it's fair to ask if this will happen to JRuby too? Again, I hope that Charles and Thomas (and the rest of the JRuby community) will keep this from happening.

Wrapping Up

Did I miss anything? Am I overstating one (or more) of the points above? What do you think?

Digg This

More thoughts on JRuby

I also got some quick thoughts from Eric Hodel about his take on the JRuby announcement:

How do you think Sun's hiring of Charles Nutter and Thomas Enebo to work on Jruby will affect Ruby as a language?

Eric: I think the most likely change will be the addition of native threading, which is something that some people believe ruby has "needed" for a long time. I imagine it would also be possible to run ruby as an embedded language alongside Java.

I don't believe it would have much of an influence on the language or standard library, though.

The Ruby community?

Eric: Rails brought an explosion of interest in ruby and brought a bunch of fresh ideas. I don't think all of them were healthy ideas, but many of the language features were re-examined with fresh eyes and lots of neat stuff came out of it.

I imagine a bunch of Java developers joining the language will have a similar impact, but the community may have to educate them as well.

Your implementation of metaruby?

Eric: I'm not sure, not much of their code has utility to us, and our code could be useful to them if it was a few more years advanced at the current pace of development, especially on the Ruby2C side. I suspect we will continue to be separate for some time.

What a Little Bird Told Me About JRuby

Shortly after I read the news about Sun hiring Charles and Thomas to work on JRuby, I talked to Kevin Tew (the developer of the recently revived Cardinal project) about it. Here are some of his thoughts:

How do you think Sun's hiring of Charles Nutter and Thomas Enebo to work on Jruby will affect Ruby as a language?

Kevin: I think it's wonderful. It will greatly accelerate JRuby's stability and execution speed.

Having people working full time on Ruby besides Matz should help eliminate the rough edges and little inconsistencies of Ruby.

I would like to see stable support for tail recursion and continuations in Ruby and JRuby. I hope the JVM can continue to evolve to meet the needs of dynamic languages and not stifle the innovation of dynamic features.

How will it affect the Ruby community?

Kevin: The greater exposure of Ruby in the Java community can have nothing but positive effects for the Ruby community.

I'm a big believer of the positive influences that diversity and new blood can have on a project. I think each new implementation of Ruby is an added benefit to the community as a whole. I'm excited to see what new ideas will come from JRuby, given Charles and Thomas's new elevated status.

How will it affect your implementation of Cardinal?

It's always nice to have more than one example to look at when writing a new implementation of a programming language.

Building Cardinal on top of Parrot is much more similar to JRuby on top of the JVM than Matz's Ruby written in C. Having JRuby as a model will help speed up Cardinal development.

Hopefully a little of the attention JRuby is receiving can rub off on Cardinal.

Any other thoughts?

Kevin: I'm looking forward to meeting and working with Charles Nutter and Thomas Enebo.