Legion Improving

I've been doing a bit of tinkering with Legion and it's already showing improvements. I've also found my first victim, err, volunteer to help get more tests running. Best of all though, Legion has found a couple of issues that can be directly addressed (one of them already has).

Improvements first: I've added another library (Ruport) to the test suite, and added the Ruby 1.9 CVS tree to implementations. The new test results look like this:

ruby ran 730 tests with 2452 assertions in 17.178 seconds.
There were 0 failures and 23 errors.
The average time per assert was 0.007 seconds.

ruby-1.9 ran 188 tests with 819 assertions in 4.549 seconds.
There were 1 failures and 7 errors.
The average time per assert was 0.006 seconds.

ruby-yarv ran 596 tests with 1425 assertions in 12.055 seconds.
There were 3 failures and 146 errors.
The average time per assert was 0.008 seconds.

jruby ran 578 tests with 1987 assertions in 96.666 seconds.
There were 29 failures and 16 errors.
The average time per assert was 0.049 seconds.

Now I need to work on some tools to drill down from this. (I can do so by hand, but I'd like to automate it.) If you've got ideas about what kinds of reports you'd like to see, let me know.

I got an email from Ben Bleything offering to help. If anyone else is interested, I'd like to get the conversation going this coming week (18-24 Sep). Drop me an email, or leave me a comment.

While I was adding Ruport, Legion identified a (really minor) problem. Gregory Brown has already fixed it.

I've fixed this in trunk and stable, revision 211. Legion is already helping out! :)

w00t! It also looks like Enumerator still needs to be built into JRuby. I'm not sure if it was on the list yet, but I've reported it to the developers mailing list to make sure.

I still have a lot of cleanup to go to get this really useful, but it's getting there.

Author Interview: David Black

Just before RailsConf Europe, I interviewed David Black, the author of "Ruby For Rails", which is rapidly becoming one of the books on Ruby that everyone recommends. Read on to get a better feel for David, and buy a copy of his book if you haven't already.

---

What role does Ruby play in your day to day work?

David: At this point, I'm doing Ruby and Rails consulting and training full-time. So Ruby is front and center. Then again, since I'm a consultant, work isn't always "day to day" at all! But I've got conference planning and writing projects and other Ruby-related things to fill in the gaps.

How did you come to write a book about Ruby?

David: Manning Publications contacted me in July, 2005, with a general question: did I, as a Ruby expert, think the time was right for publication of Ruby books? And, if so, what did I think was a good idea for such a book, and who should write it?

I took the opportunity to say: yes, it's a good time to publish on Ruby; and the book should be a Ruby-language guide for Rails developers; and I should write it!

The publisher agreed, and within a few weeks I had a contract.

What sets your book apart, why should people buy it?

David: What sets it apart is the combination of subject matter and emphasis: it's about Ruby, the language, but it presents Ruby in such a way as to be of maximum use to "the Rails generation," the people who are using Ruby mainly because of Rails.

After the book appeared, people started saying that it was actually of potential value to all Ruby programmers, not just those involved with Rails. It's gratifying to hear that. The scope of the book is not narrow: it doesn't include the whole Ruby language, but it includes a lot, and I tried very hard to provide lucid, deep explanations of Ruby. That seems to have paid off. Many people have told me that they felt they really understood Ruby for the first time after reading my book.

So I'd recommend it, first, to anyone using Rails who actually wants to know what they're doing (i.e., what their code means) and who wants to know how to do more (i.e., tap into the power of Ruby to raise the level of their Rails development). And, second, I'd encourage anyone interested in the general study of Ruby to have a look at the book.

What was the most rewarding part of writing your book?

David: It was very rewarding just to have the opportunity to sit down and write, in one place, so much about Ruby. I've always participated on mailing lists, chat rooms, conferences, and users groups; and I've always made a point of trying to help people understand Ruby, as well as expanding my own understanding of it. But that kind of communication, by its nature, is a bit scattershot. What I enjoyed most about writing the book was the process of pouring so much explanatory and expository text into one project, knowing that it would be easily available for people to read in one place.

What was the biggest challenge in writing it?

David: The biggest challenge was refining the concept of the target audience. Rails attracts people from lots of different backgrounds: programmers, Web designers, database administrators, project and team managers... and while they all play different roles, they all have a stake in Rails projects and they all want to understand what Rails actually does. It's not easy to write a technical book whose audience has the potential to be that wide-ranging.

When the going got tough, target-audience-wise, I would just remind myself that this is a book about Ruby. Yes, the book is optimized for Rails users. But it's _about_ Ruby. I came to understand that I didn't really have to worry about why a given reader wanted to learn about Ruby. My job was to explain Ruby.

That perspective helped me keep focused whenever the issue of the target audience started to seem daunting.

What did you learn from writing it?

David: I learned a lot about Ruby and Rails. I've never been a mind-dump kind of writer. For me, writing is dynamic and performative: it happens in the moment, as you go along. In part, that means that exactly how I'm going to express something, or even deciding what I'm going to express, isn't known in detail until I start writing. It also means that what I write is not just a subset of what I already know. As I work, I'm testing things, rethinking things, reading up on things, and therefore learning things. So I learned a lot about the subjects I was writing on, during the writing process. I also learned a lot about the book production process. This wasn't my first single-authored book, but the first one was in another field and much more academic. (_Law in Film: Resonance and Representation_, in case anyone's curious!) Of course, writing any given book is much like writing any other book, in many respects. But different sectors of the publishing world are very different from each other, too; and it was interesting to experience the technical publishing world close up over a period of time.

Any more books on the horizon?

David: Not at the moment, but stay tuned! I've been very busy with my change of career for most of this year, having left the academic world and set out on my own as a consultant and trainer. So my writing projects have been shorter. But I've got more books in me, I have no doubt.

What else do you have up your sleeve?

David: Lots of training. It's sort of ironic to have resigned from a tenured professorship and now be making most of my living from teaching Ruby and Rails! But I'm enjoying it, and working on plans to keep it going and growing.

I've also got a grab-bag of activities up my sleeve, if that isn't too mixed a metaphor. I'm in touch with a variety of people about a variety of small publication projects, learning materials, and other possible ventures. And in my capacity as co-director of Ruby Central, I'm working on several conferences, as well as our Regional Ruby Conference grant program and other initiatives.

I must say that, all in all, the process of shifting into full-time Ruby activity has been, so far, a very pleasant and rewarding one!

Technorati tags: Ruby interviews

Tests, thy name is Legion

It's no secret that I think we in the Ruby community can learn a lot for the communities around other programming languages. The Perl community is huge, mature, and a good source of ideas. One of the ideas that I'm trying to steal from them right now is the Phalanx Project — a testing tool for new releases of Perl.

Instead of building a large suite of unit tests, Phalanx is taking the top 100 Perl modules from CPAN and using their test suites to exercise Perl. This has several benefits. First, they can see how new versions of Perl perform in 'real world' settings. Second they can watch changes in correctness (nd performance) over time. Third, the testing wizards in the Phalanx project can help module owners improve their own testing.

I'm working on a project I call Legion, with the intent of doing much the same thing. So far, I'm using four libraries from RubyForge with a total of 4062 assertions. I'm running the tests against the latest releases of Ruby, JRuby, and YARV. As I collect data from these tests, I can feed it back into each Ruby implementation project to help them build a better Ruby.

I also want to feed the data back into the projects who's test suites I'm using. If I can identify gaps in their test coverage, I'll certainly work at building tests to close those gaps. Perhaps we'll even find some bugs through more strenuous testing.

There are a couple of things Legion isn't doing yet. With some help, I think they'll be easy to implement:

• More platforms — currently, I'm only testing on x86 Linux. I'm planning on adding PPC OS X shortly. I'd love to add x86 OpenSolaris, Win32, and any other OSes people are willing to run Legion on (as soon as it's ready for public consumption).
• Better reporting — I'm still trying to figure out how much information to record, what to report on, and where to make those reports. I need feedback (especially from Ruby implementers) to make sure I'm providing a tool that's useful.
• More libraries — I'd like to add more libraries to the test suite. Right now, they need to be pure Ruby (i.e., no C), and need to have a non-Rake method of running all their tests. I may move to Rake to automate the testing, but then will need to write a Legion specific task for each library in the suite.
• More Ruby implementations — as soo as I have things a bit more stable, I'm going to add CVS/SVN versions of Ruby (both 1.8 and 1.9 trees), JRuby, and YARV. I'f like to add Cardinal, Metaruby, and other implementations as soon as they're ready.

If you're interested in helping with any of these, please let me know.

I've already collected some data (and posted it to the dev lists for YARV and JRuby). I'd like to share the highlights here. As I can coordinate with the library maintainers and the developers, I'll open up the reporting a bit more. For now though, here's a high level view of the "big three" Ruby implementations according to Legion:

Ruby  
  total time:     115.094538 seconds
  time/assertion:    .02819  seconds
    total asserts:  4082
    total failures:   29
    total errors:     27
  
YARV
  total time:      12.012201 seconds
  time/assertion:    .00842  seconds
    total asserts:  1425
    total failures:    3
    total errors:    146

JRuby:
  total time:      97.978000 seconds
  time/assertion:    .04996  seconds
    total asserts:  1961
    total failures:   29
    total errors:     17

I noticed three things on a simple look at the tests:

• JRuby seems more compliant with Ruby than YARV right now.
• Ruby is nearly 2x faster than JRuby right now.
• YARV is ~3x faster than Ruby.

I expect to see these change over time. For example, JRuby is just starting a push for optimization now that the developer feel like they're close to being functionally complete (see Charles Nutter's blog posts, Performance: Block Variables Breakdown, Performance: Inlining Strings, and Nibbling Away at Performance).

JRuby: Thoughts from the Ruby.Net Camp

Dr. Wayne Kelly of the Ruby.NET camp, was kind enough to answer a couple of questions about the recent Sun move with JRuby for me. Here's what he said:

---

How do you think Sun's hiring of Charles Nutter and Thomas Enebo to work on Jruby will affect Ruby as a language?

Wayne: Quality alternative implementations of Ruby will hopefully change the mindset that the Ruby language is defined by a particular implementation of that language.

This will hopefully lead to a more formal specification of the semantics of the language which will ultimately benefit developers. I believe Matz likes to say that "Ruby does what you expect it to do". That's an admirable goal, but it should be backed up with a document that states exactly what that is.

Making Ruby available on new platforms such as the JVM and .NET will also enable it to be used in scenarios where it wasn't previously applicable and will allow those communities to leverage one another's resources.

Your implementation of Ruby.NET?

Wayne: In implementing Ruby on .NET we have virtually the same technical challenges that JRuby faces. I have had discussions with Charles in the past and expect to continue doing so. The more time he has to work on JRuby - the greater his insight that we will be able to leverage.

Ruby Hacker Interview: Kevin Tew

Kevin Tew is a software architect and freelance consultant. He recieved a master and bachelor degrees in computer science from Brigham Young University. Kevin's primary interests are programming language theory, large web systems, and virtualization. He has also done academic research in parallel processing, distributed systems, and phylogenetics.

---

How did you find your way to Ruby?

Kevin: Over the past couple of years I've heard about Ruby but never took more than a cursory look at it. Little over a year ago, I left full time employment to finish my Master Degree in Computer Science. While at Brigham Young University, I met Devlin Daley the current BYU Ruby Users Group president. Being the great proponent of Ruby that he is, he invited me to the BYU RUG and Utah RUG meetings. Since then I've met a bunch of very friendly and helpful Ruby Hackers.

What role does Ruby play in your day to day programming?

Kevin: Ruby definitely has its spot in my tool belt. I have a keen interest in programming language theory (PLT) so I know and use a wide variety of languages.

I'm still exploring Ruby. I use it for a lot of short quick scripts in the same way that I use Perl. I've started to use Ruby in place of Python for medium sized, more architected scripts.

I'm not the type to religiously argue that language A is better than language B. I like what I see in Ruby and hope that it will continue to evolve.

I believe that programming languages should empower the programmer to make greater and greater abstractions. Languages successfully evolve when they continue to introduce better abstractions while minimizing increases in complexity.

How did you get interested in Parrot/Cardinal?

Kevin: My interests are wide spread. I have a lot of background in systems and systems programming. I'm a big fan of Virtualization because it's another abstraction that increases my power and productivity as a technologist. Currently there are two big areas of Virtualization work as I see it. Processor and physical machine virtualization such as VMWare and language virtualization at the compiler/runtime level. I have interests in both areas as both have importance. Parrot is the most mature project attempting to build a virtual machine architecture for dynamic languages.

Parrot aims to be the JVM for dynamic languages. I also am very excited about Perl6. Parrot seemed like the right place for a newbie like me to get involved. One of the best ways to throughly learn a new language is to implement it. So I'm trying to do that for Ruby on Parrot.

So, can you give us an example of what cardinal can do at this point?

Kevin: Not a lot. It can do if, unless, while, until, it can throw a very basic exception and catch it, it can do execute a simple puts such as puts "a" or put 10, it can support functions without any arguments

parrot/languages/cardinal/t/ has the basic test suite that cardinal passes. It fails the BEGIN{}, END{}, class, and function call with args tests.

The suite is very minimal at this point in time

I realize that it's really premature to ask this, but how does cardinal compare to Ruby in terms of speed (for the things it does)?

Kevin: See Great Language Shootout:

• Ruby vs. Parrot
• Parrot vs. Perl
• Parrot vs. Python

Note, that this is parrot not cardinal. Much of parrot is still not optimized for speed. Work is primarily focused on functionality at this point.

Cardinal will run slower than the parrot benchmarks because of its focus on functionality not speed, but the promises for the future are evident.

Are you going to try to get the Cardinal Rubyforge site updated?

Kevin: Haven't thought a lot about that. Cardinal definitely needs a web site and a discussion area. If the current Rubyforge and past cardinal maintainers concur, I see no reason not to host Cardinal's web presence there. Cardinal source is currently hosted inside the parrot subversion repository. Given the pace of change in parrot and integration concerns, it is the right place for cardinal source for the foreseeable future.

What hurdles have you had to deal with in working on Parrot and Cardinal?

Kevin: Lack of formalism and standardization. Ruby could really use a formal grammar standard and a hierarchical test suite. Theses things are hard to do when you have a relatively young language and are volunteer supported. The Perl community is much more mature in years than Ruby and they are just now implementing these ideas. That said, investing time early can save a lot of headache in the future.

The Parrot community has been very accepting and positive about Ruby on Parrot. Parrot is a fast moving target. It is much more stable and feature complete than it has been in the past, but is still developing in a couple key areas. Namespace support has recently been nailed down and exception support is being worked on currently. Parrot has a functional class based object system, which is sufficient for the present. A major revision of the object system is Parrot's last major hurtle from a language implementor perspective. There are still a lot of other issues that parrot needs to address, but Parrot is finally coming to age.

If someone wanted to jump in and help with Cardinal, how should they go about it?

Kevin: Anyone and everyone is welcome to contribute to Cardinal in any way they can. We need code, tests, docs etc. Here are some places people can help:

• Writing basic tests. This is an easy place to start. We need a huge battery of tests that exercise the most basic parts of Ruby.
• Implementing core classes and modules such as Float, Fixnum, Array, Hash, String, Enumerable, etc. This requires C programming and learning a little about parrot internals.
• Implementing language features such as case, classes, modules, method_missing, etc.
• Writing test harness that compare cardinal output with Matz's Ruby.
• Writing test that verify cardinal's ability to parse and execute the Ruby standard library.

Writing tests is great way to ease into a project like Cardinal. However the more daring individuals who want to jump in and code are especially welcome.

I'm willing to mentor and help anyone that is interested in cardinal. Distance isn't a problem for me. I'd be happy to help anyone over Skype, SubEthaEdit, Gobby, IRC, screen, instant messenger, etc.

What's your take on the other Ruby reimplementation projects?

Kevin: I'm very impressed. There are obviously some very skilled and intelligent people working on both JRuby, Ruby.Net, and MetaRuby. I'm anxious to meet other Ruby implementors and learn from them. Each platform seems to have it's struggles. In many ways I'm envious of the mature platforms JRuby and Ruby.Net have to build on top of. Parrot doesn't have that maturity. I'm very curious to see the work that both Sun and Microsoft are doing to add better support for dynamic languages. In contract to the JVM and .NET Parrot's primary target is dynamic languages, while static languages take secondary position. This gives parrot advantages such as native supports for continuations. I think all Ruby implementors have advantages and disadvantages presented to them by their platform. I believe tighter cooperation would benefit all involved. Unfortunately time resources are always lacking, especially in volunteer supported projects.

How can you (the various projects) help each other?

Kevin:

• Standard Grammar Definitions and documentation for both LR and LL parsers.
• A set of standard documents like Perl6's Synopsis.
• A hierarchical test suite that has increasing levels of conformance for language re-implementors:
  • a very simple level for minimal interpreters or very early interpreters
  • additional levels where more complex language features are tested
• A minimal test engine, MetaRuby's BFTS already has an example of this.
• A sanity test suite, like Perl6 has. A sanity test suite tests only the minimal functionality needed to run the minimal test engine mentioned above. The sanity test suite should be executable using the minimal test engine.

What other projects are you working on?

Kevin: Presently, I'm writing a simple Rails application. I'm also doing some hacking to figure out how the internals of MythTV can be made to work with IPTV. (I'm trying to understand MythTV internals.) I'm looking at future XPath 2.0 functionality in Scheme.

What's next for Ruby?

Kevin: I don't know, I consider myself a newbie, but here are some things:

• Better and cleaner support for higher order functions (i.e., Procs )
• I think continuations are important and deserve more attention.
• A virtual machine. Native Thread support. Get native thread support and build whatever other abstractions you want on top. Not the other way around. This is a must for multiple processor and multiple core support.
• Software Transactional Support as a alternative to locking.

What's next for Cardinal?

Kevin: Support for functions and very basic class support.

What's next for Kevin?

Kevin:

• A short rest from School.
• Finish off backlogged TODO lists.
• Looking for opportunities to collaborate with motivated individuals on interesting projects.

Thanks for taking some time to talk with me about Parrot, Cardinal, and Ruby.

Kevin: Thanks for taking the time to show interest in cardinal.

Technorati tags: Ruby interviews