I feel like I've been ignoring JRuby a little bit, but there's good news in the JRuby world, that's worth sharing (and breaking my Jruby silence).
The JRuby Team have added a new committer, Nick Sieger. Nick's well known in Ruby circles as the master note taker at RubyConf, but his JRuby life is perhaps a bit less widely known. With his new status as a co-committer, that should change fast. Nick joins Charles Nutter, Tom Enebo, and Ola Bini — here's hoping he can keep up with them.
Over on her incomparable Slash7 blog, Amy Hoy started a poll asking for help tuning her posts for 2007. I think this is a wonderful idea, and hope everyone runs off to answer them. (If you haven't discovered Slash7 yet, go poke around the site a bit, then answer them.)
Finished answering Amy's questions? Good. I hope you're all warmed up then to answer mine. I'd like to make 2007 a great year for On Ruby by making it a great year for On Ruby Readers. I've got a feel for some of the things that have been popular this year (ahh, the joys of web log analysis), but I'd like to hear your thoughts on how I can make On Ruby a better place for you in the coming year. (Just cut-n-paste these four questions into a comment and add your answers.)
Thanks for taking the time to help out with this. I'm looking forward to reading your answers.
Update: The February Challenge has been posted. Go take a look and enter the contest.
A little while ago, I thought it would be cool to run a Ruby and Ruby on Rails contest to help generate some interesting new posts in the Ruby and Rails blogosphere. When I approached Apress with the idea, they agreed to sponsor it (heck, they even upped the ante and decided to award bigger prizes than I had initially asked about). So, starting now, and running for the next several months, I'd like to welcome you to the Apress/On Ruby Blogging Contest.
Here's what they had to say about it:
Thanks to Rails, Ruby is finally coming into its own as an awesome language. In recognition of this, Apress and its imprint, friends of ED, will be publishing a minimum of 10 Ruby and Ruby on Rails books throughout 2006 and 2007. From entry level books, such as Beginning Ruby: From Novice to Professional and Beginning Ruby on Rails: From Novice to Professional, to more specific titles like Practical Ruby for System Administration and Pro Ruby on Rails: Building Enterprise Solutions, the Apress team is ensuring that programmers will learn both the basic and advanced skills to run a clean Ruby shop.
We've seen the excitement level for Ruby/Ruby on Rails grow exponentially in the past few years, and we are extremely excited that we'll know be able to help grow the community and audience. In partnership with prolific blogger Pat Eyler, we'll be running contests for the first six months of 2007, in order to increase the knowledge base for programmers curious to know more about the Ruby/Ruby on Rails phenomenon. There are still a lot of questions out there about why and when Ruby/Ruby on Rails should be used, and our hope is that a proliferation of postings will both lessen the confusion and encourage programmers to try Ruby/Ruby on Rails out.
Sometime in the first week of the month, Pat will post, on behalf of an Apress author, a blogging challenge relating to a specific topic on Ruby/Ruby on Rails, and all programmers will be encouraged to post a thoughtful response on either his/her personal blog or on a public website. In order to be eligible, these must go up within 3 weeks of the original post, and the links must be posted as comments to Pat’s original post. The postings will be judged by a 3 person panel, which includes Pat Eyler, Apress Editor Keir Thomas Jason Gilmore, and a rotating guest judge. Within one week of the closing date, the judges will announce their decision, and the winner will receive his/her choice of 3 Apress and friends of ED titles.
The first contest will begin January 2007. The guest judge for January will be Apress author Jarkko Laine.
I'm actually sitting on the first challenge right now. I'll post it Monday morning (right after I ring in the New Year, if I manage to stay awake for that). I can't wait to see what you guys do with it.
Update: The first contest challenge is up here.
The Definitive Guide to SQLite
It has a solid chapter (nearly 50 pages) on the Core C API (for using SQLite), and another (also nearly 50 pages) on the Extension C API (for extending SQLite). While I'm partial to Ruby, and found the coverage of the Ruby SQLite library to be quite good, other languages seem to get solid coverage as well.
The book isn't just aimed at users though. There's a chapter on the architecture of SQLite and another on its internals, if you've got a hankering to start developing.
SQLite seems like a lightweight database on the rise. Its portability, small size, speed, and liberal licensing (it's in the public domain) make SQLlite an attractive choice for a number of systems. Maybe your next application is one of them.
If you're using SQLite now, or think you might be soon, you owe it to yourself to buy a copy of this book.
2006 was a busy year for URUG (the Utah Ruby Users Group — now an umbrella organization for all the Ruby related groups in Utah). We saw changes in the group, in our membership, and in the Ruby landscape around us. Here are eight things that stood out for me:
We had many new members join the URUG during 2006. Many of these were local folks who discovered the group and started attending, but some were transplants moving in from other Ruby brigades in other places. At the same time, we lost Jamis Buck one of our founders, and most famous members, when he moved to Idaho and joined the Boise.rb.
The URUG itself was transformed in 2006. It went from being a group that met monthly, and drew attendees from Salt Lake County and Utah County, to being confederation of smaller groups — the SLC.rb and UtahValley.rb with the BYU-RUG (and existing group) merging into the URUG. There were some further changes, but they happened later.
Another positive change was the growth of Ruby in the local business landscape. Several companies became much more open about their Ruby use, and some started recruiting at local Ruby Brigade meetings. It's been great to see the shift from primarily hobbyists to primarily professional Ruby hackers as the year's gone on.
In October, we had the privilege hosting Matz for a couple of days. After arriving, he spoke at a BYU CS department colloquium, then to a special BYU-RUG meeting, and finally joined 16 intrepid Ruby users for dinner at a local pizza place. The next day, Matz came into Salt Lake City to see the sights and stop by my office for a brown bag Q&A — we had 40 people show up, pretty good for a java shop.
From Utah, Matz flew on to RubyConf 2006 (with write-ups collected here), where over a dozen URUG members were in attendance. Informal polls before the conference led us to believe we'd have half that number, but people kept showing up, including a couple of Rubyists who only learned about URUG at the conference (more about them later).
Shortly after RubyConf, we announced our own regional conference, MountainWest RubyConf. In cooperation with rubyists in Idaho and Colorado, we'll be holding a regional conference on March 16th and 17th, 2007.
At about the same time, a local company held a programmers 'death match' (with more info here). A Ruby hacker was one of the contestants tied for first place, and ended up accepting an offer to work there.
Finally, 2006 saw the creation of a brand new Ruby brigade within the URUG. The Layton.rb was created by two intrepid Ruby hackers in the Layton/Ogden area north of Salt Lake City. Larry Caesar and Doug Tolton, the RubyConf attendees who discovered URUG while they were in Denver, kicked things off with a meeting of 9 Ruby minded hackers. It'll be great to see where they go in 2007.
Speaking of 2007, I'm looking forward to see what it holds for URUG. In January, we're going to run a URUG meeting in Salt Lake City on a Saturday afternoon, just to bring all the Ruby brigades together — hopefully this is something we'll do a couple of times in 2007. There are some rumblings that might result in Ruby brigades in Heber and St. George. The MountainWest RubyConf should be a huge opportunity for Ruby hackers in the region. Cardinal (Ruby on the Parrot VM is a local project poised for growth in 2007. It seems like URUG's future is pretty bright.
So, what URUG events did I leave out? What do you think 2007 has in store for URUG? Leave a comment and let me know!
I had a chance to talk with Zed about the mongrel 1.0 prereleases he's doing, and the big 1.0 release they'll lead up to. As always, Zed has some great things to say:
Getting to 1.0 has been a hard row to hoe, and Win32 has been a big part of that. How much work does Ruby need to just work on Win32?
Zed: Luis has done quite a lot of work building a nice mongrel_service for win32 that manages all the processes and does a better job of keeping things tidy. That really was the biggest issue with the win32 support for Mongrel 1.0. Part of the problem is that, well, win32 is just weird and doesn't have any real notion of a "process". Do all the hackery you want it still isn't quite the same. In order to run a Mongrel server and keep it going after you log out, you need to make a service, and that's a whole huge domain not covered by Ruby very well.
Dan Berger has made a significant dent in the problem, but it's an uphill battle since he doesn't get very much support from the ruby-core folks.
I think if Ruby is going to run on win32 very well it'll take an effort similar to ActiveState's Python and Perl. A completely cohesive install that has *everything* someone needs for win32, and then all the other goodies people want to have. Make it all built with the same compiler (instead of this three different compilers crap) and push it as the standard win32 install. Banish anyone who isn't using it as their Ruby platform on win32 and we can finally get some stability.
Just doing that would go a long way to help win32 people out.
What five things are the biggest draws for Mongrel 1.0?
Zed:
5) Your boss will have no problem with you using it now (even though it's not different from 0.3.20).
4) All the Java heads in your office who spend hours clicking around in their "app containers" can't snicker at your lousy web server's 0.3.x status anymore.
3) It works better on win32, even though we had to use FreeBASIC to make it work better.
2) Mongrel 1.0 has that fresh Enterprisey smell and will soon get TWO version numbers so it can keep up with the traditional "baffle them with versions" marketing strategy so common in the BigBiz world.
And the best reason you should use Mongrel 1.0:
1) This was recently overheard in the #django channel:
"16:25 < daibatzu> Also, is there a python based server I can use to run my Django apps, rails has mongrel"Even the django guys want a Mongrel. How hot is that?
On a more serious note, 1.0 doesn't add anything but a few tweaks and expanded static file handling (MIME types, etc.). For most people it's just a minor upgrade that helps them feel better about running an open source project.
At RubyConf, you talked about using RFuzz to find problems in Mongrel, how did you do that? How much did it help?
Zed: RFuzz helps a ton since there's many problems that I can't verify unless I have a full running Ruby on Rails stack and can hit it with a test case. In the RFuzz source I've got a full external test suite that runs Mongrel with Rails and then tests out a bunch of problems people have reported. This makes fixing the problem easier, and also makes sure the problem doesn't show up again.
It also has a bunch of thrashing I've done to make sure the parser works well, sockets getting closed violently work well, and tons of other problems related to stability and nasty clients. It's not quite as extensive as I'd like, but it's really helping keep my work load down.
About the only thing that sucks is I've got to run this before I do a release and the automated build is now taking about 4 minutes.
I know the JRuby guys are trying to get mongrel working over on the JVM. How much have you worked with them?
Zed: A bit here and there. Mostly they're waiting on the 1.0 release to get out and then I'm gonna try helping them get a Mongrel running that works under JRuby. One major breakthrough was Adrian (the author of Ragel) added a Java generator for Ragel. Using that I should be able to port the HTTP parser over to Java for them.
What's next for Mongrel? (What other Zed projects should we be keeping an eye on?)
Zed: I have a process of putting a release of Mongrel out and then letting it simmer. I judge when it's time to put out a new release based on the number of bugs I've received and how severe they are. Security problems get an immediate release, but other stuff I plan for and do meticulously. So, for the immediate future there's probably not too much more to be done to Mongrel. I have received requests to do a better process monitoring system to replace the nasty spinner/spawner/reaper/mongrel_cluster stuff, but that's about it for now.
After thinking about what should be for the "Mongrel 2.0" stream I've decided that Mongrel should really be a vehicle for cleaning up the way all of these frameworks are deployed. Big on the hit list is removing their addiction to CGI semantics and cgi.rb by coming up with a Servlets/WSGI style standard.
No, I don't mean "complex" like Servlets, but rather a standard about how applications are packaged and deployed as well as a standard API that makes using cgi.rb useless. This standard would abstract away all the IO processing and deployment so that each framework just worried about how to process requests. Chief problem to be solved is how frameworks seem to love raping and reinventing things like MIME processing, error reporting, statistics, logging, starting and stopping the *server*, clustering, and even just putting the app into a package for the deployment server.
The problem is getting the various framework developers to play ball might not happen without severe beatings and bribery. I do think Mongrel's in a unique position though to use "gentle enforcement" to make this happen, it's just a matter of whether the various framework developers are interested.
Episode I seemed popular (good thing I start at one, and didn't do episodes 4-6 first), so here's a second go at it. Thanks again to Evan and Wilson for taking part in this, and for the positive feedback I've gotten from the community.
cuby has been mentioned twice now, once in the original interview with Evan, and again in the last installment. Can you tell us a bit more about it? Maybe give an example?
Evan: cuby is a tool that we're going to beginning using soon to make the core of the VM a bit easier to work with. The core concept is that it's functions like a pretty complicated C preprocessor, but where the input is ruby code and the output is C code. The intention is not to be able to run any old ruby code into it and get C code out, it's to make write the C code that goes into the core easier by making it at least look ruby like when it's written.
A simple example would be, say that you've already setup what cuby calls a type map that looks like:
{ :Object => { :class => "object_class(%s)" } }
obj.declare :Object
obj.class
object_class(obj);
I'd probably liken it to the first C++ compiler that was really just a preprocessor that output normal C code that was then compiled.
Wilson: The basic idea is to rip off (gently) an idea from Squeak, and maybe one from the Algol-based stack computers. In other words, code the system in a simplified dialect of itself. Cuby uses a subset of Ruby to generate platform-specific C code that can then be compiled into the VM.
The Rubinius C core isn't quite boilerplate, but it is deliberately extremely simple. I don't think anyone is going to write Photoshop 12 in Cuby, but for the kind of highly-standardized C code we want to see in the VM core, I think it's an interesting idea.
What new user contributions have surprised you the most?
Evan: I'd say how much work is being done to add RSpec specs for rubinius. People really seem to like them and are putting quite a bit of work into getting them running well.
Wilson: I'm not personally surprised by the RSpec slant, because all right-thinking, freedom-loving Ruby users use RSpec. However, something in the works recently has managed to be a happy surprise. A spec technique is underway that will allow any two Ruby implementations to be compared.
I'm making up the syntax here, since it's not finalized, but..
code = "puts 5"
code.output_from(:rubinius).should == code.output_from(:jruby)
Running valgrind on rubinius is a cool feature of the project. How else are you looking to keep code quality high?
Wilson: Valgrind is amazingly great, and counts as at least three items in the code quality armamentarium.
I was toying with the idea of rigging gcov to produce code coverage stats for the C code exercised by the Ruby tests (*phew*), but that's starting to look like an excuse for grey hair.
I'm also excited by Xray, which basically amounts to a MacOS X port of DTrace. I intend to use that to peer into the guts, when it becomes available. Solaris is cool, but I'm not man enough to use it as my main development system, and I look forward to such things being available on systems with, uhh.. sound cards.
P.S. I will build a personal shrine to anyone who ports Valgrind to MacOS X. With incense and everything.
Evan: I try to impress upon anyone working on the rubinius C code to keep things as simple as possible. There will be no awards for fancy C code in the rubinius core (in fact, there may be unawards for doing so), so that helps because the simpler the code, the less chance of there being strange bugs.
I'm still looking for a good C analysis tool for OS X, so if anyone knows of one, let me know.
With Heckle being released, what are the chances of rubinius getting heckled?
Evan: I don't see why not once rubinius is self hosted. The 'kernel' of rubinius is just normal ruby code, so as long as the specs and tests perform the right checks, it will work eventually.
Btw, self hosted means that the rubinius VM is able to compile new methods on it's own. Currently it still requires 1.8 to compile methods for it.
Wilson: Heckle is awesome, and I look forward to using it. However, it will definitely need a feature to let you ignore 'safe' mutation points before it will be very helpful to Rubinius. Getting any non-trivial system to pass Heckle without any errors is basically impossible.
Wilson is a big fan of this book, so we've picked it as our sponsor for this edpisode. If you're interested in working on rubinius (or any other VM) it's probably a good addition to your library. Update: if you have ad blocking on, you've missed the 'sponsor' adds. Wilson recommended the bookVirtual Machines
Previous episodes of the Serial Rubinius Interview are available here:
I wanted to try something a little bit different with this interview. Several years ago, my son and I watched a DVD collection of old movie serials and I really enjoyed the format. Recently, I started thinking, "Why couldn't I serialize an interview? If I kept myself to three or four questions at a time, and posted episodes once or twice a week, it could be pretty fun." So I started looking around for some victims, err, accomplices. Evan Phoenix and Wilson Bilkovich were kind enough to oblige me. While they'll be key to this Rubinius interview, don't be surprised if there's an occasional guest interviewee (and maybe even an interiewer).
I don't know how long this series will last, it'll depend on how well they go over (and how patient Evan and Wilson are). Hopefully this experiment will turn out well, and we'll all have some fun with it.
Without further ado — Welcome to Part I of the Rubinius Serial Interview.
You guys are keeping an pretty open commit bit policy. How is that going?
Wilson: Extremely well. The rule (borrowed from Pugs, I think?) is that you get commit rights as soon as your first patch is accepted. So far, rubinius seems to be attracting smart people who like writing Ruby code test-first. My favorite kind of people. We're in the process of switching from test/unit to RSpec, since I now have several RSpec co-religionists in #rubinius. The inmates are running the asylum.
"@thing = 1".to_sexp.should == '[:iasgn, :@thing, [:lit, 1]]'
Along with that, Victor Hugo Borja sent in a huge patch the other day that added basically the whole of Enumerable, along with other fun goodies.
Evan: I'm not certain if Pugs has exactly the same rule, but it's certainly the same spirit anyway. One thing that has me quite excited is when I go to update my code from subversion and there are quite a few new revisions that I had nothing to do with! From the originator's standpoint, it's really so gratifying to see people commit back as much as they've started to.
Another great thing that's begun to come out of the LCB (liberal commit bit) policy is people are starting to form their own small committees and focus their work. A great example is the RSpec work that's starting to come in. I've never touched RSpec and I'm encouraging people that have worked on it to push hard on getting a testing infrastructure that they want in the product. As people step up and show interest in areas no one is in yet, they instantly become the leaders and owners of that parts.
As Wilson said, we've also started to get the patch flood! It's really awesome to see people mail in patches for code as their first email to the group.
What goals do you have for the next 3 months of rubinius developments?
Evan: My goals are to work hard to stabilize the core VM so that the guys that are implementing the standard lib in ruby have a solid platform to work off. To that end, I have two immediate goals currently on my plate: Get the rubinius compiler running natively under rubinius so that we're officially self-hosted and get RNI (Ruby Native Interface) in. The compiler is a big step because at that point, we'll be completely independent of the > current 1.8 series ruby, which is a major milestone it itself.
RNI is the working name for the subsystem in the VM that allows use to call out to C functions in a generic way, like 1.8 has. The big deal with RNI is going to be that it has the same C API as 1.8, so current C extensions will require very little to no work to compile and run under rubinius.
Wilson: Evan's goals dwarf mine, but I'm hoping to knock all the rust off of my C skills. We now have a good number of talented Ruby programmers who are banging away on the core Ruby library. In the near term, however, it is C that will get us on the fast track to 1.0. I'll let Evan talk about Cuby, but even that won't be a complete replacement for C. I'm not sure if this is in the three-month category or not, but I want to get rubinius building on Windows and PocketPC ASAP. I don't personally care about those platforms, but the second-class-citizen status of mainline Ruby on those systems drives me nuts.
What's your proudest rubinius moment so far?
Wilson: Adding support for the class << self singleton class syntax. (We're calling it 'metaclass' in rubinius.) One line of C code, and 10 or fewer lines of Ruby. That's when I knew that rubinius was both feasible and implementable.
Evan: I'd say it was what I discussed above, when I went to update and pulled down something like 15 revisions that were all new and from other people. It was a proud moment for the project too.
Because rubinius was inspired by the old Smalltalk Blue book, I'm taking that book as this episode's sponsor.
Commenting on my blog post about Bug Days for Ruby projects, Wilson Bilkovich and Charles Nutter both said "Neat idea, but every day is Bug Day on $Project". While their intent is admirable, I don't think they get my point.
While I'm not in the habit of modeling my world view on wisdom doled out by super villians (or other fictional characters), Syndrome (from The Incredibles) came pretty close to my thoughts on the matter, "[W]hen everyone's Super... no one will be". You see, both rubinius and JRuby (and a whole lot of other projects) are agile, responsive, cool, etc. — but, holding a Bug Day is special.
It's one day that you put some special effort into before, during, and after.
You see, a Bug Day isn't really about fixing bugs (though, that's a nice side-effect), it's about getting new developers, testers, writers, and users for your project. Properly run, it's a great publicity and recruiting tool. And by making it something different than the 'every day' you're making it special ... you're making it super.
It looks like groups of three are really on my mind recently. Over the last couple of days, I've seen three blog posts that have struck me as related and pertinent to Ruby. Since two of them are the Ruby is Rotting post and its sequel Ruby is Rotting, part deux by Alex Combas, this probably isn't a surprise. The third one is Become the Thing that Replaces You by Kathy Sierra, and that might not be so obvious.
Alex wrote about the complexity inherent in the MRI (Matz' Ruby Interpreter) and in YARV (Yet Another Ruby Virtual-machine), the MRI's heir apparent. He talks about the difficulty in working with it, and the slow pace of visible development/maintenance on the two systems. He also wrote about JRuby, Rubinius, and Cardinal as potentially simpler, more agile systems which could help (or potentially replace) MRI and YARV.
Kathy's post talks about how instead of bowing to an unknown, future replacement, we (or projects and products we are passionate about) need to reinvent ourselves to become our own replacement. I believe that's what we're looking at with the plethora of alternative Ruby implementations and with Ruby 2.0. They're all opportunities for Ruby the language to continue being Ruby by replacing the parts that need improvement — MRI with a faster, simpler, better interpreter/VM being the prime example.
In my last book list, I looked at three books about programming, books that should make you think about the practice of programming. This time I'd like to look at three programming books, books about technical skills that will make you a better programmer — these still aren't specific to a specific language, I'm planning on posting a Ruby specific list soon.
Refactoring, by Martin Fowler, was suggested in a comment on the last book list, and I couldn't agree more. Understanding, and exercising, the art of refactoring will make a huge difference in the quality of your programs in the long run. The only downside to this book is that so many people talk about refactoring without having read it (or understanding what refactoring really is), leading to all kinds of jargon abuse. If you haven't read it yet, you owe it to yourself to grab a copy and spend some quality time with it.
Code Reading, by Dimidis Spinellis, is a recent addition to my bookshelf. It's pretty tightly tied to the C/C++/Java triad, but the concepts it covers will also be of use to a programmer using dynamic and/or functional languages. Chapters on dealing with large projects, understanding project architecture, and tools for code reading (not to mention a 19 page example) are reason enough to pick up a copy. If your C skills are as weak as mine, the ample coverage of reading C is a wonderful bonus.
Everyone uses regular expressions, but not many people take the time to really get comfortable with them. Jeffrey Friedl's Mastering Regular Expressions is a great way to develop a good understanding of them. It also make a handy reference for the darker corners of regexp use.
As a regular user of RubyCorner, I'd like to start out by thanking both of you for all you work in putting together a great resource for the Ruby community — and then improving it with new features like the Google based search tool. Although a lot of people use RubyCorner, I don't think many people know you guys. Would the two of you mind introducing yourselves?
Edgar: Sure! First of all thank you for interviewing us. Well what can I say about myself? ... I'm from Venezuela, graduated in computer science in 1992, in 1995 I founded (in conjunction with a friend from the university) an IT company mainly dedicated to develop web applications (www.valhallaproject.com). Since then I'm mainly focused in develop web applications using Java. In 1997 I "kidnapped" Aníbal to join our team.
Aníbal: Actually I think there is not much to say about me. I am Venezuelan, who lived many years in Bloomington, Indiana. I like coding, but phone calls, mails and meetings made this my hobby. Which is good thing because I had no formal hobby at all :-)
How did you discover Ruby?
Aníbal: We were looking for an alternative to J2EE, under a Open Source / Free Software compliant license. We had a previous experience with Perl, and we were afraid just to think in our programmers switching from Java to Perl. We had a little experience with PHP for minor projects, and we didn't really liked it, and againg making our programmers switch from Java to PHP wouldn't be easy.
Then our friend Sebastian Delmont (he's Venezuelan too), told us "I don't want to program in any programming language different than Ruby" Knowing he was a big fan of Perl, we decided to give Ruby on Rails a serious trial. And RubyCorner was our first experiment.
As Java programmers, we all were seduced by Ruby.
What is the Ruby Community like in Venezuela?
Edgar: The Ruby Community in Venezuela is still in its infancy, we are promoting Ruby and Ruby on Rails by giving talks and presentations in universities across the country, and writing in our blog (www.lacaraoscura.com).
Aníbal: Yes, it is young (maybe Japan is the only place where it is not a young community) but also very enthusiatic.
How was your switch from J2EE to Rails?
Aníbal: Initially we had a really hard time setting up our development environment on Windows (now we have completely switched to Linux), but once we got over this issue we found ourselves amazed by the all these things that in Java mean to struggle with the language.
Using Rails meant for us switching from thinking in technical issues to focus on our customers needs.
Edgar: I remember being horrified because of the "nothing is closed" policy of Ruby, when in Java Object was a kind of distant God. After struggling with some weird final classes in Java, now this is something we appreciate, but try to use carefully.
Can you give us some examples of how your focus changed?
Edgar: For example in the Java world we rely in code generation techniques, but this approach isn't so flexible as I wish (after generation), now in Rails we use the dynamic nature of Ruby to do the same things with equivalent but more flexible results.
One thing I really love in Rails is the support for functional and integration testing, it's really simple and powerful. Trying to do the equivalent functional testing in Java is not easy, you have to depend on third party web testing frameworks, like Canoo or Cactus. I really enjoy writing the functional tests for RubyCorner :-)
In Java the developer depends too much on IDE automation, in Rails the magic is in the framework. A good IDE, like RadRails is a really good thing to have, but it is not mandatory.
These kind of things allow the developer to focus in the business problem domain instead on support issues and/or reiterative tasks.
Ruby and Ruby on Rails can't be all good. Other than setting up your windows environment, what have you found problems with?
Aníbal: Deploying. Initially getting the deployment process to finish with a working application was really hard. Gem versioning was also a major pain like the classloading naightmares in J2EE containers.
Our initial hosting solution wasn't reliable. We had some weird problems that were very difficult to get being addressed by the technical people of the hosting company, but now we have a really stable environment that's working great, and it is "Capistrano Powered"
Hosting is a new world for applications that require more that Apache and the "Usual Suspect Modules" and this is a field which is obviously in a quick evolution.
What other languages do you use?
Edgar: We haven been strongly developing for the J2EE stack in the last years, actually we started developing with Java for the Web a lot of time before there was a J2EE spec with tools like SilverStream and NetDynamics, and yes we are that old ;-)
Tell us about how you got the idea for RubyCorner?
Aníbal: We have been actively blogging for many years, and we found very useful a service were Venezuelan bloggers were able to register and ping to check for the updated blogs. We wanted to build a new software to bring a better service, and we started working on it, then we decided to first try it with a smaller community, and that's how the idea of RubyCorner came out.
Also we wanted to showcase our work outside of our country, because we though that we could offer a very good development alternative for a outsourced development.
Right now we started offering a high quality, low cost alternative for application development with Ruby on Rails at has_many :developers
How well do you think you've done in carrying out that initial vision?
Aníbal: Well, it works :-) We know there is a lot of space for improvement, but time is something that we struggle to find to work in RubyCorner. We had to spend a lot of time struggling with the different Feed formats, and explaining what XML-RPC pings are to the bloggers, that was something that surprised us.
Crawling all the blogs is Repeating Yourself too much, that's why RubyCorner relies on the ping mechanism, but not all the people has a CMS were pings can be automated, of ignored our facilities to ping, that includes a REST ping that can be bookmarked. We are activists of the "Ping" movement.
What kinds of improvements are you thinking about making?
Edgar: Now we are working to store the posts and their metadata (like tags, categories) in order to provide a "tagsonomy view" for the rubysphere, and to allow to identify "hot topics" or make searchs about these that don't rely on Google crawler.
Aníbal: Blogs are asynchronous conversations, I see RubyCorner and to2blogs evolving to allow users to track the conversations, and how they relate to each other.
A Google based search was the last thing you added, how is that working out for you?
Edgar: I think the Google custom search engine that we added to RubyCorner works fine, it allows to search in all the blogs registered (almost 300 to date) plus selected sites, like ruby-forum, ruby-lang, etc.
For easy use of this CSE we created a plugins that allows to add it as a search provider to your browser.
Aníbal: It is useful, but it doesn't reflect the quick rhythm that blogs have, and the order of the blogs doesn't reflect the conversation sequence, we will be working to add this kind of search to RubyCorner.
How much 'recruiting' are you doing to find bloggers for RubyCorner to aggregate?
Aníbal: Each time we find a blog that looks unknown, we check it in the RubyCorner blog search and if it is not already registered we drop a little note to invite the writer to join the community, the results are very good.
We usually suggest the best feed to register (Ruby or Rails category or tags feeds) and we are always reminding people to configure the automatic ping, this is some kind of personal crusade ;-)
Edgar: In the begining in every Ruby or Rails related blog we found we dropped a comment inviting the author to register in RubyCorner, but since a couple of months almost all the new blogs registered in RubyCorner did it by their own initiative.
I'm glad to see in RubyCorner blogs written in different languages, in my opinion this is a sign that the Ruby buzz is spreading around the world.
RubyCorner's not your only project, what else are you working on?
Aníbal: Right now we are running the same codebase at to2blogs.com, but focused on the venezuelan blogging community, it is working great and we reached the thousand blogs in the past weeks. to2blogs is quickly growing, and evolving as service very different from RubyCorner, because the usage that the community gives to the service is very different.
You guys just finished teaching a series of classes on Ruby and Rails, what's the Ruby/RoR training market like in Venezuela? How were the classes received?
Aníbal: Actually as far as we know we are first ones teaching Ruby and Rails, outside of the academical environment, this is just a starting adventure. Right now people have been very interested and entusiastic, we hope this trends keeps going on.
Classes were great, but for the people I was teaching there was not easy to jump from Microsoft tools to Ruby programming, I needed to do a lot of emphasis in object orientation the way it is implemented in Ruby and the functional aspects.
The good thing is that I'll continue coaching this group, and now they have a very good base ground to start their journey On Rails.
Edgar: Ruby is the programming language I had liked to use when I was a student at the university. The mix of OO + functional that Ruby provides is great. Ruby and Rails is like a fresh air to remind you that exist more than one options to resolve a problem. Like I said to my students in the university, "If you only know a hammer you see everything like a nail".
I also understand that you're in the process of making a bunch of 2 hour presentations about Ruby at various universities. What kind of stuff are you covering at these? How are the audiences responding?
Aníbal: We have been invited to talk about Ruby and Rails at the major Venezuelan universities and other organizations, usually we introduce the principal features of Ruby and later we demonstrate Rails building a simple application featuring a little testing and Ajax.
In the previous opportunities the feedback from the audience has been great.
These kinds of activities seem like a great opportunity to show the seeds of local Ruby Brigades. Do you know of any groups that are springing up in your wake?
Aníbal: We have been asked to lead this process, and we are just looking for a break from our activities to found a local user group, we think that maybe we will be doing it in our spare time between 3 and 4 of the morning ;-)
Technorati tags: Venezuela Blogs Ruby Rails interviews
This year I've read (or reread) and enjoyed three 'programming' books that I thought were worth pointing out. They didn't pertain directly to any programming language, so they might not be on a lot of people's "I've gotta read this" lists, but myabe they should be. One of them is is from 1999 and the other two have roots that go back well before that.
The first one is The Pragmatic Programmer, by Dave Thomas and Andy Hunt. Even if you've read it before, this is a book well worth (re)reading. Many of the seventy tips (on a convenient Quick Reference Card even), color a lot of today's Ruby community: #11 Don't Repeat Yourself; #17 Program Close to the Problem Domain; #29 Write Code That Writes Code; #46 Test Your Estimates; #47 Refactor Early, Refactor Often; and #62 Test Early, Test Often, Test Automatically; and #66 Find Bugs Once should all be familiar to Ruby programmers. Some of their tips haven't gotten as much play: #4 Don't Live With Broken Windows; #36 Mimiize Coupling Between Modules, and #65 Test State Coverage, Not Code Coverage all deserve more respect and more application.
If you don't own this, get it onto your gift list (or just buy yourself a copy). If you do own it, buy a copy for someone else then go and reread your copy.
Software Conflict 2.0 (published in 2006) is a new edition of Software Conflict: Essays on the art and Science of Software Engineering (pubished in 1990). For this edition, Robert Glass (the original author) wrote new, retrospective essays looking back on each chapter of the original book. The new edition is a great 'jumping of point' for a reader unfamiliar with the classics of the field. Bob covers a huge amount of ground, peering under rocks and behind bushes as he goes. He avoids the meaningless detail of specific architectures, methodologies, and languages and instead deals with the questions that underly them.
Software Creativity 2.0 (another one by Robert Glass), was a late addition to my reading stack this year, but one I couldn't really pass up. I've been entertained and educated as I've read through the updated essays that make up this new edition. I still think the third chapter is my favorite, and especially "In defense of Ad-Hocracy", but there are a number of other great pieces in there.
What books are you reading? And what are you getting out of them?
On Dec 16th, the Parrot team will be holding their first ever Parrot Bug Day. This sounds like a great way to get more people involved in Parrot development (maybe some cardinal work will get done too, eh?), but it also sounds like an idea worth stealing for JRuby, rubinius, and a host of other cool Ruby projects.
Okay, you're probably asking yourself "What is a Bug Day?", and "How can it help JRuby (or some other project)?". Those are fair questions, so let's take a look at them.
A Bug Day is really just a day set aside to fix bugs. Some simple things will help it work well:
Bug Days can help a project in a number of ways. Obviously, the most direct is that bugs get fixed. Less directly, getting bugs fixed (and release criterea met) also means more momentun (and visibility) for the project. Another significant outcome should be that new developers are brought into the project and junior developers get more experience — in the long run, this might be the most important outcome of them all.
I'd love to see projects pick up the Bug Day idea and run with it. If the days are carefully chosen, they could even tag along on local Ruby Brigades' Hacking Nights (Tuesdays seem good for this). If you think the idea of a Bug Day on it's own is cool, just imagine what would happen in five or ten Ruby Brigades jumped onto the irc channel and each knocked down a bug or two ... shoot, even if they just built the project, tried it out, and reported any problems (and filed bug reports as appropriate) that would be a win for whoever was holding the Bug Day.
Update: You can read more at rubinius in the ongoing rubinius serial interview:
Rubinius is a project that's been in a lot of people's sights (and sites) lately. From an article at InfoQ, to a donation from Geoffry Grossenbach, rubinius is making waves. I was able to corner Evan for an interview over email recently. Here's what we talked about.
Evan, thanks for agreeing to do this interview. Would you mind starting out by introducing yourself?
Evan: I'm a computer scientist who for the past 3 years has had the pleasure of working on ruby as my full-time job. Currently I work for a small company in Seattle called Lockdown Networks, building network access control appliances.
I began using ruby in 2002 while working as a system administrator. What began as using an elegant language for writing automation scripts became the desire to use ruby for all projects. Before ruby, I used quite a bit of C, Python and Perl.
And what about rubinius?
Evan: rubinius is a ruby implementation built from scratch. It is architected very similar to the original smalltalk-80 virtual machine, but takes a different approach to various things to make it easier for ruby. The idea behind the whole thing is that a simple core engine means a more powerful language. As little of the ruby runtime is implemented by opaque code or data in the engine, exposing even more of the whole runtime to the user to play with. A good example is that there is almost nothing that the user interacts with that are not first-class objects. That includes stack frames used to perform methods (called a MethodContext) to the methods themselves (all methods are instances of CompiledMethod).
You've been involved in the Seattle.rb for quite a while. What value does a strong, local Ruby community have?
Evan: I've found a strong, local ruby community to be indispensable for a few reasons.
What kinds of things do you do to help foster that community?
Evan: As I said above, having a young community with a focus on development and inclusiveness really makes people feel special and important. It's that feeling that keeps people coming back each week. Each week, everyone is encouraged to bounce ideas off the group and ask for help.
For example, with rubinius, I've had people gather around my laptop as I explain how the internal parts of it work. I believe that that is going to help people get excited about rubinius and hopefully bring them to the project to help.
What sets rubinius apart from the other implementations?
Evan: I think that simplicity is what sets it apart. While it's still not the simplest program in the world, I believe it is vastly simpler than the other implementations. Simple has it's problems too though, there isn't nearly as much optimization, but when there is a simple way to add them, they will get added.
Have you been able to get anyone else hacking on rubinius yet?
Evan: I'm happy to report that I've handed out commit rights to second core developer, Wilson Bilkovich. He's been ramping up and already has made changes to the compiler and runtime. Interest has been growing in the past few weeks and more and more people are coming to me, asking how they can help. It's really quite amazing to see the ruby community all pitch in and help get this project off the ground.
Do you have a set of (fairly) simple tasks for a budding rubinius hacker?
Evan: I'm really working to try and get some simple tasks written up so that people have a place to dive in. With a project like this, it can be hard because a lot of the tasks like things like "Fix GC to allow for resizable memory use." I'm not always the best judge of whats easy for a newcomer either, and thus far it's certainly been a learning experience trying to manage how people can come on to the project.
But for people to dig in and start, all the code can be checked out of subversion (code.fallingsnow.net/svn/rubinius/trunk/) and can be built and tested to get people a feel for the whole thing. Instructions on building and testing are available at blog.fallingsnow.net/rubinius/.
There are 3 things I happily accept from anyone: patches, failing tests, and bug reports. In fact, if they submit all 3, they'll get a prize from me! (not sure what the prizes are yet, but I'll figure something out). All 3 can be submitted via the rubyforge bug tracker
That being said, one task that I'm really looking for is a cheerleader. I'm not the best self promoter as I've got a tendency to put my head down and just code, hoping that people are able to follow along. What I really need is someone who can help me get people excited about the project and help organize the tasks a bit. Anyone who is interested, email me!
What have been your biggest surprises (good and bad) in reimplementing Ruby based on the Smalltalk model?
Evan: I'd say that my biggest surprise is really how close the internal semantics of ruby are to Smalltalk. Before rubinius, I'd never really seen or programmed in smalltalk, just heard about it. But the more I looked into how the original smalltalk-80 interpreter was written, the more it sounded a lot like ruby. It diverges at a lot of places too, but the overall semantics are quite close.
I'm trying to stay pretty positive about the whole experience, making it as fun as I can, so I don't think there is a big, bad surprise. Maybe the bad surprise is how much fun it is and therefore how much my other projects suffer since I want to work on rubinius all the time. :D
During your presentation at RubyConf, you talked about this version being a 'cheating' prototype since you're writing the C by hand. What do you want to get ironed out in this release before you move on to the automatic translation of Ruby to C?
Evan: Currently, I'm doing a lot of hand written C work on the interpreter called shotgun, which is my hand recoding of the ruby version. I think that it's important that I have running code so that people who get involved with the project has something tangible to play with. So my plans for shotgun are to get all the biggest features that are necessary to make rubinius usable. My current TODO list covers most of them. At that point, rubinius will be running and hopefully quite usable. Then I'll begin the overall process of replacing the shotgun with code that is auto-generate from the existing ruby code.
That being said, I'm actually working on a tool right now I call cuby. It's a dialect of ruby that generates C code directly. It uses the same parser that rubinius does, but has a processor on it that generates C code directly using vastly simpler semantics that normal ruby. My idea is to use cuby to starting writing parts of shotgun in something that sort of looks like ruby. Then when I go back to to get translation, it will be a lot easier because the code will sort of in between C and ruby already.
What can you tell me about cuby?
Evan: The idea behind cuby is to provide a language that feels like ruby, and has a lot of the same semantics, but breaks them where it needs to. I would describe cuby as a 'system language'. The idea is that the code it generates is able to run standalone without any helper libraries to provide core functionality.
I modeled the original idea after pyrex, a tool for python that makes it easy to write python extensions in a python-like language that converts itself to C.
I want to cuby to be able to do the same, provide a simple interface/language for writing rubinius extensions. I for see that parts of the rubinius core will in fact use cuby to simplify development.
How does it compare to Ruby2C?
Evan: It's quite close to Ruby2C in idea, but it's simpler and cheats more. IE, you can hardcode methods to implements in cuby, so that you can say that "whenever someone calls method method 'to_i', I want you to call the C function 'convert_to_int', no matter what the receiver of the method is."
When do you think you'll be able to make a 1.0 release of rubinius?
Evan: It's still too early to tell. My hope is to have a version 1.0 of rubinius out by christmas next year, to coincide with the release of 1.9.1.
What do you think of the budding ruby on strongtalk and ruby on smalltalk efforts?
Evan: I think that they're excellent work and that the ideas and/or code that comes out of them will be useful to all implementations. At a personal level, I see it more as a way for smalltalk programmers to bring ruby libraries and such to their environment, a goal which almost all languages have (being able to interface to code written in other languages). I'll be the first on to admit that ruby shares an huge number of concepts with smalltalk, but it also differs from them in many ways. The current movement is to get the most basic of programs translated to run on smalltalk, but once they get to the more complicated ones, it might get a lot more complicated. That being said, it might be easy, there are some really smart and intrepid programmers out there that might make it work.
What do you think are the two or three most important things that can come out of the Ruby implementers summit/discussions?
Evan: I would say the first and foremost is a healthy dose of mutual respect. We all want our own implementation to succeed, but at the same time, we respect and work with individuals who don't share our exact goal.
Playing off the point on the same goal, we all do have the same, overarching goal, to see ruby succeed. And to that end we've begun to figure out how we can all help each other to achieve that goal.
Concretely, the establishment of the central test repository is big. It's going to mean that as we all move further down our respective roads, we're all able to maintain overall consistence for the users of our implementations. One thing that was brought up over and over again at rubyconf is the danger of fragmentation. I'm not so naive to think that it won't happen, and I'm sure it will. Fragmentation is a side effect of change and forward progress, it happens when one implementor decides that they believe that their way to do things is better than the current way, or that some currently missing feature is needed. By having tests that define the core ruby behavior in the form of tests, we can be help to avoid fragmentation in the core.
What facets of the other alternative Ruby implementations do you envy?
Evan:I envy the jruby developers for not having to debug segfaults, thats a biggy.
What Ruby development tools and libraries are you using as you work on things like rubinius and cuby?
Evan: One of the core libraries that is used is sydparse, a standalone ruby parser that I extract from my work on Sydney. It wraps up the parser from Sydney in an extension and provides a way to view the code as data using a version of ParseTree I got from Ryan Davis and Eric Hodel. It provided the basis for writing the compiler, since I wasn't currently interested in writing a ruby parser, since it's that step that has faltered quite a number of projects like this. In fact, I've extract the parser from sydparse and retrofit it to fit directly into rubinius so that for the time being, we don't have to write our own parser.
RubyInline also formed a big part of the prototype. I used to wrap a number of raw system operations, like allocating memory, read and writing values directly to address, etc. I built all of the objects on these primitives. That way, the prototype behaved a lot like it would when it's translated directly to C.
I understand that you're also using valgrind with rubinius. How are you using it?
Evan: We've started to use valgrind to validate shotgun, the rubinius engine thats in C and standalone from the normal ruby. It's an amazing tool thats provided us with a lot of information about programming errors in C code that could crash the interpreter. I suspect that in the future, we'll integrate valgrind into a daily test run to make sure that the code is continually tested for completeness.
If you weren't working on rubinius, which Ruby projects would you be tempted to get involved in? Why?
Evan: Well, I'm really interested in ruby building up library and tools so that it becomes an attractive platform for all kinds of developers, hobbyist to enterprise developer.
To that end, I've always been interested in DRb and encryption. I've got a partially started project that wraps an awesome C library called libtomcrypt to bring ultra fast, simple encryption to ruby as an alternate to OpenSSL.
If you're interested in making a contribution to rubinius, please paypal the money to PayPal to evan (at) fallingsnow (dot) net.
While researching an article on Springy and JRuby, I spent some time with Jan Berkel of Trampoline Systems (Spring's developer). Since it didn't all fit in the article, here's 'the rest of our discussion'.
What made you decide to write springy in JRuby?
Jan: The goal for springy was to create a flexible and easy to use DSL to set up Spring contexts. Ruby just happens to be well suited for this task (Rails being the best example) and JRuby provides the necessary integration layer between the two languages.
What made JRuby a good fit fit for this?
Jan: JRuby makes it very easy to extend existing Java code: it is getting to a point now where the boundaries between languages almost completely disappear, at least from a Ruby perspective. You can use and extend Java classes in a very natural way whilst retaining Ruby's power and flexibility.
What problems did you have using JRuby?
Jan: The documentation is sometimes not very helpful or does not exist at all, so I often had to dig through the source code. However, there's a JRuby wiki now and as the community gets bigger things will certainly improve in this area. JRuby is moving very fast at the moment, so keeping up with recent developments (new features, bug fixes) takes up a lot of time.
Do you have any feel for how many people are using Springy outside of trampoline systems?
Jan: It has just been released, so I'd say there's little chance of someone else using it at the moment. I get the impression that a lot of Java developers still see Ruby more as a threat than as a chance to make their own development environment more productive. XML-based configuration will be around for a very long time as it is the established convention and supported by many tools. However, people already familiar with Ruby will find that writing their own 'mini-tools' can be a lot more powerful.
What other opportunities do you see for JRuby to impact Java development/developers?
Jan: In the short term I see interesting opportunities in areas like configuration management (as alternative to XML), build systems (as alternative to ant/maven) or testing (RSpec). In the long term JRuby on Rails will certainly play a big role, giving developers access to one of the most innovative web frameworks without having to sacrifice their previous investments in Java.
On a less technical note, I think that JRuby could also help to bridge the cultural differences between the two development communities. Java users will use more Ruby for various tasks, and Rubyists will (hopefully at some point) make full use of a JVM-based Ruby implementation, where C extensions can be replaced with Java.
How much is Trampoline systems using (J)Ruby?
Jan: We use a hybrid development strategy, using (C)Ruby on Rails for the frontend and Java for all backend related things (actually with bits of Lisp in it as well). It's really a "best-tool-for-the-job" approach, Ruby/Rails is agile and productive whereas Java is the big guy in the background who does the heavy lifting. They get along very well at the moment. We're looking forward to a production-ready JRuby on Rails, it will simplify a lot of things.
Our build system is based on Rake with some Java-related extensions (which we've opensourced recently — Jerbil). It reduced the complexity of our build system from ca. 1500 lines of XML to 180 lines of Ruby, largely thanks to Ruby's metaprogramming capabilities.
Work on my book hasn't been going as well as I'd wanted (it's amazing how procrastination and laziness can throw you off a schedule). Recently, I've been doing a little bit better though. Last week I was working on some material on benchmarking Ruby code, and suddenly I started to channel Zed.
I realized that in my discussion of how to use the Benchmark library, I was completely ignoring all of Zed's great advice about applying statistics to performance testing. I really wanted to get that advice in front of anyone reading my book, so I wrote this little sidebar (below).
When I was done with it, I realized two things. First, I wanted to get the information out to a wider audience sooner, rather than later; and second, I think there's an opportunity for an enterprising tool writer to take the idea (and sample code) below and create a tremendously useful benchmarking tool. I talked with my editor, and he agreed to let me post the sidebar here — that takes care of the first realization. Hopefully the lazyweb will take care of the second.
I don't have a deep background in statistics, so I relied on a lot of advice while writing this (any mistakes are certainly mine though). If you think I'm all wet, I'd love to collect some feedback. Please drop a comment here explaining why you think the concept, method, or math needs work. (Of course, if you want to say something nice about it, I certainly won't mind.)
It's important to note (but beyond the scope of this book to do much more than that) that running a single benchmark of one method against another is not sufficient to determine whether or not you've really made a performance improvement. Zed Shaw has talked about this at length.
A quick way to do this is to run a series of tests for your original and new code and calculate the deltas between the results. Then you can calculate a mean and the standard deviation to see if 0 is in the range of the mean +/- 2 times the standard deviation.
The Benchmark library actually makes it fairly easy to build something like this. Here's a quick-n-dirty example:
require 'benchmark'
iterations = 10
loops = 100_000
deltas = []
def mean(ary)
ary.inject(0) { |sum, i| sum += i }/ary.length.to_f
end
def std_dev(ary, mean)
Math.sqrt( (ary.inject(0) { |dev, i|
dev += (i - mean) ** 2}/ary.length.to_f) )
end
iterations.times do
original_result = Benchmark.realtime do
1.upto(loops) { |num| num+1 }
end
new_result = Benchmark.realtime do
for num in (1..loops) do num+1 end
end
deltas << (original_result - new_result)
end
deviation = std_dev(deltas, mean(deltas))
mean_delta = mean(deltas)
printf "The deviation in the deltas was %f\n", deviation
printf "The mean delta was %f\n", mean_delta
max = mean_delta + 2.0 * deviation
min = mean_delta - 2.0 * deviation
if min <= 0 || 0 >= max
puts "There's no statistical difference"
end
With this example, you can see a more accurate picture of the performance differences between the for and upto methods:
The deviation in the deltas was 0.005296
The mean delta was 0.006870
There's no statistical difference
For more information (and better tests) go cuddle up with a good book on statistics because my brain hurts from doing this much.
I took a few minutes to interview Charles Nutter, Michael Studman, and Aslak Hellesoy for an article I wrote for InfoQ. I tend to ask a lot more questions than I'll have space to print in an article like that, and I hate to throw stuff like this away, so here's the whole interview. Enjoy!
Why do you think RSpec on JRuby is a big deal?
Michael: I think this development is important from a couple of perspectives. Firstly RSpec working on JRuby is proof, visible to the wider Ruby community, that JRuby is a solid Ruby interpreter and a viable alternative to MRI. It's rapidly approaching the point where big, complex, important apps and frameworks like RSpec just work. The fact that we're almost there with RSpec speaks volumes about how good JRuby's implementation is today - this is a cause for celebration!
Secondly, it's important because it opens up the possibility of BDD to Java developers in a way that's largely been denied to them in the Java language: a simple, clean, natural means of expressing intent. Many developers in the enterprise will, for a long time to come, be denied the ability to develop the production code in Ruby (JRuby or not). I'm personally encountering this at the moment and the myopia mistaken for common sense and risk avoidance is really frustrating. This means for many there's no chance of taking advantage of the substantial benefits Rails, Camping and the like give you. Where I've found more wriggle room (again in the enterprise) is in areas like BDD and unit testing - the prejudice against dynlangs isn't as strong here. So I think many Java developers can at least make use of the substantial benefits of tools like RSpec today while delivering a 100% Java product. In this way I hope tools like RSpec running on JRuby become Trojan horses for Ruby in the enterprise.
Charles: Behavior-driven development, using a tool like RSpec, is important for two reasons: It's a higher level of abstraction than is typically found in standard unit-testing frameworks, and it's closer to how we think about such problems. When using RSpec, we also benefit from using Ruby with all its capabilities. And of course RSpec under JRuby ties everything together: behavior-driven development, agile and elegant Ruby code, and access to all available Java libraries. It's the best of all worlds, and a pretty exciting new way to test applications on the Java platform.
Aslak: I think there are several big deals here.
First, it will help bring Ruby (the language) to the Java world (see below).
Second, having RSpec running on JRuby is a big step for JRuby itself. RSpec is used to spec itself and it pushes Ruby to its limits by using some fairly exotic metaprogramming in the internals. If JRuby is able to run RSpec's spec suite this is a very good indication that JRuby is on par with standard Ruby.
Third, RSpec is the de facto library for BDD in Ruby. Being able to use it on JRuby is a big deal because more people will be able to get familiar with BDD and realize the benefits of writing executable documentation. This is actually more of a big deal to the BDD cult than to RSpec or JRuby. These tools are just vehicles for a grand plan to take over the world >:)
What were the hardest bits in making it happen?
Michael: Frankly I think the hardest bits were all done by other people — the many JRuby developers over the years and particularly the current three committers. It's a testament to their efforts that RSpec just mostly works right now and that getting even further along this path is not proving a substantial effort. I guess the only hard bit for me was sneaking development in between work, sleep and life!
Charles: Hmm, I dunno, downloading JRuby? When I tried out RSpec a couple months ago, everything I tested worked fine. Aslak (or someone) also discovered that all the sample specs in the RSpec distribution ran fine. RSpec's own tests showed a number of errors, but Michael and I managed to resolve the vast majority of them with a simple method-visibility fix. I think we're down to only a couple remaining failures out of hundreds of tests, so official 100% compatibility is probably days away. JRuby's mature enough now that most pure-Ruby apps like RSpec "just work", so we'll see more and more such apps "officially" supported.
Aslak: When I first started to run RSpec's own suite of specs with JRuby there were somewhere around 50 errors. I submitted a couple of bug reports and within a few days they had been fixed. It turned out that most of the errors were caused by Kernel.raise having wrong (private) visibility. So within days the number of failing specs were down to about a dozen.
I figured that the main reason the remaining bugs had been allowed to survive this long was that they would only surface under very rare conditions. And since I was sitting on the code that could pry them out I decided to dig into the JRuby internals and submit patches. I'm really impressed by the JRuby team's responsiveness. The patches usually make it into trunk in less than 24 hours!
For me the hardest part was to become familiar with the JRuby internals, but the code is really well tested (good tests are great documentation) so I managed to get to the bugs much quicker than I had anticipated. I've made an extra effort to include tests along with my patches - to prove that something is wrong until the patch gets applied. There is no better way to get a patch in quickly.
How does doing things like this help JRuby in general?
Charles: Without a complete testing kit for Ruby (though the RubyTests project is starting to make progress in that area), we tend to use applications' own test suites to measure Ruby compatibility. RSpec represents another set of tests we can use to ensure we don't regress, and once it's 100% Aslak has offered to start running future tests against JRuby as well as MRI. That will help us all ensure RSpec continues to run in future releases...and ensure we don't break anything.
Michael: I agree. It also helps JRuby because it builds confidence in the wider Ruby community that JRuby is ready for the big time. The more of these big applications and frameworks we can say "JRuby's 100% compatibility with that" frees people to choose to use it for the reasons that should matter: because it's fast (coming soon!), because it has a native threading model that can give your app greater throughput, because it seamlessly integrates Rails with the 4 million lines of backend Java code you've spent the last 5 years developing.
Aslak: Cooperation between JRuby developers and developers of other Ruby projects helps JRuby become more compatible with the standard Ruby runtime. I hope more Ruby developers will join the effort. I'm sure they will.
What affect do you see RSpec on JRuby having on the wider Java world?
Charles: I think it's certain that dynlang-based testing frameworks — especially RSpec — are going to make TDD and BDD far easier on the Java platform than anything of the Java-based tools available today. There's no doubt in my mind.
Aslak: You can use RSpec running on JRuby to write executable specifications for standard Java code! Consider this example:
require 'java'
include_class 'java.net.ServerSocket'
context "ServerSocket" do
specify "should know its own port" do
server_socket = ServerSocket.new(5678)
server_socket.localPort.should == 5678
end
end
RSpec is a powerful alternative to JUnit and other similar tools. Being able to generate documentation that expresses the code's intent is really, really powerful. I'd almost go as far as saying it's a paradigm shift in programming. For example, running the spec above with the --format specdoc option would produce:
ServerSocket
- should know its own port
Another impact of RSpec on JRuby is general adoption of Ruby in the Java community. People are usually less sceptical to introduce new tools in the build environment than in the production environment. Since RSpec fits in the java build environment (you can call it from Ant) it's going to be easier to try out for people who just want to get a feel for what Ruby is - without having to deal with as much red tape as they'd typically have to deal with if they were to introduce it in the production environment.
This way people on Java projects can use RSpec on JRuby to get familiar with the Ruby language in a relatively risk-free context. This is an important step towards more widespread understanding of the powers of Ruby and its general adoption. I think this will make more people willing to take the next step - develop full-scale applications in Ruby and run it on JRuby (or on classic Ruby for that matter).
Michael: RSpec is a beautiful example of what's possible in Ruby: a natural, logical, consistent & delightful API. I hope that RSpec on JRuby, besides becoming the de-facto way to develop BDD for the Java platform, also causes more Java developers to sit up and notice Ruby as a language with incredible potential for all manner of problems. The 10 minute 'build a blog in Rails' videos have been a useful draw card but what better way to convey Ruby's power to Java developers than to have them experience it themselves by putting JRuby and RSpec to work in a current project?
Anything else you'd like to add?
Charles: I hope other Ruby developers will start looking at JRuby as well. The reality now is that JRuby is very close to full Ruby 1.8 compatibility, and more and more folks are going to be using it. Ruby app and framework developers should embrace that fact and start designing their apps with implementation-independence in mind. We promise to help however we can.
Aslak: Both SUN and Microsoft have started to take dynamic languages seriously. I recently attended Microsoft's TechEd conference in Barcelona, and there was a lot of buzz about Python on the CLR. We're seeing the same kind of buzz around Ruby on the JVM. When the big ones embrace dynamic languages, the developers sitting on the fence will follow suit. Ruby and Python are coming out of the closet now.
The past 2-3 years have seen a big drift towards "lightweight" java web and persistence frameworks - away from the complex J2EE stack touted by SUN's design-by-committee enterprise architects.
Today it's possible to write much simpler Java applications than 3 years ago. I think JRuby will push this trend even further. Ruby's language features will to some extent obsolete the need for separate AOP libraries, dependency injection and other "glue" features provided by lightweight frameworks that Java people (me included) have invented in order to simplify things.
Less infrastructure means more focus on code delivering business value. Ruby/JRuby makes it easier to write loosely coupled code - and less code. Couple this with increased adoption of automated testing (where RSpec plays a role) and you have a bright future for outcome-focused programmers and people who are willing to hire them.
Michael: I'd like to put a thought out there and hope someone runs with it.
Charles mentioned the work being done by RubyTests in creating a testing kit for Ruby implementations. He's also the creator of the RubySpec wiki — a community effort to document what Ruby is. Matz has also made positive noises about RubySpec and seems to approve of the movement towards specification. I see tremendous potential synergy between the two efforts.
My fear with something like RubySpec is that if it's really successful and has hundreds of pages documenting Ruby, its core and standard library etc, that this may become difficult to maintain over time. On its own it will require constant revision to ensure it stays accurate. Ideally I'd like to see the output of RubyTests drive a large part of the content in RubySpec but I know that standard unit tests don't provide the right abstractions for this - this is where RSpec comes in. What if a large part of the effort to document 'what is Ruby' was driven by RSpec specifications? We'd have an executable set of specifications we could point at any Ruby implementation to determine how well they stand up. We could also extract human friendly prose from such specifications and publish those to the wiki along with reports indicating which implementations fall down and where they do so.
Granted, there are aspects of 'what is Ruby' you simply can't express in a specification language built on top of the language you're trying to specify - and you need to write that in prose or in the language of maths. On the other hand there are plenty of things that could do well with RSpec-style specification - the core and standard libraries are prime candidates.
A trivial example is TCPSocket.new(remote_host, remote_port). Remote_port can be specified as a fixnum _or_ as a string so long as the string can be converted to a fixnum or is a well-known service name. Where is that specified? It's not in the rdoc and its support could change between any given release of MRI or JRuby or RubyCLR or Ruby on Symbian.
In a comment on my Profiling and ruby-prof: Getting Specific post, Thorsten asked:
If you profile just a part of your code, is it possible to accumulate over multiple "parts"? For example, I want to profile a Rails action invocation, but I really want to profile 100 invocations. So I want to start/stop the profiling 100 times just around the action invocation and at the end I want to print out a single profile (not 100). Can this be done?
Instead of answering him there, I thought it would be worth devoting a post to the answer, not because it's hard to answer but so that the answer doesn't get lost. (And maybe to help people see how to find things like this for themselves.
Let's start with a little code sample:
require 'profiler'
Profiler__::start_profile
puts "profiling"
ary = (1..10).to_a
puts ary.join(" ")
Profiler__::stop_profile
Profiler__::print_profile($stderr)
profiling
1 2 3 4 5 6 7 8 9 10
% cumulative self self total
time seconds seconds calls ms/call ms/call name
100.00 0.01 0.01 1 10.00 10.00 Array#join
0.00 0.01 0.00 1 0.00 0.00 Enumerable.to_a
0.00 0.01 0.00 4 0.00 0.00 IO#write
0.00 0.01 0.00 1 0.00 0.00 Range#each
0.00 0.01 0.00 10 0.00 0.00 Fixnum#to_s
0.00 0.01 0.00 2 0.00 0.00 Kernel.puts
0.00 0.01 0.00 1 0.00 10.00 #toplevel
Now, let's change the code so that we're profiling two different things:
require 'profiler'
Profiler__::start_profile
puts "profiling"
ary = (1..10).to_a
puts ary.join(" ")
Profiler__::stop_profile
puts "not profiling"
Profiler__::start_profile
puts "profiling again"
1.upto(100) { |i| i += 1 }
Profiler__::stop_profile
puts "back out of profiling"
Profiler__::print_profile($stderr)
profiling
1 2 3 4 5 6 7 8 9 10
not profiling
profiling again
back out of profiling
% cumulative self self total
time seconds seconds calls ms/call ms/call name
100.00 0.01 0.01 1 10.00 10.00 Integer#upto
0.00 0.01 0.00 1 0.00 0.00 Kernel.puts
0.00 0.01 0.00 2 0.00 0.00 IO#write
0.00 0.01 0.00 100 0.00 0.00 Fixnum#+
0.00 0.01 0.00 1 0.00 10.00 #toplevel
Okay, so the first bunch of profiling data got stomped on. That answers Thorstens question. But maybe not the question he meant to ask. If he's trying to benchmark his code to see how fast it is, the Benchmark library is a much better choice than Profile, Profiler__, and ruby-prof (but not a perfect answer either ... hmm, that seems like another blog post I need to write). If he wants to analyze Rails stuff and not worry about the Ruby code itself, he might want to look at Rails Analyzer by Eric Hodel (and especially at Action Profiler).
If you found this post helpful, you might want to look at my ruby-prof post collection.
Recently, I spent some time interviewing Google Summer of Code participant Kevin Clark (of mkrf fame) and his mentor Caleb Tennis. Being able to talk to these two about there experiences was a great opportunity. I think there's some real value for potential participants for future participants and mentors. Take a quick read through and see what you think.
To really get us going, would you mind introducing yourselves?
Caleb: I'm Caleb Tennis, a married 20-something Ruby hacker living in Columbus, Indiana. My formal background in in Electrical Engineering, but I spend a majority of my time in the software realm.
The company I work for, AEI is involved in the diesel engine industry. We make products that are used during engine testing, and we also run a facility that provides rooms (commonly known as test cells) to perform the testing. My main job is focused around the hardware and software that keeps the testing facility running. This includes things like our main software controls ateach test cell and the engine inside of it.
A large portion of our software is written in C++, but many of the utilities we use are Ruby based. In particular, we make heavy use the QtRuby toolkit (I just happened to have written a book on it ...) and utilize Rails for a number of internal websites.
Kevin: I'm Kevin Clark. I'm 21 and a few weeks ago decided to cut my senior year at UC San Diego short to work with my friend Josh Susser at Powerset Inc in San Francisco.
I'm known for one of my Rails plugins, ARTS, and my Ruby and Rails centric blog, Gluttonous.
I've been working professionally with Ruby and Rails for about a year and a half, and continue to use it with my work at Powerset.
How about introducing mkrf too (for anyone that wasn't at RubyConf)?
Kevin: mkrf (make rakefile) is a library intended to make building extensions to Ruby easier. It's modeled after mkmf which is in the standard library and generates Makefiles to build C extensions for Ruby. The problem with mkmf is that it's large and unweildly, and nearly impossible to clean-up due to both the coding style and the lack of automated tests. mkmf also isn't object oriented or modular which makes reuse of its code severely difficult at best and impossible at worst.
mkrf tries to fix the issues that exist in mkmf and at the same time make it easier to use for everyone involved. It's object oriented, well tested, and much cleaner than its predecessor. This allows for reuse and make maintenance easier. It also uses rake as a build system rather than make which is easier for many rubyists to maintain.
How did the two of you get involved with mkrf and the Summer of Code?
Kevin: I was finishing my Junior year at UC San Diego and wanted a chance to hack on a Ruby (non Rails) project and become more involved with that side of the community. I had heard about the Summer of Code the year before and been disappointed that Ruby wasn't included in the sponsored groups. This year, with Ruby as a mentoring organization I began looking around for an interesting problem I though I could solve.
I decided on mkrf after hearing Eric Hodel complain about mkmf for 10 or 15 minutes on IRC. I took a look at the codebase and decided it needed a good replacement.
Caleb:David Black put out a call for help early on asking for volunteers to be mentors. I, along with others, replied that I would be interested in helping out. Over the next few weeks applications started trickling in. Google provided a web based interface that allowed potential mentors to look at all of the project proposals and give them a score. Scores ranged from +4 ( very excited about the project and would be happy to mentor it ) to 0 (neutral about the project) to -2 ( the application isn't acceptable ).
Rubycentral received 84 applications, which the group of mentors spent a couple of weeks evaluating and discussing the relative merits. Overtime, based on the scores assigned by potential mentors, the applications became ranked based on which ones had the most interest. In the end, Google allocated 10 projects to Ruby central, meaning that the top ten ranked projects would be accepted and funded.
When the ranking process was complete, David Black then went through each of the 10 funded projects and assigned a mentor based on their comments and, most notably, willingness to mentor. I was initially assigned a different project, however one of the other mentors had expressed a lot of interest in that particular project, so I 'traded' with them and received mkrf (which I had also rated highly).
Wow! I'm kind of shocked that there were 84 submissions this year. Caleb, are there any of those that you think should get some love (either unpaid, or sponsored through some other mechanism)?
Caleb: There were, unfortunately, a lot of really good ones that didn't make it. I think the decision of which ones just barely didn't make the cutoff came down to mentor interest and the fact that we only had ten to choose. Some of the close calls included a neural network toolkit for Ruby, refactoring tools, an XML schema library, and a unix-shell like environment for Ruby. These proposals were quite good; we just didn't have the right people or capacity to support them.
It also seemed like this year there were a LOT of applications for building Rails based sites, like for classroom management, forums, and other full blown sites utilizing Rails. Most of the mentors agreed that these weren't the kinds of projects we were looking for from a RubyCentral standpoint. We wanted projects that were good for the Ruby community as a whole - if they could build off of Rails, then that's great. But we didn't see a lot of value as far from the SoC in supporting the building of pure Rails based CMS systems (and there were quite a few proposals for this very thing).
Tell me a little about how the day to day process worked?
Caleb: After the process was started, Kevin and I had some discussion regarding the types of things he wanted to tackle. He certainly had a very good idea of the types of things he wanted to address in this program. One of them was that the program mkrf was replacing, mkmf, was quite outdated and still using very old Ruby idioms that could be improved upon.
The day-to-day process for us was probably atypical from other projects, though still quite manageable. First, Kevin and I are on opposite US time zones. This means that my evenings are his afternoons, and as such the free time to meet up for discussion wasn't always there. As well, I'm not a big fan of online chat/IM clients because I find myself spending more time using them than actually getting work done, so a lot of our conversation was relegated to e-mail. I'm sure this was a bit of a distraction to Kevin, but he certainly was able to work around it.
I viewed my position as a mentor more of a spiritual guidance provider than a teacher. Kevin is already well skilled in Ruby, so he didn't require any Ruby lessons. There were many times when Kevin would ask my thoughts on which of two ways to proceed, and I was happy to provide them.
The overall process of being a mentor didn't require a lot of effort, but I think that's due to Kevin's knowledge of the domain. I basically left it up to him to define the project, pick what things to work on, and implement them. I was there to download his code, provide some peer review, and make sure the tests all passed.
One of the most defining points of the SoC that Google stressed was that the quality/quantity of code is all relative to the project and not even the most important part. They are genuinely looking to get people more integrated into the open source communities. So, an important part of the overall project was to ensure that the names of the participants, their projects and contributions, were being integrated into the community as a whole. To help with this, I encouraged Kevin to write about the project on this blog, to announce released to the Ruby newsgroup, and to find other ways to get his project and his name known. Speaking about mkrf at Rubyconf was a fantastic way to do this.
Kevin: I had a pretty good idea of how I wanted the code to feel after looking at as many gems' extconfs (extension configuration files for mkmf) as I could track down. Caleb and I talked about how I could approach the problem and I started working at it.
As I made progress (or didn't) I'd check in with Caleb who would take a look at the source and offer suggestions wherever he could. Outside of the source he encouraged me to get a RubyForge project setup and to start talking about the project in the community. Through his coaxing I pulled in a few early users who were able to give great feedback which changed many of the assumptions I had starting out.
Now that the sponsorship is over, where does that leave mkrf?
Caleb: Well, in good hands I hope. I hope Kevin continues to work on it, but even if not that's the beauty of open source - someone else can come in and pick it up and run with it.
Kevin: Unfunded, but not abandoned ;). Work has sucked up a good amount of my time, but mkrf is still on my mind and I want it to get it that little bit further so it can start being adopted throughout the community.
What did each of you learn from the Summer of Code?
Kevin: Being closer to the Rails part of the Ruby community, I hadn't really taken advantage of RubyForge or been active on Ruby-Talk. My project forced me to work with RubyForge and other tools that many Rubyists use on a daily basis, and I started taking notice with respect to what was happening in Ruby outside of Rails. It was really nice, and I feel like I'm closer to Rubyists as a whole and am able to contribute back more easily.
Caleb: It's truly a big project. A quick glance at the website shows just how many projects were involved. And in some cases, the allocation of projects was surprising. For example, Gentoo ( of which I'm a developer ) received more projects than Debian, which is arguably a bit more ubiquitous of a Linux distribution.
I'm not sure if people are aware of how much Google spent on the SoC, but this year it was over $3 Million US dollars. That's purely in payments to students and mentoring organizations, and doesn't include any administrative costs like Google employee salaries, T-Shirts, postage, etc.
What do you think could have been improved about the Summer of Code program?
Caleb: I hate being critical of Google here, because they were great sponsors to have, but I think some folks were really unhappy about the payment system. I know that the first round of student payments were very delayed, causing some frustration. As well, a lot of mentors and students not based in the US found out very late in the game that they would have to file various forms of tax paperwork or be subject to a large withholding in taxes. I know some found it easier to just take a 20% reduction in their payments vs. diving into complicated US tax forms.
The SoC time frame is based somewhat around the US college schedule, which doesn't always mesh well with non-US based students. There were many students who were still in school, with pending final exams, for the first few weeks of the project, and weren't able to get started at the same time as everyone else. As well, some had travel plans or service requirements for a few weeks during the middle of the program, which caused problems with turning in evaluations on time. I don't know how to address this particular problem, but it's definitely something that will have to be improved upon.
But it's only in its second year. I think as time moves on, Google will only get better at these things. I hope their sponsorship continues well on into the future.
Kevin: I'm with Caleb on this, the payment system is where the process broke down. Payments were months late at times which puts a burden on students to secure other sources of income rather than working on their projects. I had to spend time taking little bits of contract work in order to pay rent when I would have rather been working on mkrf. I know others who felt like they needed to drop from the problem because the payment process was delayed so far. That aside, I'm sincerely grateful to Google for putting on the program and would do it all again.
What advice would you give to next year's Summer of Code hopefuls?
Kevin: Keep on task. When things start to slip it's easy to let them get out of hand.
Caleb: This is a very good question. From a generic SoC standpoint, you need to address a couple of things:
Kevin, what do you think you'd bring to a mentoring role?
Kevin: I think it helps that I've been through the process as a student. It gives me some perspective into how that side of thing works and feels and also the de-motivators that keep you from getting the job done. I've also worked with various parts of the community through SanDiego.rb (and now SF Ruby Meetup), the different conferences, and publishing gems and Rails plugins. I feel like I could help make the journey into our little world comfortable and fun.
What advice would you give to other potential mentors?
Kevin: I think to work on any project like this you need to be truly excited about what you're doing. The payments are a good carrot, but that will only get you so far. I think mentors need to be able to help motivate the students by showing them just what sort of an impact they can have on things if they work hard. It's the classic "pulling ones self up by their bootstraps" story in a true meritocracy. It's why I love the hacker sub-culture and the Ruby community in particular. Everyone has a chance to do something new and interesting, and get other people excited about it. I think the Summer of Code gives students that ordinarily wouldn't have that sort of exposure a glimpse into that world and mentors should encourage that.
