Thursday, November 30, 2006

Summer of Code Interview

Recently, I spent some time interviewing Google Summer of Code participant Kevin Clark (of mkrf fame) and his mentor Caleb Tennis. Being able to talk to these two about there experiences was a great opportunity. I think there's some real value for potential participants for future participants and mentors. Take a quick read through and see what you think.


To really get us going, would you mind introducing yourselves?

Caleb: I'm Caleb Tennis, a married 20-something Ruby hacker living in Columbus, Indiana. My formal background in in Electrical Engineering, but I spend a majority of my time in the software realm.

The company I work for, AEI is involved in the diesel engine industry. We make products that are used during engine testing, and we also run a facility that provides rooms (commonly known as test cells) to perform the testing. My main job is focused around the hardware and software that keeps the testing facility running. This includes things like our main software controls ateach test cell and the engine inside of it.

A large portion of our software is written in C++, but many of the utilities we use are Ruby based. In particular, we make heavy use the QtRuby toolkit (I just happened to have written a book on it ...) and utilize Rails for a number of internal websites.

Kevin: I'm Kevin Clark. I'm 21 and a few weeks ago decided to cut my senior year at UC San Diego short to work with my friend Josh Susser at Powerset Inc in San Francisco.

I'm known for one of my Rails plugins, ARTS, and my Ruby and Rails centric blog, Gluttonous.

I've been working professionally with Ruby and Rails for about a year and a half, and continue to use it with my work at Powerset.

How about introducing mkrf too (for anyone that wasn't at RubyConf)?

Kevin: mkrf (make rakefile) is a library intended to make building extensions to Ruby easier. It's modeled after mkmf which is in the standard library and generates Makefiles to build C extensions for Ruby. The problem with mkmf is that it's large and unweildly, and nearly impossible to clean-up due to both the coding style and the lack of automated tests. mkmf also isn't object oriented or modular which makes reuse of its code severely difficult at best and impossible at worst.

mkrf tries to fix the issues that exist in mkmf and at the same time make it easier to use for everyone involved. It's object oriented, well tested, and much cleaner than its predecessor. This allows for reuse and make maintenance easier. It also uses rake as a build system rather than make which is easier for many rubyists to maintain.

How did the two of you get involved with mkrf and the Summer of Code?

Kevin: I was finishing my Junior year at UC San Diego and wanted a chance to hack on a Ruby (non Rails) project and become more involved with that side of the community. I had heard about the Summer of Code the year before and been disappointed that Ruby wasn't included in the sponsored groups. This year, with Ruby as a mentoring organization I began looking around for an interesting problem I though I could solve.

I decided on mkrf after hearing Eric Hodel complain about mkmf for 10 or 15 minutes on IRC. I took a look at the codebase and decided it needed a good replacement.

Caleb:David Black put out a call for help early on asking for volunteers to be mentors. I, along with others, replied that I would be interested in helping out. Over the next few weeks applications started trickling in. Google provided a web based interface that allowed potential mentors to look at all of the project proposals and give them a score. Scores ranged from +4 ( very excited about the project and would be happy to mentor it ) to 0 (neutral about the project) to -2 ( the application isn't acceptable ).

Rubycentral received 84 applications, which the group of mentors spent a couple of weeks evaluating and discussing the relative merits. Overtime, based on the scores assigned by potential mentors, the applications became ranked based on which ones had the most interest. In the end, Google allocated 10 projects to Ruby central, meaning that the top ten ranked projects would be accepted and funded.

When the ranking process was complete, David Black then went through each of the 10 funded projects and assigned a mentor based on their comments and, most notably, willingness to mentor. I was initially assigned a different project, however one of the other mentors had expressed a lot of interest in that particular project, so I 'traded' with them and received mkrf (which I had also rated highly).

Wow! I'm kind of shocked that there were 84 submissions this year. Caleb, are there any of those that you think should get some love (either unpaid, or sponsored through some other mechanism)?

Caleb: There were, unfortunately, a lot of really good ones that didn't make it. I think the decision of which ones just barely didn't make the cutoff came down to mentor interest and the fact that we only had ten to choose. Some of the close calls included a neural network toolkit for Ruby, refactoring tools, an XML schema library, and a unix-shell like environment for Ruby. These proposals were quite good; we just didn't have the right people or capacity to support them.

It also seemed like this year there were a LOT of applications for building Rails based sites, like for classroom management, forums, and other full blown sites utilizing Rails. Most of the mentors agreed that these weren't the kinds of projects we were looking for from a RubyCentral standpoint. We wanted projects that were good for the Ruby community as a whole - if they could build off of Rails, then that's great. But we didn't see a lot of value as far from the SoC in supporting the building of pure Rails based CMS systems (and there were quite a few proposals for this very thing).

Tell me a little about how the day to day process worked?

Caleb: After the process was started, Kevin and I had some discussion regarding the types of things he wanted to tackle. He certainly had a very good idea of the types of things he wanted to address in this program. One of them was that the program mkrf was replacing, mkmf, was quite outdated and still using very old Ruby idioms that could be improved upon.

The day-to-day process for us was probably atypical from other projects, though still quite manageable. First, Kevin and I are on opposite US time zones. This means that my evenings are his afternoons, and as such the free time to meet up for discussion wasn't always there. As well, I'm not a big fan of online chat/IM clients because I find myself spending more time using them than actually getting work done, so a lot of our conversation was relegated to e-mail. I'm sure this was a bit of a distraction to Kevin, but he certainly was able to work around it.

I viewed my position as a mentor more of a spiritual guidance provider than a teacher. Kevin is already well skilled in Ruby, so he didn't require any Ruby lessons. There were many times when Kevin would ask my thoughts on which of two ways to proceed, and I was happy to provide them.

The overall process of being a mentor didn't require a lot of effort, but I think that's due to Kevin's knowledge of the domain. I basically left it up to him to define the project, pick what things to work on, and implement them. I was there to download his code, provide some peer review, and make sure the tests all passed.

One of the most defining points of the SoC that Google stressed was that the quality/quantity of code is all relative to the project and not even the most important part. They are genuinely looking to get people more integrated into the open source communities. So, an important part of the overall project was to ensure that the names of the participants, their projects and contributions, were being integrated into the community as a whole. To help with this, I encouraged Kevin to write about the project on this blog, to announce released to the Ruby newsgroup, and to find other ways to get his project and his name known. Speaking about mkrf at Rubyconf was a fantastic way to do this.

Kevin: I had a pretty good idea of how I wanted the code to feel after looking at as many gems' extconfs (extension configuration files for mkmf) as I could track down. Caleb and I talked about how I could approach the problem and I started working at it.

As I made progress (or didn't) I'd check in with Caleb who would take a look at the source and offer suggestions wherever he could. Outside of the source he encouraged me to get a RubyForge project setup and to start talking about the project in the community. Through his coaxing I pulled in a few early users who were able to give great feedback which changed many of the assumptions I had starting out.

Now that the sponsorship is over, where does that leave mkrf?

Caleb: Well, in good hands I hope. I hope Kevin continues to work on it, but even if not that's the beauty of open source - someone else can come in and pick it up and run with it.

Kevin: Unfunded, but not abandoned ;). Work has sucked up a good amount of my time, but mkrf is still on my mind and I want it to get it that little bit further so it can start being adopted throughout the community.

What did each of you learn from the Summer of Code?

Kevin: Being closer to the Rails part of the Ruby community, I hadn't really taken advantage of RubyForge or been active on Ruby-Talk. My project forced me to work with RubyForge and other tools that many Rubyists use on a daily basis, and I started taking notice with respect to what was happening in Ruby outside of Rails. It was really nice, and I feel like I'm closer to Rubyists as a whole and am able to contribute back more easily.

Caleb: It's truly a big project. A quick glance at the website shows just how many projects were involved. And in some cases, the allocation of projects was surprising. For example, Gentoo ( of which I'm a developer ) received more projects than Debian, which is arguably a bit more ubiquitous of a Linux distribution.

I'm not sure if people are aware of how much Google spent on the SoC, but this year it was over $3 Million US dollars. That's purely in payments to students and mentoring organizations, and doesn't include any administrative costs like Google employee salaries, T-Shirts, postage, etc.

What do you think could have been improved about the Summer of Code program?

Caleb: I hate being critical of Google here, because they were great sponsors to have, but I think some folks were really unhappy about the payment system. I know that the first round of student payments were very delayed, causing some frustration. As well, a lot of mentors and students not based in the US found out very late in the game that they would have to file various forms of tax paperwork or be subject to a large withholding in taxes. I know some found it easier to just take a 20% reduction in their payments vs. diving into complicated US tax forms.

The SoC time frame is based somewhat around the US college schedule, which doesn't always mesh well with non-US based students. There were many students who were still in school, with pending final exams, for the first few weeks of the project, and weren't able to get started at the same time as everyone else. As well, some had travel plans or service requirements for a few weeks during the middle of the program, which caused problems with turning in evaluations on time. I don't know how to address this particular problem, but it's definitely something that will have to be improved upon.

But it's only in its second year. I think as time moves on, Google will only get better at these things. I hope their sponsorship continues well on into the future.

Kevin: I'm with Caleb on this, the payment system is where the process broke down. Payments were months late at times which puts a burden on students to secure other sources of income rather than working on their projects. I had to spend time taking little bits of contract work in order to pay rent when I would have rather been working on mkrf. I know others who felt like they needed to drop from the problem because the payment process was delayed so far. That aside, I'm sincerely grateful to Google for putting on the program and would do it all again.

What advice would you give to next year's Summer of Code hopefuls?

Kevin: Keep on task. When things start to slip it's easy to let them get out of hand.

Caleb: This is a very good question. From a generic SoC standpoint, you need to address a couple of things:

  1. Your project needs to be of the right scope. The mentoring team is quite familiar with the project being represented, so as such they will have a pretty good idea of how complex your proposed project is. If it's way too simple, you probably won't get picked. As well, if it doesn't seem possible to complete the project in the scope of the summer, it won't get picked as well. Remember, mentors want to pick projects they think will be completed.
  2. The quality of your application is critical. If English isn't your strongest language, you really need to get help in making the application read well. Some applications were very difficult to understand, so they received little attention. Introduce yourself, make your project goals clear, and explain why you want to be involved in doing this.
  3. You need to be committed to doing the project. This year's SoC had a completion rate of 82%, meaning that 18% of the project didn't make it to the final evaluation. In general, this is due to the abandonment of the project by the applicant. Some just got in over their heads, some just lost interest. Think of the SoC as any other paying job, one in which you are being evaluated by a large diverse group of peers. Make sure you're willing to see the project through, even if it ends up being difficult or time consuming. You were the one who proposed doing it.

Kevin, what do you think you'd bring to a mentoring role?

Kevin: I think it helps that I've been through the process as a student. It gives me some perspective into how that side of thing works and feels and also the de-motivators that keep you from getting the job done. I've also worked with various parts of the community through SanDiego.rb (and now SF Ruby Meetup), the different conferences, and publishing gems and Rails plugins. I feel like I could help make the journey into our little world comfortable and fun.

What advice would you give to other potential mentors?

Kevin: I think to work on any project like this you need to be truly excited about what you're doing. The payments are a good carrot, but that will only get you so far. I think mentors need to be able to help motivate the students by showing them just what sort of an impact they can have on things if they work hard. It's the classic "pulling ones self up by their bootstraps" story in a true meritocracy. It's why I love the hacker sub-culture and the Ruby community in particular. Everyone has a chance to do something new and interesting, and get other people excited about it. I think the Summer of Code gives students that ordinarily wouldn't have that sort of exposure a glimpse into that world and mentors should encourage that.

No comments: