Tuesday, June 30, 2009

Ruby Hoedown 2009 mini-Interview with Jeremy McAnally

Jeremy McAnally (@jm) is a good friend, and we've worked together on regional Ruby conferences and other projects. With the Ruby Hoedown looming, I thought it was about time to sit down with him for a mini-interview about his free conference.

How have the community and your sponsors responded to making the Ruby Hoedown free?

Jeremy Everyone has largely been in two camps: "Wow that's awesome!" and "What? Are you crazy? How are you going to do that?" I find myself somewhere between the two.

You've even announced a plan to help encourage registration, where did this idea come from?

Jeremy I was thinking about how to get more sponsors involved and how to get them involved with everyone more effectively. Book giveaways and things like that are great, but then only 5-6 people are getting to benefit and the sponsor is really only "touching" those people. Around that same time I was trying to think of another way to differentiate the Hoedown and add value to the conference, and the ideas sort of gelled together around the time one of my co-workers was talking about the MacHeist bundles he'd gotten this year.

You're getting really close, but this is the hairiest time as you get all the last minute stuff moving. How are you feeling about this year's Hoedown?

Jeremy I feel great. There's actually a lot less hassle this year than last year, so it's been going pretty smoothly thus far. I may not feel the same way in a month or so, but for now, I'm feeling good!

There are just hours left for people to get proposals in. Is there anything specific you're looking for?

Jeremy Anything even vaguely Ruby-related. I love a good technical talk, but I'd also like to hear why pomodoro timers are the best things since sliced bread or why eating celery can help improve not only your test coverage, but also your overall quality of life.

Wednesday, June 24, 2009

A Ruby Couple: Interview with James and Dana Gray

It's another week without a Questions Five Ways discussion, but I've got another great interview that more than makes up for it. James Gray (@JEG2) is very well known in the Ruby community. His wife, Dana, is less well known, but won't stay that way for long. Fresh from her Ruby presenting debut, a lightning talk on Ruby regular expressions at MWRC, the two of them are embarking on a joint training session at the Lone Star Ruby Conference.

I asked the two of them if they wouldn't mind doing a short interview with me, and am really happy that they agreed. Here's what we covered.

Dana, if I understand correctly, you gave you first presentation at MWRC this spring. Now you're teaching a class with James. That's a pretty steep curve. Other than having a great team member, what's helped you navigate it?

Dana I think it goes without saying that James is an amazing mentor and a very gifted programmer and without his patience and enthusiasm, I certainly wouldn't be where I am now. But you asked aside from him, so...

Ironically, teaching comes naturally to me, especially in the tech world. Before I came to work with James, I was, among other things, the software trainer for sales reps at a big food company. I was responsible for not only developing the software system for managing their business relationships but for teaching them how to use it as well. So training at a tech conference where I get to teach people who already speak my language, well, that seems pretty much like having cake and getting to eat it too.

I think the other thing that has really helped me grow as a programmer is simply doing it every day, five days a week. If I don't work and learn every day, I don't pay the mortgage and that is a pretty strong motivator. This industry wakes up in a new world every other month or so and to survive you have to be willing to learn new stuff all the time. I'm just glad LSRC gives me the opportunity to share some of what I've learned with others.

James, you've been a member of the Ruby community for really long time, and you've done a lot for it. Which experiences stand out as things you want to pass along in your training?

James The Lone Star Ruby Conference trainings, which this will be my second year doing, are a neat opportunity for guys like me. I'm really just a pretty average programmer, but it turns out that I can teach a little. In fact, I use to teach chess for a living.

I have seriously looked at doing private trainings where I live in OKC, but that's a really big gamble. To get it going I would need to commit a lot of resources and then just pray that 20 or so people would be willing to fly out here for a few days and pay what I'm asking. Otherwise, I could take some pretty heavy losses. I haven't been brave enough to try that yet.

At the LSRC though, all I need to do is show up and teach. Plus with hundreds of programmers coming in, surely a handful will be interested in taking the class. It's easy for them too. LSRC, which I jokingly call The Foodie Conference, will even feed all of us. That makes it a great fit all around.

How did the two of you come up with the idea of putting on a training course together?

James I did a training at last year's conference with Gregory Brown called The Ins and Outs of Ruby I/O. It being our first big training, we did make a few mistakes. The main error was that we only asked for three hours since we were scared to commit to a full day. Then we planned about two days worth of content and didn't end up getting to everything. However, we had a terrific group attend. Even Matz dropped in and helped answer some questions. The end result was that it turned out pretty great, in my opinion.

This year Gregory and I both had good ideas for new trainings and we couldn't decide what we liked better. So we found more help and split into two teams. Gregory, with the help of Brad Ediger, will be doing a pair of trainings inspired by Ruby Best Practices (his new book). Dana and I joined forces to deliver the Moving to Ruby 1.9 Workshop.

We have all been waiting for Ruby 1.9 to be ready for the road for so long, I think some of us have actually missed the fact that it has happened. It officially has a production release, Rails runs there now, and we're definitely seeing people start to make the move. The time has come. It's a big jump though. A lot has changed. For example, I was able to write a series of eleven blog posts about just one of the big changes. The training gives us a great opportunity to dig into all the new stuff and show people both what you have to know and also just how you can use the new features to improve your Ruby. Hopefully that makes the training a good source of knowledge that a lot of us are looking for right about now.

Dana Last year at LSRC, I took James and Greg Brown's training and was excited to see how well it was received. It was pretty different from my previous training, where the sales reps I trained didn't really get a choice. Here were people who not only volunteered to take this class, they paid to do so. So when James said Greg was thinking about doing his own training, I volunteered to help James give his full day training. I was excited by the idea of being involved in the knowledge-share of Ruby's future. And it is a good opportunity for me to learn about Ruby 1.9, since I have to teach it. :) Besides, my experience as a trainer will help James stay on track. He tends to hot air balloon. :) I'll help develop and break down the material into digestible pieces and lots of hands on labs.

James She's right, I need the help.

That was probably the second mistake of last year's training: we built it too much as a huge brain dump from Gregory and myself. The attendees endured and even steered us a bit with their discussion, but it was definitely an endurance test for them. This year, Dana and I are planning a much more interactive environment that will be much better for relaxed learning.

What other joint Ruby activities do you foresee yourselves working on?

Dana Most likely more conferences and running the OK Ruby users group. Not to mention running our company, which keeps us plenty busy.

James That's a great question I'm not sure I know the answer to. Dana and I are best friends who really enjoy each other's company, in addition to being married. We are those rare people that can spend all day in each other's company and not get tired of ourselves. That makes us well suited to work together.

For now, we've been traveling to the Ruby conferences together, working on our speeches together, and obviously building applications together. Now we have this training to teach together. Who knows what we will find to try next.

Dana doesn't do as much open source work as I do, but I do have one project I keep hoping I'll be able to drag her into eventually. . .

There are a lot of under represented (or misrepresented) groups in the Ruby community (and other tech communities). The two of you seem to be doing really well. What are some of the keys to your success?

James My secret weapon is actually Dana.

I ran this business for many years on my own. It did OK. It took care of me and things gradually improved. However, when Dana came to work for me, almost exactly a year ago now, is when things really took off. We have all the work we can handle, we're working on some killer cutting edge projects, and we're making it to more conferences. I guess she's what my business was always missing!

Dana I think the biggest key to our success is simply us. James and I are very close. We work well together in just about everything we tackle, from buying a new car to running a business. We understand each other and we listen to each other. We are intimately aware of each others strengths and weaknesses and I think we are extraordinarily lucky that we compliment each other like that. We work with some great people out there, like the guys at Highgroove, which helps us focus on the things we like to do. And we stay away from anything that we feel is not a good fit for our goals. We work hard but try not to get too focused on work at the expense of anything else.

James We did make some good decisions when we built our programming business. My mother does accounting for small businesses and she really helped me out there. You would be surprised what a difference that makes. I see people around me frequently make classic mistakes she steered me away from right from the beginning. It's hard to be successful when you start off with big disadvantages. I think I could teach another class just on the right ways to build a small business.

Friday, June 19, 2009

People Behind GoGaRuCo, Josh Susser

I didn't get a Questions Five Ways discussion done this week, hopefully I'll get that back on track next week. I did finish up another project I've been working on for far too long though, an interview with Josh Susser (@hasmanyjosh), one of the GoGaRuCo organizers. Josh is a longtime member of the Ruby community, and a very smart guy. I'm grateful that he took a the time to talk with me, I hope you enjoy this interview too.

What motivated you to organize a regional Ruby conference?

Josh I started joking last month that if you don't think you're getting enough email, you should organize a conference.

Actually, I got the idea for doing a Ruby conf in SF about two years ago. We've had some other Ruby events in the area, but with all the stuff we have going on here in SF it seemed there was a huge community that was being under-served. It was almost embarrassing that all these other cities had awesome local confs, but we didn't. I also wanted to do something to help energize the local Rubyists. I think I felt like I wasn't getting enough out of the local community, and I feel that the best way to change that is to put more into it yourself.

Who else is/has been involved in organizing GoGaRuCo?

Josh Leah Silber has done most of the heavy lifting for organizing the conference, dealing with sponsors and logistics. I've been focusing on the technical program and wrangling speakers. We're a pretty good team. We've also had a lot of support from our employers, Pivotal Labs and Engine Yard - while they aren't technically producing the conference, they are doing a lot more than sponsors usually do for a conference. And we had a great team of volunteers too - without them we would have had the lamest conference ever.

Now that you've recovered from the first go-round, are you ready to get back on the horse and hold GoGaRuCo 2010?

Josh Sure, both Leah and I are up for that. We're really happy with how our first year went, but already have ideas for how to make next year even better. We'll probably expand from 200 to about 250 people because you can do more that way, but we want to stay intimate and single-track.

Who's the target audience for GoGaRuCo? How are you reaching out to them?

Josh We are definitely focused on people who write Ruby code for their living, or at least for a lot of their hobby time. Some shows have a lot of content for managers, VCs, and business people, but that's not us. If you're not a programmer, you might find this conference boring. But if you are a programmer who loves writing Ruby code, this is the conference you don't want to miss. We also assume that by now we don't have to have a lot of introductory content. There's a place for that kind of material, but we felt for what people were paying they deserved to see advanced material, and stuff that hasn't been seen anywhere else.

Reaching out has been pretty easy. We marketed the conference with blogs, Twitter, announcements at local meet-ups, and posting to email lists. Basically word of mouth and social networking tools.

A lot of regional Ruby conference organizers tout something unique about their conference. What makes the GoGaRuCo special?

Josh I think San Francisco is pretty special! And we tried hard to put together a program that represents that SF character. It's not just about improving your technical skills, but also what you use those skills in service of. So we had a couple of talks that weren't as much about the technology as they were about what you can use all this amazingly powerful technology to accomplish, and what kind of impact that can have on people's lives.

We also tried a few experiments with how we ran the conference. Things we'd always thought we'd like to see, improvements we wanted to try out. One of them was how we put together the program. We didn't do the typical CFP then have a couple people select from the proposals. I knew from the start there were certain people I wanted to present, so I just invited them to speak. That filled up half the program. The other half was selected from proposals through voting by registered attendees. I think that got us some speakers that otherwise might not have submitted proposals or had them chosen by a small committee. People really liked that idea and the program seems to have benefitted from it. We did get some surprises in the voted talks, which can be good or bad depending on what you were expecting. So far, the feedback has been mainly positive. We rushed the whole voting thing, but with a little preparation I'm sure we can make it work much better next year.

I loved the GoGaRuCo Wrap ... Where did the idea come from? Has it had the payoff/impact that you were hoping for?

Josh Thanks. The idea for the Golden Gate Ruby Wrap came when we were thinking about doing color printed programs, which everyone seems to do but costs a lot and has low value for both attendees and advertisers. You don't really need a program for a single-track conference anyway. But you do need some place to put the sponsors' ads! So we thought we should do a PDF zine to save paper and also create something of lasting value. It's good for everyone, including the advertisers. Instead of 200 people looking at a program for 30 seconds then throwing it on the floor, many more people get to spend a while reading the zine and will keep it around for reference, maybe for a long time. We got some very good write-ups for the talks, have some great photos taken by Andy Delcambre, and kept the focus on the community by including interviews with a number of attendees. So far we've had over a thousand downloads, so I think the impact is already pretty good. We didn't really get to include everything we wanted to, but next year we'll know how to do it better. I'm also hoping the idea catches on and we see more conferences produce what amounts to proceedings-in-brief. We don't mind folks copying here us at all.

It seems like the regional Ruby conference field is getting congested. What, if anything, should we be doing about it?

Josh That seems like asking what we should do about there being so many reality TV shows. I don't think there is a "we", and there isn't anything to do either. The field of regional conferences is an ecosystem, and those things tend to take care of themselves. Rubyists will go to the conferences they like, sponsors will help fund the conferences that are worth their support to them, and those conferences will continue to be produced. The others won't. We've seen that happen already. I think it's a positive thing, evolution selects the ones people want to see continue.

What other language(s) would you like to see have a presence at regional Ruby conferences? (Or should we be soley focused on Ruby?)

Josh I think keeping the main focus on Ruby is what people want in a Ruby conference, but there are certainly complementary languages and other technologies that can add to the value people get from Ruby. Nathan Sobo's talk on Unison and June was a great example of how Ruby and JavaScript frameworks can work together to overcome a difficult problem. Ruby doesn't exist in a vacuum, so it makes sense to show things that can help you use Ruby better.

If someone wants to get a regional conference started, what advice would you give them?

Josh The thing nobody ever tells you about running a conference is how hard it is on your body. I've said that to other organizers and they all smile painfully and nod. It's also demanding on your personal life, and can be really hard to work out with your job too. So I'd say you really have to want to do it, and you have to be ready to make a few sacrifices and deal with a lot of stress to make it happen. But it's also really worth it, so I'm not complaining.

The most important advice is that, like anything worth doing, you can not do it alone. It's crucial to have a good core team of two or three people who can keep things moving, and a larger team of people who can help get things done.

And lastly, make sure you have fun or you're not doing it right. Leah and I took this on to create the conference we ourselves always wanted to go to. If you're going to do something this big, it has to be an expression of yourself and what you're passionate about. Don't just try to be like everyone else, but use your own perspective and creativity to make something that is your own. Otherwise, you might as well go to someone else's conference.

Click here to Tweet this

Wednesday, June 10, 2009

Reading and Testing Your Legacy Code

In this week's installment of Questions Five Ways, we're talking about testing and code reading as tools for dealing with legacy code. This week's participants are Cory Foy (@cory_foy), Dave Thomas (@PragDave), Antonio Cangiano (@acangiano), and Tim Bray (@timbray).

What is the relationship between testing and code reading when dealing with legacy code, and how can we use the two processes to improve each other (and the code we're dealing with)?

Cory When Pat first ran the question by me, I really liked it. I think primarily because it goes to the heart of what we've been saying for years, what Bob Martin says in Clean Code, and some of the core of the Craftsmanship movement — we have to be good at code, which means we need to be good at reading it.

Dave Sometimes I look at legacy code because I have to change it. Sometimes I look at it because I like to learn interesting new techniques (or learn not to do "interesting" new techniques). In both of those circumstances, I find code reading skills and unit testing to be complementary and mutually supportive.

Code reading lets me grok the structure of the code: to see how it hangs together, what the structure is, the idioms used by the developer, and so on. But all that research really just leads to a set of assumptions. And whenever I find myself building up a backlog of assumptions, I like to validate and verify. And that's where the tests come in. If the code permits, I'll slap in a quick test to document and validate an assumption I've made. If I think to myself ("OK, so she's storing dates as ISO strings" I'll see if I can write a test to check dates are indeed stored that way.)

If my reason for looking at the code is simply research and interest, I probably won't go much further than putting the occasional test stake in the ground. If instead I'm planning to make some change, I'll go further, and attempt to isolate the area where I'll be making the change with a ring of tests before making any changes. This is often tricky (sometimes to the point where there's a pragmatic decision to relax the standards a bit—there are few times where it's worth refactoring 10,000 lines of code simply so you can test a change that turns a field from green to blue). But every test you do manage to inject is one more fixed point. And the more fixed points there are, the fewer permutations you have of things that can go wrong.

You ask you you can use testing and code reading to improve each other: I think the answer is to use them in this way, iterating between the two, each time around the loop reducing your uncertainty.

Antonio Code reading is a required step to approach and understand any codebase. The existence of tests greatly simplifies the process of investigating features and how they work, while aiding the evolution of the existing code, thus providing some degree of confidence. In the excellent book "Working Effectively with Legacy Code", Michael Feathers characterizes legacy code as hard to work with, particularly due to the lack of regression tests. In my experience, while this isn't always the case, I've found that it's fairly common to deal with code that has no form of automated testing. In such situations, code reading and testing complement each other particularly well.

In the same book, Feathers defines "characterization testing" as the process of observing and capturing the current behavior of the application, so that tests can be added when none are already in place. Reading the code, running it, and creating some sort of safety net through testing allows programmers to gain insight into the codebase while attempting the arduous task of refactoring and improving legacy code. As Dave mentioned, it's a fundamental way to validate and verify assumptions.

Cory The interesting thing is that it is so natural to make the assumption that the skillset exists that we rarely mention it. I'm glad Antonio brought up Characterization Tests from WELC, because that was my first thought. I can come into a team that has a big heap of legacy code dropped on them and quickly show them how to write characterization tests. But if they don't already have the knowledge of how to read code, how to make the inferences, how to scan code blocks and branching structures - then we are going to have a much harder time.

The big joke is that Perl is "write-only" code. But I think too often any code we come across is write-only because we don't want to take the time to master the skill of reading code. Which really is the core of the NIH (Not Invented Here) syndrome - "I don't understand this code in 23 seconds of looking at it, therefore I reject it and declare we need to rewrite the entire module"

In fact, in my current team I fought the team hard against rewriting one of our applications. Oh, I agreed it needed to be rewritten - it was a monstrosity. But embedded in that monstrosity were business decisions and logic that had been built up over the past 6-7 years - domain knowledge that came from people that were no longer around.

I feel pretty confident that if our team had been much more skilled on the art of reading code - and not just producing it - we could have been able to be much more productive.

This is getting a bit long winded, but I want to tie this back to one other thing. In Lean Manufacturing (perhaps cliche, but bear with me) the concept of "stop the line" relies on people being able to know enough about what is supposed to be produced to recognize when something is wrong. I think we give lip service sometimes to "stop the line" in the software world, because we aren't calling it on other people's code. And while some of that is dysfunction of not wanting to challenge each other, some of it is because /reading/ code is considered a chore, and isn't nearly as glamorous as writing it.

You can't test what you don't understand, and you can't understand without reading and comprehending.

Dave I think that a lot of developers have that mindset, which in unfortunate, bordering on unprofessional. Perhaps that should be a qualifying interview question: "read any good code lately?"

Tim First, I haven't seen anyone echoing the so-true cliché: Reading code is harder than writing it. Furthermore, it gets harder as code gets older and grows a coat of defenses against corner cases and insane user input and so on; these obscure the original design, which is often hard enough to see through the code.

Dave I'm starting to learn the more formal side of music: I can just about read sheet music, but I read it beat by beat: it's slow and stuttery work. My oldest son is a really good pianist, and he sight reads very well. I get the impression that he kind of sucks in the shape of the whole page, and then just uses the notes to fill in the gaps. That's where fluency lies.

I think the same kind of fluency is there for the taking when it comes to reading code. When you first start, you're reading line by line. But when you get good at it, the program kind of speaks to you. You look at the shape of it, the indentation, the module and method structures, and somehow get a good feeling for how it all hangs together. The individual lines can then be read a lot faster, just as my son reads music.

Cory I think the music analogy is quite good. Great musicians can site read just about anything, but still take the time to learn it well when they need to actually do something with it.

I remember in high school our band competitions — both individual and as a group — were based on both rehearsed pieces and site reading pieces. An interesting exercise might be to pit individuals and teams in recreating logic in their language of choice — and then making the logic work in a language they've never seen before.

Tim How about reading test code? I think that's a big deal. In fact this very week I found myself forced to spelunk in among some of my own not-that-old code and had that familiar sickening "WTF-was-I-thinking" sensation. Fortunately, those particular classes had good test coverage, and ../test/* helped me a lot in figuring out why on earth, at the top of a really important loop, I'd said

next if path == '/'

Sometimes test code is easier to read than code code. Many of us go to a lot of work to eliminate duplicate logic and parameterize algorithms and encode behavior in data structures, and these are often good things to do, but they're not necessarily friends to readability. On the other hand a lot of us also tend to write unit tests in a fairly brute-force linear abstraction-free manner which while less elegant is maybe more obvious to the casual eye; and the eye is always casual, first time through.

Dave I personally read test code when I want to know how an API works. But generally, test code isn't satisfying to read, because you lose the context. You're not looking at the whole story of the program: instead you're looking at individual sentences, out of context. As such, I think test code has to be read very differently to the program itself think it's a more focussed, analytical kind of reading.

Antonio On a slight tangent to the subject, I think it's also worth considering the effect that testing has on the code that others (or ourselves three months later) will have to read. Code that was incrementally developed following a scrupulous TDD approach looks quite different from code that was developed without the aid of any unit tests, no matter how skilled the programmer.

Among many other obvious benefits, TDD leads to code that is far more modular and readable. This greatly benefits anyone involved with a code reading or code reviewing session. Reading tests surely aids in learning how an API works, but the sheer presence of good tests is a preliminary indication of the readability of the code itself.

Dave There are lot of code reading tricks. Ward Cunningham likes to load all the source into Word, shrink the font down to 2 points, and just look at the shape. I like to scroll it by really fast and just look at the patterns. But ultimately, I think the key to mastery is simply doing it. Choose code and just read it. But read it actively, not passively. Challenge yourself to answer questions about it, or find bugs in it, or look for refactorings, or whatever. But the more you do it, the better you'll get.

Cory I think the key is just do it. An adage that has been extremely helpful to me has been, "If something hurts, do it more". That holds true for TDD, for automation of tests, and for reading code. Code reading is something we all find painful, yet we don't bother to invest the time to make it right - which is as simple as looking at other code.

In fact, another helpful adage has been to learn a new programming language every year. By default you end up reading a lot of code to learn the new syntax and semantics, by default making you a better code reader.

Click here to Tweet this

Friday, June 05, 2009

More of the Ruby Mendicant: Gregory Brown's Back with Another Great Project

For the second interview in this morning's double-header, I've got a quick one with Gregory Brown (@seacreature) about his recently announced second run as the Ruby Mendicant. This is a big opportunity for the ruby community. I hope you'll pitch in a few bucks.

RubySpec has been a huge success in the Ruby Community, but it's just one part of a vision that Charlie Nutter put forward at the 2006 Ruby Implementors Summit. Another leg was an live look at how any given Ruby implementation performs against the spec . You're talking about making this happen as part of a second instance of Ruby Mendicant. Can you tell us a little more about your plan?

Greg I will essentially be trying to build and deploy a web based reporting system that aggregates RubySpec definitions and results across every Ruby implementation and language version we can get our hands on. This will help people know what the state of the specs actually are, and how well various implementations and Ruby versions run against them. I made a rough mockup of what one of these reports might look like. We already have some changes to MSpec and a skeletal database model in place that could facilitate this sort of report. The full scope of the project will emerge once I have a chance to talk with implementers and Ruby community members about what their needs really are, and I expect to have those details collected together by June 15th. But basically, I hope to support in one way or another most or all goals that Brian Ford mentioned in his idealized case of the needs RubySpec would serve.

Whatever the final roadmap is, I hope to implement something functionally useful to the community over the course of just 8 weeks. Based on the ideas I have in my head right now, if I just hacked on this in my spare time, it might be something I could put together over the course of 6 months or a year. But with me taking hard math classes in the fall, I'd rather fill my summer lows with some open source Ruby awesomeness. That's where a sustained, community funded effort, will come in handy. The cool thing is that it'll be easier to do it this time around than it was with Prawn....

What did you learn from being the Ruby Mendicant last time that will change your approach this time around?

Greg Okay, so there are two things that'll change pretty radically:
Funding and Organization.

When it comes to organization, I plan to have a fairly clear road map set up before I begin on this project. Ruby Mendicant collected 100% of funds before a *project* was even selected, which made it a very different proposal. The upshot here is that since we're mainly just building a web application to interface with an existing technology, there are less unknowns in the core needs for this project. While there are a ton of directions we could take it that may lead into dark spots, I think I can define with some accuracy a useful set of features from the get go. I don't want to get all ponies and rainbows about what you can expect from this project, things will change and I have no way of promising to deliver a certain set of features within a rigid time-line. But generally speaking, there are less variables here so long as we keep the project scoped tightly enough. This new way of organizing things might appeal more to those who were afraid of the ambiguity of purpose that the Ruby Mendicant project had.

Then there is funding, a topic I absolutely hate, but over the course of the years, I have realized is pretty important to successful free software development on all but the most sexy projects.

Since the time I finished up with Ruby Mendicant, I've been doing most of my work through a devshop called Madriska Media Group run by my friend Brad Ediger. Brad has been pretty cool about letting me spend some company time on open source here and there, which has been great. When I mentioned my ideas for what I wanted to do with RubySpec, it went over well. Madriska will be dedicating some time, money, and resources to helping me get this new project up and running. This will help out a lot, but due to the aggressive timeline and scale of the project, won't by any means cover the full costs.

So while I'm most of the way covered here in terms of funding and available resources, I'd like to help make sure I don't run into financial troubles again, like I did with Ruby Mendicant. That means I'll need to open up a donation drive even though I have the effort partially funded.

How can people contribute?

Greg Code help will definitely be desired a little later on, but right now, the community at large can most easily help out by throwing me some money. I am asking for $500 a week, for which I will guarantee at least 10 hours of dedicated work on this project. This will of course be in addition to any work that Madriska pays for, and not just a weekly minimum.

I plan to do the funding in two drives, one that's already open, and another that will open July 15th. Each of these cover 4 weeks of funding (a total of $2000). This time around, I won't need to raise all the money up front before I start, so those who are skeptical can wait to pitch in if they'd like. While work will begin officially on June 15th, I'll be preparing a roadmap and doing some preliminary planning over the coming weeks. Unlike Ruby Mendicant, you'll have a pretty clear sense of at least what my goals are before this project kicks into full gear.

Of course, those who trust me to do good work, and enjoy what I did with Prawn can go ahead and donate now if they'd like. I'm not going to complain :)

Have any organizations stepped up to help?

Greg When it comes to funding, MountainWest Ruby will match dollar for dollar on the first $500, so thanks for that Mike :) Obviously, if other organizations want to sponsor this effort monetarily, I'd be happy to take their money, so do get in touch if you're interested. What I do need to let folks know is that this is not a bounty, or a hired job, so consider your donation a gift if you make one at all. I can do my best to make sure you won't regret it, but be sure to ask questions if you don't know what to expect.

When it comes to slinging code, I've been having good conversations with the JRuby and Rubinius guys, and will reach out to the other implementations soon as well. I mostly just need their guidance on certain aspects of things because I'm fairly new to the happenings of Ruby implementations. While there is no formal agreement, I think that these guys will help out when they can, and that alone will be huge.

I think the best bet is to spend a few weeks getting together a function alpha for the community to poke at, test, and provide feedback for. After that happens, regular contributions via github would be most welcome. BTW, the working title of this project is Unity, which hopefully describes its intentions pretty well :)

Click here to Tweet this

Feedzirra and Typhoeus: an Interview with Paul Dix

With my recent Questions Five Ways series, I've gotten away from my regular interviews. To break that dry spell, I've get a pair great interviews for you. The first is with Paul Dix (@pauldix), the developer of Typhoeus and Feedzirra and budding author (see below). He and I had a good talk about how he builds such great libraries. Read on, I think you'll like it.

What kind of hacking do you do when you're not building cool new libraries for the community to play with (and argue about)?

Paul At work I'm doing Ruby stuff with a mix of Sinatra and Rails. The company I work for, kgb, is building a new web product in the aggregation/search space. That area has been my primary focus for the last three years so I get to play with stuff like machine learning and natural language processing. We're a bit early on so I'm not sure yet what I'll be using for that. Probably Java because of the availability of libraries, more speed, and better memory use/cleanup. Six months ago I did some research into the methods being used by competitors in the Netflix prize. It was really interesting stuff and I did most of the heavy lifting in Java. I started out with Ruby, but found that I needed a little more efficiency. However, I still did all my scraping and data preparation in Ruby.

During my free time I'm not getting to do much hacking these days. I'm working on a new book for Pearson called Service Oriented Design in Ruby and Rails so that takes up quite a bit of time. However, things like my new library are directly related to my day job and the topic of the book. So free time hacking along those lines will probably be fair game for the next six months.

Have you considered using JRuby to make the bridge between Ruby's accessiblity and Java's libraries easier to cross?

Paul I definitely have considered JRuby and I've used it before. However, with machine learning tasks it really comes down to speed of execution. Some jobs can take minutes to hours even with Java. Speedup factors of 2-10 times or more matter in these cases. Of course, I pulled that number completely from thin air. Depending on how it's used JRuby may be a viable competitor to Java, but I'd have to test to make sure. For interfacing with Java libraries it's definitely a good option.

How did you discover Ruby? Why have you stuck with it?

Paul In 2005 I was working as a C# programmer at McAfee. I had arranged to quit work and go back to school in the fall. Since I didn't have to worry any longer about making my living day to day in the Microsoft world, I decided it was a good time to switch up languages. My first stop was Python. I read Mark Pilgrim's Dive Into Python [now available for Python 3] and wrote a few scripts. I thought it was ok, but kept looking around and found Ruby.

I kind of picked up Rails and Ruby at the same time. I read through the first edition of Agile Web Development with Rails [now on it's third edition] and Programming Ruby [also in a third edition]. I liked it, but that's not what really made me stick with the language. I had also moved to NYC and started going to the NYC Ruby users group (nyc.rb). I think that's what really made me stick with Ruby. I enjoyed the community and how passionate everyone was. The development gains with Rails didn't hurt either.

It sounds like finding Ruby was something of a process. I'd assume you're still looking at languages as the process continues. What other languages have moved beyond catching your eye and are holding your attention?

Paul I try to be deliberate about what skills I focus on and languages are definitely a part of that. At this point nothing is really holding my attention as a replacement for Ruby. I haven't seen anything that fits that bill so instead I'm looking for supplemental languages that cover Ruby's weak spots. I'm playing around with Scala at the moment and using it to implement a something at work that will be very high traffic. I think languages geared towards parallel, distributed computing are very interesting right now.

How did you get interested in Ruby's http and xml performance?

Paul I've had an idea to build a feed aggregator/search service for a while now. Most of my free time hacking has revolved around playing with different pieces of that system. Obviously, a big part of that is pulling stuff down from the web and parsing it. I got even more into the HTTP thing with my current job. The system we're building is based on different HTTP services written in a variety of languages. I want Rails to be the front end for these, so a performant HTTP library is the first step.

Could you walk us through your process for building highly performant code?

Paul Generally I start out with something that I think is slow. Like feed parsing, for instance. I didn't like the existing libraries and I thought it could be done better. Luckily, Nokogiri had recently been released and was touted as super fast (which it is). From there I wrote a spike. No TDD, no elegant design. Just some ugly spaghetti that will actually do the basic task. That's enough to wrap a benchmark around so I run that. If my approach is fast then I figure I've proved out the concept. From there I rewrite the whole thing using TDD and and take a little more thought for API design.

On Typhoeus it was a bit different. There were already some speedy options out there, they just didn't work perfectly for me for one reason or another. So I built on the work that Curb had done and completely relied on libcurl for the real speed.

With any of my libraries where speed is a concern, some of the processing is done in C. It's just a fact of developing with Ruby. If you want performance you need to drop down to C. The truth is that my libraries really just piggyback on other developers' awesome code. It's just a matter of bringing it into Ruby or exposing it through another API.

There are two other things that I generally do as I write libraries. One is quantifiable and the other, not so much. I try to think about how many times certain sections of code will be run. This is the standard stuff you find in a data structures and algorithms course. Is an algorithm O(n) or n^2 or whatever. I'm not actually doing the big-oh calculation but I definitely think about if I'm executing something in an inner loop. It helps to have it in your head what the performance of a hash vs. .include? on array, if you're doing an eval, or any of the other things that might have an impact on speed.

The more quantifiable approach is to use ruby-prof for checking how long different calls are taking and where your memory is going. That library is crazy awesome.

Other than profiling, what kinds of code analysis are you doing, and what tools are you using to do it?

Paul I like to bounce ideas off of other people in the Ruby community. We have regular hackfests and I'll break out my code to see if people can point out where it sucks. I think code review is something that more people need to focus on.

I probably should be using things like flay, flog, and reek, but I haven't started yet. Another thing I really want to check out is Aman Guptu's perftools.rb. He gave a lightning talk on Google Perftools and his Ruby bindings to them at GoRuCo [ed. Aman's talk isn't posted yet, but hopefully will be soon].

Since you work TDD style on production code, which testing/mocking libraries do you use and why? Have you looked at tools like cucumber to guide your test writing?

Paul I've been using Rspec for a while now. I'm not a ninja, but it does what I need. I prefer the general style of 'describe' blocks and 'it' calls. I like the built in matchers and how the test code reads. I also used FactoryGirl for fixture data for a while, but my current work isn't backed by ActiveRecord so it's not applicable.

I looked at cucumber briefly, but I honestly don't like it. I know I'm in the minority in the Rails community right now for that opinion, but I found that I spent way too much time writing regexes and building up my test suite to work. I've talked to people that do client consulting that find value for client communication with cucumber style stories. I just don't have that need. I find that the regular test code is plenty readable for the people I work with and I don't have to spend a bunch of extra time testing. Ultimately, testing is just a means to an end. It's easy to get bogged down and spend an inordinate amount of time writing tests. I like to focus more on the implementation and what the code can actually do.

What approach do you use in working with C — Ruby-Inline, FFI, the traditional C API, or something completely different?

Paul I use the traditional C API. I should probably move over to FFI to make things play nicely with JRuby and Rubinius, but I haven't gotten to that yet.

What approaches to concurrency/parallelism are you finding most useful?

Paul I think the reactor pattern is really good. It's what EventMachine and libcurl multi use to fake parallelism. I find that's much easier than trying to deal with managing a thread pool and running IO through that. It's fine to run single threaded since we're using multiple processes anyway. You'll get a chance to peg your CPU even with multiple cores.

Are there other patterns (not necessarily concurrency related) that you find especially powerful/useful in Ruby?

Paul Ruby lends itself well to creating DSLs. ActiveRecord, DataMapper, and many other libraries have great little DSLs for building up classes with complex behavior with fewer lines of code. I copied those styles when creating SAXMachine and Typhoeus.

The other thing that I really like is the use of proxy objects for lazy evaluation. I used this technique in Typhoeus to avoid making HTTP calls until absolutely necessary. When you make a call it actually puts it on hold and gives you a proxy object. The remote HTTP call isn't actually made until you access something on the proxy. That style made it much easier to hide the details of gathering a group of HTTP calls before calling out to libcurl-multi.

Earlier, you mentioned that you are working on a book. When should we be looking for it to hit the shelves?

Paul I'm really early on in writing the book so it won't be out for a while. The pre-release chapters will be put on Safari Bookshelf as I finish them. The first bit should be there in the next month or so. The final version of the book probably won't be in dead tree form until March of 2010.

Click here to Tweet this

Wednesday, June 03, 2009

Good Programmers and How to Become One: Questions Five Ways

The guys behind the Ruby Best Practices Blog are my guests for this installment of Questions Five Ways. I'm a big fan of both the RBP book and the blog, so I was really excited when Gregory and I talked about doing a discussion with the team.

Robert Klemme, Gregory Brown (@seacreature), Lakshan Perera (@laktek), and James Britt (@jamesbritt) were the panel that ended up discussing my my question about good programmers.. I hope you enjoy this just as much as I have.

What makes a programmer a good (or great) programmer? What should Rubyists be doing to become better programmers?

James Very few of the programmers I've thought of as really good or great thought of themselves that way. They thought well of their skills, but scarcely presented themselves as "rock stars" or "gurus". Very rarely did they proclaim a solution to a problem as the only way, or the even the best way; they explained why it was good, and invited people to find flaws.

It's not a question of simple modesty. You get good by not being complacent, by not believing you've got all the answers.

Robert While I pondered the question(s) it occurred to me that "good" and "great" on their own do not mean much. Without further context there are a ton of possible answers in different directions. You picked the personality direction and I agree wholeheartedly to what you wrote. It is likely that people who are interested in their profession turn out better than those who are just doing a job.

Lakshan Yeah..I agree with this. However, I believe passion is the main driving factor for one to become a better programmer than their profession. If you treat programming as a profession, it could hinder you from expanding your skill-set or taking a paradigm shift. Professionalism is always bound to time, money and ROI. Open mindset with lot of passion is what ignites the real innovation.

Matz started working on Ruby in 1993, but it took him until 1995 to make a public release. Yet, it took at least another 5 years to catch the eye of the mainstream developers. DHH, also did worked more than 1 1/2 years on Rails behind closed doors. During this period, Matz or DHH surely wouldn't have imagined how their work would be accepted by the public or where this would end up. They could commit mainly because they found pleasure in working on it. For me a Great Programmer is the one enjoys, feels passionate about his own work. If something is fascinating enough to yourself, surely it would be fascinating for a majority of like minded people.

Robert The important word in my statement was "interested" and not "profession" - we probably wanted to convey the same but somehow my English seems to have failed me here. I wouldn't go as far as to claim people should get passionate about programming partly, because I'm from the northern parts of Europe and partly, because passion can narrow your mind too much. I do agree, that a strong motivation helps.

James It's said that the truth shall set you free; actually, doubt sets you free. Being convinced you've found The Truth keeps you from growing.

Doubt frees you from being trapped by a single methodology, a single programming paradigm, a single language, whatever it is you've decided the The One True Way. Even if you decide to stick with that methodology, paradigm, or language, routine questioning will help you determine that it's the still the right choice at the time.

Healthy doubt can be tricky; it's easy to slide from scepticism to cynicism, from being self-aware to endlessly self-critical. A key is to be able to recognize and appreciate the good points of what you have and what you know, while understanding that, odds are, you are almost certainly ignorant of much more. It's the difference between, "This is no good", and, "I bet there is better"

So, for example, a good developer is justified in appreciating and enjoying Ruby, but also considers that Haskell or Prolog may be worthwhile too; that one should make use of object-oriented programming, but not be so conceited as to think it is the One True Way and never bother to explore functional or logic-based programming.

Rubyists should foster the habit of constructively questioning what's around them. When someone says "This is the best [practice, tool, library, framework, whatever]", there should be a demand for concrete, verifiable proof, followed by an examination of the alternatives, just to be sure. Rubyists should not assume that everything some Well Known Rubyist says is true, or proper, or even sensible.

Including this.

Greg I agree with all of this, but I think doubt has too much of a negative connotation to it. I tend to think of what you're talking about as curiosity (and maybe you do too, just under a different name). But the important thing for me is that someone is able to conditionally accept something but still want to dig deeper. That drive is what separates those who copy and paste code from those who go back and learn how things work under the hood.

Curiosity leads to a lot of things, and one place curious developers may end up is reading the source of free software projects. Though it seemed like a scary and impossible task to me at one point, this is now the number one way that I learn. See how real problems get solved, and you'll learn a lot more than you could in a book. Naturally, when you see some unfamiliar technique, if you are the least bit curious, you'll try it out in your own code. You'll then learn where the holes were in your understanding, and fill the gaps in as necessary.

Over time when you repeat this process, it becomes easy to see what code is good in what contexts, and what practices are generally going to lead to pitfalls no matter what the context. If you make the effort to teach others about these things, whether through talks, blog posts, or even writing a book, you'll learn even more about what a not-so-great programmer you are, which will in turn, help you find ways to become better.

In the end, hard work pays off. You only get better by reading code, writing code, and then sharing what you've learned with others. The last part may seem like something extra, but it's really not. If you don't put your ideas out there, you're only a great developer in your own mind. What good does that do for you? Usually, while folks like this are too busy being awesome, the true hackers are out writing code and learning from one another.

Robert Unfortunately this attitude is not appreciated everywhere - which I painfully experience right now in the project I am currently involved in. I won't go into the details here but communication is a major issue. This stresses the importance of what you said because the attitude you described helps in these situations: it will keep you open for what the other tries to convey and help you understand his ideas before you even start criticizing them.

Another "direction" is the corporate perspective: from a company's point of view a good programmer is someone who produces code with minimal cost over the software's life cycle. This implies that the person is either extremely cheap (we often see this scenario in outsourcing situations) or produces code with less bugs and a more maintainable structure than equally paid colleagues.

I do believe though that in practice companies often do only apply short term math, e.g. when they cut costs in training or press developers to come up with results quickly which almost automatically produces suboptimal (performance, maintenance) code which needs to be fixed later.

Lakshan Becoming a better programmer is not something that you can achieve overnight. It is something you could attain by enjoying what you do and by doing it constantly. Patience and strong commitment will matter, more than the technology or the tools you've got.

Robert And you need to be open to pick things up and learn.

Lakshan There was a notion back in the day, programming is for people with strong analytical and mathematical skills. In the era memory was scare and data transferring speeds were painfully slow, writing efficient code would've been a must.

But since there are almost no such barriers today, programmers seems to enjoy lot more freedom in creativity and make their lives easy. It is evident from the beautiful DSLs and intuitive frameworks exist today.

Perhaps this is gone to the extent that one might tend to feel creativity is the most important aspect of programming. Yet again we have seen cases how performance and scaling nightmares could haunt us back, when the full focus has been on the beauty of the code rather than it's efficiency.

So according to your view, to be a good programmer what sort of balance one needs to have in terms of analytical and creative skills?

Robert I believe you need both — probably analytic skills a bit more than the creativity. The reason is twofold: for one, you do not always write new code — at least in the corporate world. Then, if you do not have the toolbox to analyze problems and craft (!) solutions your creativity will probably get you nowhere. Compare it with a carpenter: if he does not know his tools and how to actually put his ideas into wood, you won't see his ideas.

Click here to Tweet this

Like this article? reddit it!

Monday, June 01, 2009

Communities, Publishers, and Conferences.

JavaOne [h]as a *very* different feel than that of a Ruby show, obviously :PLeah Silber

Really starting to believe that small, short, regional conferences are the way to go.Andrew O'Brien

I think events of many sizes can be worthwhile -- they just have different profiles and risks/rewards.David Black

Some of the discussion recently on Twitter has made me think about how we organized MountainWest RubyConf. We've been very focused on keeping it intimate and engaging. From the comments I hear, it seems like we did a good job. It certainly feels like we've for the community behind us, and that can only help us get better each yer.

It also reminded me of some of my earlier writing about publishing. I think there's a lot of overlap in building community for a conference and for a book/publisher.

To me, I guess it boils down to conferences and books with strong ties to the community feel better. What do you think?

This post is part of a collection of articles about Publishing, Growing Markets, and Books.

Publishers Can Improve Their Markets and Sell More Books

As a writer (blogs, magazines, and a book), I've got an unhealthy interest in publishing. A number of my blog posts spring from that attraction. Here's a list to get you started: