Monday, December 22, 2008

Author Interview: Relax with CouchDB

O'Reilly is at it again, getting an open content book out there to cover an emerging technology. This time it's Relax with CouchDB by Chris Anderson (@jchris on twitter), Jan Lehnardt (@janl on twitter), and Noah Slater (@nslater on twitter) -- the book is also available as a rough cut, if you'd like to support O'Reilly and the book.

Last spring, Jan came out to MountainWest RubyConf to talk about CouchDB, and I've been interested ever since. Now that the book was on the way, I asked Chris, Jan, and Noah to sit down for a quick interview. Here's how it went:


I know this is the title of your first chapter, but I feel like we should start things out with it here too: Why CouchDB?

Noah I'm a bit of a hypertext fetishist. Serving up documents via HTTP is great, but I became interested in CouchDB because it lets me serve up documents, and then lets me edit them too! There are a few other technologies, like WebDAV, that let you do that over HTTP, but here is this amazingly elegant solution that doesn't require protocol extensions or any other annoying cruft.

Of course, CouchDB also happens to be a very powerful database! A document database isn't for everyone, but if your application revolves around organising and serving up documents, CouchDB hits a real sweet spot.

Chris I'm pumped about CouchDB because it has the potential to make a bunch of new freedom. It's replication feature make offline databases at least as powerful as hosted data. Because everyone already knows the API (whether they've heard of it or not) writing apps on it is (will be) child's play. I say "will be" because right now we're on the edge of it. CouchDB is not yet 1.0 so we are still learning how to write apps against it. I am focussed on finding the simplest path from user to documents. Ajax seems to be the answer.

Jan I gave a few presentations on CouchDB. In the introduction, I usually ask who has built database-backed web applications. I usually get 80-90% hands. I follow up with "... and who likes it and doesn't have any issues with the database?". Most hands go down and I get a few laughs.

This is a nice skit to start a presentation with and win an audiences sympathies, but it is also giving RDBMSes a bad reputation, which they do not really deserve.

Today, relational databases are used everywhere on the web (where I'm coming from). For a long time they were the only sensible choice to solve a lot of the problems you get with concurrent access to your data in a high-traffic manner. But as demand grows, relational databases begin to be used in ways they were not intended to. JOINs are broken up, data is de-normalized. Disk space and insert-speed is sacrificed for concurrent read-speed etc. They are no longer the ideal technological choice.

CouchDB is built for the web scenario. Storing huge amounts of semi-structured data is the default case in CouchDB. It makes excellent use of modern multi-core machines and multi-machine setup. CouchDB views are built using Map/Reduce, the concept that made Google. Replication allows a user to take data offline and work with it locally, without a network connection; and it can be used to synchronize machines in a load-balanced or highly-available setup or both. And the HTTP REST API makes talking to CouchDB as easy as opening a browser.

And it is written in Erlang which gives us high concurrency (20k requests per second on a single machine?), fault tolerance and live code-upgrades. Erlang is worth an entire interview in itself.

Oh hey, that isn't exactly a concise answer and I didn't even tell you half of the good stuff :-)

You're approaching this book in a very open fashion (the book is online and available as you write it). How did you convince O'Reilly to go that way?

Jan We are developing CouchDB. Writing documentation comes with development. We figured that if we have to do the writing anyway, we don't want to do it twice.

O'Reilly is pretty open about (heh) open books. They have the Subversion book that everybody knows and recently released a book on Haskell. They have experience doing this kind of thing and they have seen good results in the past. When we asked about the possibilities, they just said "sure".

Chris As Jan says, it wasn't hard. O'Reilly understands the value of open information, and the value proposition around publishing, which is part of why they've done so much to enter the wider arena of sharing ideas.

JanNoah ultimately pushed us in the direction of writing the book in the open. Once decided, I couldn't see it done any other way and I am glad for the pushing.

Noah We had been talking about doing a book for a while and I was always very adamant that whatever we did would be released under a free license.

The biggest deficiency in free operating systems is not in the software—it is the lack of good free manuals that we can include in these systems. Many of our most important programs do not come with full manuals. Documentation is an essential part of any software package; when an important free software package does not come with a free manual, that is a major gap. We have many such gaps today.
   —Free Software and Free Manuals, Richard Stallman

Since Stallman wrote this essay, the technology publishers have started to wake up a little bit. A growing number of manuals and books are being released under free licenses, and this is absolutely marvellous. O'Reilly is really leading the way on this, so we were very lucky to get a deal with them.

Our editor told us a surprising rule of thumb, that releasing a good book under a free license makes it sell more copies, and releasing a bad book under a free license makes it sell less copies. Let's hope that we are the former!

What benefits do you expect from this approach?

Chris For me, the hardest part is knowing what people just learning CouchDB will need to know. The authors mostly communicate with people who already understand and use CouchDB, but I'd like the book to be capable of drawing you in, even if you're new, so that you feel comfortable building Couch apps.

Noah I am very lucky to be writing the book with two people a lot smarter than me. When I make mistakes or write something a little silly, I get corrected. Similarly, when you develop something in the open, you get all these really bright people reading your stuff and picking you up on things that you've missed or didn't think about properly. Open collaboration is hugely beneficial like that.

Jan That the world fixes our typos. Oh wait, no! Actually, that's what our editor will take care of. You saw in my answer to the "Why CouchDB" question that it is not a one-liner. Add a very diverse potential readership and you get even more ways to put things We can't cater everybody, but we are trying make this book worth a read for a lot of people, from developers to administrators, system architects and hardcore RDBMS lovers.

By opening the writing process, we get decent feedback very early on and as a result will produce a much better book. Besides, we have been open source developers since forever, we just couldn't do it any other way. Release early, release often.

So far there's been a lot of feedback on the book's mailing list. How is that affecting your writing process?

Jan In early drafts you know that there are passages that try to explain an idea and that these passages don't make a good job. If you know the topic at hand pretty well, you might not notice immediately. Public feedback will tell you pretty quickly what works and what doesn't and that helps with the review proces. The writing process, at least mine, is not really affected.

Noah I think the most significant thing I have taken away from this process is the level of confusion caused when trying to compare CouchDB with traditional databases. I mean, fundamentally, this comparison makes about as much sense as comparing cheese and bread. Sure, you can eat both of them, and one may be applicable in one situation and not in the other, but they also taste really nice when you put them together. That's kind of how I see the CouchDB versus RDBMS debate. Meaningless, essentially. CouchDB isn't a panacea, nor are relational databases, and both of them have their uses. I think we're going to work on improving this clarification in the book.

Having said all that, we've only been taking comments for five days now and the response has been just overwhelming. Who knows what other issues, or points of confusion we're going to find along the journey? I'm pretty sure whatever happens, it's going to be just as rewarding. Thanks for the input everyone!

Chris I've been happy about how people are reacting to the figures. I basically hand-sketched some basic drafts of technical drawings for the book. I was thinking, "what's the simplest thing that could possibly work?" and some of the drawings were even captured with the iPhone built-in camera.

I was surprised because people seemed to like the hand-made feel. I've started refining that style, with the rough drafts for the code-example sections. I'm a big fan of the Head Rush books, so I'm hoping that when O'Reilly brings their artists in, they can approach it with the same playful style.

You've posted three chapters of the book already. What should we be looking for next, and how soon should we be seeing it?

Noah We're aiming to submit another six chapters to O'Reilly in early January, so you should see things trickle through to the website as we complete that process.

Jan We are aiming some time in January for the next batch. We are trying to do the book sequentially and the ToC is online already, so you can see what's coming :o)

Chris We're working on a description of the example blog application we'll be releasing. The app right now is pure-Ajax, but by the time CouchDB hits 1.0, we'll be able to do major portions of it without JavaScript at all. CouchDB it getting HTML generation capabilities so expect that to change some of how we build applications.

What kinds of projects should developers be looking at CouchDB for?

JanAfter my talks or in discussions with developers I often get asked what "niche" CouchDB fits in. I usually say "The Web" and that's me trying to be funny again because the web is nowhere near a niche.

Everything that stores messy data that users (no offence) submit (Facebook, flickr, YouTube etc.) and need efficient access to that data. Systems that handle document structures (CMSes, Blogs ... ). Situations where offline-work is preferable. The internet is available everywhere, except when the Wifi is not working or the hotel charges $5 a minute. Not using The Net to get to your data is usually faster, too.

Finally, database are a fairly boring topic (among non-database nerds). CouchDB makes databases fun again.

Chris Yes and +1 to that, Jan!

Look at replication, think about the new opportunities it opens up. Think about applications with the power of the desktop (and location) but written in the language of the web. HTML, JavaScript, REST, JSON, these have become essentially the lowest-common denominator for web services. When offline mode is not a service downgrade, and the source code is at the desktop, people have new affordances. I'm hoping Couch apps become the new Excel macro.

While CouchDB is written in Erlang, there are a lot of libraries to use it from other languages. Which ones have you dealt with? What's good and bad about them? Which ones will show up in your book?

Chris We're concentrating on the HTTP API for the book, so most of our examples are written in JavaScript or Curl. However, there are good libraries in most languages these days, so to the extent a language handles dynamic JSON-like objects well (and can access HTTP) it's a good fit for CouchDB.

Jan I think I wrote three PHP libraries and only one didn't completely suck, but that were the early days. There is the excellent (documented, unit-tested) PHPillow that has been extracted from a real-world project, which is always a good thing.

I used couchdb-python by our very own contributor Christopher Lenz for a number of projects now and it is very solid. If you are more of a Twisted person (heh), David Reid hosts Paisley.

Good and bad? CouchDB introduces new paradigms and we are still in the process of finding out what client-library abstractions work best (hint, ActiveRecord doesn't work well at all). Like all pioneering, this includes some stabbing in the dark. But in general, the libs I've seen are at least well suited for a single job or class of jobs. Maybe there is no one library to rule them all. We'll find out.

If you want to get into CouchDB (and Erlang) what's the best way to learn?

Jan For CouchDB, you don't need to know any Erlang. If you are familiar with the web, you are good to go. Start out by reading through the wiki and of course our book (*cough*). For more specific and tutorial style documentation, check Planet Couch, it aggregates CouchDB related blog posts and includes already a wealth of information.

If you still want to get into Erlang, there is Thinking in Erlang 30-page free PDF that gives you rough overview. For deeper diving, Joe Armstrong, Erlang's inventor, wrote an excellent book Programming Erlang which is a fun read and packed with all you need to set out writing your first (and second) applications.

Chris I learned Erlang by working my way into CouchDB's source code, based on my experience with the HTTP API. The HTTP callbacks are easy to find, and since you already know what they do, its a good way to learn Erlang. There are plenty of ways to get into CouchDB without touching Erlang — we're working on a feature that allows you to write arbitrary controller logic in JavaScript, so really, you can customize CouchDB, as well as use it, by programming any language that understands JSON.

Friday, December 19, 2008

Reading List for the Holidays

I'm not sure I can call the next two weeks the holidays since I'm going to be spending a great deal of time at work, but I hope to get some serious reading in while I'm at it. Here's what's on list.

Hopefully, I'll be chock full of good ideas and armed with new tools in 2009.

What are you reading over the next couple of weeks?

Wednesday, December 10, 2008

First Post: VeloSLC: Getting the Ball Rolling

I attended several LISA conferences back in the 90s, and was a member of SAGE for a while, but sort of slid away from the whole thing when I took a turn in the ranks of IT management. When I went back to systems/infrastructure admin/engineering, I never really got back into it.

Then, I read a post by Luke Kanies which suggested that LISA was stagnating and recommended Velocity as a better alternative.

Now, I fondly remember the time I spent at LISA, and the things I learned at the various conferences, but a conference devoted to web performance and optimization seems a lot more focused on what I do. Luke's points that the Velocity has more of a community feel make me want to get on board. So, how best to do that?

I like to think of myself as a community hacker. I helped start the very first Ruby Brigade, I've helped put together Perl Monger groups, and I've helped organize the MountainWest RubyConf. So what would be more natural than trying to start a local Velocity minded group — VeloSLC.

Let's start with a mailing list. As we build to critical mass, we should talk about getting together, maybe over lunch, or one evening a month. If we can develop the interest, a local 1 day conference wouldn't be out of the question. It all needs to start with a conversation though, so let's get the ball rolling.

Tuesday, December 09, 2008

Apprenticeship Patterns Interview with Dave Hoover and Adewale Oshineye

Dave Hoover and Adewale Oshieye are working on a book called Apprenticeship Patterns, which is due out in out in March of 2009. The book is about learning how to learn, and pushing yourself along the path toward a higher level of craftsmanship. While it's not Ruby-centric, I think it's the kind of book that will be widely read within Ruby circles. To get a jump on things, I've started an interview with Dave and Ade. Here's the first part:


Why don't you start out by telling us a bit about yourselves

DaveSure. So, I was a practicing child and family therapist until 2000 when I wanted to try my hand at programming. I learned Perl and was a Perl/CGI developer until 2002 when I "discovered" a bunch of different things like Jerry Weinberg, Extreme Programming, Ruby, and Java. Agile and Java were in their hayday so focusing on these helped me get a job at ThoughtWorks in 2004. My 2 years at ThoughtWorks was a great experience, it was during that time (2005) that Ade and I met (via email) and started writing the book. In 2006 I left ThoughtWorks to join smaller, more locally-focused consultancy, Obtiva. I'm now a principal of Obtiva, where I pioneered our Ruby competency, and founded our Software Studio practice.

AdeI started programming back in the 8-bit days using Basic and 6502 assembler. I then took a long detour into Political Theory and journalism until the late 1990s. I studied for an MSc and started working at a dotcom consultancy. In 2000 I stumbled upon Lightweight Methods (which is what the Agile movement used to call itself before the invention of the Agile brand) and through that the Pragmatic Programmer book. That one book introduced me to Ruby, Ward Cunningham's C2 wiki and the Extreme Tuesday Club.

At the time the Extreme Tuesday Club was Europe's epicenter for all things related to Agile, TDD and Mock Objects. We would meet up in a central London pub and people like Martin Fowler, Erich Gamma and Kent Beck would just drop by for a pint.

I eventually joined ThoughtWorks because back then they were recruiting all these really fantastic people and there was no way I was going to turn down the chance to work with people like Aslak, Nat Pryce, Obie, etc. The funny thing is that I think I'd already bumped into Dave on bookshelved.org by the time he joined ThoughtWorks. I have this vague memory of sending him an email in his first week at the company and saying something like "are you the Dave Hoover from bookshelved?"

Up until I joined Google I'd spent my entire career at various different kinds of consultancies across western Europe. I'm now a Technical Solutions Engineer which basically means that when it comes to the company's big strategic partners my group are responsible for everything from javascript problems right down to broken trans-Atlantic cables. The best part of the job is that I get to work with people across the entire company. As you can imagine this is a really good way to learn humility.

What are these apprenticeship patterns you're writing about, and why are they so important?

Dave These apprenticeship patterns were originally extracted from my experiences teaching myself how to become a software developer. I switched careers when I was 26 and taught myself Perl and JavaScript, and then Java and Ruby, and I've played with Scheme and Erlang. But the patterns aren't about any specific languages or technology, and they're not design patterns. These are behavioral patterns. For example, the first pattern I wrote was "Be the Worst", which was based on a quote from guitarist, Pat Metheny, who advised new musicians to try to always be the worst guy in their band.1 This obviously doesn't mean to try to play poorly, it means to surround yourself with great musicians. When I read that quote in Chris Morris' weblog, it immediately resonated with me because it helped me describe why I had joined ThoughtWorks the previous year. I wanted to work with people who were better programmers than me, it's one of the best ways to learn.

I believe these patterns are important because they have a strong theme of staying technical, of being dedicated to one's craft, and pursuing the mastery of that craft. There are too many talented, yet short-sighted, developers who grab their first management opportunity and end up hating their job just a few years later. These patterns are important because we don't need a bunch more programmer-turned-manager people, we need a bunch more insanely experienced, talented software craftsmen that can wield their vast knowledge and experience to create innovative, robust software, and shape the next generation of software developers.

Ade These apprenticeship patterns are about capturing the common techniques that experienced people have used to build their careers and making them available to newcomers to the software industry. It turns out that a lot of the common techniques are grounded in certain values e.g. feedback is important, take responsibility for your career and fate, learn by working with people who know what they're doing on real projects, share what little you know in other to build a community, etc.

These patterns are important because they make it more likely that someone starting out today will be able to surpass their elders. If every software developer doesn't have to start their career from scratch and reinvent every technique that skilled people have been using for decades then it becomes reasonable to talk about the software industry moving forwards.

We're not claiming that this wisdom is ours. We've merely followed in the footsteps of books like the The Pragmatic Programmer and Software Craftsmanship. These are books that people keep telling us changed their lives. We're not quite that ambitious. But hopefully if we introduce new software developers to ideas like software craftsmanship and show them the most powerful ideas we know of then those people can use them to change their lives.

Dave,you mentioned the pattern "Be the Worst". One problem with "being the worst" is that there is always a senior member of any team. How do those people find ways to be the worst?

Dave Remember, these are apprenticeship patterns. They're for people who identify themselves as apprentices. So, if you don't identify yourself as an apprentice, then this pattern is not appropriate for you, instead you should be looking for opportunities to mentor more junior programmers. But if you do identify yourself as an apprentice, and you find yourself leading a team, and this is not uncommon for talented apprentices, then it's time to consider a couple apprenticeship patterns. First, "Use Your Title", meaning: if you've been programming for 2 years and your title is Senior Application Developer, then use that title to gauge your organization. Chances are your organization is sub-par if someone with so little experience could rise so quickly. Sure, you may be a natural, and you may be insanely smart, but if your learning is slowing and you're the best on your team, it's the right time to "Be the Worst", and look for a new team to join.

How does this compare to the more traditional kind of apprenticeship practised at Role Model Software?

Ade Sadly I've never experienced a proper apprenticeship or studio environment. The nearest I've got has been working with some of the stellar folks at ThoughtWorks. Dave's a much better person to handle this question. i will just mention that one of the goals of the book is to provide guidance for the vast majority of us who aren't lucky enough to get that kind of explicit mentoring.

Dave These patterns are different from the kind of apprenticeship that was (is?) practiced at RoleModel. That's because these patterns were extracted from the experiences of people who weren't in traditional apprenticeships. These patterns are going to be most useful for newcomers to software development who find themselves in less than ideal circumstances, the sort of circumstances that keep Dilbert in business. For example, one of the patterns is "Nurture Your Passion" and is written for the enthusiastic developer who finds himself in demoralizing situations at work and needs to the take the time to keep his passion for the craft alive. That pattern would be more relevant for an inexperienced developer working in a large, inept IT organization than an apprentice at RoleModel.

For those of us team leaders, project managers, and development directors who have the power to create more ideal circumstances for apprenticeship, Pete McBreen's Software Craftsmanship is a more relevant book. Reading about the ideal scenarios in Pete's book inspired me to scrape my own informal apprenticeship together out of non-ideal circumstances. And then I used Pete's ideas, combined with my own, to create an apprenticeship program similar to RoleModel's in Obtiva's Software Studio.

How well are these patterns going to transfer to the technical non-programmer (say, a sys admin like me)?

Dave I would think they would transfer well to system administrators, database administrators and web designers. I know that my editor at O'Reilly, who is non-technical, found some of the patterns useful in her own career. The patterns are mainly about learning, attitude and motivation, so the details that are specific to programmers shouldn't be too much of a hindrance.

Ade I think you will find that these patterns are very useful to someone in your position. A lot of the ideas in the book are drawn from a wide range of sources and expressed through patterns. For instance we cite Twyla Tharp ( a dancer turned choreographer) and Atul Gawande (a surgeon) qute often because they're highly skilled practitioners of their disciplines who have written about the path they took.

I would probably quibble with your description of yourself. You're more than your job title. The best sys-admins I've ever known were people who spent time reflecting deeply to find trends in the problems they were fixing, created feedback loops, dug deep below the surface of issues/systems and did many other things that could have been taken straight from our book. I suspect that technical people in the software industry have far more in common with each other than our arbitrary job titles would lead us to believe.

You will probably find our perspective (as developers) a bit different to your perspective. However you should find that the patterns are useful to you precisely because they talk about things you're already doing or planning to do. The example implementations of the patterns, which we've called Action sections, might not be as relevant to you though.

In fact we tackle this issue in the preface. The preface points out that patterns are meant to be a mapping from a family of problems to a family of solutions. The way you apply this mapping will depend on your current context. Even though we assume that the majority of our readers will be full-time software developers the fact that you're a sys admin or a DBA is just a slight modification of this context. I look forward to seeing how that affects the solutions generated by your usage of the patterns.

Ade, I liked your reminder that developers and sys-admins/DBAs have more in common than we often think. It reminded me of a blog post by Andrew Shafer. How do you think we can best build on that commonality?

AdeI found that blog post about the tribal nature of the software industry to be very insightful. Unfortunately we tend to have teams who don't maintain the systems they build and who feel that production support is somebody else's job. As the systems we all build/maintain/administer/support start to handle ever more data, need to scale better and users develop expectations based on the consumer-facing web-apps they use we're not going to have the luxury of that kind of horizontal specialisation. We're all going to need to develop a more thorough sense of the stack on which we're building rather than just throwing code over a wall.

The way we can build on this commonality is by arranging conferences that span the tribes so that we can start to realise how much value other people's skills have. Those of us who don't arrange conferences can do our bit to help by trying to go to at least one conference a year which is filled with people who have an 'alien' but effective perspective. Find a community of web designers, embedded sensor network programmers, unix sys admins, kernel hackers, etc and just lurk at one of their conferences.

Soaking up the ideas that other groups of highly skilled people care about can bridge the divide but it can also give you valuable new insights into your own work. For instance I use the techniques in Tom Limoncelli's book Time Management for System Administrators to help me manage my day. It really helped me when systems like GTD didn't seem to fit the way I live or work.

Andy Hunt's new Pragmatic Thinking & Learning book talks a lot about the Dreyfuss model and learning to push your way up it. Where does self-imposed apprenticeship fit in that process?

Dave I still need to read all the way through Andy's book. I had to put it off so I could finish mine, but I've heard rave reviews about it. I honestly don't know anything about the Dreyfuss model, so I'll have to defer to Ade on this one.

Ade I haven't read Andy's book but it's on my reading list but I've read the presentations that both Andy and Dave Thomas have given about the Dreyfus Skills Acquisition model. I think both books complement each other.

We had a candidate pattern called Learn How You Learn which isn't likely to make it into the book since only a few of our interviewees had used it. This pattern was about finding out which techniques for learning were the most effective for you and then trying to make sure you prioritised their usage accordingly. The idea being that if you're aware of the set of potential learning styles then you can detect situations where you're using an inappropriate style and fix it. However the people who had tried this seemed to see it mostly as choosing to learn from reading books or by trying to build things.

Compared to Andy's our work tends to focus more on the factors that propel someone through the different levels of the Dreyfus model using a system of values. In effect we're trying to place learning in a wider social context. Even though I'm a big fan of the Dreyfus model I think that that the shuhari model has some advantages over it for the kind of people we hope will read our book.

The Shuhari model (first you obey rules, then you try to break the rules, then you just do whatever makes sense in context) claims that each level still retains the qualities of the earlier levels. These are not so much steps on a ladder as concentric rings. This is important to software craftsmanship because of the tension between tacit knowledge (where somebody just knows the right thing to do in a given context) and the transmission of skill. If you're wandering around just doing whatever makes sense and you're getting great results then I'm happy for you. However in the craftsmanship model you need to actually pass your skills on. You have to break out of your unconscious competence by creating new rules or heuristics so that new apprentices can match and eventually surpass your achievements. If we don't do this we run the risk of falling into the same trap as people like Stradivari where the quality of their workshop's output declined after they died. As software craftsmen we have a duty to help to create teams that can sustain and even improve their skill level when we leave.

The only way this can work is if you retain an awareness of all the places where you don't have unconscious competence and have to blindly follow rules. You may be the world's greatest C programmer but the memory of all the times you have to cling to the rules for normalising a database schema is what makes it possible for you to start thinking about representing your C skills in way that others can imitate and enhance.

Atul Gawande's book, Better, has a wonderful section where he talks about the importance of checklists for people who are highly skilled. The checklist is how they make sure that they don't leave instruments inside the patient when an operation develops complications or forget to wash their hands after seeing a patient who is terminally ill. His point is that tacit knowledge is easily forgotten by experts in moments of great stress but the checklists can be used by the less skilled members of a team to keep everybody honest. In fact some of my fondest memories of ThoughtWorks projects are the moments when someone I've taught TDD asks "is there a test for that?" At that point I've either been caught as I'm about to cut a corner or I've been caught applying some meta-heuristic without explaining it first. Either way everybody learns something.

The Dreyfus model seems to be focused on helping the teacher tailor lessons based on their assessment of the level of their students. I agree about the importance of "directed skill acquisition" and that working with someone who knows what they're doing is definitely the best way to learn. However most apprentices won't have a teacher or mentor because there are surprisingly few people in the industry who want to provide guidance to others. You have to consciously set out to find these mentors who can guide you up the levels of whichever model you wish to use.


1 Chad Fowler read the same quote in the same blog post and wrote about the same concept in My Job Went to India.

Monday, December 08, 2008

An Authors Story: a cautionary tale

A few days ago, I was IMing with a friend who's working on a technical book. Things started out like this:

So, how's the book coming?

Yeah, I have some editing to do still..I can't speak for when it'll be published. I've pretty much lost all willpower to work on it :( so it's hard to finish up the deathmarch.

Why the lack of willpower? Have things been too long in the pipeline, or is it something else?

Then, a sort of cautionary tail rolled out over IM. I've reduced it to the following points. I wanted to post it so that others could benefit from the mistakes my friend and his publisher made.

It looks like there were several things that went wrong:

  • The publisher really wants things turned around faster than my friend is able to work on them, and they want them in discreet discrete (thanks anon) chunks that don't match well with the agile mindset a lot of programmers work from. If you're thinking about writing a book (or contracting a programmer to write one) watch out for mismatched expectations
  • After writing several chapters, the the publisher made some very major decisions about changing the style of the book without asking my friend about it. He wanted to write a narrative using idioms to guide the reader along, they wanted a more granular approach. Make sure both sides know what they're looking for up front, then stick to it.
  • At this point, my friend felt like he was losing steam and ownership. Ennui had set in, both the book and the schedule suffered. If the author isn't happy, I don't see how the publisher will be happy in the long run. The opposite certainly applies as well.

My friend has already written other books, but this was a new publisher and a new situation. Had he and the publisher partnered more, maybe this would have turned out better, I don't know. I do know that they both would have been happier, and a better book would probably be on the shelves already.

Update! You might also want to read A Tale of Two Books.


This post is part of a collection of articles about Publishing, Growing Markets, and Books.