Dean Wampler and Alex Payne: Author Interview

There are several good looking Scala books on their way to market. I'd like to help you get an early look at the books and their authors, so I'm working on a series of Scala author interviews for the FP side of On Ruby. Here's the second of several interviews.

Programming Scala authors Dean Wampler (@deanwampler) and Alex Payne (@al3x) have been good enough to answer several questions for me in this interview.

---

Functional Programming languages seem to be an increasingly popular topic. Why? What do FP languages teach or do for us?

Alex FP forces you to think differently about solving problems in code, and in a highly structured way. If object-oriented and procedural languages offer a more linguistic approach to programming, functional programming suggests a more mathematical approach. Once you wrap your head around the functional approach to handling collections of data, you'll never think about programming quite the same way. That's the reason that MapReduce, a combination of two fairly simple functional programming concepts, is so essential to what Google does. FP is rich in these powerful abstractions.

Dean It seems that every "paradigm" that goes mainstream lurks in the background until some "killer app" comes along for which the paradigm seems tailor made. GUI's did that for OOP. Now, it seems that growing concerns about pervasive concurrency, sometimes called the "multi-core problem", are driving interest in functional programming.

FP is a promising approach for concurrent programming because in pure FP, variables are immutable and functions have no side effects. If you have immutable variables, no synchronization is required for shared access. All the hard, error-prone programming problems in multithreaded code involve synchronizing access to shared, mutable state. If you get rid of mutable state, those problems just go away.

Another hallmark of FP is that functions are side-effect free — they don't change any global or object state. Such functions are inherently simpler to reason about, test, invoke concurrently, you name it.

Even if you don't care about concurrency (rare, these days), immutable state and side-effect free functions improve code quality, in general. For example, if I have an immutable object "graph", I don't have to worry about passing it to a client that wants to read it. If it's mutable, I have to worry that the client will screw it up in some way. I'd rather screw it up myself, thank you very much!

What makes Scala the right FP language for people to pick up?

Dean When I decided to learn an FP language, I picked Scala for very practical reasons; it seemed to be a very good candidate for widespread adoption. Because it runs on the JVM and because Scala code interoperates with Java code so seamlessly, I felt that a lot of Java developers would eventually migrate to Scala, first as a "better Java" and then later as a hybrid object-functional language, all without losing their existing investment in Java technology.

Alex It's worth clarifying that Scala is a hybrid Object-Oriented/Functional language. It's the best of both worlds: the power of FP with the familiarity of OOP. That familiarity makes Scala is a great first step into FP for programmers who haven't yet tried a functional language. For those who have, the practicality of doing domain modeling with OOP and interfacing with other JVM languages makes Scala a compelling alternative to strictly functional languages like Haskell and Lisp. All languages seem to gravitate towards an object model over time; see the Common Lisp Object System and the myriad efforts to turn JavaScript's prototyping into more traditional OOP. Why not pick a functional language that already has a mature object model baked in?

Why is the JVM a good platform for Scala and FP?

Alex The JVM is a great platform for any language for two reasons: maturity and community. It's widely acknowledged that the JVM isn't just the most mature VMs out there, it's simply one of the most mature of pieces of software out there, on par with the Linux kernel in terms of its history and extensive deployment. The JVM can be tuned, instrumented, profiled, and debugged using a variety of seasoned tools on a multitude of platforms. What's more, there's over a decade's worth of libraries for most any programming task available for the JVM, readily accessible from Scala with essentially no performance penalty. The Java community has already been through the process of producing code to tackle big-time, big-money problems, and that's value you can put to work immediately with Scala.

Scala's author had a hand in more than one functional programming language for the JVM before embarking on Scala. Coincidentally, he's also responsible for many of the performance gains the JVM has made over the years. Other big FP brains like Clojure's author have found the JVM to be a great host for functional languages. It still has a way to go before it's friendlier to dynamic languages, but the JVM's support for strong typing is a good fit for the FP paradigm.

Dean The JVM is arguably the most powerful VM in the world. The global, runtime optimizations it performs are quite extraordinary, doing things a compiler can never do, since it has to rely on static analysis of the code. Of course, the Java ecosystem is also rich with libraries and tools for almost every possible need. Hence, the JVM is great platform to build upon.

That's not to say that the JVM is perfect. It has a number of limitations that make FP harder, like functions are not "first-class" citizens (usable as values, arguments to other functions, and able to stand alone, without being a member of a type). Also, the JVM doesn't do tail-call optimization, which is very useful for recursive functions, which are more common in FP. However, language designers have found ways around these limitations and the JVM is definitely going to address these issues in the future.

By the way, Scala also compiles to .NET byte code, so the .NET community can enjoy Scala goodness, too.

What is the status of Scala on the CLR? Do you know anything about it's performance?

Alex Personally, I haven't worked with Scala on the CLR. My understanding is that CLR support is a lower priority for the developers working on Scala's implementation, and that support may not be complete at this time.

Dean I haven't worked with the CLR version, either. I have heard that Microsoft has given a grant to Martin Odersky's team to keep the .NET support current with the Java support.

With the upcoming changes to the JVM (tail call optimization and first class functions among others), will there be changes to Scala?

Alex I can't speak for the language's authors, but my understanding is that they track changes to the JVM closely to wring as much performance as they can out of Scala's implementation. As the JVM evolves to embrace its newfound role as host to myriad languages, I'm sure Scala will take advantage of those changes. No other language on the JVM has a team that's as capable of doing so.

Dean I've read some detailed blog posts by Charlie Nutter, the JRuby project lead, on the issues they have encountered supporting Ruby features on the JVM. One area of inefficiency is the need to wrap "blocks" (loosely speaking, "bare" functions) in little classes, consuming lots of resources. Some of the features planned for the JVM will certain benefit Scala by eliminating such workarounds.

What kinds of problems are a good fit for Scala?

Dean Scala's flexible syntax is surprising good for defining "internal" DSL's, for which Ruby is justly famous. If you need an "external" DSL, then the combinator parser library will probably be all you need to build a custom parser. This flexibility also allows Scala to provide some very elegant built-in API's for manipulating XML, building thick-client interfaces with Swing, etc.

I can't think of another language that has quite the flexibility for small to large scale problems.

Alex Scala's name is a shortening of "scaleable language". The goal of its creators was to design a language that scaled from small scripting tasks to desktop GUI applications and all the way up to huge enterprise systems. While Scala is certainly capable of all that, its sweet spot seems to be in producing maintainable back-end systems, particularly those that require non-trivial concurrency. Essentially, it's good for the things that languages like Java and C++ are good for, but without suffering the pains and hassles that often come with those languages.

That said, I really do believe that Scala scales from scripts on up. I've written tiny scripts that rip through giant XML files that are just as terse as anything I've written in Ruby and many, many times faster (even accounting for the JVM startup time). I've written a small blogging system of the sort that people usually write in Perl, Python, or PHP and found the code far more maintainable. At Twitter, we're using Scala for critical parts of our service's infrastructure, such as our message queue. I certainly believe in the right language for the right problem, but there's not much I wouldn't at least attempt to solve in Scala.

If you were putting together a development team for a scala project, what are some things you'd look for in the candidates?

Alex We're actively hiring engineers at Twitter, and this is definitely something we've had to think about when it comes to Scala experience, or lack thereof. We know the Scala community is small, and that hiring someone with prior Scala is unlikely today. I think that in a couple years, that's going to be a very different story.

Generally, in-depth experience with OOP, dynamic languages, and a smidge of functional language experience means that a programmer can get up to speed with Scala quickly. If a candidate can tell me what a closure is, or what a combinator is, or describe some basics of type theory, they're going to do fine with Scala. If they can't, though, chances are that they've used those features in other languages and just aren't familiar with the terminology.

The more experience with the Java SDK a programmer has, the easier it'll be to navigate the landscape of classes that Scala is built on. That's hardly a prerequisite, though, and I can say that from personal experience. I've mostly avoided Java over the past five year of my programming career, and it hasn't taken long to come up to speed with "Javaland" by way of Scala. When I need it, it's there. When I don't (which is most of the time), it's not.

I think the ideal Scala programmer is passionate about the language she works with every day. You can write perfectly passable code in Scala if you treat as just another tool, but if you really explore its rich set of features, there's so much to take advantage of. It's been great to see our engineers go from zero familiarity with Scala to writing code that uses the language's most advanced features, like parser combinators and actors.

Dean Over the years, I've found that really good developers tend to be very curious about other languages. So, even if they have no prior exposure to Scala, they have probably seen variations of its features in other languages. In my "day job", I train and mentor teams trying to adopt agile methods, like the XP practices (TDD, pair programming, etc.). I see so much damage in teams that lack these practices that I would also want my Scala new hires to be good at XP!

It seems a bit strange to talk about so much about a language without showing it off a bit. Could one of you provide a small program that shows Scala off a bit?

Alex Here's an example I used in a presentation:

"showing a nonexistant user returns 404" in {
  withFormats(List("xml", "json")) { format =>
    val response = User.show("jack", "asonetibaosneitb", format)

    response mustNot beNull
    response.code mustEqual 404

    response
  } mustNot beEmpty
}

In it, you can see Scala's support for Domain Specific Languages (via use of the Specs BDD library), anonymous functions, and the clarity and fluidity of Scala's syntax. Given how much time a programmer spends (or should be spending!) writing tests, I think this beautiful test code really shows off what Scala can do in just a few lines.

Dean Scala's support for building nice DSL's doesn't get enough attention, IMHO. It's a killer feature, for the same reason it's so seductive in Ruby. Here's another DSL example, a snippet of code for a build tool I've been working on.

target('compile -> List('clean, 'build_dir)) {
    scalac(
        'files     -> files(srcDir+"**/*.scala", specDir+"**/*.scala"),
        'classpath -> environment.classpath,
        'd         -> buildDir,
        'opts      -> "-unchecked -deprecation"
    )
}

target('clean) {
    deleteRecursively(buildDir)
}

target('build_dir) {
    mkdir(buildDir)
}

It's inspired by Ruby's "rake". You specify targets, named with Scala symbols (e.g., 'compile) and there are built in functions for invoking the compiler, manipulating files and directories, etc.

By the way, it seems to be an unofficial Scala community requirement that when you do a project to learn Scala, it must either be a build tool or a Twitter client...

Design patterns and algorithms look differently when implemented in different languages. Can you give us some examples of elegant patterns or algorithms in Scala?

Alex Scala's implementation of inheritable, extensible cross-cutting concerns in the form of Traits is one of the language's best features. Imagine Ruby modules on steroids and you're getting there. That's a difficult problem for most languages to handle well, and Scala does a great job at it.

Scala's Actor library is another superbly-implemented part of the language. While Actors in Scala feel "baked in" to the language, in actuality it's a completely self-contained library. The Actor model can be found in Erlang, Io, and a number of other experimental languages, but it feels solid and right at home in Scala.

Dean Indeed, the Actor library is another example of the flexible support for building internal DSL's.

Just for emphasis, let me echo what Alex said about Traits. It's one of the scalability features (in terms of application size and complexity) that I'm most excited about. Here's another way to describe traits; it is Scala's mechanism for supporting "mixin"-style composition, originally made famous in Lisp and more recently in the form of Ruby modules. You have your main class hierarchy (single inheritance, like Java), but you can mix in abstractions and implementations in a modular way, defined as traits. With Java, you can only "mixin" the abstraction part as interfaces. You have to resort to ad-hoc hacks to add implementations. This composition mechanism solves some of the issues that Java users turn to Aspect-Oriented Programming to solve and it promises to encourage more modular designs that will grow gracefully over time.

The question of Design Patterns is an interesting one right now. Patterns are getting a lot of bad mouthing from some reputable (and not so reputable) people. Without rehashing the arguments, let's just say that the central argument seems to be that patterns are just workarounds for language deficiencies. I think that misses the point that a good pattern stands on its own merits. If a language supports it "natively", then so much the better!

Because of Scala's support for FP, especially closures, some design patterns just aren't as important as they are in Java. (I'm looking at you, Visitor!) Rather, API designers can build in elegant hooks for clients to pass in closures to customize behavior, etc.

Also, traits make patterns like Observer very elegant. I've heard there are some good examples of Observer in a Scala book O'Reilly is publishing, available now as a Rough Cut!!

Click here to Tweet this article