In this week's episode, Brian Ford (brixen) is joining us. He's been doing a lot of work on the testing and RSpec side of the rubinius house, so that's where we'll start today.
Evan, in our last interview, you talked about 4 tasks you'd like to tackle, at the time, you mentioned that you'd like to work on RNI first. Shortly afterwards, the same topic came up on #rubinius. and there was a pretty strong call for stabilizing the compiler and adding method lookup caching. How does this affect your plans. In general, how do you balance public requests with your own vision of rubinius?
Evan: I ask for public opinion on my task list to help me gauge what's important. There are tasks that I find quite interesting, but they're not always the ones that help the project the most.
I don't allow public opinion to totally drive the order of my list, if I did, I'd probably have to jump from task to task. Everyone on the project has a different list of what they see as important.
I think that giving the public input into the dev process, even if it's just their opinion, helps develop a feeling of community. That's important because I want to get more people to believe in rubinius and want it to succeed. As the number of people interested grows, the interest grows and the project has a better chance of success.
Back in the 2nd episode, Wilson mentioned a spec technique that would allow two implementations to be compared. How is that coming?
Wilson: A great deal of progress has been made. Here's an example spec. In this case, for Array#&:
specify "& should create an array with no duplicates" do
example do
p(([ 1, 1, 3, 5 ] & [ 1, 2, 3 ]).uniq!)
end.should == nil
end
example takes a block, and then evaluates that block using a particular Ruby implementation; MRI, JRuby, or Rubinius. When you run the specs, you decide which runtime you would like to use. At the moment we just run these manually, but eventually we would like to have a continuous integration system for it. If you have 100 failures already, it can be hard to notice that you (for example) fixed one but created a new problem.
Brian: Actually, there are currently a couple of approaches. One is to write a single set of specs that describes Ruby both at the language level and also at the core and standard library level. When I first wanted to add specs to drive my work on rubinius, I knew that rubinius could not run RSpec. Looking at the tests, I saw that Evan was running rubinius in a subprocess from the unit tests. That was all the inspiration I needed. I initially wrote a method named 'rubinius' to wrap the subprocess machinery. That lead to specs that looked like this:
specify "some behavior" do
rubinius do
p [1, 2, 3].first
end.should == '1'
end
This really bugged me because I wanted the specs to look like this:
specify "some behavior" do
[1, 2, 3].first.should == 1
end
But I realized that in MRI (Matz' Ruby Interpreter), I could define 'rubinius' like so:
def rubinius
yield
end
"Sweet!" I thought. But then the rubinius name didn't make sense anymore. That's when I decided on the a method named 'example'. And since I was writing all my spec bodies in irb and copying the result into the spec, it made a lot of sense to just make it run under MRI. So, I decided to make a host/target configuration where the 'host' would run RSpec and the 'target' would execute the body of the spec. Making that work with MRI/MRI meant that I could write specs and run them directly instead of doing everything in irb first. That was a big productivity gain for me.
Of course, I was asking a lot of questions and getting a lot of feedback on my initial work on the specs. A couple folks, lypanov and mae in particular really disliked the string comparisons I thought we were forced to do. Finally, one day I was chatting with headius and he asked why I didn't just put the 'p' call into the 'example' method and eval the result. That was a big jump because now specs looked like this:
specify "some behavior" do
example do
[1, 2, 3].first
end.should == 1
end
Still not ideal, but much closer. And changing these to the ideal form would be really simple, we'd probably even be able to automate it. So anyway, showing off what I had accomplished with headius' prompting to nicksieger inspired him to whip the jruby_target.rb (the file that implements the machinery for running the specs with JRuby) into shape. So now we can target MRI, JRuby, or Rubinius.
Another approach to compatibility has been pursued by mae (Matthew Elder). His idea was that it would be great to just have files of many different examples of Ruby code and have an automated process to run those through RSpec. His work on this can be found in the directory spec/compatibility. I think it's still rough at the moment because I think mae is pretty busy with school work.
Last week, when we talked about STM, it was mentioned that rubinius core classes should use STM to make sure they're thread-safe. I think that using STM like this is easier said than done. Could you show us simple example of using STM to implement something (say, Array#pop)?
Wilson: OK. Let's say that there is a core method, 'atomic', that takes a block, and yields to it. Behind the scenes, it wraps that code in a transactions, and either completes it or retries it, depending on what happens. For those familiar with database transactions, it's just like that, but in RAM.
I'm not sure that Array#pop is a method that would really be 'automatically' wrapped in a transaction. I think it is more likely that users would wrap a section of code that they needed to be re-entrant in an 'atomic' block, at a higher level. We don't want to slow down serial code unnecessarily. For now, though, let's pretend like we wanted a guaranteed-safe version of Array#pop:
class Array
def pop
return nil if self.empty? # Popping an empty array is fine.
e = self.last # Fetch the last element
@total -= 1 # 'resize' the array.
return e
end
end
Without some kind of safety net, two different tasks could end up popping the same element from the array. Thread #2 could end up hitting e = self.last before Thread #1 updates the array size.
One (not that great) way to implement this would be to allocate a Mutex when creating an array, and have every critical method synchronize on it.
That would probably work fine in ruby 1.8, because Array is implemented in C, and Ruby code cannot directly tamper with it. In Rubinius, Array is a .rb file, and user code can (potentially) re-open the class and muck about with the internals. User code could then bypass the Mutex, and directly access the '@total' instance variable. (Not really; we'll be protecting that before we are done, but you get the idea.)
With STM, you would simply wrap the body of the 'pop' method with 'atomic { }' and move on. The visible state of the '@total' variable will change 'all at once' when the transaction commits.
Finally, I understand that rubinius can now load archives, and perform partial recompiles. What new opportunities will this open up for rubinius and its developers?
Brian: This is a feature that I really lobbied for when Evan was asking what he should tackle next. This gives us the ability to stabilize parts of the rubinius implementation and thereby involve a wider range of folks. The core libraries needed to run rubinius are now included, so to get going, people only need to compile the C source for the rubinius VM (shotgun) . Some people will jump right in and build rubinius, chasing down compile errors and that sort of thing. Others will start reading source and asking questions. Others will be most excited to run their own carefully typed 'puts "hello, world!"'. We want as many people at every level to help us get rubinius into shape as quickly as possible since we're surely playing catchup with MRI, YARV, and JRuby.
Wilson: High performance virtual machines (such as Microsoft's CLR, Sun's JVM, and Jikes) have shown that there are many different execution scenarios for dynamic programs. The machine needs to be able to choose amongst several different optimization plans, and even switch back and forth as circumstances dictate.
Given code like this:
if almost_never_happens
self.something_expensive
else
nil
end
nil
The evolving Rubinius features are, at least partly, the support infrastructure that will allow this kind of thing. A file containing puts 'Hello, world' should probably just be interpreted directly, to reduce overhead. A Rails app that runs for a year without a restart needs aggressive optimization and caching.
Evan: One scenario I'm hoping to see archive loading used is deployment. The deployment strategy for rails apps and the like is hit or miss, relying on tools like capistrano and rsync to push new code out. My hope is that archives will be able to replace some of those mechanisms, allowing a rails app to deployed by simply replacing the app's main archive on the production systems.
Deployment of real applications also becomes simpler. A developer building an OS X application in ruby can easily package up all the ruby code in an archive to make it easy to release new versions.
This week's episode is brought to you by Test Driven Development: A Practical Guide
Previous episodes of the Serial Rubinius Interview are available here:
- Episode 1, in which we talk about the rubinius community
- Episode 2, in which we talk about cuby and testing tools
- Episode 3, in which we talk about rubinius documentation
- Episode 4, in which we talk about cooperation with the JRuby team and YARV
- A diversion, in which Ola Bini, MenTaLguY, Evan, and I talk about rubinius and lisp
- Episode 5, in which we talk about STM and rubinius (with special guest MenTaLguY).
No comments:
Post a Comment