The Rakudo “nom” hacking marathon

Originally published on 2011-06-06 by Jonathan Worthington.

Here in Sweden they have these things called “red days”. They’re basically days off, and June has a load of them. When I first heard the term, I wondered if it was some socialist connection, but got the much more boring answer: “it’s because they’re in red on the calendar”. :-) Anyway, with Thursday and today being red days, and Wednesday somewhat being one too, and a free weekend, there was only one thing for it: hack loads on the Rakudo “nom” branch!

The “nom” branch – short for “new object model” – is where I’m re-building a bunch of Rakudo things on top of 6model. Last time I gave an update of it here, I’d barely got going: the build procedure was updated somewhat and I was digging into a symbol handling refactor and starting to flesh out meta-objects, but there wasn’t anything you could actually take and play with. By now, while there’s still lots to do, things are starting to get interesting. :-)

Over the last 5 days, I’ve made approximately 130 commits to the Rakudo repository, and 20 to the NQP one (mostly 6model core improvements). Happily, others have also been joining in on the fun: moritz++, pmichaud++ and tadzik++ have all committed stuff in the last few days too, and they along with many others have provided useful feedback and comments. Here’s a selection of the things we now have working in the “nom” branch:

• String and integer literals. Unlike in master, we don’t re-build boxed objects every time we need them; instead we build them once at compile time and then just reference them from the serialization context (which, amongst other things, serves as a kind of constant table).
• Signatures. They’re not as complete as in master yet. However, a critical difference is that we build them at compile time now. In master, we build them at load time, and the compiler just keeps its own representation of them that is just enough for code generation. This change means the compiler is now dealing with the real thing, so if it wants to analyze them to do optimizations, it can just as happily work with ones from other compilation units as well as the current ones. This will be important  for implementing a wide range of optimizations in the future.
• Class declarations work along with inheritance, method declaration and attribute declaration. The .” sigil and rw` trait for accessor generation also work.
• Multiple dispatch of subs and methods, using the new proto semantics that master never caught up with. At the moment, declaring a proto is mandatory, but we’ll relax that before the branch lands.
• The :U and :D type-modifiers for dispatching by definedness are now implemented (never had these in master)
• Traits on classes and attributes. Unlike in master, we really do dispatch these at BEGIN time, using multi-dispatch now. In master, they are deferred to INIT time.  Thus we only handle the is repr trait in the compiler now, and none of the rest are magical any more.
• Many basic operators for Int, Num and Str, as well as a few methods. These were mostly filled out by moritz++ and tadzik++. They’re defined using the new multis and box/unbox with 6model primitives. Note that those three types are also far more memory efficient and GC-friendly than in master.
• Scalar binding and assignment. The setting up of containers is now rather more efficient. Binds and assigns to lexicals and attributes are type-checked at runtime; we have a primitive in place (though don’t use it yet) for a type-based optimizer to short-circuit type checks that it can prove are not needed. Note that unlike in master, binding is now a cheap operation (though assign is also decidedly cheaper, and we’ve plenty of scope for improvement on it). Note that some semantic issues with binding in master are also resolved.
• if / unless / while and probably others that I didn’t check – we didn’t start over from scratch, so a lot of stuff we just get from it having already worked in master, or can fix up pretty quickly
• Closures work. We’ve sometimes had…fun…getting those right in the past, so I decided to try and deal with them early. Thankfully, thanks to pmichaud++’s good ground work in this area in the past, it wasn’t so painful to do. :-)
• Probably a few other bits that I forgot…

Of course, we’re still missing a load of things and “nom” isn’t ready to start testing anything serious on yet, but it’s exciting that so many of the primitives – including some of the tricky ones – are back in place. Of course, some tricky ones are yet to come: parametric type stuff, lists and deferral to name three of the biggies.  Happily, my supply of hacking time is looking to continue to be good for the week ahead, and others are joining in with the hacking, so I’m looking forward to another exciting week ahead. :-)