Rakudo: this week’s release, and the next Rakudo Star

Originally published on 2011-11-20 by Jonathan Worthington.

On Thursday, tadzik++ cut release #46 of Rakudo. This time, we named it for the London Perl Mongers, organizers of this year’s outstanding and very well attended London Perl Workshop. I’d not been to one for a couple of years, and I’d forgotten what a fun event it is. This one felt better than ever. So, thanks folks! :-)

So, what was in the Thursday release?

• Big Integer Support: the Int and Rat type now support very large values without loss of precision/coercing to floating point. This means you can do such things as compute factorial to 1000 (which stringifies to 2568 characters, by the way :-)). You don’t need to get any libraries set up for this – we bundle a slightly extended libtommath with NQP, and it’s exposed by some ops and an extra 6model representation, which the Int type inlines into its body (so Int is still a single garbage collectable object at the VM level). Thanks go to moritz++ for doing much of the work on this.
• *Protoregexes with LTM: **it’s taken us a lot longer to get protoregexes back into the nom development branch than we’d initially hoped. Here’s the story. A while ago, *pmichaud++ worked on real Longest Token Matching support – driven by an NFA – in a re-development of the regex engine. We’d only ever had “cheating” protoregexes in Rakudo before – ones that could work on literal prefixes, but nothing more. The improved engine is a really nice piece of work – amazingly little code for what it does, and very elegant. Sadly, pmichaud has not been able to work so much on Rakudo of late, so the work lay not-quite-done and unmerged for a while. Just ahead of the release, I picked it up, and found it was far enough along and sufficiently easy to get in to that I could get a first cut integrated in time for this month’s release. Since the release, along with diakopter++, I’ve been hacking away at regexy bits, so there’ll be many improvements in this area in next month’s release. It was nice to get something in place for this month’s, though.
• CATCH improvements: mls++ did a bunch of work that made our handling of CATCH blocks far, far more in line with the Raku specification. Naturally, this makes them much more useful; you now can write when blocks in your CATCH, and any exceptions you don’t match will get re-thrown to the next handler. Write a default block to swallow everything. We also do the stack unwinding at the correct point now. Great work!
• Improved MAIN argument parsing: japhb++ has been leading the way on improving our support for the MAIN subroutine, including nice usage message generation. This is a really nice Raku feature for writing command line applications, so it’s good to see work here. :-)
• 6model REPR API 2: this is very much a behind the scenes thing, and deserves a post of its own for those who follow the blog in order to know more about metamodel design. However, in short: I re-worked the REPR API in 6model somewhat, and updated the various representations to use it. The immediate result was that we could efficiently have Raku’s Int type do big integer stuff. However, it’s also the foundation for some forthcoming work on compact structs and packed arrays. Hopefully at a user level, though, you’ll notice nothing at all about this change; it’s all guts. :-)
• Various other fixes and improvements: bugs fixed, performance improved, missing stuff implemented…we had quite a few smaller, but very worthwhile things done in this release too.

So, this is great, but the next question on everybody’s mind is: when is a nom-based Rakudo Star release coming out? The answer – epic disasters aside – is “December”. Yes, next month. Two big things have happened in the last week: grammar support in the nom development branch drastically improved, and this unblocked tadzik++ to get our ecosystem tools (for module build and installation) working with it. The focus from here is…

• Native call support: get our support for loading and calling into C libraries re-established. The NativeCall module basically worked…apart from it made a load of guts-y assumptions that don’t hold true any more, and was somewhat restricted in what it could handle (for multiple reasons). Now we have 6model and native type support, we can do far better. My task for over the next week is to get our native call support back in shape, all being well with a bunch more features.
• More regex work: while we’ve come a really long way in recent days – we have many things working that never worked in the previous development branch – there are still some issues to address, including one nasty backtracking issue.
• Fixing Modules: quite a few modules already work just fine on nom. Some block on native call support, others on the remaining regex stuff. Others will point to bugs or will be using outdated semantics or relying on previous bugs. tadzik++ has figured out how to automatically run through the module ecosystem, build modules and run their tests, so we can get a good sense of what needs doing.

And, of course, the usual round of feature addition, bug fixing and performance work.

And, with that, it’s back to the hacking. :-)