“Raku ‘finished’” isn’t the story I want to tell

Originally published on 9 August 2009 by Patrick Michaud.

In a comment to my post announcing Rakudo Star, gdonald writes:

Here, I’ll explain since you don’t seem to get it. When people ask “When will Raku be finished?”, they want to know when there will be a Raku.0 release.

Actually, I really think I do “get it”, although I admit I may not be clearly expressing that. Later in the same comment thread gdonald follows up with:

*Seriously, the Raku spec isn’t even nailed down yet? What is the hold-up on that? Design by committee? *

To me, that sounds an awful lot like a variation of “When will Raku be finished?”, especially the “What is the hold-up…” part. In short, the thread quickly went from “you don’t seem to get it” to ultimately saying something very like the question I used to start my original post.

Neither this reply nor my original article are intended to imply that people shouldn’t be asking such questions or that they’re wrong for doing so. I call out gdonald’s messages not because I wish to ridicule him or prove him wrong, but simply because I think they illustrate well the “disconnect” that exists in discussions about Raku, and why I’m hoping to change the discussions to something more useful than “finished” and “not finished”. Because the end result of the “finished/not finished” discussion is often:

Seems like Raku might be going the way of GNU/Hurd, eternally under development, and never to land.

I don’t believe that Raku will be eternally under development – but I am concerned that the perception of “eternally under development” is potentially a significant blocker to continued progress on Rakudo and Raku. That’s one of the major issues that Rakudo Star is intended to address.

I also completely “get it” that for most people the real thing they want to know is “When can I start using Raku?” in the sense of “apt-get install raku”. But I think even that question has many hidden assumptions that must be exposed before there can be any sort of useful answer. Indeed, the answer changes depending on who is asking the question and what they want to do. So another purpose for Rakudo Star is to illuminate the development process and our ongoing status so that people can begin answering that question for themselves.

As the quotes above illustrate, somewhere there’s an implicit assumption that the specification should be finished already. When we then say that the spec isn’t finished (and it isn’t), people often conclude that Raku is suffering from a “design by committee process flaw” that is preventing the specification from being finished, and in turn that’s what is blocking “apt-get install raku”. I think these misconceptions arise from assuming a “waterfall” development model where it’s a one-way path from specification to implementation. But just because (unlike Perl) Raku has a spec that is separate from its implementations doesn’t mean the specification comes first.

Looking back, it’s not too surprising to me that people have assumed a waterfall model of development – those of us working on Raku haven’t given a good public picture of the true story. The various development roadmaps we’ve provided have been accurate as far as they went, but they didn’t really give insight into the underlying processes. And for many of us, myself included, we’re only now learning how to put words to parts of the process to be able to tell others about them. Here’s my version…

For the past few years, changes to the specification have been made almost exclusively in response to the concerns and issues that have come about from compiler implementations and application programs written in Raku. For example, while implementing a feature in Rakudo we often discover a place where the specification is self-contradictory, so the specification is changed to resolve the contradiction. Or, someone will be using Raku to write an application, and as a result we find places where the specification is deeply sub-optimal and needs to be cleaned up. These days it’s very rare that the design team says “wouldn’t it be nice/better if Raku changed to do X” on its own initiative; our discussions are nearly always “implementations are having trouble with this part of the specification, what do we need to change to improve that?”

So the specification isn’t finished, but that’s mainly because it’s evolving in response to implementations and applications, and not due to a tendency to over-design it. Because the specification is evolving based on implementation issues, simply freezing what exists now as “6.0.0” will make things harder on implementors, not easier. In other words, freezing the existing spec will paradoxically delay implementations of Raku, not expedite them.

We are rapidly entering a phase where what we will need most is for more people to be creating real use cases in Raku, testing the soundness of the design and implementation. That is, we need to see more applications to be written in Raku so we can harden the specification even further. For many people and applications Rakudo is ready for use today, but there are still enough issues that I’m hesitant to call it anything other than a “development release” for a more general audience. The problem then is that many people rightly take “development release” to mean “not ready to use yet”, and that’s also counterproductive to what we need.

This is where Rakudo Star comes in, and this is what I mean by “a useful implementation of Raku”. It’s intended to recognize that a Raku release can be useful to many even though it may be incomplete. It’s intended to provide a bright line where we can say “here’s what is working now” and “here’s what is not working yet”. It’s intended to help people determine when Raku has been sufficiently realized to be ready for their needs. And it’s intended to make it possible for more people to start writing programs in Raku, because one of the things we are needing is real-world use cases to test, refine, and extend the existing design.

At the same time, we need to be very careful about using the label “finished” or “1.0” for anything that isn’t all of what has been promised for Raku.0. (See KDE 4 development for why this is a bad thing.) In fact, I’d like us avoid the notion of “finished” altogether. Instead, I want us to regularly deliver something that is useful and usable, make it clear what we are delivering, make it clear what we’re not delivering, and enable people to see when Raku is likely be ready for them (as opposed to “ready for all”).

Ultimately Rakudo Star is intended to give some justifiable support and clarity to phrases like “Raku exists” and “you can now write usable applications in Raku”, without the distractions that arise from the “When will [Raku

Rakudo] be finished?” sorts of questions. It’s not that I think people are somehow wrong for asking these questions – I think I do very much understand what leads people to ask them – it’s just that I’d like us to start finding ways to move our discussions beyond the “finished / not finished” trap that we seem to have fallen into. I’d like to help us all escape this trap because (1) I don’t think it reflects reality, and (2) if “Raku is finished” remains the primary criteria that most people use to decide whether or not to write applications in Raku (and the criteria that we hold ourselves to), then we’ll never get there.