Perl 6: July 2008 Archives

Today was this week's Rakudo day, and I spent it continuing my work on the S12 (objects) implementation. The biggest new addition this week is that you can now write anonymous classes.

I've got tests written for anonymous classes with attributes and methods; you should be able to inherit and compose roles too, but I didn't get tests for those done yet. Anyone who feels like writing such tests, please do feel free - they go in S12-classes/anonymous.t (in the t/spec, in the Pugs repository).

We've been able to write multi-dispatch subs where the dispatch was type rather than arity based for a little while. As I start to think about implementing the real Perl 6 multiple dispatch algorithm soon, I wanted to make sure that multi-methods worked with the current Parrot algorithm. I'd never thought to try it, but it occurred to me that there wasn't really any good reasons why it wouldn't work either. It turns out it didn't work, but the fix wasn't hard either. So, I did what was needed and now you can write multi methods in classes. Arity based dispatch should be fine; type based has all the issues multi-subs do until I get the real Perl 6 multi-dispatch algorithm in place that knows about roles, constraints and so forth.

After that, I set about adding the .^ method call operator. This means "call on the meta-class". S12 changed recently to make things a bit saner here. Before it was specified that you would be able to write things like:

To introspect the methods of the object $obj. However, this doesn't quite work out, since you are invoking "methods" on the meta-class, and there's no promise that just one class will have one meta-class; the meta-class may be shared amongst many classes, depending on your meta-model. So now you have to pass in the object to introspect as the first parameter to methods on the meta-class.

Which is a bit of a mouthful, which is why there is the .^ operator, allowing you to write the above like this:

This operator sorts out getting the meta-class by calling HOW, and inserting $obj as a parameter to the meta-class method. Note that "methods" on the meta-class isn't implemented yet, though it's probably not far off being done. Today I modified our existing meta-class methods to match the new S12 changes, then implemented the .^ operator. So this gets us on the way towards more introspection support. While I was in that area of the code, I moved the auto-vificiation (WHENCE) stuff out of the P6Object code (which is meant to be more generic than what Rakudo needs) and put it into the Rakudo code, which should make some upcoming refactors pmichaud has planned on P6Object a tad easier.

I also looked at starting to get a few other things in place to flesh objects out. One of them is .WHERE, which simply returns the memory address of an object. That didn't take long. Then I read up on .WHICH, which returns an object's identity value, which is what will allow us to implement value type semantics to go with the "===" comparison operator. For normal objects, it just returns the memory address of the object; I put this in place. Next us is to override it in the value types, and then the "===" operator can be added. This is probably fairly low hanging fruit for anyone who fancies digging into Rakudo some, by the way.

In other bits, I've applied a patch from Carl Masak during the day, and this evening helped Chris Fields, who has happily implemented transliteration for us and is now working on .match on strings, to debug a rather subtle Rakudo bug. I got to the bottom of what's wrong, but resolution is to be discussed. But hopefully we have one soon and then we can have .match defined on strings - another piece in the S05 puzzle! We've a long way to go yet, but bit by bit, day by day, Rakudo is coming together. And last but not least, I'd like to thank to Vienna.pm for funding my weekly Rakudo day.

This week's Rakudo day (Tuesday was catching up last week's) came around and this time I worked on some more high level, visible stuff. A while ago I quickly implemented the anonymous enum constructor, which essentially is a hash constructor on steroids. This week I got a lot of progress in on the (rather more advanced) named enums. This means that you can now write things like:

Notice how the names lie under the Day namespace, but are also imported into the current namespace too (and if you have an enum from some external library in the future, and you import that, the same thing will happen). There are some subtleties relating to symbol collisions that we aren't handling yet (we need a type name registry in Rakudo first). You can also use pairs to set the starting values or the values at any point in the enum too, as with the anonymous ones. A cool thing is that we evaluate the list at compile time to get the set of values, and then can construct the role/class and so forth at compile time too, rather than having to work that all out at runtime.

One of the trickier bits of enums is working out what they actually are. You are in fact introducing a role named Day in the above example, and that means you can mix it into anything else using "does" or "but". The "but" keywords a like "does", but it knows how to take one member of an enumeration and generalize it to the enumeration role, then mix than it. Additionally, it operates on a copy of the original value rather than being destructive. So we can do things like:

Note here that .No and .Yes return true only if .Maybe (the property, which you can also assign to) matches that value.

Note that you can use the "but" operator whenever you would use "does" but want a copy of the value operated on rather than the original (the expression as a whole evaluates to the copy that had the mixin applied). The "but" operator actually is implemented in terms of the "does" operator under the hood.

Also note that "0 but True" doesn't quite work yet, or at least isn't affecting the outcome of if statements. This isn't a problem in the enumerations implementation, but rather seems an inconsistency in the spec. Hopefully a mail to perl6-language will get that cleared up - I'll write it tomorrow, when I've got some sleep.

In other minor happenings today, I found and fixed a segfault in Parrot, helped trace a couple of other Parrot issues and fixed the Rakudo "does" operator bug where it lost the association with the proto-object in the modified object. Yesterday, I had some quite long and detailed discussions with both Larry and Patrick over signatures. I think we've got the spec side of things, where it wasn't completely clear before, worked out. However, we still have some details of the implementation left to completely work out (basically, issues about how data-ish and how procedural-ish signatures really are). I'm hopeful we can resolve these in the next week, so I've got a good base to start building the Perl 6 MMD implementation on top of.

As usual, thanks to Vienna.pm for funding my Rakudo hacking.

The first of this week's two Rakudo days (as I catch up from not having one last week) was today, and I dedicated it to working on signature objects. These are something I need to implement MMD, but also come into play in various other situations. For example, if you write something like:

Then this is constructing a signature object on the left hand side, which you can bind to; assignment to it will also work at some point in the future (but not yet). Signatures also appear, of course, on methods and subs, and we've had those working in some form for quite a while in so far as you could pass parameters, get them type-checked and so forth. We weren't constructing real signature objects, however. And to provide the rich information about the signature and the types that an implementation of Perl 6 MMD needs, we need to start constructing them. This will also allow you to introspect the signatures of subs amd methods at runtime (it's not a lot more work to get that in place from what I've got done today).

While a lot of todays work was beneath the surface - actually constructing the signature objects rather than using them - there is some visible stuff. For one, you can now write declarations like this:

Note that you can't do assignment to many things at once just yet - that's part of list assignment. It's high on Patrick's todo list, so we should have that soon. I'll probably get some parts of it in (implementing assignment to the signature) next Rakudo day.

I also implemented the :(...) syntax which parses what is between the parens as a signature and creates a signature object. This works now, though not all signature syntax is parsed yet.

Finally, you can get hold of the constructed Signature object by doing something like &foo.signature, though at the moment you can't do much at all with it. Well, I guess you can "say &foo.signature.WHAT" and see that it's a Signature object. The introspection interface isn't fully specified, but I asked about it on list and Larry provided an idea as to what he expects to see, so I'll use that to guide my implementation.

Signatures can be incredibly powerful; once we get a full implementation, you will be able to unpack trees nodes right there in a signature and all sorts. For now, I just want to get an initial implementation that builds signature objects holding all the information needed to get on with an MMD implementation, though I'm sure things like smart-matching against them and the more advanced features of them will follow in the not too distant future too.

Thanks to Vienna.pm for funding today's work, and also for knowing a really nice Mexican restaurant to go to after their tech meet last night. Good food, good beer, good fun. I like living in a city with a Perl Mongers group, and just an hour down the road from Vienna, which also has a good, active one. :-)

This is the third "monthly" report for my development grant from the Mozilla Foundation and The Perl Foundation. As regular readers will have surmised by now, the definition of "month" has been stretched a bit for a variety of reasons, but as this report and the other reports will show, it's not been a hindrance to our progress.

To review, the primary goals of this grant are:

1. To have a Perl 6 on Parrot implementation that supports commonly-used Perl 6 constructs;
2. Improvements to the Perl 6 test suite;
3. To substantially complete the Parrot Compiler Toolkit, including documentation;
4. Increased community participation in Perl 6 and Parrot development, including development efforts on other languages utilizing Parrot and the Parrot Compiler Toolkit.

It's now clear that the work under this grant has been (or otherwise shortly will be) successful in meeting all of the above goals. As before, in this report I'll highlight the major events and milestones that have been reached since the previous report, and let my other article postings provide increased details.

Progress, April 2008 to June 2008

* Of course, one of the biggest news items is that in May 2008 The Perl Foundation received a $200,000 philanthropic donation from Ian Hague, roughly half of which will be used to support further development of Perl 6 and to build upon the work performed under this grant.

* In addition, Jonathan Worthington received a grant from Vienna.pm to continue his work on developing Rakudo; this grant led to implementations of type checking, multimethod dispatch, regex and grammar support, public and private methods, Ranges, scalar variables, runtime role composition, enums, and a lot more.

* During this period we continued to improve the documentation for the Rakudo (Perl 6 on Parrot) compiler, although more focus in this area is definitely needed. In April we published a list of Rakudo milestones as a basic "road map" to guide continued development. We also generated numerous articles and blog postings that describe the various features of the compiler and how it's all being put together.

* In May we started measuring progress on Rakudo by the number of passing tests from the official test suite. As of June 30, Rakudo Perl is passing 1126 tests from the official test suite, and we're averaging 100 new passing tests per week. I'm hoping this trend will continue.

* To facilitate development Moritz Lenz and Jerry Gay refactored the test harness to provide a "make spectest_regression" target. Now Rakudo developers can verify that changes to the compiler are not breaking any tests that were previously passing. Based on this we're able to to maintain a spectest-progress file, and Moritz created an excellent utility to display the progress as a graph:

http://www.pmichaud.com/perl6/rakudo-tests-2008-06-30.png

* Work on refactoring, improving, and updating the Perl 6 test suite is being handled by Adrian Kreher, Moritz Lenz, and Jerry Gay under a Google Summer of Code grant, along with test contributions and suggestions from many others. We now have a better process in place for reviewing and updating tests; this has enabled progress in other areas to also proceed more rapidly.

* Many of the basic Perl 6 statement types and constructs are now in place -- the primary notable exception being list assignment and lazy list operations. Implementing list assignment properly will require some modifications to the underlying grammar engine -- that work is expected to occur later this summer. Lazy lists and operators are awaiting some improvements to Parrot's exception subsystem (expected in early July).

* The Parrot Compiler Toolkit (PCT) and Not Quite Perl (NQP) tools developed in the first months of this grant continues to demonstrate its power and effectiveness. Most of the HLL translators for Parrot have either adopted or are planning to using PCT/NQP for their underlying code generation. In particular, both the Ruby and PHP implementations (Cardinal and Plumhead) have made significant progress by using the Parrot compiler tools. PCT now has support for basic "return" control exceptions -- other types of control exceptions will be added shortly.

* We continue to gather more active contributors to Parrot and Rakudo Perl. There has been a substantial increase in patch submissions -- so much so that we've held discussions about how we might improve our ability to respond to code contributions more quickly. I've given presentations about the recent improvements at FOSDEM 2008, The Texas Open Source Symposium, DFW Perl Mongers, and YAPC::NA 2008. Each of these presentations have increased participation and enthusiasm about Rakudo Perl and Parrot. More presentations about the project will be made at OSCON 2008 and YAPC::Europe 2008.

* At YAPC::NA 2008 Jim Keenan, Will Coleda, and others on the Parrot team organized a "Perl 6 and Parrot Workshop", in which we helped approximately 20 people download and build Parrot and Rakudo Perl and run through the basic test suite. This has further increased interest in the project; indeed, some of the participants came across some bugs (and filed bug reports and/or patches), and many are planning to hold similar workshops in their local user groups. We plan to repeat the workshop at other venues, including the Pittsburgh Perl Workshop, YAPC::EU, and likely other workshops and user groups.

* All of the "specific tasks" targeted in the end of the last grant report have been achieved.

Where things are headed next

The goal for the next few weeks will be to continue (and perhaps improve) the excellent development momentum achieved during the past couple of months. In particular, we will continue improving the test suite and Rakudo's ability to pass tests from the suite. In addition, as some new Parrot features become available (e.g., improved exception and lexical variable handling) we will be able to take advantage of them in the compilers and compiler toolkits. We will also begin identifying the specific individuals and tasks to be engaged under grants from the Ian Hague donation.

Specific tasks for the remainder of this grant:

• Continue improving the official test suite
• Develop a more complete implementation of Perl 6's exception model
• Implement basic lazy lists and operators
• More refactoring of basic operators, functions, and classes according to recent changes in the language specification
• Allow compiler and builtin library components to be written in Perl 6
• Continue convergence efforts with other Perl 6 implementations
• Write the final grant report, documenting the work performed, quantifying results achieved, and outlining the next phases of development

As things stand today, I expect to publish the final report for this grant sometime around OSCON 2008 (July 2008). Of course, I will continue to post articles at use.perl, rakudo.org, and parrotblog.org.

Thanks for reading!

Pm