Rakudo Perl: July 2008 Archives
I took Friday off this week, so that I could work Saturday on Rakudo, matching up my Rakudo day with the post-OSCON Parrot/Rakudo hackathon. I started off the day by going through the Rakudo RT queue and trying to resolve some of the tickets in there. Rather than doing any big new thing today, I instead went to the RT queue for Rakudo and addressed some of the tickets in there, to resolve a range of bugs that people had reported.
- Applied patch from Carl Masak that avoids a NULL PMC Exception if the return value of sleep was evaluated.
- You can now (as per the spec) write just "multi foo { }" rather than "multi sub foo" to introduce a multi-sub.
- Writing "multi sub foo { }" (note there is no signature) now works the same as "multi sub foo() { }" rather than giving a Null PMC access.
- Fixed a bug in Parrot that caused an assertion failure when trying to write to a closed file-handle; now it throws an exception, so we can implement the correct Perl 6 semantics in Rakudo.
- Trying to inherit from a non-existent class now gives a meaningful error message, rather than a rather less informative null PMC access exception.
- Fixed a bug in PCT which could cause an exception to get lost in the return_pir control handler. This allowed at least one ticket to be closed out, and will likely help people working on other compilers written in NQP too.
I then turned to another ticket with a bigger task: getting class attributes to work. While there will be more bits to do on this, the basics are working. Here's what I've just put into the regression tests.
class Foo {
my $.x is rw;
}
my $x = Foo.new();
$x.x = 42;
is($x.x, 42, "class attribute accessors work");
my $y = Foo.new();
is($y.x, 42, "class attributes shared by all instances");
So essentially, you're introducing a lexically scoped variable and getting an accessor method generated for it.
During the day, I've also had various chats about other little bits of Perl 6 stuff on IRC, and been re-reading bits of the spec as food for thought on implementing some more bits in the future. So, that's this week's Rakudo day. Thanks as usual goes to Vienna.pm for funding!
Today was this week's Rakudo day, and I spent it continuing my work on the S12 (objects) implementation. The biggest new addition this week is that you can now write anonymous classes.
my $c = class {
has $.x;
method foo { say "OH HAI" }
};
my $obj = $c.new(x => 42); # Instantiate it
$obj.foo; # OH HAI
say $obj.x; # 42
I've got tests written for anonymous classes with attributes and methods; you should be able to inherit and compose roles too, but I didn't get tests for those done yet. Anyone who feels like writing such tests, please do feel free - they go in S12-classes/anonymous.t (in the t/spec, in the Pugs repository).
We've been able to write multi-dispatch subs where the dispatch was type rather than arity based for a little while. As I start to think about implementing the real Perl 6 multiple dispatch algorithm soon, I wanted to make sure that multi-methods worked with the current Parrot algorithm. I'd never thought to try it, but it occurred to me that there wasn't really any good reasons why it wouldn't work either. It turns out it didn't work, but the fix wasn't hard either. So, I did what was needed and now you can write multi methods in classes. Arity based dispatch should be fine; type based has all the issues multi-subs do until I get the real Perl 6 multi-dispatch algorithm in place that knows about roles, constraints and so forth.
After that, I set about adding the .^ method call operator. This means "call on the meta-class". S12 changed recently to make things a bit saner here. Before it was specified that you would be able to write things like:
my @methods = $obj.HOW.methods();
To introspect the methods of the object $obj. However, this doesn't quite work out, since you are invoking "methods" on the meta-class, and there's no promise that just one class will have one meta-class; the meta-class may be shared amongst many classes, depending on your meta-model. So now you have to pass in the object to introspect as the first parameter to methods on the meta-class.
my @methods = $obj.HOW.methods($obj);
Which is a bit of a mouthful, which is why there is the .^ operator, allowing you to write the above like this:
my @methods = $obj.^methods();
This operator sorts out getting the meta-class by calling HOW, and inserting $obj as a parameter to the meta-class method. Note that "methods" on the meta-class isn't implemented yet, though it's probably not far off being done. Today I modified our existing meta-class methods to match the new S12 changes, then implemented the .^ operator. So this gets us on the way towards more introspection support. While I was in that area of the code, I moved the auto-vificiation (WHENCE) stuff out of the P6Object code (which is meant to be more generic than what Rakudo needs) and put it into the Rakudo code, which should make some upcoming refactors pmichaud has planned on P6Object a tad easier.
I also looked at starting to get a few other things in place to flesh objects out. One of them is .WHERE, which simply returns the memory address of an object. That didn't take long. Then I read up on .WHICH, which returns an object's identity value, which is what will allow us to implement value type semantics to go with the "===" comparison operator. For normal objects, it just returns the memory address of the object; I put this in place. Next us is to override it in the value types, and then the "===" operator can be added. This is probably fairly low hanging fruit for anyone who fancies digging into Rakudo some, by the way.
In other bits, I've applied a patch from Carl Masak during the day, and this evening helped Chris Fields, who has happily implemented transliteration for us and is now working on .match on strings, to debug a rather subtle Rakudo bug. I got to the bottom of what's wrong, but resolution is to be discussed. But hopefully we have one soon and then we can have .match defined on strings - another piece in the S05 puzzle! We've a long way to go yet, but bit by bit, day by day, Rakudo is coming together. And last but not least, I'd like to thank to Vienna.pm for funding my weekly Rakudo day.
This week's Rakudo day (Tuesday was catching up last week's) came around and this time I worked on some more high level, visible stuff. A while ago I quickly implemented the anonymous enum constructor, which essentially is a hash constructor on steroids. This week I got a lot of progress in on the (rather more advanced) named enums. This means that you can now write things like:
enum Day <Mon Tue Wed Thu Fri Sat Sun>;
say Mon; # 0 - the underlying value
say Day::Wed; # 2 - the underlying value
Notice how the names lie under the Day namespace, but are also imported into the current namespace too (and if you have an enum from some external library in the future, and you import that, the same thing will happen). There are some subtleties relating to symbol collisions that we aren't handling yet (we need a type name registry in Rakudo first). You can also use pairs to set the starting values or the values at any point in the enum too, as with the anonymous ones. A cool thing is that we evaluate the list at compile time to get the set of values, and then can construct the role/class and so forth at compile time too, rather than having to work that all out at runtime.
One of the trickier bits of enums is working out what they actually are. You are in fact introducing a role named Day in the above example, and that means you can mix it into anything else using "does" or "but". The "but" keywords a like "does", but it knows how to take one member of an enumeration and generalize it to the enumeration role, then mix than it. Additionally, it operates on a copy of the original value rather than being destructive. So we can do things like:
enum Maybe <No Yes>;
my $x = 0 but Yes;
say $x; # 0
say $x.Maybe; # 1, since it's Yes
say $x.No; # 0, because $x.Maybe is not 0
say $x.Yes; # 1, because $x.Maybe is 1
Note here that .No and .Yes return true only if .Maybe (the property, which you can also assign to) matches that value.
enum Maybe <No Yes>;
my $x = 0 but No;
say $x; # 0
say $x.Maybe; # 0, since it's No
say $x.No; # 1, because $x.Maybe is 0
say $x.Yes; # 0, because $x.Maybe is not 1
Note that you can use the "but" operator whenever you would use "does" but want a copy of the value operated on rather than the original (the expression as a whole evaluates to the copy that had the mixin applied). The "but" operator actually is implemented in terms of the "does" operator under the hood.
Also note that "0 but True" doesn't quite work yet, or at least isn't affecting the outcome of if statements. This isn't a problem in the enumerations implementation, but rather seems an inconsistency in the spec. Hopefully a mail to perl6-language will get that cleared up - I'll write it tomorrow, when I've got some sleep.
In other minor happenings today, I found and fixed a segfault in Parrot, helped trace a couple of other Parrot issues and fixed the Rakudo "does" operator bug where it lost the association with the proto-object in the modified object. Yesterday, I had some quite long and detailed discussions with both Larry and Patrick over signatures. I think we've got the spec side of things, where it wasn't completely clear before, worked out. However, we still have some details of the implementation left to completely work out (basically, issues about how data-ish and how procedural-ish signatures really are). I'm hopeful we can resolve these in the next week, so I've got a good base to start building the Perl 6 MMD implementation on top of.
As usual, thanks to Vienna.pm for funding my Rakudo hacking.
The first of this week's two Rakudo days (as I catch up from not having one last week) was today, and I dedicated it to working on signature objects. These are something I need to implement MMD, but also come into play in various other situations. For example, if you write something like:
my ($a, $b);
Then this is constructing a signature object on the left hand side, which you can bind to; assignment to it will also work at some point in the future (but not yet). Signatures also appear, of course, on methods and subs, and we've had those working in some form for quite a while in so far as you could pass parameters, get them type-checked and so forth. We weren't constructing real signature objects, however. And to provide the rich information about the signature and the types that an implementation of Perl 6 MMD needs, we need to start constructing them. This will also allow you to introspect the signatures of subs amd methods at runtime (it's not a lot more work to get that in place from what I've got done today).
While a lot of todays work was beneath the surface - actually constructing the signature objects rather than using them - there is some visible stuff. For one, you can now write declarations like this:
my ($a, $b, $c); # declares all three variables
class Foo { has ($.x, $!y); } # declare both attributes
Note that you can't do assignment to many things at once just yet - that's part of list assignment. It's high on Patrick's todo list, so we should have that soon. I'll probably get some parts of it in (implementing assignment to the signature) next Rakudo day.
I also implemented the :(...) syntax which parses what is between the parens as a signature and creates a signature object. This works now, though not all signature syntax is parsed yet.
Finally, you can get hold of the constructed Signature object by doing something like &foo.signature, though at the moment you can't do much at all with it. Well, I guess you can "say &foo.signature.WHAT" and see that it's a Signature object. The introspection interface isn't fully specified, but I asked about it on list and Larry provided an idea as to what he expects to see, so I'll use that to guide my implementation.
Signatures can be incredibly powerful; once we get a full implementation, you will be able to unpack trees nodes right there in a signature and all sorts. For now, I just want to get an initial implementation that builds signature objects holding all the information needed to get on with an MMD implementation, though I'm sure things like smart-matching against them and the more advanced features of them will follow in the not too distant future too.
Thanks to Vienna.pm for funding today's work, and also for knowing a really nice Mexican restaurant to go to after their tech meet last night. Good food, good beer, good fun. I like living in a city with a Perl Mongers group, and just an hour down the road from Vienna, which also has a good, active one. :-)
First of all, sorry that it has taken me so long to write up last week's Rakudo day. Normally I write the report at the end of the day, but I stayed up debugging and fixing stuff until I was too tired to do anything, but sleep. I'd also been having some insomnia over that week, and had been down with some virus (as in, me personally, not my computer) around the start of last week, so by the time the weekend arrived I just wanted to rest. I'm happy to say that I'm now in much better shape, so there's nothing to worry about. The last couple of days have just been full of catching up with $DAYJOB stuff I couldn't do at the start of last week.
Second, I'm sorry to have nothing much exciting to speak of. Thursday was one of those days spent doing needed stuff rather than fun stuff. Patrick had discovered various issues with Parrot's lexicals implementation that were in the way of making progress on various things in Rakudo. Since getting that right was important in getting some fairly fundemental stuff fixed, and given I have Parrot internals knowledge, I took the day away from doing the cool stuff and focused on these issues. Along the way I fixed at least one other long-standing issue. With some additional contributions from chromatic++ and NotFound++, I think we are now free of the issues that were blocking things.
One of the issues I solved was that methods were unable to see lexicals in enclosing scopes. That is now resolved at a Parrot level, and Patrick did the changes in PCT too and thus it should be resolved in Rakudo. The more serious issue was that outer scopes weren't being identified uniquely enough in the emitted code. This was mostly seriously an issue for multi-subs, but also could be exposed when multiple namespaces were used in one file (a common occurence). This is now resolved.
I also started laying the foundations for getting us to be able to identify Subs, Blocks and Methods as different types, and having somewhere to attach signatures. I'm not completely there with this just yet, and have an uncommitted patch that does some work on this (there is checked in some initial work on it too that sort of works, but not entirely). Getting signatures attached is working towards mutli-method dispatch, which I will be working on this month.
This week I'm overseeing the launch of a fairly big project for one of my clients, and won't get a good day's peace this week for Rakudo hacking. So, rather than having a Rakudo day this week where I keep getting distracted, I'll push it to next week and have two then. I will also be working on the MMD grant this month, so after the slow start to the month this week I will be working on Rakudo two days a week throughout the rest of the month and probably into August too. I will also be at YAPC::Europe::2008.
This is the third "monthly" report for my development grant from the Mozilla Foundation and The Perl Foundation. As regular readers will have surmised by now, the definition of "month" has been stretched a bit for a variety of reasons, but as this report and the other reports will show, it's not been a hindrance to our progress.
To review, the primary goals of this grant are:
- To have a Perl 6 on Parrot implementation that supports commonly-used Perl 6 constructs;
- Improvements to the Perl 6 test suite;
- To substantially complete the Parrot Compiler Toolkit, including documentation;
- Increased community participation in Perl 6 and Parrot development, including development efforts on other languages utilizing Parrot and the Parrot Compiler Toolkit.
It's now clear that the work under this grant has been (or otherwise shortly will be) successful in meeting all of the above goals. As before, in this report I'll highlight the major events and milestones that have been reached since the previous report, and let my other article postings provide increased details.
Progress, April 2008 to June 2008
* Of course, one of the biggest news items is that in May 2008 The Perl Foundation received a $200,000 philanthropic donation from Ian Hague, roughly half of which will be used to support further development of Perl 6 and to build upon the work performed under this grant.
* In addition, Jonathan Worthington received a grant from Vienna.pm to continue his work on developing Rakudo; this grant led to implementations of type checking, multimethod dispatch, regex and grammar support, public and private methods, Ranges, scalar variables, runtime role composition, enums, and a lot more.
* During this period we continued to improve the documentation for the Rakudo (Perl 6 on Parrot) compiler, although more focus in this area is definitely needed. In April we published a list of Rakudo milestones as a basic "road map" to guide continued development. We also generated numerous articles and blog postings that describe the various features of the compiler and how it's all being put together.
* In May we started measuring progress on Rakudo by the number of passing tests from the official test suite. As of June 30, Rakudo Perl is passing 1126 tests from the official test suite, and we're averaging 100 new passing tests per week. I'm hoping this trend will continue.
* To facilitate development Moritz Lenz and Jerry Gay refactored the test harness to provide a "make spectest_regression" target. Now Rakudo developers can verify that changes to the compiler are not breaking any tests that were previously passing. Based on this we're able to to maintain a spectest-progress file, and Moritz created an excellent utility to display the progress as a graph:
http://www.pmichaud.com/perl6/rakudo-tests-2008-06-30.png
* Work on refactoring, improving, and updating the Perl 6 test suite is being handled by Adrian Kreher, Moritz Lenz, and Jerry Gay under a Google Summer of Code grant, along with test contributions and suggestions from many others. We now have a better process in place for reviewing and updating tests; this has enabled progress in other areas to also proceed more rapidly.
* Many of the basic Perl 6 statement types and constructs are now in place -- the primary notable exception being list assignment and lazy list operations. Implementing list assignment properly will require some modifications to the underlying grammar engine -- that work is expected to occur later this summer. Lazy lists and operators are awaiting some improvements to Parrot's exception subsystem (expected in early July).
* The Parrot Compiler Toolkit (PCT) and Not Quite Perl (NQP) tools developed in the first months of this grant continues to demonstrate its power and effectiveness. Most of the HLL translators for Parrot have either adopted or are planning to using PCT/NQP for their underlying code generation. In particular, both the Ruby and PHP implementations (Cardinal and Plumhead) have made significant progress by using the Parrot compiler tools. PCT now has support for basic "return" control exceptions -- other types of control exceptions will be added shortly.
* We continue to gather more active contributors to Parrot and Rakudo Perl. There has been a substantial increase in patch submissions -- so much so that we've held discussions about how we might improve our ability to respond to code contributions more quickly. I've given presentations about the recent improvements at FOSDEM 2008, The Texas Open Source Symposium, DFW Perl Mongers, and YAPC::NA 2008. Each of these presentations have increased participation and enthusiasm about Rakudo Perl and Parrot. More presentations about the project will be made at OSCON 2008 and YAPC::Europe 2008.
* At YAPC::NA 2008 Jim Keenan, Will Coleda, and others on the Parrot team organized a "Perl 6 and Parrot Workshop", in which we helped approximately 20 people download and build Parrot and Rakudo Perl and run through the basic test suite. This has further increased interest in the project; indeed, some of the participants came across some bugs (and filed bug reports and/or patches), and many are planning to hold similar workshops in their local user groups. We plan to repeat the workshop at other venues, including the Pittsburgh Perl Workshop, YAPC::EU, and likely other workshops and user groups.
* All of the "specific tasks" targeted in the end of the last grant report have been achieved.
Where things are headed next
The goal for the next few weeks will be to continue (and perhaps improve) the excellent development momentum achieved during the past couple of months. In particular, we will continue improving the test suite and Rakudo's ability to pass tests from the suite. In addition, as some new Parrot features become available (e.g., improved exception and lexical variable handling) we will be able to take advantage of them in the compilers and compiler toolkits. We will also begin identifying the specific individuals and tasks to be engaged under grants from the Ian Hague donation.
Specific tasks for the remainder of this grant:
- Continue improving the official test suite
- Develop a more complete implementation of Perl 6's exception model
- Implement basic lazy lists and operators
- More refactoring of basic operators, functions, and classes according to recent changes in the language specification
- Allow compiler and builtin library components to be written in Perl 6
- Continue convergence efforts with other Perl 6 implementations
- Write the final grant report, documenting the work performed, quantifying results achieved, and outlining the next phases of development
As things stand today, I expect to publish the final report for this grant sometime around OSCON 2008 (July 2008). Of course, I will continue to post articles at use.perl, rakudo.org, and parrotblog.org.
Thanks for reading!
Pm
Subscribe
-
Subscribe to rakudo.org
