Perl 6: May 2008 Archives
The big headline for this week is that The Perl Foundation received a very large grant to support Perl 6 development, so we're all very excited about that.
I'm also pleased at the increasing number of contributors we have to Rakudo and Parrot. Some highlights over the past week:
* Moritz Lenz refactored the test harness so that "make spectest_regression" gives far more usable output.
* Eevee contributed an implementation of the 'reduce' method to List objects. Thus one can now write:
## long version of [+] operator
my $sum = (1..6).reduce { $^a + $^b };
* Vasily Chekalkin added a .kv method for hashes, and .sort and .map for lists.
* Tene (Stephen Weeks) continued to improve placeholder variables and topic handling in blocks.
* Jonathan continued his amazing work by implementing private methods, .?, .+, and .* forms of method calls, specifying the name of an invocant in method signatures, the prefix:<^> operator, and much more.
This week I spent most of my hacking time (1) brainstorming; (2) implementing a new Perl 6-like metaclass system for Parrot, the compiler tools, and Rakudo; and (3) answering questions and closing RT tickets.
Most of the brainstorming I did was on "major pieces" that we will need for implementing Perl 6 on Parrot. First, thanks to the new metaclass system (described below and next post), I finally figured out how to refactor PGE's base Match, Regex, and Grammar classes to be much less convoluted than they are now. In the process I also figured out how we can implement protoregexes from Synopsis 5, which should improve our parsing speed a fair bit and also allows us to bring Rakudo's grammar much closer to STD.pm. The change I have in mind still doesn't bring us full "longest-token-matching" capabilities yet, but it will still be a substantial improvement.
PGE also needs a little bit of refactoring for its operator precedence parser in order to properly handle list vs. item assignment. Since that's one of the highest priorities for Rakudo at the moment, I'll be tackling that this week also.
The other thing that finally started to make sense to me was how to handle variables (mutables) in Parrot. After discussing the ideas with Jonathan last week, our plan now is to create a Mutable PMC that can serve as the base implementation for variable types like Scalar, Array, and Hash. It should simplify a lot of things in Rakudo's implementation.
The new P6object.pbc metaclass library replaces the old Protoobject.pbc library that PGE and PCT were using, as well as the various "Perl6Protoobject" classes that Rakudo perl was using. This means we now have common metaobject model underpinning the various Parrot tools that have Perl 6-like behaviors. In particular, every class gets a protoobject, objects in the class have standard Perl 6 methods like .WHAT and .HOW, and the like.
On Tuesday I converted PGE and PCT over to the new metaclass library, and then Jonathan, Jerry Gay, and I did a massive hacking session yesterday to convert Rakudo over to the new library. Moritz addition of the "spectest_regression" target was a huge help to making it all work correctly. Thanks to everyone's help we were able to make this massive internal change in the span of just a few hours.
I'm quite happy with the final result -- the underlying implementations are all much cleaner and more regular than they were before, and we were able to eliminate a fair bit of cruft that had been accumulating in Rakudo and PCT. So it was well worth the effort, and will greatly simplify things for people who later work with the compiler tools or Rakudo.
I'll post more details about the P6object library in my next use.perl.org post and on parrotblog.org. I still also need to write up the details of the PCT code generation changes I made last week, but that's a bit more involved and so I may save that for much later (so I can keep up my momentum on writing code that we really need sooner rather than later).
Questions and comments welcomed as always.
Pm
After yesterday's day of Rakudo hacking, I was too tired to write up what had happened coherently, so I left it until today. I did quite a few different things throughout the day, so rather than taking them in order I'll group them in a way that makes it a bit easier to follow.
First, I dealt with some things from the perl6-compilers list. There was a patch from Moritz to factor out a large chunk of one of the methods in the parser actions file to help make it more readable, which I applied. There was also a bug report from Chris Fields, pointing out that typed variables had stopped working. After trying to work out why from some debugging, I then started working out which check-in had caused the problem, which soon led to a fix. So, type annotations will now work again.
I had tried to get some basic capture support in place before, but it was based upon the Capture PMC. For some reason, this didn't work out so well, and I was keen to have captures for something else I was going to implement later on. Therefore, I did a quick PIR version. I also implemented the $( ), @( ) and %( ) contextualizers (which put whatever is inside the brackets into item, list and hash context respectively). Therefore, you can now do things like:
my $x = \(1, 2, 3, foo => 42);
say $x[1]; # 2
say $x<foo>; # 42
for @($x) -> $val {
say $val; # 1\n2\n3\n
}
There's plenty more to do on captures, but you have the basics now. And of course, the contextualizers work much more generally than just with captures, but we've some work to do in order to make these useful in the general case.
Each week I'm implementing a little more of S12 (the objects specification). This week was no exception. We now have basic support in for declaring and calling private methods.
class A {
my method private {
12;
}
method public {
self!private
}
}
say A.new().public; # 12
Notice how they are called with an entirely different syntax, so its obvious when looking at the code that you're calling something private. Talking of different calling syntaxes, there are some other variations to just writing "." to call a method. .? will call a method of that given name if there is one, or just return an undef (rather than throwing an exception) if there is no such method. .* will call all methods in the class hierarchy with that name, including inherited ones, and return a list of captures containing the values that were returned, and if there are no methods you will just get an empty list. .+ is like .*, but if it can't find at least one method to call it throws an exception. Think of them like quantifiers on regexes, but relating to method calls instead.
class Dog {
method bark { say "WOOF" }
}
class Puppy is Dog {
method bark { say "woof" }
}
my Puppy $x .= new();
$x.?bark(); # woof\n
$x.?doesnotexist(); # no exception
$x.+bark(); # woof\nWOOF\n
I haven't collected return values into captures in these examples, but it works too.
Something else that had been on the want list for a while was allowing explicit specification on a variable to hold the invocant of a method. This turned out to be trivial to implement, so rather than just having the self keyword you can now say:
class Foo {
method test($inv: $x, $y) {
say $inv.WHAT();
say $x + $y;
}
}
my $x = Foo.new();
$x.test(14, 28); # Foo\n42\n
As a slightly ad-hoc thing, I implemented the prefix ^ operator. On the name of a class, this gets the meta-class. On a number, it generates a range from 0 up to that value minus one. So the following prints the numbers 0 through 4, each on a line of their own.
for ^5 -> $n {
say $n
}
This was actually just a trivial task I spotted while persuing the specification for ranges. We just fake them up in Rakudo at the moment to produce a list, but they are supposed to be lazy and have a whole load of other semantics. I stubbed in a Range class to get us started in that direction, but it needs quite a bit more work before we're ready to actually get Rakudo's ".." operator to construct it. For now it just stores its start and end points and knows how to give a Perl representation of itself.
Once again, thanks for Vienna.pm for sponsoring this work. Also, thanks to everyone else contributing to Rakudo, be it with patches, bug reports or general feedback. :-)
I started out the day by looking through the RT queue for Rakudo. Two tickets were already dealt with, so I just closed those. Another was a bug report concerning assigning undef to typed variables. Doing:
my Int $x = undef;
Would give a type check failure. This is now resolved. Furthermore, if your type is a class name, then assigning undef at any point to it will result in it holding the protoobject for that class again. I also took a moment to post to Perl 6 Language to get some clarifications on what "my TypeName $x;" left $x being when $x was a role, subset type or a junction of types.
Last week I started getting grammars in place. I got so far as having the regex live in the correct namespaces, but that didn't make grammars at all class-like, which is how they should be. This week I set out to fix that. Grammars now get protoobjects too, which you can call .WHAT and so forth on. Furthermore, I got inheritance working and also smart-matching against a grammar, which runs it's TOP rule. Therefore you can now run the following example in Rakudo.
grammar Loads { regex Lots { \d+s } };
grammar Many is Loads { rule TOP { of } };
if "100s of 1000s" ~~ Many {
say $/; # 100s of 1000s
}
Having closed four RT tickets so far, I took a look through there to see what else there was. There was one that did most of what was needed to implement the .perl method on Junctions (which you can call, in theory anyway, on anything to get a Perl representation of it). I did the required fixes and applied it, but realized in the process that we were missing .perl for any of the really fundamental types, so I added it for Num, Int and Str.
With that done, I spent a little time on the S12 tests. I added fudge directives to get one of the that failed to parse to do so, and added an extra test. I plan to add much more here and flesh out the tests quite a bit over time.
Turning back to the OO support, I did some updates to the grammar that both brought us closer to STD.pm - the official grammar - and added the ability to parse a range of extra things. The first grammar changes were related to method calling. Normally you call a method just with ".", but private methods are called with "!". Additionally, there are ways to call sets of methods with quantifiers (.?, .+ and .*, with meanings analogous to those in regexes). Finally, I added the ability to scope declarations of routines too, so we can now parse lexical subs and private methods. I stubbed in conditionals for all of these cases that throw unimplemented exceptions, so people didn't use them expecting that because they parse, they will also work.
So, now there's a bunch of stubs in there for another bunch of OO features and, if nobody beats me to it, I'll be filling some of those out on my next Rakudo day, or maybe before then if time allows (though I'm moving apartment - and country - over the coming week, so I'm not expecting to have much time). A big thanks to Vienna.pm for funding today's work.
Today's work has been a mixture of refactoring and clean-ups that had been on the want list for a while, but just hadn't happened, as well as making some new things work.
First, the initial work I did on types attached them to variables, but what we really needed was a more general way to attach properties. Therefore, there is now a hash of properties instead, where we can stash other stuff.
Next up, I had based pairs on the Parrot Pair PMC, though as Patrick pointed out it's so far off being right for Perl 6 (for example, it's mutable, the Perl 6 one isn't) that we might as well just have our own. Dropping the Parrot Pair PMC and doing that took me all of ten minutes of work, and we get the semantics of pairs a bit more correct too. So that's much cleaner now.
A few days back, dakkar sent in a bug report regarding inheritance. It was almost correct code, but didn't work on Rakudo, since initialization of parent attributes was not yet implemented. I've now implemented this, and I'll borrow the example from the bug report to demonstrate it.
class Foo {
has $.x;
method boo { say $.x }
}
class Bar is Foo {
method set($v) { $.x = $v }
}
my Foo $u .= new(:x(5));
$u.boo; # 5
$u= Bar.new(Foo{ :x(12) }); # This is what now works
$u.boo; # 12
$u.set(9);
$u.boo; # 9
This is not some magical hacky syntax just to make constructors work; you can use it in the general case to associate some vivification data with a proto-object, which gives you a copy of it back with the data attached. It's a bit like currying the object instantiation. So after making the above work, it wasn't much more work to get the following working.
class Foo { has $.x }
my $foo42 = Foo{ :x(42) };
my $test = $foo42.new();
say $test.x; # 42
Note that the original Foo itself isn't changed. We'll have to revisit this again later, because the way I've done it now doesn't have the lazy semantics it's eventually meant to have. It makes the common use case work, though.
With some time spent on objects, I moved onto some improvements to regex stuff. The upshot of this is that you can now use grammar to group regexes into a namespace.
grammar Test {
regex Load { \d+s };
rule Loads { <Load> of <Load> };
}
if "100s of 1000s" ~~ Test::Loads { say "yes" }
yes
Note that this is just the start of grammars; inheritance doesn't yet work and you can't smart-match against them yet. It's a stop forward, though.
If you're an avid Rakudo follower, you'll have noted that regex, rule and token all (wrongly) did the same thing before today. I've fixed that too now (there was some behind the scenes work in being able to pass options to the compiler that will be useful elsewhere, as to users of the Parrot Compiler Toolkit in general). In a nutshell, token and rule don't backtrack, where as regex does, and additionally rule translates spaces to the
# Demonstrating :ratchet semantics (rule like token here).
regex WillBT { a*a }
token WontBT { a*a }
if "aaa" ~~ WillBT { say "yes" } else { say "no" }
yes
if "aaa" ~~ WontBT { say "yes" } else { say "no" }
no
# Demonstrating :sigspace semantics.
regex Test1 { \d \d };
rule Test2 { \d \d };
if "12" ~~ Test1 { say "yes" } else { say "no" }
yes
if "1 2" ~~ Test1 { say "yes" } else { say "no" }
no
if "12" ~~ Test2 { say "yes" } else { say "no" }
no
if "1 2" ~~ Test2 { say "yes" } else { say "no" }
yes
So, that's what got done today. I'd like to thank Vienna.pm for funding this work, and hope you'll have fun playing with it, breaking it and reporting bugs. :-)
Subscribe
-
Subscribe to rakudo.org
