Parrot: June 2008 Archives

After taking last week off for workshops, this week I got back to having my weekly Rakudo day. There were various things that were just about ready, and I've spent much of today getting those in place. The result is some new features, some fixes and some work on getting the semantics correct in places that we didn't before.

One of the big changes is that the range operator, "..", will now create a Range object, which is a lazy iterator. I'd done some of the initial work in writing the Range class before now, but using it blocked on some other list refactoring that was needed. That has now been done (thanks to Patrick for this), and today I was able to switch ".." over to creating the Range object rather than just a flat list. It still will be eager in a lot of places, because we haven't got a full lazy lists implementation just yet. However:

for 1..1000000 -> $x {
    say $x;
}


Will now not create an array of a million elements. In fact, it will run in constant space. I also implemented smart-matching against Range objects. This means that you can do things like:

if $x ~~ 1..100 { say "valid percentage" }

Again, this will not create a list 100 elements long, but instead just one Range object that knows its endpoints. There are still various things to do with ranges, including the operators for constructing ranges that are exclusive of their endpoints (and Range objects need some work for this themselves too). Also, we need to implement the :by adverb, but we can't parse adverbs on operators just yet. Oh, and then there's the fun of infinite ranges, but I'm hoping they will just fall out naturally with little to no changes to Range once the Inf type is in place.

I've been mumbling for a while about a bunch of work under the title of mutables. It has very much been an under-the-hood thing that was needed to make other things possible. Today I got us fully switched over to this model for scalars with no additional tests failing. And I also started taking of advantage of it to get a few more things in place.

Before, parameters were passed and were modifiable. However, unless you used "is rw" this should not have been so; they are meant to be readonly by default. Now we have the correct semantics, and "is rw", "is copy" and "is readonly" work too (but writing "is readonly" is a waste of time here, since it's the default anyway; I haven't bothered in the example below).

sub foo($x) { $x = 42; say $x; }
my $a = 100; foo($a); say $a; # Cannot assign to readonly variable
sub foo($x is rw) { $x = 42; say $x; }
my $a = 100; foo($a); say $a; # 42\n42\n
sub foo($x is copy) { $x = 42; say $x; }
my $a = 100; foo($a); say $a; # 42\n100\n


I also got the VAR($x) and $x.VAR macros in place (they are actually compiler macros, not sub or method calls), which let you get at the underlying implementation type - in this case the default Scalar object. In the end, this will let you work with traits and tied containers, though we are some way off being able to implement those yet (quite a few dependencies). For now you can do things like $x.VAR.readonly to find out if a variable is readonly, for example.

I've also started doing some preparations for the Next Big Thing I intend to do some work on during my Rakudo day next week: roles. We already have some basic composition support. I fixed using roles as type constraints on a variable. However, what we have so far isn't even scratching the surface of what we need to fully do roles. Attributes surely aren't done right, for example. I've spent some time reading the appropriate bits of S12, trying to get into my head what it means. I also got to discuss some bits of it with Larry (especially the type parameterization bits), and it seems I'm along the right lines. So, some work coming on roles next week, all being well.

I also fixed a range of smaller bugs.

• The prefix + and - operators were not preserving integer type when performed on an integer.
• The truncate method was not returning an Int, as its signature said it should.
• Parrot's String Array PMCs were missing the get_iter vtable method to get an iterator for them; applied a patch from chromatic to add them and wrote a test.
• Made subset ... of ... where syntax without a block work (you can just mention something there that is an expression to smartmatch against, which can be a load neater for some use cases)
• Worked out what was wrong with my previous patch written over the weekend for a Parrot GC bug, and applied a corrected one, resulting in one less spectest failure and improved stability in interactive mode. Plus this helps everyone else using Parrot.
• "my Foo $x" already correctly put a Foo protoobject in $x if Foo was a class, but did the wrong thing for roles and constraints. Now it does the right thing - sticks a Failure object in there.

So, a pretty productive day, if a bit all over the place. Thanks to Vienna.pm for funding this work, and for introducing me to a nice place to eat curry in Vienna after their Monday tech meet, which I popped along to. :-)

The Perl 6 design team met by phone on 28 May 2008. Larry, Allison, Patrick, Jerry, and chromatic attended.

Larry:

• mostly off for this long weekend elsewhere
• continuing to work on quoting roles
• they need generic support to mix in roles with parameterized strings as starter and stopper sequences
• that's an interesting problem, when mapped into Perl 5
• no support for generic roles or parameterized types yet
• I'll probably just do an eval

Allison:

• reworked continuations in Parrot, based on our discussion last week
• I have very high regard for the value of reading through the commit logs
• found the source of the problem that way
• someone committed the problem about a year and a half ago
• now continuations work the way they should work
• we now don't try to make CPS respect the dynamic environment stack
• it's largely unused now that we don't store exceptions there anymore
• tracking down one last bug
• I can duplicate it
• it's an exception thrown, caught, and resumed from an :onload subroutine
• looks like they get executed differently from regular execution
• first draft of the Bylaws and Articles of Incorporation for the Parrot Foundation
• hired a lawyer in Washington state for incorporation there

Patrick:

• added quite a few things in the Rakudo implementation
• added a new metamodel that looks like the Perl 6 metamodel
• with Jonathan's and Jerry's and Moritz's help, we converted the compiler tools to that model
• cleaned up some of the inheritance hierarchy in Rakudo
• builtin types report correct methods and types
• a Parrot Integer reports itself as an Int in Rakudo, down to the metaobject information
• fixed pair handling, especially for calling functions
• people are testing those in great detail now
• now it's obvious that Rakudo's and Parrot's and Perl 6's argument handling there somewhat disagree
• doing some PGE refactoring
• primarily fixing up its class hierarchy; looks pretty easy
• may move Rakudo's grammar to use protoregexes
• might help get the metaoperators working -- French brackets, for example
• if we don't get this soon, people might submit patches doing things the wrong way (multiplying new operators for various combinations)

Jerry:

• Google Summer of Code started this week
• the aggregated feed of Perl GSoC 2008 student and mentor blogs is available
• the students will report weekly status at the #parrotsketch meeting
• the students and mentors will report at #perl6-soc on irc.freenode.net on Wednesdays at 18:30 UTC
• all of the students seem to be on track now
• made some spectest changes
• made some changes to rakudo List methods, applied some patches
• made some Rakudo test changes, should make test management easier and allowing us to create private and shared lists of tests we want to run
• should make it simple for rakudo developers to specify a subset of test files, making focused development and focused testing easier

c:

• not a lot of energy for hacking for some reason
• applied a few patches
• cleaned up a few tickets
• talked with Andrew quite a bit yesterday about GC plans
• he has some good ideas, including a clever marking scheme for which I have high hopes
• mostly on the right track
• will write up some guidelines on profiling and optimizing
• have one idea to optimize keyed object attribute access
• otherwise really need to work on the PIR profiler

Richard:

• trying to plan a legitimate way to ascertain Perl 6 grants, given the donation from Ian Hague
• planning a process around that
• thinking about YAPC::NA and a grant process BOF

Allison:

• Patrick mentioned argument passing in his report
• we don't quite understand how it needs to work in Perl 6
• Parrot will support whatever Perl 6 needs, though it may not necessarily be the default
• one goal of argument passing is to make it lazy
• you don't have to parse the whole list to start pulling out parameters
• required parameter positionals are clear
• required positional parameters are clear
• you can pass in a required positional parameter as either a positional or named argument
• Parrot maps the positional arguments to positional parameters
• when it runs out of positional arguments, it starts checking named arguments

Patrick:

• that's not really what Parrot does
• Parrot fills positional parameters with positional arguments
• then it fills in named parameters with positional arguments

Larry:

• what do you mean when you say "named parameter"?

Allison:

• pairs

Patrick:

• all parameters are named in Perl 6
• they can be filled by a named argument
• Parrot has parameters that you can fill only by position
• then there are parameters that you can fill by position or name
• but Parrot fills them opposite from how Perl 6 would expect; it chooses the positional argument over the named one.

Allison:

• I thought you needed a way to skip over a named parameter in a list
• and then fill in its value from the named argument list
• or scan the entire named argument list first, and then fill in positional parameters
• it sounds like this is different

Larry:

• the compiler should recognize arguments intended as named parameters
• it can set up a data structure for efficient access
• the other constraint is that any argument can depend on the value of a previous argument in the list
• you have to bind them in the order of declaration
• you go down the list
• for each of them, you look up in the named argument list
• to see if it's bound by a name
• if not, you take the next one from the positional list

Patrick:

• that can be lazy

Larry:

• yes
• that's the abstract view from the Perl 6 end
• one reason we invented prototypes is so that the compiler can know that the first n arguments are positional
• map any named arguments into positional without having to do the lookup
• that's primarily a sop to efficiency
• for a very small number of named arguments, brute force is probably faster

Allison:

• do you never have any parameters which are strictly only positional?

Larry:

• there's no way to write that
• other than writing a bare sigil in the declaration
• you're not giving it a name
• but you can't access it, because it needs a name

Allison:

• sounds like we want three different flags in Parrot
• one for a purely-positional argument

Larry:

• the optimizer might use that, if it sees a prototype

Allison:

• we keep the :named flag
• for named parameters

Patrick:

• named-only

Allison:

• we don't really have that
• we have named that acts like positional

Patrick:

• we have named that prefers positional, and then throws an exception if a matching named argument is also supplied

Allison:

• might have four flags then
• one that's either named or positional
• maybe one that gives priority to the positional
• not sure that's useful

Patrick:

• I couldn't come up with a case where that's useful

Larry:

• some languages want that

Patrick:

• you can almost simulate that with slurpies in your code
• or sticking in dummy arguments

Allison:

• could come in useful if you have optional parameters
• you don't have to pass anything at all

Patrick:

• I proposed what you described as numbers 1, 2, and 4
• the existing :named flag is named-only
• the :positional flag is a named parameter that accepts positional arguments

Allison:

• I'm trying to get down the ideas
• I probably wouldn't use :positional for that
• important to get the categories right

Patrick:

• I couldn't figure out a case where you have a named parameter that prefers the positional over the name

Allison:

• make it a positional and don't worry about it
• if you make it positional, you don't have to scan the argument list

c:

• look at Python's default parameter handling
• I think it may follow the one case you couldn't figure out
• not entirely sure

Patrick:

• is the answer that Parrot's underlying model will likely change to support what Perl 6 needs?

Allison:

• yes
• exact details yet to be determined
• as soon as possible so that you can use it

Patrick:

• it's been the subject of at least five or six RT tickets in the Rakudo queue
• they look like different problems, but they call come down to the same problem
• it's becoming a pain point
• I'm willing to implement it
• there are probably three or four related tickets in the Parrot queue

Larry:

• there was a question about writing Pairs various ways as arguments
• are they intended to be named arguments or pairs?
• how do you write the prototype for list, by the way?

Patrick:

• I suggested putting colons in front of it

Larry:

• but :a is always a valid term

Patrick:

• you can always look ahead for a fat arrow

Larry:

• I'd almost rather write :> for that
• originally we always used the fat arrow
• then we put in the colons
• we wanted to use the fat arrow for arguments
• people seem to be used to the colon for named arguments, not the fat arrow
• maybe we should just bite the bullet
• you need the name as the identifier
• you don't need the general pair form
• the fat arrow makes a Pair not intended for a named argument, unless you do processing on it
• that'd be a simplification
• you can tell by just looking at it
• one is a named argument
• the other is just a positional parameter

Jerry:

• is there a method on a Pair that allows it to be passed as a named parameter?

Patrick:

• prefix vertical bar?
• S06 has options for making a Pair a named argument
• or a hash as a list of named arguments

Larry:

• we're trying to guess it based on the context of the argument list
• if you pass the list to a function call, it assumes you want named arguments
• otherwise you don't
• that might be guessing too far
• the list function might want to parse more like parens
• but it behaves like a function
• you don't want to pass those as named arguments
• how do you tell the compiler that it's special
• maybe you want to make it a macro, but that's kind of weird
• maybe it's time to look at the cultural issue of how people want to write named arguments
• maybe we can simplify that

Patrick:

• there's a similar issue with the hash constructor
• you can kinda get away with it, because order is less important
• but it still exists
• it'd be interesting to look at the existing tests and Synopses to see the results
• there is some historical baggage toward using the fat arrow

Larry:

• the guy who wrote that Synopsis was familiar with Ada's fat arrow for named arguments

c:

• is there some sort of prototype declarator that says "It was an ordered list in the source code, and that's what I want!"

Larry:

• macros are a lot like prototypes

Patrick:

• Rakudo for now will treat them as special in the way that Pugs does
• it'll treat list as a macro
• do something special, it's not a normal argument list

So, after some time "on the road", mostly spent going to Perl events, I'm finally back home, where I'll be pretty much staying for the next two and a half months. Certainly, I don't plan to fly anywhere before YAPC::EU::2008; no doubt I'll enjoy a couple of weekend breaks that I can take by train first, though. Home is currently Bratislava (capital of Slovakia), so having so many interesting bits of central and east europe so close to hand will be far too much to resist. I plan to make it over to the western Ukraine (I loved Kiev and want to check out Lvov), and hopefully see Poprad and take a bit of hiking in the nice mountains in that area.

I've now got all of my slides uploaded to my talks page, and here are a few direct links with summaries to help you get to the juicy stuff faster.

• Implementing Perl 6, in Perl 6, on Parrot - I gave this one at the Nordic Perl Workshop, French Perl Workshop and to Vienna.pm. The slides were very nearly the same for all of them, so I just linked to the Vienna.pm version, which was the final one.
• Perl 6 Tutorial - from the French Perl Workshop. This was delivered in the space of two hours, and assumes no prior knowledge of Perl 6 (but some basic Perl 5 knowledge). The vast majority of what it covers is usable today in Rakudo.
• Understanding Perl 6 - this one could also have been called Perl 6 For Computer Scientists (there were variuous people who were doing academic research in computer science at the workshop). It's aimed at explaining how the features of Perl 6 fit together and giving a big picture of the language rather than dealing with syntax and practical usage.
• Using Perl 6 And Parrot In Teaching - a look at how Parrot, PCT and Perl 6 could be used in teaching.
• All Your Dynamic Languages Are Belong To Us - this was a guest lecture at a university in Stockholm to a bunch of undergraduate students. I introduced both Parrot and PCT, and then built a compiler live on stage. Overall, the talk lasted an hour; the compiler was built in just over 35 minutes.

Thanks to Best Practical and The Perl Foundation for giving me a grant to travel to help fund my travel to these events, to Claes Jakobsson for very kindly letting me stay at his place in Stockholm (and also for cooking some really delicious Swedish meatballs) and to the many people I met along the way who made the last couple of weeks a lot of fun.