chromatic: June 2008 Archives

The Perl 6 design team met by phone on 11 June 2008. Larry, Allison, Patrick, Jerry, Jesse, Nicholas, and chromatic attended.

Larry:

• slowly improving
• seems like I'm getting smarter, but appearances can be deceiving
• my brain's not really back to a lot of spec work yet
• continuing to refactor and debug the gimme5/STD5/cursor5 galaxy of pain
• making progress on that
• parses all of radix.t, though I'm not sure it's doing that correctly
• starting to make progress on parsing STD.pm itself, though it's not getting far
• hope to be able to parse most of STD.pm using STD.pm by my talk next week
• my future plans are to switch over from using lists (which cannot easily be lazy in Perl 5) to using lazy scalar iterators
• that'll probably make the produced Perl code look a little prettier too
• unless your idea of prettiness is a lot of nested map statements
• going to Chicago on Friday
• giving a talk a week from tomorrow at Google in Chicago

Patrick:

• things are going very well
• Rakudo and PCT now support the return statement
• hooks for the other types of exceptions implemented
• have a version of fail implemented, but it fails in certain cases
• want to clean that up

Larry:

• it fails to fail?

Patrick:

• it fails badly
• mostly need to find where we did things the Parrot way and layer the Perl 6 way on there instead
• fixing little bugs here and there to get more spectests running
• wrote my own small version of Test::Harness to get the output results in a format that makes it easy for us to chart our progress
• this one tells us how many we passed
• running a test right now to find out where we were this morning
• essentially, we're passing around 700 spec tests this week
• last week we were passing 624
• we may be quite a bit more than 700
• all of this will show up in a journal post in the next day or two
• writing code distracts me from journal posts, or vice versa
• no other distractions
• looking forward to YAPC and the hackathon
• putting together a list of things to work on at the hackathon
• one big thing is how people can help us with the spectests
• doing more thinking about how to update PGE to do protoregexes and longest token matching
• seems doable this summer
• maybe between YAPC and OSCON
• tomorrow is Jonathan's Rakudo day
• we'll do lazy lists in Parrot

Jerry:

• applied a few patches
• answered some questions
• we set up the Parrot Foundation, a new non-profit for supporting Parrot in specific
• assigned directors, incorporated, set up a bank account, will announce at YAPC and OSCON
• involved in the logistics of setting that up and so on
• finalized the purchase of parrot.org
• we'll put up a site there as soon as the domain transfer happens

Jesse:

• what are you transferring over?

Jerry:

• the Parrot milestone funds

Jesse:

• are you getting your own infrastructure for this, or will it run on perl.org?

Jerry:

• need to talk to TPF about that
• just met with Auzon about the Perl 6 tests
• have some questions for discussion
• Kevin Tew's back on track with his project as well
• fighting some PGE brainos
• #parrot helped him move on
• he can parse a 6000 line C++ header file in a minute and a half

c:

• fixing some bugs
• still working on the string system
• made some improvements there
• cleaned up some grotty code, more to go
• might look at the string PDD after the hackathon
• continue to mentor Andrew's GC work; he's on the right track and I have high confidence in him
• helped Kevin profile his code
• found some spots in PGE that could use optimization
• Patrick could put in a couple of hours and get some improvements for everyone

Patrick:

• I spent a couple of hours on that this morning
• too much backtracking
• found some other stuff I want to clean up in the process
• we'll get more speed from longest-token matching though

c:

• and a better garbage collector, Parrot-side
• but a 20% improvement now is a good thing

Patrick:

• also for people who aren't using protoregexes and such

Larry:

• the xor problem is between whether you treat them as binaries or list precedence
• do you think treating them as lists makes optimization more effective?
• it's not the only thing in that precedence level
• if we treated them all as list precedence, does that open up more general optimization improvements?
• I'm getting the feeling that the answer is yes

Patrick:

• PGE's issue is implementation
• with eleven alternations, it stored ten backtracking points

Larry:

• it's simpler to implement short-circuit logicals as binaries
• implementing them as a list operator might not buy you anything
• but they degrade to left-associative anyway, if the operator changes
• I could change all of those left-associative ops to list-associative with no change in the binary case
• I'm going back and forth on that for infix:xor
• should they have the useless semantics of parity?
• it's a tradeoff

Patrick:

• if they were listops, I don't think we'd get that big an improvement
• it wouldn't bother me if it were there
• it's not hard to implement them that way

Larry:

• the listop could implement them in terms as binary, if it wanted
• that's fine

Patrick:

• having the one sense would have been really handy

Larry:

• you have an explicit one now, but that's just junctions

Patrick:

• and it doesn't short-circuit

Larry:

• if you have one that's true, tell me which one
• that's probably more useful

Patrick:

• I could do parity easily

Nicholas:

• would be nice to restart the Perl 6 list summaries
• the Perl 5 summaries have produced some productive work
• David Landgren spends about a full day of work on the summaries every week
• the Perl 6 lists have a bit more traffic combined than p5p

Jesse:

• we've had trouble finding anyone interested in doing the summaries last time

Nicholas:

• there's also the huge problem of trying to summarize IRC
• David's reaction was "ugh"

Patrick:

• I'll try this and report it in the next two weeks
• I haven't been reporting on my progress as frequently as I'd like
• weekly might be better, especially given our velocity
• I prefer that someone else does the summaries
• getting more people to participate is more important
• and it's easier for someone involved to summarize than someone who isn't
• I'll look for some interested people at YAPC
• if that doesn't happen, I'll set aside a day a week to write a summary report

Jesse:

• I'd happily pay in beer or whatever for that

Jerry:

• conference season is a great time to find someone

Patrick:

• if we don't, I'll make that one of my deliverables for my next grant, if I get one
• one question from the SoC meeting before this
• how do we handle design questions?
• I feel badly dumping these on Larry, but I don't know where else to go for resolution of these things
• what should my approach be?

Larry:

• I treat p6l as a queue of things I ought to have thought about

Patrick:

• I'll start posting to p6l then
• I was thinking of p6c, but if p6l works better, I'll use that

Larry:

• p6c tends to read to me like a lot of implementation things my eyes glaze over

Patrick:

• we'll use p6l from now on

Larry:

• you get a little more noise from the theory folks at times

Patrick:

• if things still get Warnocked, I'll bring it up again

Larry:

• that's one reason we have a weekly phone call

Jesse:

• speaking of that, no phone call next week while everyone's at YAPC

The Perl 6 design team met on 04 June 2008 by phone. Larry, Allison, Patrick, Jerry, Jesse, and chromatic attended.

Jerry:

• things are going well with GSoC
• Kevin's on vacation, and due back tomorrow
• he'll be getting to work then
• the other students seem to be working according to plan
• we'll start using the Pugs Trac to track additions to the Perl 6 test suite
• a place to request new tests

Jesse:

• is there a really good reason to use two different ticketing systems?

Jerry:

• it's already there and already has a category for Perl 6 tests

Patrick:

• already in use for Pugs
• and the Pugs repo already hosts the tests
• Rakudo still uses RT

Jerry:

• and there's nothing Perl 6 spec-related I know of that lives in any other ticketing system
• applying some good patches from contributors
• modified the test suite tools to make it easier to work with the spec tests in Rakudo
• you can specify a local list of tests to run
• added the ability to run the spec tests purely as they are, or fudged
• you can use a test config file for that

Larry:

• fudge already kinda has that built in

Jerry:

• if there are no directives in the file
• my addition allows you to ignore implementation-specific directives and run the file directly

Larry:

• and take the consequences
• that's fine

Jerry:

• talked to Patrick about Rakudo management
• we're considering adding some new committers
• looking for a better way to handle tickets for patches
• it's getting difficult for Patrick to apply all incoming patches
• interrupting his work
• we want to be able to appoint a person or persons to apply approved patches

c:

• I'm happy to apply anything once you've approved it and it's ready to apply

Jerry:

• that's what we're looking for
• that should make things easier for Patrick

Patrick:

• mostly Parrot time this week
• refactored list and array handling in Rakudo
• list assignment now works!
• the various composer works
• bare curly hash composition doesn't work yet
• that won't be hard to do
• just need to write some rules to detect when to build a block versus build a hash
• it'll happen soon
• Jonathan implemented a Mutable PMC, which implements Perl 6 scalars
• forwards its operations to what it contains
• we can assign to it and do type checks in the process
• some issues in Parrot's type management
• not always clear when we want to talk about the scalar as a scalar versus what it's containing

Allison:

• container type versus value type?

Patrick:

• yes
• the particularly nasty vtable entry for that is find_method
• we'll look more into that in the next couple of weeks
• it's not impeding progress on us anymore
• lots of cleanup of the AST building routines
• some duplicated code, some refactoring there
• more of that coming in the next few days
• lots of little improvements and changes
• small cleanups
• more tests pass
• starting to keep a daily count of the number of passing tests of the official test suite
• I'll report improvements on a daily basis
• as of yesterday afternoon, we had 55 files and 638 passing tests
• that's an improvement of 344 over the passing week
• Jerry and Moritz deserve special compliments for the work they've done on the tests and the suite
• much easier to work with now
• much easier to find low-hanging fruit
• optimistic about our progress over the next few weeks
• Jonathan is doing a lot of work on classes and types and subsets
• you can declare subset types and have them work
• a variety of improvements are on the way there too
• things are coming together a little faster than I expected
• like Jerry said, one day this week, I spent a few hours applying patches
• that's a good problem to have
• would be nice to read over it for sanity then have someone else test to see that it doesn't break things and perhaps apply it
• not always easy for me to do without colliding with what I'm doing currently

Larry:

• did a little bit of spec work
• trying to remain responsive
• due to a confluence of very many factors, I've been running as hard as I can to stay in one spot

Allison:

• closing out a few last bugs in the concurrency branch
• adding a few new features
• PGE uses bsr and branch heavily
• they don't play well with continuations
• added a new feature for continuation-safe branching
• about to submit the paperwork for the new foundation this week or next

c:

• fixing bugs, closing tickets
• mostly adding polish
• did some work on optimizing string usage; will have more benefits there shortly
• watching over Andrew and his GC work
• he's on the right track, hasn't needed much guidance
• otherwise preparing for YAPC

Jesse:

• not much to report
• just work
• what do you need?

c:

• maybe encouraging people on #parrot to apply patches and test them locally and report their results would help out

Jesse:

• maybe doing that as an example would help?

Patrick:

• we're going to require all patches to go through RT first
• that gives us a little process

Jesse:

• how many patches need massaging?

Patrick:

• some 80% apply and work, but make incorrect assumptions
• I can apply it and say we'll fix the assumptions later
• or I can respond and give guidance to resubmit the patch
• or I can make the minor changes and fix them as I apply it
• pretty rare to get a 100% patch
• but I don't want to become the bottleneck

Jesse:

• what's the time-consuming part of the apply and test cycle?

Patrick:

• I'm trying to avoid something
• someone comes out with a patch that solves a test case
• they submit it somehow
• you want to give them instant gratification
• if I'm in the middle of working on a feature myself
• and many that I work on aren't small
• I need a separate checkout to apply and test and commit
• that takes a few minutes, maybe ten
• then I have to pick up where I was, merging their changes into mine
• the low-hanging fruit is all in the same files

Jesse:

• it doesn't sound like tooling is the important thing to fix here

Jerry:

• it's the process

Patrick:

• a lot of it will fix itself in a short period of time
• part of the problem is that I was absent from code review for a couple of months
• I'm glad we made progress during that point
• but now it's the time to refactor away some of the assumptions there
• it may not be as big a deal if I get caught up soon
• that will happen sooner if I don't have to deal with the other things

c:

• about 80% of the patches I see for Parrot in general are high quality
• not a lot adding new features, which may affect that
• but the quality is good, and I'm pleased

The Perl 6 design team met by phone on 28 May 2008. Larry, Allison, Patrick, Jerry, and chromatic attended.

Larry:

• mostly off for this long weekend elsewhere
• continuing to work on quoting roles
• they need generic support to mix in roles with parameterized strings as starter and stopper sequences
• that's an interesting problem, when mapped into Perl 5
• no support for generic roles or parameterized types yet
• I'll probably just do an eval

Allison:

• reworked continuations in Parrot, based on our discussion last week
• I have very high regard for the value of reading through the commit logs
• found the source of the problem that way
• someone committed the problem about a year and a half ago
• now continuations work the way they should work
• we now don't try to make CPS respect the dynamic environment stack
• it's largely unused now that we don't store exceptions there anymore
• tracking down one last bug
• I can duplicate it
• it's an exception thrown, caught, and resumed from an :onload subroutine
• looks like they get executed differently from regular execution
• first draft of the Bylaws and Articles of Incorporation for the Parrot Foundation
• hired a lawyer in Washington state for incorporation there

Patrick:

• added quite a few things in the Rakudo implementation
• added a new metamodel that looks like the Perl 6 metamodel
• with Jonathan's and Jerry's and Moritz's help, we converted the compiler tools to that model
• cleaned up some of the inheritance hierarchy in Rakudo
• builtin types report correct methods and types
• a Parrot Integer reports itself as an Int in Rakudo, down to the metaobject information
• fixed pair handling, especially for calling functions
• people are testing those in great detail now
• now it's obvious that Rakudo's and Parrot's and Perl 6's argument handling there somewhat disagree
• doing some PGE refactoring
• primarily fixing up its class hierarchy; looks pretty easy
• may move Rakudo's grammar to use protoregexes
• might help get the metaoperators working -- French brackets, for example
• if we don't get this soon, people might submit patches doing things the wrong way (multiplying new operators for various combinations)

Jerry:

• Google Summer of Code started this week
• the aggregated feed of Perl GSoC 2008 student and mentor blogs is available
• the students will report weekly status at the #parrotsketch meeting
• the students and mentors will report at #perl6-soc on irc.freenode.net on Wednesdays at 18:30 UTC
• all of the students seem to be on track now
• made some spectest changes
• made some changes to rakudo List methods, applied some patches
• made some Rakudo test changes, should make test management easier and allowing us to create private and shared lists of tests we want to run
• should make it simple for rakudo developers to specify a subset of test files, making focused development and focused testing easier

c:

• not a lot of energy for hacking for some reason
• applied a few patches
• cleaned up a few tickets
• talked with Andrew quite a bit yesterday about GC plans
• he has some good ideas, including a clever marking scheme for which I have high hopes
• mostly on the right track
• will write up some guidelines on profiling and optimizing
• have one idea to optimize keyed object attribute access
• otherwise really need to work on the PIR profiler

Richard:

• trying to plan a legitimate way to ascertain Perl 6 grants, given the donation from Ian Hague
• planning a process around that
• thinking about YAPC::NA and a grant process BOF

Allison:

• Patrick mentioned argument passing in his report
• we don't quite understand how it needs to work in Perl 6
• Parrot will support whatever Perl 6 needs, though it may not necessarily be the default
• one goal of argument passing is to make it lazy
• you don't have to parse the whole list to start pulling out parameters
• required parameter positionals are clear
• required positional parameters are clear
• you can pass in a required positional parameter as either a positional or named argument
• Parrot maps the positional arguments to positional parameters
• when it runs out of positional arguments, it starts checking named arguments

Patrick:

• that's not really what Parrot does
• Parrot fills positional parameters with positional arguments
• then it fills in named parameters with positional arguments

Larry:

• what do you mean when you say "named parameter"?

Allison:

• pairs

Patrick:

• all parameters are named in Perl 6
• they can be filled by a named argument
• Parrot has parameters that you can fill only by position
• then there are parameters that you can fill by position or name
• but Parrot fills them opposite from how Perl 6 would expect; it chooses the positional argument over the named one.

Allison:

• I thought you needed a way to skip over a named parameter in a list
• and then fill in its value from the named argument list
• or scan the entire named argument list first, and then fill in positional parameters
• it sounds like this is different

Larry:

• the compiler should recognize arguments intended as named parameters
• it can set up a data structure for efficient access
• the other constraint is that any argument can depend on the value of a previous argument in the list
• you have to bind them in the order of declaration
• you go down the list
• for each of them, you look up in the named argument list
• to see if it's bound by a name
• if not, you take the next one from the positional list

Patrick:

• that can be lazy

Larry:

• yes
• that's the abstract view from the Perl 6 end
• one reason we invented prototypes is so that the compiler can know that the first n arguments are positional
• map any named arguments into positional without having to do the lookup
• that's primarily a sop to efficiency
• for a very small number of named arguments, brute force is probably faster

Allison:

• do you never have any parameters which are strictly only positional?

Larry:

• there's no way to write that
• other than writing a bare sigil in the declaration
• you're not giving it a name
• but you can't access it, because it needs a name

Allison:

• sounds like we want three different flags in Parrot
• one for a purely-positional argument

Larry:

• the optimizer might use that, if it sees a prototype

Allison:

• we keep the :named flag
• for named parameters

Patrick:

• named-only

Allison:

• we don't really have that
• we have named that acts like positional

Patrick:

• we have named that prefers positional, and then throws an exception if a matching named argument is also supplied

Allison:

• might have four flags then
• one that's either named or positional
• maybe one that gives priority to the positional
• not sure that's useful

Patrick:

• I couldn't come up with a case where that's useful

Larry:

• some languages want that

Patrick:

• you can almost simulate that with slurpies in your code
• or sticking in dummy arguments

Allison:

• could come in useful if you have optional parameters
• you don't have to pass anything at all

Patrick:

• I proposed what you described as numbers 1, 2, and 4
• the existing :named flag is named-only
• the :positional flag is a named parameter that accepts positional arguments

Allison:

• I'm trying to get down the ideas
• I probably wouldn't use :positional for that
• important to get the categories right

Patrick:

• I couldn't figure out a case where you have a named parameter that prefers the positional over the name

Allison:

• make it a positional and don't worry about it
• if you make it positional, you don't have to scan the argument list

c:

• look at Python's default parameter handling
• I think it may follow the one case you couldn't figure out
• not entirely sure

Patrick:

• is the answer that Parrot's underlying model will likely change to support what Perl 6 needs?

Allison:

• yes
• exact details yet to be determined
• as soon as possible so that you can use it

Patrick:

• it's been the subject of at least five or six RT tickets in the Rakudo queue
• they look like different problems, but they call come down to the same problem
• it's becoming a pain point
• I'm willing to implement it
• there are probably three or four related tickets in the Parrot queue

Larry:

• there was a question about writing Pairs various ways as arguments
• are they intended to be named arguments or pairs?
• how do you write the prototype for list, by the way?

Patrick:

• I suggested putting colons in front of it

Larry:

• but :a is always a valid term

Patrick:

• you can always look ahead for a fat arrow

Larry:

• I'd almost rather write :> for that
• originally we always used the fat arrow
• then we put in the colons
• we wanted to use the fat arrow for arguments
• people seem to be used to the colon for named arguments, not the fat arrow
• maybe we should just bite the bullet
• you need the name as the identifier
• you don't need the general pair form
• the fat arrow makes a Pair not intended for a named argument, unless you do processing on it
• that'd be a simplification
• you can tell by just looking at it
• one is a named argument
• the other is just a positional parameter

Jerry:

• is there a method on a Pair that allows it to be passed as a named parameter?

Patrick:

• prefix vertical bar?
• S06 has options for making a Pair a named argument
• or a hash as a list of named arguments

Larry:

• we're trying to guess it based on the context of the argument list
• if you pass the list to a function call, it assumes you want named arguments
• otherwise you don't
• that might be guessing too far
• the list function might want to parse more like parens
• but it behaves like a function
• you don't want to pass those as named arguments
• how do you tell the compiler that it's special
• maybe you want to make it a macro, but that's kind of weird
• maybe it's time to look at the cultural issue of how people want to write named arguments
• maybe we can simplify that

Patrick:

• there's a similar issue with the hash constructor
• you can kinda get away with it, because order is less important
• but it still exists
• it'd be interesting to look at the existing tests and Synopses to see the results
• there is some historical baggage toward using the fat arrow

Larry:

• the guy who wrote that Synopsis was familiar with Ada's fat arrow for named arguments

c:

• is there some sort of prototype declarator that says "It was an ordered list in the source code, and that's what I want!"

Larry:

• macros are a lot like prototypes

Patrick:

• Rakudo for now will treat them as special in the way that Pugs does
• it'll treat list as a macro
• do something special, it's not a normal argument list