Perl 6: April 2008 Archives

The Perl 6 design team met by phone on 23 April 2008. Larry, Allison, Patrick, Jerry, Will, Jesse, Nicholas, and chromatic attended.

Jerry:

• we're in a period where everyone's trying to break Parrot
• they're adding new features and accidentally breaking thing
• but they're fixing it
• it's a good part of the cycle
• people fix it
• we don't have a build farm, so we can't test everywhere before committing to trunk

Patrick:

• I thought that was the point of the release cycle

Jerry:

• some people have suggested that we always keep trunk building and passing tests
• but we don't have the means to do that
• especially when we're playing with config
• moving on, the big news is that TPF has six slots in Google's Summer of Code
• one of them is fleshing out the Perl 6 test suite
• we've needed someone to spearhead that
• having a funded new contributor is wonderful
• two Parrot-related projects
• one is generating NCI stubs
• Kevin Tew, a long-time contributor
• the other is the incremental GC specified in the PDD
• that's Andrew Whitworth
• there's also an ASF project for integrating the GC from Apache Harmony into Parrot
• they've wanted to release it as a standalone library
• Parrot's the first test of a standalone system

Nicholas:

• nice that it doesn't count against TPF's slot list

Jesse:

• and it's nice visibility for Parrot from another group

Jerry:

• I finally have six weeks of no plans to travel
• should be able to devote more time to Parrot and Rakudo
• looking forward to that

Larry:

• getting some hacking in on my two pet bugaboos, the longest token matcher and match object generator
• I refactored the matcher
• it still uses TGE
• instead of lumping all of the expect term possible tokens (that is, all of them) into one bucket it separates them into buckets based on their first letter
• it's a one-level tree
• we can build a much smaller DFA for the regexes that start with that letter
• it caches that, of course
• can get an instant reject if the next token can't possibly start with that letter
• also flattened out all of the rules such that the list of tokens is easy to read
• if the first probe with the DFA engine fails, I can take that small set of tokens that start with the same letter, run all of those rules, and sort from longest to shortest
• preserves the token matching order without building huge DFA structures
• as a backoff strategy, that will scale pretty well
• refactored the parameter passing on the matcher side (STD 5)
• instead of passing an initial array of random things, I have parameters
• constructs match objects more correctly
• in the sense that it gets all the information it's supposed to have
• also has some attachments where it shouldn't have
• I'm cleaning that up
• that should scale pretty well

Jerry:

• is there a drop in memory usage?

Larry:

• I haven't measured
• I'm sure that not feeding 800 regexes to TRE at once will make it allocate 17 megabytes on the stack
• it might still be allocating too much for some of the larger things
• I'm still aiming for correct, as opposed to fast
• just trying to bear fast in mind
• the longest token matcher now returns a linked list of states
• not a string
• should be a lot faster; easier to cache
• functionally it's the same as before
• one of those things you don't even have to measure to know it'll be faster
• trying to avoid the bugaboo of premature optimization by doing what I know will be efficient to begin with
• all the while trying to make the thing work
• it has a good chance of being pretty speedy
• my talk in Tokyo will be about all of the places where the current grammar allows extensibility
• it'd be nice to be able to demonstrate some of that

Allison:

• getting work done again
• launched the Strings PDD
• list of tasks for concurrency that I'm breaking down into smaller pieces
• may post what I have now, and leave other people to break them down

Patrick:

• are they parallelizable? :)

Allison:

• many of them are
• there are some bigger things, like switching the exception system over to the event handler
• otherwise, just life stuff

Patrick:

• had paying work come up this past week, so not a lot of actual coding
• need to type the milestones document
• it's all in my head
• managed to remove a lot of unused code thanks to chromatic's post about possible optimizations
• mostly just cleanup
• but helped me figure out things which will feed into my redesign of PGE for longest token matching
• should be able to return to direct Rakduo hacking later this week

Will:

• various Parrot cleanups
• TPF quarterly grant proposals are due at the end of the month
• haven't seen anything come in yet

Allison:

• they're queued in an RT queue
• I don't know if grant members have access to that queue

Will:

• we do
• please, get your proposal in now, sooner than later
• that goes for you on the call as well as people reading the minutes

c:

• mostly spent the past week optimizing Parrot and Rakudo
• looks like it's the building speed is twice as fast as when I started
• runtime is faster too, but the optimization is compilation time
• found some infelicities that need more design thought
• but I'm happy to put these improvements in now and take them out later when the design improves around them
• hope to start adding new features again soon

Patrick:

• most of the test execution time is in compilation
• how useful would it be to compile Rakudo to standalone PIR?

c:

• I'd find it useful
• but I'd find about ten things useful with all I work on
• so not a blocker at the moment

Jesse:

• how far will that get you toward native executables?

Patrick:

• the existing trick for building perl6 would work
• but it's not the same

c:

• if it takes an hour or two, it would help me with debugging and profiling
• if it takes more time, it's not that important

Patrick:

• we have to figure out runtime deployment issues for the Perl 6 runtime library

Will:

• we could add the requirement to run from languages/perl6/ right now

c:

• that's fine by me for now

Patrick:

• that's an afternoon job, not too bad
• what do we need to do to get the Perl 6 and Parrot pages up to date?

Will:

• I'll work on that

Nicholas:

• why is C99 useful to Parrot and the compiler tools?

c:

• front-end parsing for C header files to build NCI declarations automatically
• the backend is pretty easy, that's thunk generation
• the front-end keeps people from having to write boilerplate code by hand
• generate the front-end once, where you have the headers, and then you can run the generated code anywhere even if you don't have the headers

Jesse:

• how does that compare to Python's ctypes?

c:

• as I understand it, they have the nice backend stuff
• not so sure about the front-end
• my P5NCI is nicer, if incomplete
• just haven't had time to work on it...

Jesse:

• if you put in for a TPF grant, that would be very useful

c:

• get me a clone first, and you have a deal

Jerry:

• Allison, we talked about implementing return
• that requires tying in exceptions to the concurrency scheduler?

Allison:

• yes
• just not implemented yet
• when we did exceptions, we didn't have concurrency
• so it's on the top of my list to tie them together

Will:

• Tcl's already using exceptions to handle return, break, and continue

Allison:

• right now, you can't have an exception handler which is a full subroutine

Patrick:

• I'm not sure we need one for that feature
• every subroutine block decorated as such in PAST puts an exception handler in the block
• if any nested block throws a return type exception, it grabs the arguments, does what it has to, and then does a Parrot return

Allison:

• if that's what you need, go ahead and do it
• I thought there are some features you didn't have yet

Patrick:

• I thought there might be some opcodes I needed, like handled
• but we can do something now
• might not be completely optimal
• but it's just packaging things up now
• I have something I think will work
• it's not trivial, but I'm 80% confident
• just a matter of sitting down and doing it

Allison:

• the concurrency stuff will be there before the next release
• might not want to roll it in before the release
• but it'll be there soon

Patrick:

• I want to get return in for the April 15 milestone we're behind on

Jerry:

• have you put in tickets for the breakdown of specific tasks?

Allison:

• I've never done tickets for that
• just sent mail to the list of the tasks
• handed them out to people as they volunteered

c:

• can you put them on a wiki page?
• some of the other committers wanted that

Allison:

• I can do that

The Perl 6 design team met by phone on 16 April 2008. Larry, Allison, Jerry, Will, Nicholas, Jesse, and chromatic attended.

Jerry:

• the release went pretty smoothly
• had some help
• the make release target is broken on Windows, and that's the only platform I had out here
• we'll fix that before the next release from Windows
• talking to Andy Armstrong about getting Test::Harness to 3.0 and subclass TAP::Parser so that we can report Rakudo's fudge tests better
• every fudged test is a failure right now, even if all subtests pass
• I talked Jonathan into implementing simple MMD in Rakudo
• chromatic wrote about it
• but it's broken in the release (and only the release)
• put some work into internationalization
• need to figure out the make rules
• then I can put more work into localization too
• trying to secure the parrot.org domain too
• TPF is likely to get five slots for GSoC
• some will be Parrot and Perl 6 related
• the official announcement is on Monday
• trying to encourage others to take on responsibility
• seems to be working
• some of the committers I've mentored are becoming mentors themselves

Patrick:

• mostly reviewing different things
• working with Jonathan on his various objects and MMD implementations
• will check in my Rakudo milestones document tonight or tomorrow morning
• cleaning up bug reports, closing tickets, catching up on patches
• need to spend a little more time on paying work this week

Will:

• trying to get the most recent release bundled as a macport
• there's apparently a build issue since the previous macport
• should have an easy way to install Rakudo as perl6 after that gets straightened out too
• still trying to cut dead things out of Parrot

c:

• poked at optimizations per Patrick's request
• sped up OO and Rakudo by about 40%
• not as much as I wanted, but you notice it
• looked at a few more optimizations, but they're bigger and take more work
• GC is the biggest, so let's hope we get that as one of the GSoC projects
• think I can get the profiling core to emit Callgrind-compatible output for PIR
• I know mostly how to do it now

Larry:

• spending a lot of time tweaking a new laptop and getting it all set up and customized nicely
• feebly trying to keep up with the onslaught in p6l
• I haven't been keeping up, but I'm keeping my eye out for things going off track badly
• plotting how to get rid of TRE as my longest token matcher
• want something that will scale better
• the handwriting was on the wall the first time I went in it with the debugger
• for the regex matching where a token is expected, TRE was allocating 17 MB on the stack
• I have some ideas for something with better semantics and less memory usage
• TRE optimizes for running one regex a lot over a lot of data, rather than running a lot of regexes

Jesse:

• spent a lot of time in discussions about funding Perl 5 hackers
• I see François Perrad has released a Win32 binary of Parrot

Jerry:

• he's been doing that for the past few releases
• just a release in binary form, not a fork or branch

c:

• what's the status of Parrot in Debian?

Allison:

• one final build bug in IA-64
• the guy with the box should be getting to it this week
• we're planning to put up the 0.6.0 release
• if it doesn't go soon, I'll just put the F<.deb>s on the site
• we might put IA-64 builds on our platform wishlist

c:

• we need someone who knows how to fix them too

Allison:

• it was a PGE bug unrelated to the arch, I think, in 0.4.0
• we just need the bug confirmed fixed on that arch for Debian

Jerry:

• PAUSE kinda stinks
• I hate that we don't know if we'll have an authorized release of Parrot until it gets uploaded
• PARROTRE lacks permissions on eight modules
• all of which have been refactored on something else
• they're all related to the configure or test system

Allison:

• we don't have to distribute through PAUSE

Jerry:

• we should not index those modules anyway

c:

• they're not that useful outside of Parrot anyway

Will:

• the release link on the Parrotcode site, http://www.parrotcode.org/release/devel, links to the CPAN download
• there's a lag between the update and the availability

Jerry:

• we could upload to Parrotcode first
• upload to the CPAN from that site

c:

• I'm not sure why we should index the config::* modules
• the only ones I care about are in Parrot::Embed

On the train over to Stockholm after the hackathon and on the plane back to Spain a day later, I implemented various cases of 'handles' (not all of them, since the wildcard ones are trickier - they will get done probably along with a whole load of other work on attributes that needs doing). The 'handles' trait verb is the thingy that lets you auto-generate methods that delegate to a methods on an attribute. Here's some examples of what you can do with this:

class Bar { method a { say "a" }; method b { say "b" } }
class Foo1 { has $x handles 'a' } # one method
my $test = Foo1.new(x => Bar.new()); $test.a()
a
class Foo2 { has $x handles <a b> } # several
my $test = Foo2.new(x => Bar.new()); $test.a(); $test.b()
a
b
class Foo3 { has $x handles :mya('a') } # rename one
my $test = Foo3.new(x => Bar.new()); $test.mya();
a
class Foo4 { has $x handles (:mya('a'), :myb('b')) } # rename many
my $test = Foo4.new(x => Bar.new()); $test.mya(); $test.myb();
a
b


So, that's another small piece of the Perl 6 implementation puzzle put into place.