Rakudo Perl: September 2008 Archives

The Perl 6 design team met by phone on 03 September 2008. Larry, Allison, Patrick, Jerry, Jesse, Nicholas, and chromatic attended.

Larry:

  • I was at the Lightweight Languages Conference in Tokyo last week
  • about a thousand people there
  • a large Ruby presence, as well as some of the other "scripting" languages
  • gave another variant on my talk
  • seemed well-received
  • mostly just vacationing and touring around otherwise
  • did some work getting STD and the test suite to agree with each other
  • now have symbol table checking in STD
  • a global scale, not lexical
  • can tell when you call an undefined function
  • found surprisingly few difficulties with the existing parsing
  • that makes me happy
  • hanging out on IRC and email, doing a little bit of spec work
  • the standard parser can parse itself in under 30 seconds
  • I have other ideas to make it faster
  • not sure I want to implement them yet
  • it might adversely impact other work with installing a real LTM
  • they remain ideas for the moment

Patrick:

  • most of you saw the announcement about Paula
  • we celebrated last Friday
  • mostly answering questions and debugging problems on Rakudo
  • expect to be back full-time on hacking tomorrow
  • Jonathan's out on vacation this week
  • Moritz and Carl and other people have been continuing to make progress
  • I've been responding to them
  • not a lot has happened beyond testing improvement
  • the primary coders are on vacation or busy with other things

Allison:

  • another nice quiet and productive week
  • we have a new, much cleaner dispatch route in Parrot
  • it uses signature objects, which are like Perl 6's Captures
  • everything you need to invoke a sub is all in one place now, not spread out over lots of calls and varargs and unpacking varargs and...

c:

  • you realize that code has always been haunted, right?

Allison:

  • I'm ripping out some of the old code, but some of it will stick around
  • we can make another branch to clean it up
  • the branch has some problems in TGE
  • I probably haven't caught all old dispatch instances yet
  • we're ready to launch the new Parrot website
  • doing some work on that
  • almost ready to launch the new Parrot mailing lists too
  • just need to ask Ask or Robert for subscriber lists

c:

  • started a couple of arguments online
  • need to work on the "How to Help the Patchmonster" document
  • worked on vtable cleanup in Parrot, had MMD problems, may have to wait a bit
  • otherwise applying patches
  • should have more time in the next couple of weeks

Nicholas:

  • I love parallel tests in blead, and I'm missing them in 5.10

Jesse:

  • I've been working and traveling for the past two weeks
  • also been trying to goad chromatic to post suggestions to p5p

Allison:

  • which seems to have worked

Jesse:

  • is there anything I can do to help you?

Patrick:

  • I just need time and energy
  • so maybe saying "Get back to work!"

Jesse:

  • I can do that

Jerry:

  • haven't had any commits recently
  • helping lots of people with implementation questions
  • talked to Evan Phoenix about the Rubinius VM
  • he had some questions about Parrot's architecture and design decisions
  • good chat
  • we started talking about bytecode and AST translation
  • they're lacking a well-defined AST now
  • just S-expressions
  • we'll chat more frequently
  • there are only so many people interested in implementing VMs
  • we should all get together
  • still talking to Microsoft
  • they want to know what an ideal testing environment would look like in their labs
  • I need to figure that out
  • it could be good for Perl 6, Parrot, and Perl 5

Patrick:

  • is that worth taking to the mailing list?
  • Allison mentioned something about old MMD and new MMD
  • what's the difference from a PIR perspective?

Allison:

  • absolutely none

Patrick:

  • there should be no difference in PGE and TGE then

Allison:

  • there are a few places internally which are C code
  • if there's a direct C call into the old MMD system, bam!
  • I've been replacing them with the new C-style calls

Patrick:

  • it's just that PGE is exposing the bugs
  • it's not a source

Allison:

  • that's why it's so perplexing
  • I haven't been able to trace it back to a particular call

Patrick:

  • I'll look at it too
  • I'm happy to see Parrot taking this new model

Allison:

  • we can avoid a lot of indirection

Larry:

  • there have been discussions of different levels of test harness support
  • we need to straighten out the namespace
  • "What does use Test; mean?"
  • it has overloaded meanings for many people
  • should that be the minimal thing, and everything else be larger
  • or should the minimal test thing be built into the language

Jesse:

  • that would be fascinating

Jerry:

  • and everything's multi and can be overridden?

Allison:

  • ok() and is() are easy to write
  • easy to include in the language

Larry:

  • just in the standard Prelude
  • it's like POD's play to bring documentation into the language to make it easy
  • it could be a module that Prelude brings in

Patrick:

  • just a module that implements these?
  • they print what they print now?
  • I'm happy with that
  • I'd like to define the minimal set of functions that the test suite can rely on

Larry:

  • or the fancy set should be the default set

Patrick:

  • I can argue it that way too
  • but that seems to be the wrong way
  • I'm happy for any clarity the Perl 6 designers throw in there

Jerry:

  • for a Perl 6 implementation to be a real implementation, it uses the core tests only
  • and builds on top of those
  • any other fancy test library is out of the core and builds on Perl 6?

Patrick:

  • the test suite functions are part of the official test suite

Nicholas:

  • that scared me slightly in how the perl 5 core bootstraps itself to testing, and trusting/testing its test system
  • the very early tests run inline
  • you can't be sure that require works
  • the next step uses a file named test.pl
  • doesn't use ++ for example
  • everything beyond the core ops tests are free to use Test::More
  • written in proper grown-up Perl
  • uses constructs which may have failed earlier

Larry:

  • we still have two of those things
  • the third level is potentially possible
  • the core definition of standard Perl 6 could have a fancier test module
  • the test prelude we're talking about is the equivalent of that test.pl

c:

  • and if you're implementing this, you can hardcode these as special cases in the compiler
  • and then you can rewrite them in your Prelude when you have that working

Patrick:

  • we can get rid of the line use Test; in our tests now

Larry:

  • that's part of the motivation

Patrick:

  • we don't have to worry about getting our namespaces working first
  • I'm in favor of that
  • that would have sped up a lot of stuff in Rakudo development

Larry:

  • the only other major argument against it is the notion that the function names are Huffman-coded for testing
  • not necessarily general use in a language
  • a user might want an ok() function that does something else
  • that's okay, if we do the overriding properly
  • but there's a conflict with the current spec because there are two different meanings of is
  • I can change the meaning of is Dog;> within a class
  • it can be specific, so it only parses on is followed by a type name

Jerry:

  • or it's only a single parameter

Larry:

  • multidispatch could handle that, but it's declaration
  • declarators need compile-time resolution
  • it needs compile-time distinguishing
  • the other is has multiple arguments
  • this one never does

Patrick:

  • it hasn't been a problem in Rakudo yet

Larry:

  • it's doable
  • the question is if it's too much of a hack
  • or if there's a better way

Patrick:

  • that depends on what the meaning of "is" is
  • what kind of equivalence does it do?

Larry:

  • in the header, it's either a trait
  • or if it's a type name, it turns into an isa
  • there's some argument of some kind

Patrick:

  • what if I say is $a, 3?

Larry:

  • or we can do is_eq, is_eqv
  • or just ok

c:

  • but you do lose diagnostic messages

Larry:

  • unless it's a macro
  • but that could be too clever

c:

  • what if someone thinks is is infix?
  • equivalent to == or eq?

Nicholas:

  • or is can do smart match
  • you have smart match in Perl 6

The Perl 6 design team met by phone on 20 August 2008. Larry, Allison, Patrick, Jerry, and Jesse attended.

Larry:

  • decommuting from Copenhagen
  • couple of tiring days wandering city
  • mostly hacking on STD.pm grammar to speed it up a bit
  • mostly propagating ratchet optimization around
  • compiles itself in under 90 secs
  • it's an improvement
  • also working on adding rudimentary symbol table checking
  • can figure out where it's misparsing things as function calls
  • getting ready for trip to Japan next week
  • lightweight languages conference, for some definition of "lightweight"

Patrick:

  • returning home from Copenhagen, which was quite productive
  • on the plane ride I implemented pieces for precompiled modules in Rakudo
  • finished that up last night, so now Rakudo can use precompiled modules
  • spectest_regression on my box went from 12 minutes to under 4 minutes
  • it also means that use statement can start doing load_bytecode things
  • today been working on lazy lists with jonathan
  • much of my time in the next couple of days will be writing reports and blog posts, catching up on events at OSCON and YAPC::EU
  • also, at YAPC::EU hackathon Jesse and I put in some refactoring on Rakudo's ROADMAP, identifying task dependencies and some relative effort required

Jerry:

  • GSOC update; mentors reviewing and submitting final reports
  • looks very good that all three are completing
  • very successful SOC
  • we have good NCI stubs for Parrot, no longer need to write function signatures for things in C header files
  • have 700 new tests added by Adrian for Perl 6 spectests
  • have mostly complete GC implementation (some debugging needed)

Jesse:

  • the recent blog post sounded a little "unhappy" with outcome

Jerry:

  • yes, that the student's self-perspective
  • however, it's design-wise complete, just not debugged fully

Allison:

  • we didn't expect total rewrite of GC system
  • we can do the GC integration with Parrot
  • he did what we needed him to do, indeed he went much farther than I expected

Patrick:

  • chromatic also mentioned that things are farther along than intimated
  • just need a bit more debugging before we can pull it back into trunk
  • didn't want to potentially block others' progress

Jerry:

  • trying to catch up on work done at YAPC::EU
  • much didn't happen online
  • missed out on some good conversations
  • today discussed exportation in Parrot on IRC with Patrick

Jesse:

  • getting way too little sleep at YAPC
  • very nice conversations with Patrick
  • developed ROADMAP, put estimates of relative effort on them
  • there's a lot less left to do than I would've estimated
  • I also talked to Jonathan about what he'll be up to after his current grants expire

Allison:

  • reverse commute from Copenhagen
  • diving right into mmd branch, have tasklist
  • Andrew and smash very helpful
  • finished parsing of multi declarations
  • build multi-dispatch table from C declarations
  • then deprecate a bunch of C code to get to a single mmd impl
  • roll that branch back into trunk
  • should finish MMD milestone this week or next
  • then it's onto IO implementation

Jesse:

  • Perl 6 IO specification was flagged in Saturday's discussions
  • Where does that belong?

Larry:

  • I've looked at it a few times, it's certainly not complete
  • "It's just a library issue!"
  • it needs some TLC

Jesse:

  • Are there others worth harassing for proto-TLC?

Larry:

  • where IO interacts with concurrency I tend to sweep under the carpet
  • I tend to not stress about it
  • let's do one step better than P5 and not worry about it

Patrick:

  • I also think that as long as we can soon provide an implementation that can do basic file I/O and sockets, then people can explore and play in the design space

Larry:

  • as long as it's properly OO to begin with, there's lots of room to play

Allison:

  • Parrot's IO model is becoming much more OO, yes

Jesse:

  • could Rakudo take care of providing an OO interface to Parrot's IO?

Allison:

  • independent of Rakudo, Parrot IO needs to be OO
  • the old design seems to be "reinventing multiple systems to do the same thing"
  • also wanted to mention volunteers working on test.parrot.org
  • moving pages over to new parrot.org domain
  • as soon as pages are moved over, we just need security certificate

I came back from a trip to my local curry house tonight determined to get my recent work on implementing the Perl 6 multiple dispatch algorithm being used by Rakudo. I had already got the algorithm itself implemented and got some PIR tests for it, but Rakudo was still using the Parrot multiple dispatch algorithm. This didn't provide the Perl 6 candidate sorting algorithm, nor did it allow dispatch based upon roles as types or with subset type constraints acting as tie-breakers. So, I sat down and started hacking. About 2am, a series of patches hit the repository, the final one switching all code compiled with Rakudo over to using the Perl 6 multi-dispatch algorithm. :-)

I'm a little tired now, but here's a couple of examples that now run. This first one decides which one to call based upon one of them having a constraint that can disambiguate them.

multi sub foo(Int $x) { 1 }
multi sub foo(Int $x where { $^n > 42 }) { 2 }
say foo(12); # 1
say foo(45); # 2

And here's one where we have a role involved.

role Explode { }
class Firework does Explode { }
class Kitten { }
multi sub bar(Explode $x) { 1 }
multi sub bar(Kitten $x) { 2 }
say bar(Firework.new); # 1
say bar(Kitten.new); # 2

I've discovered that there are some problems that have come up with the candidate sorting, but I'm too tired to track them down right now. Curiously, there are PIR tests to exercise this and they pass. It's only when using it from Rakudo that the problems appear. So there's something not quite right there, and I'll investigate it in the next couple of days. In the meantime, all spectests that were passing before continue to pass (and we've got two more unexpected passes now too), so I'm quite hopeful that this isn't going to cause too much disruption to folks while the kinks are ironed out. Anyway, enjoy, and bug reports (other than the candidate sorting issue that I'm aware of) are most welcome. :-)

Thanks for funding me to stay up until this crazy hour hacking on this (OK, to be fair, I slept until mid day today^Wyesterday) goes to DeepText.

The Perl 6 design team met by phone on 06 August 2008. Larry, Jerry, Nicholas, Jesse, and chromatic attended.

c:

  • started the release management expectation for post 1.0
  • want to get that nailed down before the end of the year
  • applying patches, fixing bugs
  • more small projects, the better
  • helped Allison get the concurrency branch merged, looks fairly good

Jerry:

  • agree with that timeline for getting documents sane on the release process
  • lots of traveling for several Parrot and Rakudo hackers
  • work has slowed down because of that
  • the concurrency branch merge was pretty smooth
  • able to dedicate more time to Parrot and Rakudo lately
  • reviewing Kevin Tew's GSoC project as it nears completion
  • running into a miniparrot build problem
  • trying to help him debug that
  • taken over mentoring for Adrian while Moritz is on vacation
  • he's been pretty quiet this week

Larry:

  • the standard grammar parses 99+% of t/
  • four of the six failing tests are the fact that it does not install new grammar rules for user-defined operators
  • one of the tests fails because the author of the test thought that Perl 6 as should read your mind as to where to apply adverbs
  • straightening out some of the parsing snafus
  • we need to be able to distinguish between types and variables all over the place
  • now :: really isn't a sigil
  • it gets in the way all over again
  • special symbols like ::?CLASS are specially parsed at this point
  • responding to questions on p6l about exception handling with defined-or and MMD semantics
  • mostly just hacking on the standard grammar and the tests
  • always have to decide whether the test or the grammar is wrong, when they conflict
  • thankfully it's often the tests
  • going to Copenhagen next week

Nicholas:

  • part of the problem of Perl 5 and the CPAN gets solved with Perl 6
  • which allows different versions of modules by different authors

Jesse:

  • had several chats with Audrey, who's hacking on Perl 6 again
  • she's taking Pugs apart into separate useful packages of Haskell code for Hackage
  • it's a Voltron-style development method for Pugs
  • the current version passes about the same number of tests as the version of two years ago
  • it now passes them in 15 minutes, instead of two hours
  • a dramatic performance improvement already

Nicholas:

  • why?

Jesse:

  • some internals changes
  • she's documented some of them
  • she's waiting for the 6.10 version of GHC, which adds built-in DSL support
  • it'll be a lot easier to port the standard grammar for running in Pugs atop GHC
  • it's easier to drop pure Perl 6 implementations of Perl 6 on top of Pugs and get them to work then
  • now a lot of cleanup and stabilization is going on
  • she wants a lot of people hacking on Rakudo, v6.pm, and the tests
  • also Haskell hackers
  • Flavio's back to hacking, but we haven't caught up
  • there's a lot more activity now than three weeks ago
  • I'm presuming there's no call next week, with so many people at YAPC::EU

Nicholas:

  • applied a patch to get parallel tests working in the Perl 5 core
  • running full tests in parallel is only slightly faster than serial

Jesse:

  • use -j 9 on a dual-core machine with the new Test::Harness
  • --fork also works well
  • --save and --slow front-loads the slowest tests, which causes things to run faster

Nicholas:

  • Andy Armstrong says this is moderately brittle
  • I think he pays more attention to issues from p5p at the moment than Parrot

Last year I received a Perl 6 development grant from the Mozilla Foundation and The Perl Foundation. Below is a copy of the final report I've submitted to close out the grant. It's also available in PDF if anyone wants a more printable version.

My thanks to everyone who helped me with this grant, and I'm looking forward to the next one.

[2008-09-11 update: the original version of this report mis-attributed the Squaak tutorial to Kevin Tew; the corrected author is Klaas-Jan Stol (kjs). My sincere apologies for the mistake, it is now corrected below.]

----------

Perl 6 Development Grant
Final Report

Patrick R. Michaud
September 10, 2008

INTRODUCTION

This is the final report for the Perl 6 and Parrot development grant provided by the Mozilla Foundation and The Perl Foundation. This report summarizes the work performed under the grant, what has been accomplished, and where things are headed from here. As this report illustrates, we have achieved all of the goals and outcomes intended for this grant.

REPORT

The primary focus of the project was to get Perl 6 on Parrot development "over the hump" and acquire a critical mass of infrastructure, developers, and tools from which sustained further development of Perl 6 can take place. The project proposal identified five specific expected outcomes:

  1. A working Perl 6 on Parrot implementation that supports commonly used Perl constructs and operators
  2. Review and improvements to the Perl 6 language test suite
  3. A substantially complete Parrot Compiler Toolkit, with documentation
  4. Ongoing, active development efforts for other languages based on the Parrot Compiler Toolkit
  5. An increased number of participants in Perl 6 and Parrot design, implementation, and testing

1. A working Perl 6 on Parrot implementation

We now have a substantial Perl 6 implementation on Parrot. At the beginning of this project, we had a rudimentary Perl 6 compiler that could handle some of the basic features and syntax of Perl. However, some key features such as arrays and hashes were only marginally implemented, and most of the translation components of the compiler were written in Parrot's intermediate assembly language (PIR).

Today the Perl 6 on Parrot compiler has been substantially rewritten using the Parrot Compiler Toolkit (described below), and is now known as "Rakudo Perl" or "Rakudo" [1]. Rakudo currently supports arrays, hashes, classes, objects, inheritance, roles, enumeration types, subset types, role composition, multimethod dispatch, type checking, basic I/O, named regular expressions, grammars, optional parameters, named parameters, slurpy parameters, closures, smart match, junctions, and many other features expected from Perl 6. Rakudo has progressed to the point that others are now using Rakudo as an Apache handler ("mod_perl6") [2], a wiki engine ("November") [3], and recently to build applications on Xlib [4]. The Perl Foundation also recently awarded a grant to Vadim Konovalov to implement a Tk GUI interface for Rakudo [5].

During the project there have been many contributors to the development of Rakudo Perl; some of the major contributors include chromatic, Vasily Chekalkin (bacek), Jerry Gay, Jeff Horwitz, Moritz Lenz, Carl Mäsak, Cory Spencer, Stephen Weeks (Tene), and Jonathan Worthington. Jonathan Worthington's efforts deserve special mention: he is primarily responsible for the bulk of the work on classes and types in Rakudo Perl, and much of his work was supported by a grant from Vienna.pm [6].

2. Review and improvements to the Perl 6 test suite

At the beginning of this project, the test suite that existed for Perl 6 had been implemented as part of the Pugs effort and contained approximately 16,000 tests. However, in many cases the tests were out of date with respect to the current Perl 6 language specification, and we needed a way for multiple independent Perl 6 implementations to be able to easily make use of the test suite.

In December 2007 I proposed a structure for reorganizing the test suite [7]; Jerry Gay and Larry Wall then extended this proposal and developed tools to make it easier to share the tests among multiple implementations. In May 2008 Moritz Lenz refactored Rakudo's test harness to provide a "make spectest_regression" target -- since then this has become our primary measure of Rakudo progress [8]. Moritz Lenz also developed a tool to display the progress in graphical form:

    

Adrian Kreher received a Google Summer of Code grant (with Moritz Lenz and Jerry Gay as mentors) to continue the test suite refactoring [9]. As of early September 2008, the official, refactored test suite contains a little over 8,000 tests, or approximately half the size of the original Pugs test suite. Furthermore, the official suite includes many new tests for object-oriented and typing features of Perl 6 that weren't present in the original test suite. As the graph above indicates, Rakudo is currently passing over 3,200 of the tests in the official suite, and work is continuing on refactoring the suite and increasing Rakudo's pass rate.

3. A substantially complete Parrot Compiler Toolkit, with documentation

As mentioned previously, the compilers that existed for Parrot at the beginning of the project (such as Perl 6) were primarily written in Parrot's intermediate assembly language. This made working on the compilers less accessible to new developers, as well as increasing the time needed to build a compiler. Therefore, the first couple of months of this project were spent on developing the Parrot Compiler Toolkit (PCT) and a simple Parrot language called Not Quite Perl (NQP). PCT provides an abstract syntax tree representation and code generator for Parrot languages; NQP enables compiler writers to easily create compilers and builtin functions for Parrot using a simplified Perl 6 syntax.

Once PCT and NQP were substantially complete, we were then able to convert Perl 6 (now "Rakudo Perl") and many other Parrot compilers to use the new toolkit. This went surprisingly quickly -- most of the existing compilers were converted within just a couple of weeks. In addition, a few new compilers and languages arrived on the scene: Will Coleda and Simon Cozens quickly created an implementation of LOLCODE [10], and Klaas-Jan Stol created a language called "Squaak" as a demonstration and tutorial for the toolkit [11]. A couple of quotes from the period give a sense of how these tools opened up Parrot development to others:

"The real fun, though, has been digging into perl6, the Parrot Perl 6 implementation. Recently, Patrick Michaud has been doing some incredible work building NQP (Not Quite Perl 6), a bootstrapping language for implementing Perl 6, and extensively refactoring the existing Perl 6 on Parrot compiler to fit with it. I'm still very much getting my head wrapped around the whole thing, but it's been easy enough to start digging into and implementing and fixing a few things."
- Jonathan Worthington, December 2007 [12]
"It's really, really true. Parrot lets you implement your own languages using Perl 6 rules for the grammar and Perl 6 for the compiler."
- Simon Cozens, January 2008 [13]

Both PCT and NQP stabilized early in the project; recent changes have been primarily to optimize existing features or make other minor improvements. We expect these tools to continue to evolve as needed to support compiler development; however, given the wide variety of languages being implemented using the toolkit surprisingly few changes have been needed. Primary documentation for the toolkit consists of a Parrot Design Document (PDD26) for the abstract syntax tree representation [14], the Squaak tutorial [11], and numerous example languages in the Parrot repository. More detailed documentation and examples for the toolkit are expected to be developed over the coming months.

4. Ongoing, active development efforts for other languages based on the Parrot Compiler Toolkit

Currently there is active development on at least three languages for Parrot; these include Perl 6 (Rakudo Perl), PHP (Pipp), and Ruby (Cardinal). In addition, there is ongoing but less active development on implementations of Python (Pynie) and Smalltalk (ChitChat). All of these are based on the tools from the Parrot Compiler Toolkit. The Perl 6 and PHP implementations are usable from mod_parrot [2], and our next steps are to improve the toolkit and library conventions to support loading multiple languages simultaneously in Parrot.

5. An increased number of participants in Perl 6 and Parrot design, implementation, and testing

There can be little question that momentum for Perl 6 and Parrot development continues to grow, and the work supported by this grant has been a major catalyst for that growth. For the twelve months prior to September 2007 the Parrot subversion repository had a total of 6,634 commits; in the subsequent twelve months Parrot had 9,686 commits -- a year-over-year increase of 46% in the commit rate.

At the beginning of this project there were perhaps three or four active contributors to the Perl 6 on Parrot compiler (i.e., Rakudo Perl); over the course of the past year that number has increased to at least ten active contributors and at least as many more occasional contributors to Rakudo Perl. Parrot and the Parrot Compiler Toolkit have also garnered new contributors, including several new committers. There are also new teams of developers working on applications based on Rakudo Perl and Parrot, such as the November wiki engine and the Rakudo/Tk GUI interface.

Lastly, recent improvements in the Rakudo compiler architecture will allow much of the Perl 6 runtime library currently written in PIR to be rewritten substantially in Perl 6. This will enable even more new contributors to participate in Perl 6 development.

PROJECT EVALUATION

The project proposal identified five criteria by which the success of this project could be measured:

  • Ability to write and test Perl 6 programs and language features
  • Passing rate for Perl 6 language test suite
  • Improved coverage and accuracy of the Perl 6 language test suite
  • Increased number of participants in Perl 6 and Parrot development
  • Active development of at least two other languages using Parrot compiler tools

This report demonstrates the success of this project along all of these measures. Perl 6 on Parrot ("Rakudo") is being used to write and test Perl 6 programs and language features, and has become a strong driver for improving the coverage and accuracy of the test suite (and indeed, of the Perl 6 language specification itself). Continual reporting and publication on Perl 6 and Parrot activities continues to attract increased interest and participation from the wider community.

Moreover, this project helped jump-start even larger fund-raising efforts. In May 2008 the Perl Foundation received a $200,000 philanthropic donation from Ian Hague; roughly half of this donation is intended to continue the Perl 6 development efforts that have been part of this project [15]. And, as mentioned earlier, Jonathan Worthington and others are receiving grants for continued work on Rakudo Perl and Parrot [6,8,16].

CONCLUSION AND FUTURE WORK

The funding provided by the Mozilla Foundation and The Perl Foundation has indeed enabled us to get Perl 6 on Parrot development "over the hump" in development. We now have a robust platform for higher-level language development on Parrot, along with an active and growing development and support community (which is the hallmark of any successful open source project). All of the desired outcomes of this project have been realized.

Our next steps will be to continue to extend and build upon the work of this project -- increasing Rakudo's coverage of the Perl 6 language and bringing all of our efforts much closer to production releases. In fact, we recently developed a "road map" for Rakudo Perl that identifies the major steps to be taken in the upcoming months and assists in coordinating the remaining development activities [17].

Lastly, I want to express my sincere appreciation to the many people who contribute time and energy to Perl 6 and Parrot, and to give special thanks to the people of the Mozilla Foundation and The Perl Foundation for their ongoing support and enthusiasm for this project. It is a great honor to work and correspond with such a terrific and professional group of individuals.

REFERENCES

  1. P. Michaud (January 16, 2008), "The compiler formerly known as 'perl6'", http://use.perl.org/~pmichaud/journal/35400
  2. J. Horwitz, "Mod_parrot website", http://www.smashing.org/mod_parrot/
  3. C. Mäsak, "Announcing November, a wiki in Perl 6", http://use.perl.org/~masak/journal/37212
  4. http://svn.perl.org/parrot/trunk/examples/nci/xlibtest.p6
  5. Alberto Sim&otild;es (August 30, 2008), "2008Q3 Grants Results", http://news.perlfoundation.org/2008/08/2008q3_grants_results.html
  6. "Vienna.pm funds Jonathan Worthington to work on Ra[kudo]" (April 23, 2008), http://use.perl.org/article.pl?sid=08/04/23/2314234
  7. P. Michaud (December 20, 2007), "Proposal: refactor the test suite according to synopsis, http://groups.google.com/group/perl.perl6.compiler/msg/ed748490f030da2f
  8. P. Michaud (June 16, 2008), "Rakudo test suite progress", http://use.perl.org/~pmichaud/journal/36695
  9. A. Kreher, "Auzon's Blog: gsoc2008", http://auzon.blogspot.com/search/label/gsoc2008
  10. "I HAZ A PARROT" (January 3, 2008), http://lolcode.com/news/i-haz-a-parrot
  11. K. Stol (March 9, 2008), "PCT Tutorial Episode 1: Introduction", http://www.parrotblog.org/2008/03/targeting-parrot-vm.html
  12. J. Worthington, "Chipping away at perl6", http://www.jnthn.net/cgi-bin/blog_read.pl?id=589
  13. S. Cozens (January 3, 2008), "Parrot is really quite wonderful", http://blog.simon-cozens.org/post/view/132
  14. "Parrot Abstract Syntax Tree (PDD 26)", http://www.parrotcode.org/docs/pdd/pdd26_ast.html
  15. R. Dice (May 16, 2008), "TPF receives large donation in support of Perl 6 development", http://news.perlfoundation.org/2008/05/tpf_receives_large_donation_in.html
  16. J. Worthington (August 5, 2008), "Multiple Dispatch Design Work", http://use.perl.org/~JonathanWorthington/journal/37101
  17. "Perl 6 Development Roadmap", http://svn.perl.org/parrot/trunk/languages/perl6/ROADMAP

A little while ago I wrote about starting work on my Rakudo MMD implementation grant. At that point, I was just working out the algorithm and hadn't started on the implementation. Since then, with some hacking under the funding and some random extra hacking during YAPC::Europe, I have made some progress, which I'll talk about in this post. Before I do, I'd like to thank DeepText, who are providing the grant, for graciously allowing me to continue work under it this month, after (for reasons mentioned in the last post) I didn't start work on it until late in August, which was too close to the original deadline.

At this point, I have succeeded in subclassing the Parrot MultiSub PMC and have implemented the majority of the dispatch algorithm in the PMC, as planned. The dispatch currently takes into account the arity, class and role types ordered with type narrowness in mind and tie-breaking on subset constraints. Left to do is tie-breaking on the "is default" trait, which should be trivial, and then handling proto fallbacks, which will be less trivial but shouldn't be too hard; I'm leaving that for a bit later.

To make sure all of this code essentially works, I have written some tests in PIR that mock up subs with signatures, add subs to the Perl6MultiSub PMC and then invoke it, running the dispatch algorithm. It's quite tiring and verbose to write these tests in PIR, so there will be many more tests to be written once we can write multi subs using the dispatcher in Perl 6. And this is pretty much my next task - to start using this in Rakudo.

For those interested in something more technical, here's a little bit on the guts of what I've just implemented. The first time a call is done on a multi, we take all of the candidates and sort them in order of type narrowness, using the topological sort described last time. The only difference from a normal topological sort is that we care which candidates are equally narrow. It turns out that this is fairly straightforward to keep track of: we take the DAG that we built and remove together all of the things that do not have any incoming arrows and put them in the sorted result list, then stick a NULL value in, and then continue until we have taken all of the nodes from the graph. We then throw an extra NULL on the end, so a double NULL is our sentinel for end of list. We can then at dispatch time cheaply chase through the list, working out which candidates are admissible and knowing to stop if we had one or more admissible candidates of the same narrowness. It turns out that checking for admissibility based on arity is very cheap, but type-based admissibility checks are more costly. I think there will be ways to improve that a bit in the future, however.

So, the roadmap from here is:

  • Clear up some memory leaks in the candidate sorting, while I remember that they're there! (Next day or two.)
  • Get Rakudo properly attaching Perl 6 signature objects to subs with all of the correct information in them that the dispatcher needs. (Within the next week.)
  • Do what is needed in Parrot to ensure that when we generate code with MultiSubs we end up with Perl6MultiSub PMCs rather than the default Parrot one (HLL-Map style things). Once this is done, we're using the new dispatcher. (Within the next two weeks.)
  • Write more tests in Perl 6 to exercise the dispatch algorithm. (Within the next two weeks.)
  • Add the "is default" trait and start using it for disambiguation, plus more tests. (Within the next three weeks.)
  • Get proto and only to do something close to the Right Thing and make the dispatcher fall back to them as required. (Within the next three weeks, I hope.)

The Perl 6 design team met by phone on 30 July 2008. Allison, Jerry, Will, Jesse, and chromatic attended.

Allison:

  • OSCON was a success
  • now I'm completely free from all responsibilities until at least December
  • launched the PIR PDD
  • now working on the last few failing language tests before I merge back the concurrency branch
  • also spent some time talking to PhD advisors at Cambridge this week
  • might start fall 2009, if I can get funding and such
  • leaving for London on Saturday
  • speaking at the BBC on Tuesday
  • may miss next week's meetings

c:

  • was very busy with OSCON
  • trying to apply a few patches
  • trying to get other people to the place where they can help with that
  • mostly I need tasks I can do in 30 minutes when I can grab an hour or two
  • fixed some small stuff at the hackathon

Allison:

  • I need to split up the MMD PDD into bite-sized tasks

Jerry:

  • OSCON was fabulous
  • our talk was successful
  • good feedback and enthusiasm
  • the lighting talks with Jeff were successful
  • most of my hacking was on mod_parrot
  • trying to refactor its configure system into something more robust and portable
  • it's just about working in the same state it was before
  • hope to reach that point this afternoon
  • met with some folks at Microsoft's open source lab on Monday
  • promising beginning to a relationship between them and the Parrot Foundation
  • the hackathon was productive as well

Jesse:

  • I talked to some Sun people about what makes CPAN tick

Will:

  • mostly absent the past couple of weeks
  • trying to resurrect Tcl, which was untouched for some time
  • making progress there on spec tests
  • identifying what's left to do

c:

  • what's our roadmap look like for the rest of the year?

Jesse:

  • I asked Patrick about that
  • he said there's a good chance to get that from the hackathon
  • if not, he said I could get his attention at YAPC and get his milestones to break down soon

Jerry:

  • he spoke on IRC about that today
  • he wants to break it down into better granularity and put dependencies between tasks
  • have a feeling it'll happen at YAPC::EU
  • Jonathan is keen to get together with Allison and Patrick there to discuss related designs

c:

  • I've noticed projects in Parrot like Reini's attempt to get installation and loading sorted out
  • seems like time to get those sorted out

Jerry:

  • it's difficult to have willing volunteers and not get them the right guidance
  • and to make "eventually" now

c:

  • thinking we should ask Reini for a list of concerns, needs, and use cases
  • then try to make a design that addresses more of those

Jerry:

  • especially as he's about to leave on a two-week business trip
  • sounds like a PDD to me

Allison:

  • sounds like several

Jerry:

  • library loading, runtime components

c:

  • does he have commit access?

Jerry:

  • we can encourage him to apply

c:

  • thinking about bindings to Prophet
  • probably client-side

Jerry:

  • we need sockets

Will:

  • we have experimental sockets
  • we need to make tickets or mark up PDDs for unimplemented things
  • I did some of that with PDD 19 today
  • stuff's deprecated and only exists in the PDD
  • just to warn people who are going to write code that docs don't match reality

Jerry:

  • that merits a CAGE task for reviewing the PDDs
  • we could spread that out among many people

Will:

  • I'll send an email

c:

  • and post it on parrotblog

Will:

  • no one reads those

c:

  • they won't if we don't publish them

Jesse:

  • I see them posted places like Reddit fairly often

Allison:

  • probably will roll that into parrot.org eventually

About this Archive

This page is a archive of entries in the Rakudo Perl category from September 2008.

Rakudo Perl: August 2008 is the previous archive.

Rakudo Perl: October 2008 is the next archive.

Find recent content on the main index or look in the archives to find all content.

Subscribe

    Subscribe to rakudo.org

Powered by Movable Type 4.21-en
Technorati Profile