Recently in Rakudo Perl Category

The Perl 6 design team met by phone on 25 February 2009. Larry, Allison, Jerry, Patrick, and chromatic attended.

Patrick:

• continuing to prepare Rakudo for its next release
• expect that to happen today
• for now, the releases will be alphabetical: words beginning with A, B, C, D, etc.
• partially to emphasize that I don't consider these user-ready releases
• putting a number on that may imply that
• no easy way to tie releases to numbers right now anyway
• some people have said "What happens when you get to X?"
• I hope we're releasing official releases by then
• updated Rakudo's test harness
• no longer requires Parrot's test harness
• it now passes over 7000 spec tests
• Jonathan gets most of the credit for that
• he's been closing tickets and fixing bugs while I've been working on build issues
• totally unplanned to hit that number
• updated the spectest progress files to show our progress
• haven't updated the graph today
• following along conversations and trying to keep track of various changes

Allison:

• spent the week mainly on install stuff
• various languages can build from an installed Parrot without depending on the build directory
• still some hard-coded paths within Parrot's configuration system for C compilation
• Rakudo and Partcl can definitely build from an installed Parrot
• need to do Pipp next
• set up a languages repository on svn.parrot.org
• waiting for some admin work on that
• came up with a saner version number strategy

Patrick:

• I find it much easier to say that January and July are the memorable releases than to attach numbers to them

Allison:

• with Ubuntu, it's likely we'll get 1.0 and 2.0 into the distribution and not say 1.4
• they have an upstream packaging freeze about a month before we put out our release
• the release that goes into their 09.10 will be 1.0, not 1.4, because the latter won't be out in time

c:

• why not 1.3?

Allison:

• that's a developer release

c:

• we're going to have to talk about that

Allison:

• we have a final copyright assignment from the Perl Foundation to the Parrot Foundation

Jerry:

• have some small pending changes to S19
• nothing much to report there
• Google's giving a GSoC presentation at the University of Washington
• they've asked me to speak about our experiences last year with TPF
• I'll invite people to join the Perl and Parrot Foundations project this year
• hope to have an ideas page to point people at at that point

c:

• have you spoken to Jonathan Leto about that?

Jerry:

• only heard about it yesterday
• it's on my list
• looking for presentations on GSoC to find slides to borrow
• about all I've had time to do lately

Larry:

• having a really nice problem
• a lot of people are helping to flesh out the specifications
• the nice problem is that they have to be managed
• trying to oversee what people are hacking into the new specs
• want to make sure things don't go too far afield
• weighed in on event processing and inexact numeric matches on the mailing list
• nothing there is quite spec-worthy yet
• doing name-whackage of various sorts
• killed off the $*DEFIN, $*DEFOUT, $*DEFERR symbols which emulated Perl 5's old IO select states
• not it relies on that those are contextual vars: $*IN, $*OUT, $*ERR
• any scope can override them
• that's simplification
• many compiler variables that were redundant and error prone with regard to nested scopes
• you now chase links through your symbol tables to find out your lexical or package nesting
• unhandwaved how a setting can loop around user code
• you can mark it so that it knows how to dump the lexical scope surrounding user code
• there's a stub in there called YOU_ARE_HERE
• I'm open to better names
• clarified associativity at the same precedence level
• that can happen with user-defined operators
• non-associative in that case, so you get an error
• noticed that the tighter and looser precedence generators only produced precedence levels that were close to one that exists
• wanted the ability to define something in the middle
• if you use both in the same precedence level, it splits the difference

c:

• sounds like you need a synonym: between

Larry:

• tried that, but you have to specify two things anyway, and there's already syntax for that
• after the discussion of time semantics, put in two core types: Instance and Duration
• defined to be well-behaved in terms of time
• most of my standards hacking has been in user-oriented error messages
• now an error message that warns you if you try to use a C++ constructor

c:

• can we backport that one?

Larry:

• it's also a valid Perl 5 constructor
• just thought it would be more insulting to call it a C++-style constructor :-)

c:

• I still want it

Larry:

• previous patch to catch Haskell-style open-ended ranges where you leave out one of the terms
• that prevented talking about any .. operator in square brackets
• we couldn't use the notation where you put square brackets around infix operators
• I had to fix that
• fixed up the warning when interpreting if() as a function call
• introduced enum names into the symbol table
• it's a bit of a hack
• anonymous enums now parsed correctly
• no more complaints of missing package when you use CALLER:: or CONTEXT::
• can now introduce term symbols, as well as prefix, infix, and postfix
• other random warning-suppresion bugs

c:

• working on project management, version numbers, deprecation, triaging guidelines
• trying to fix as many bugs as possible
• reviewing SKIP and TODO tests

Jerry:

• I read your Setting work this week
• can you roll back a Setting?
• can you access something outside the current Setting?
• I don't have a use case
• is it as simple as calling OUTER::Setting?

Larry:

• if there are multiple settings, they're just enclosing lexical scopes
• any mechanism to navigate lexical scopes works
• the outermost is CORE:: and so forth

Jerry:

• to cut off a file in mid-stream processed with -n, could I close the handle in CORE::?

Larry:

• chances are that's a dynamic variable, a $*ARGV equivalent

Jerry:

• can I find out which Setting I'm currently in?
• finding out its name
• inside my current runloop, can I tell which Settings I have loaded?

Larry:

• we probably want to have a way of naming them, much as CORE:: has a name
• lexical scope should have a way of saying "My lexical scope has a name, and it is..."

c:

• O-S-C-A-R?

Larry:

• ... but I haven't thought about that yet

Jerry:

• any status on protoregexes and stuff?

Patrick:

• I end up being the person of last resort to do things that need doing
• had hoped that other people would pick up the ball on build issues and stuff
• I can make predictions for the next few months, but life catches me off guard
• once I get back to working on PGE, it won't take too long
• it would bug me more if the lack of those features were holding Rakudo back
• that doesn't appear to be the case
• we're making good progress and we're getting new features
• we could use them for adverbs, but we can work around that now
• if that reaches a pain threshold, it'll move it higher in the queue
• that's where I'm focusing my efforts

Jerry:

• Jonathan's very good at moving things forward

Patrick:

• it's fortunate that each of us tends to like doing what the other person hates
• with the exception of lexicals and the build system

Allison:

• no one ever wants to do a build system

Patrick:

• delayed binding helps this though
• seeing grammar changes as they come up makes it easier to figure out what to do
• with the Setting code now, and features written in Perl 6, we're closer to finishing one of my milestones

Jerry:

• how will the install process be for Rakudo?
• it seems like Parrot is doing the heavy lifting there

Allison:

• I've been working on independent building
• what would you like for installing?
• an install script where you identify your language files, and it puts them there?
• you have a languages/ directory in the Parrot lib directory
• you have configuration files that tell you where they are
• you just copy files after that

Jerry:

• certain things might need changing
• grouping the docs?
• cross-indexing?
• if you're doing dynops or dynpmcs?

Allison:

• haven't been doing any doc installs, just executable files
• maybe they should go in with the Parrot docs
• installation works via a MANIFEST
• you specify the type of file with a tag in the MANIFEST, whether doc or lib
• are you happy to have a MANIFEST in your language directory
• let the installer take care of it for you?

Patrick:

• if there's a script to do the install, we'll probably use that
• I just want to get it so that people can build and run Rakudo
• I'll probably look at installation in a month or two

Allison:

• I'll look at a simple MANIFEST for Rakudo
• will patch your Makefile for that
• will build a basic installer script
• it'll be a basic, working install

Jerry:

• the mk_languages_shell.pl script needs an update to make a MANIFEST
• François has been keeping that up to date

Allison:

• I haven't added MANIFEST to any other languages yet

The Perl 6 design team met by phone on 18 February 2009. Larry, Patrick, Allison, Jerry, Nicholas, and chromatic attended.

Larry:

• fixed STD so that if you added A::B, it added A as a subpackage
• so as not to complain about a missing package declaration
• can now have $! as a parameter
• mostly working over a lot of the error messages to be friendly to Perl 5 programmers
• if you use do/while or do/until it now complains
• if you use if or one of the keywords as if it's a function, it tells you what the problem is without saying "I don't recognize that function name"
• people coming from a Haskell background think they can write 1.. to get an indefinite range
• it now tells them what they should write instead
• fixed a bug which reported runaway strings which start and stop on the same line
• not really a runaway
• now it reports that only if the string crosses a newline
• in the spec space
• to accompany the ability to use a bare sigil in declarations as an anonymous name, now you can use a bare :: to signify an anonymous package name or type
• allows us to have a package named is without ambiguity
• blew away the Main package; combined it with the GLOBAL package
• everything in the main program comes up starting in the GLOBAL namespace

Patrick:

• oh, I'm happy!

Larry:

• thought the distinction served no useful purpose
• continued doing spec work combining context variables with globals
• at least in terms of twigils
• realized that filehandles belong to the PROCESS namespace, not the GLOBAL namespace
• continuing on the vein of .Whatever on most normal operators builds a closure of one argument
• made *.method do the same thing
• even in places which are not syntactically special, we can use *.prime in a grep for example
• equivalent to putting .prime in the curlies
• now we have a fairly general mechanism of writing closures of a single argument
• actually reads pretty well if you don't want the curlies
• we could go as far as to undo the special syntax on whens and ~~ so you have to say *.foo to call a method
• still thinking about that
• would regularize the syntax slightly

Patrick:

• had a little emergency over the weekend, but things are fine
• improving Rakudo's build process to make it easier to build
• someone who wants to build it can get a copy of the repo and pass a special option to Rakudo's Configure.pl
• that'll download a copy of Parrot from the right revision and build it for you
• people who want to play with Rakudo don't have to play with Parrot dependencies
• Jonathan and I have most of the guts ready to start writing Setting code
• we can start to write methods in Perl 6 instead of in PIR
• we'll gradually migrate methods which make sense to rewrite in Perl 6
• moving those over into the Setting code
• that'll make it easier for people to hack on
• some things will remain in PIR, but we don't know what those are yet
• most of my plans for this week is doing more cleanups in the build process
• rewriting Rakudo's t/harness to remove dependencies on Parrot files
• have that mostly done
• writing some articles discussing Rakudo's new home
• how to get it, how to build it, and updating websites

Allison:

• working on Parrot's install process
• making it so that Patrick doesn't have to build and install a whole copy of Parrot to build Rakudo

Patrick:

• we'll need that for some time

Allison:

• trying to make it so that people who build and install packages have an easier time
• you won't have to depend on a Parrot build directory
• you can run against an installed Parrot

Patrick:

• I would like to get rid of the build tree dependency

Allison:

• the patch I have gets rid of most of the build tree dependencies
• there are a few header files in weird locations
• they don't get included in PMCs and dynops
• I've fixed all of the build tools
• all that's left is C-level stuff
• should probably send you to the patch to experiment with
• lots of stuff preparing for the release
• seems to be encouraging that we've been getting more novice questions
• seems to be an influx of interest here

c:

• talking to Richard Blackwell about YAPC
• Parrot and Perl 6 hackathon needs
• working on TODO/SKIP test review for Parrot
• also pondering bug triaging guidelines for Parrot
• having some discussions about release policies and deprecations (though mostly Perl 5)
• setting expectations early seems to help a lot

Nicholas:

• what projects did you look at, or did you look at things afresh?

c:

• Subversion's really early milestone release process was a real inspiration
• the Linux kernel's backwards compatibility policy was also
• not much besides that

Jerry:

• Ubuntu's long term support is pretty nice

c:

• I'm a little less thrilled there
• even six months time is a long span between releases

Allison:

• that's kind of a corporate support guideline

Patrick:

• I like knowing that Hardy will be supported for a few years

c:

• that's a risk for them
• how long are they going to support KDE 3 when upstream doesn't?

Nicholas:

• RHEL and other enterprise distributions support Perl 5.8 until 2011 or so
• but we haven't heard anything from downstream there
• and we only support critical security fixes there

c:

• I put in expicit language about vendors promising long-term security fixes
• that's your vendor's problem, not the problem of volunteers

Nicholas:

• in Perl 5, you used to do package; with no namespace
• removed somewhere in 5.8.x
• was such a thing considered for Perl 6?
• you basically banned all unqualified variables

Larry:

• it was a hack in Perl 5
• I want to make new mistakes

Patrick:

• I expect to do Rakudo's next release next Wednesday
• that's a target anyway
• I'll be out of town this weekend
• we'll start its regular release clock with that release
• we'll do monthly releases
• that exact date will vary by a few days here or there until we get the release process down

Jerry:

• will you have a similar release structure as Parrot
• multiple release managers?

Patrick:

• yes
• I don't want to be the single release manager
• I'll probably do the first couple
• I want at least one other person to do some by May or June
• ideally a team
• the Rakudo release needs to happen very quickly after the Parrot release
• we won't be able to do this this month
• what do we need to think about for POD in Perl 6?
• what we have now is "DRAFT DRAFT DRAFT"

Allison:

• can you say "We'll target this version and update if necessary?"

Patrick:

• that's exactly what I've done so far

Larry:

• I agree

Patrick:

• I expect to generate or write a fair bit of POD

Larry:

• it's amenable to mechanical translation
• someone should write a standard parser for POD

Allison:

• Parrot might as well stick with the standard

c:

• are you suggesting a Parrot-based POD6 parser?

Allison:

• if you develop a POD6 parser in Perl 6, could we compile it back down to PIR and use it?

Patrick:

• we'd more likely target NQP
• Rakudo expects the Perl 6 runtime available
• it'd be nice to say that that's just a PBC, but it's also PMCs and dynops

Jerry:

• makes me wonder how small we can make a Parrot distribution that only contains NQP as a language

c:

• that's the Parrot 2 proposal

Jerry:

• I meant ripping out all ops and PMCs you don't need
• maybe no IMCC

c:

• it's not PBC compatible though

Allison:

• he's talking about a version of Parrot built on as few basic opcodes as possible

c:

• no PMCs or ops written in C

Nicholas:

• I thought the computer scientists proved that you only need one operation to make things work, sort of a compare and jump

Jerry:

• then you have Lisp

c:

• but you need infinite storage space

Jerry:

• disk space is cheap, and getting cheaper

c:

• but not infinite
• you can climb that asymptote all you want, but it keeps getting steeper

Patrick:

• the ultimate slippery slope argument

Today was Rakudo day, and I used it to get a bunch of fixes in place as well as applying some patches from other folks. Before digging in to the details, I'm happy to point out that we're now passing over 7,000 spectests - in fact, we got past that last night, before today's round of fixes, so we're well over it now.

• Fixed a bug involving the === operator and proto-objects; specifically, if you had an attribute "has Foo $!x" and in a method - without ever having changed $!x - did "$!x === Foo", it wrongly gave False.
• Whatever wrongly inherited from Object rather than Any, as S02 says it should. Changed this, which allowed me to resolve a ticket.
• Was happy to find an enums related bug report that had already been fixed - this time, by the type registry coming into existence last month. Added a couple of tests to make sure the bug didn't come back and closed the ticket.
• Got bare sigils in signatures working. This turned out to be quite easy, resolved an RT ticket and made ten or so tests that were skipped in assign.t pass too. So you can now do things like:
  my ($, $, $, $four) = 1..4;
my ($one, $two, *@) = 1..4;
To only assign some things. Note that *@ is just the slurpy array * - it's a signature (and in fact you would now be able to write *@ in a signature on a sub too).
• masak discovered that:
  say "a".."c" Z "?", "a".."b"
Gave the wrong output and reported it. bacek sent in a patch. Mortiz wrote a test. Which just left me to review and apply the patch and untodo the test. Teamwork for the win.
• Calling .parse on a grammar that is in a namespace did not work. I fixed it and added a test.
• Our multi-method dispatch has never been quite right, and has long been on my fix list. Now it's at least a lot more correct, in that if it finds no applicable candidates in the current class it now goes looking up the MRO.
• While I was in the multi-dispatcher, I realized that every time we got a cache miss, we leaked a little memory. Fixed the leak. Was glad that Rakudo only has a little bit of C...I'm too stupid to get manual memory management right.
• Did some tweaks to the way we construct subsets to make them act a bit more type-ish, and make sure that the type object works like a proto-object (that is, it's undefined). May need some more tweaks, but it fixes what .defined does and passes all the current spectests for this.
• Applied some documentation patches from leto and Chris Dolan. Easy work thanks to the github fork queue.

Additionally, I tracked down where the perl6 executable often crashes with a double free or got into an infinite loop on exit (it was a problem that only showed when using the executable, and not when using the bytecode file on Parrot - not because the problem wasn't there, but because Parrot didn't bother to reclaim the memory on exit so it never ran the cleanup code that exploded). I wrote a patch which sucked, but prevented the double frees. I showed it on #parrot, at which point pmichaud - now knowing where in Parrot the problem lay - churned out a much smaller patch that was more correct and more efficient. It's in, we bumped up the recommended Parrot version for Rakudo and I closed some tickets about segfaults.

So, thanks to Vienna.pm for sponsoring Rakudo Day this week. There won't be one next week, since I'll be taking some vacation. This weekend is the Belgian Perl Workshop, and the weekend after is the Ukrainian Perl Workshop, so I'm going to spend a bit of time in each of those countries between them, relaxing and enjoying the sights, the beer and the food. And of course, very much looking forward to seeing folks and talking about Perl 6 and Rakudo at the workshops too!

The Perl 6 design team met by phone on 11 February 2009. Larry, Jerry, Patrick, Nicholas, and chromatic attended.

Larry:

• hacking symbol table support into the STD parser
• so that I can tell when things have and haven't been defined
• largely complete
• started moving the lexical pads out to a separate file
• they have more information in them
• the Setting file, formerly the Prelude, is now parameterized so that other Settings can be set for the lexical settings
• lots of work on various error messages associated with declaration, non-declaration, and redeclaration errors
• just checked in a unification of the + and * twigils
• threatened that for a long time
• prototyped that in the STD parser
• it seemed to work
• hacked that in
• now we only have the * twigil
• it essentially does the contextual variables
• it looks in global and process symbol tables if it doesn't find them locally
• it looks in the environment only after failing to find it in global and process
• it's very easy to hide that, or substitute a vetted environment table somewhere in your dynamic scope
• cleaned up the definition of how environment is passed on to the subprocess
• dealing with the twigil changes in the Spec
• cleaning up the pseudopackage names somewhat
• regularizing the setting and file scope and current compilation unit and their relationships

Patrick:

• went to Frozen Perl this weekend
• gave one major presentation on Perl 6
• a couple of lightning talks on PCT
• had a hackathon on Sunday
• that went well
• there's a lot of enthusiasm for Rakudo and Perl 6
• Andy Lester focused on that for his keynote
• lots of people talking about it outside of that
• lots of people starting to look at Perl 6 again
• "We're getting to an implementation"
• "These features are nice"
• "I'm looking forward to using this in my business"
• trying to update the instructions to download and build Rakudo from its new repository
• coming along more slowly than I'd like
• noticed at the hackathon that invoking Rakudo with the Parrot command line — parrot perl6.pbc — is confusing for folks
• Parrot's not in a common location they can get to it from
• we'll try to focus more on the executable form
• people can understand that more
• noticed that pbc_to_exe took an inordinately long time to do its work
• hacked on that, and the new version is a lot faster
• learned a lot about IO in Parrot
• tweaking the build scripts for Rakudo
• plan to do more of the same
• will write up more documentation, READMEs, guides, blog posts, etc

Jerry:

• interested in the work on the Setting
• might work with one of the S19 options I've toyed with
• -E to include an environment variable
• can be generalized into a one-liner to add a setting
• had some minor updates to S19
• don't have the split/comb update quite right yet
• been fighting with Parrot's Windows compatibility through changes in the config system
• discovering some things this week about static and shared library linking
• working toward a solution there
• started creating an Ubuntu VM for distribution to interested hackers
• has all sorts of Perl 6-related projects
• Pugs, Rakudo, Parrot, November, Apache, mod_parrot
• VMWare image that should fit on a 16 GB thumb drive
• will make that online
• should make it easier for people who want to hack on Rakudo to do so
• provided we can set up tools to manage and update these distributions

c:

• closed some bugs
• checked in the support, deprecation, and release policy
• brainstorming ideas on how to make a VM with less C code

Nicholas:

• does a VM really take 16 GB?

Jerry:

• I don't think it will
• that's how big I have it now
• it can probably be smaller
• I'm not familiar enough with a minimal install of Ubuntu with enough space for GCC and the build tools

Nicholas:

• it wouldn't fit on a CD
• it might fit on a DVD
• assumed that burning a DVD is a more disposable way of giving it away
• they're effectively free

Jerry:

• I'll consider that
• hadn't thought terribly about distribution
• I can resize it smaller at any time

Patrick:

• just did an Ubuntu install yesterday
• GCC, Git, Subversion... no Parrot yet
• it's a fresh install
• looks like 3.5 GB
• I can generally work in 6 GB
• that'd fit on a dual-layer DVD

c:

• maybe look at a live DVD
• add to that

Patrick:

• Ubuntu lets you boot off a USB drive
• you keep your storage with you

Jerry:

• want to give Jesse credit for Prophet and SD, which syncs with RT, Hiveminder, and Trac
• offline access to bug queues and TODO lists
• it's nice to travel to a venue and then sync so everyone sees how productive you are when you're not connected
• going to put that in the Perl 6 dev VM

Patrick:

• I want that!
• especially with all of the traveling I'm doing

Rakudo day today was mostly about getting going with a Perl 6 setting (what other languages tend to call a prelude) for Rakudo. This allows us to write built-ins in Perl 6 rather than in PIR, which will speed development and, we hope, encourage more contributions. It is also a needed step on the way to other things.

After Patrick told me The Plan that he'd worked out for this, I dug in to doing the grunt work. The first task was to split the compilation into two steps: first, compile the core of Rakudo, then use that to compile the Perl 6 prelude, then bundle the two together to get the final Perl 6 compiler. It turned out that this step was fairly easy. We'd done plenty of work leading up to this to provide features that would make it possible, so within a couple of hours I had the changes to make this work comitted. The first two Perl 6 methods in the prelude were .perl and .ACCEPTS on the class Whatever - easy, for sure, but enough to prove the concept.

I thought things were looking good - but then it all came to a screeching halt when I discovered that because of the way we were building things, we were ending up with duplicates of some things that were supposed to be unique per compilation unit. Happily, Patrick jumped on the PCT changes required to fix this one up while I had dinner and did some shopping, and when I was back from all of that I was able to continue porting a few more things over to Perl 6. It's a start, but of course there's a long way to go yet. Help is very welcome - I really do hope this will help more people to get involved with Rakudo, since now you don't have to learn PIR to write built-ins!

As well as working on the setting, I dealt with a few RT tickets, to pull our queue back under the 250 mark that it had sneaked above again.

• Recently masak pointed out that you couldn't create an anonymous class that inherited from another class, or had traits and so forth. The syntax "my $x = class is Foo { }" gave a parse error, since the 'is' was taken as the name and then it didn't know what to do with the Foo. Larry tweaked STD.pm and declared that the way to do this is "my $x = class :: is Foo { }" - the :: token indicating anonymity. Today I put the matching change into Rakudo's grammar, did the work to support it and added a couple of tests.
• Getting good test coverage isn't always so easy. There is some syntax for initializing the parent attributes of a class when calling new ("Child.new( Parent{ x => 42 })"), which worked. Well, in that simple case - but as soon as you got two different parents in there, it wouldn't work, or you'd end up putting the wrong values in the wrong places. A little effort later, I found the logic bug in our BUILDALL implementation and corrected it.
• As well as "class Foo does Bar { }" you should be able to say "class Foo { does Bar }". Now you can.
• Did some tweaks to get Rakudo building better on Win32, including the perl6 executable, which didn't really work on Win32 since we moved out to GIT.
• Applied a patch from bacek++ to get "min= and "max=" working ("$foo min= 100" will assign 100 to $foo if it's smaller than what is in $foo).
• Applied another bacek patch to fix the aossiciativity of the ** infix operator - it's right associative.

Thanks to Vienna.pm for funding my work on Rakudo today.

The Perl 6 design team met by phone on 04 February 2009. Larry, Jerry, Will, Patrick, Nicholas, and chromatic attended.

Allison:

• successful migration of the Subversion repository to svn.parrot.org
• took up chromatic's ticket challenge and closed a handful a day, applying patches, fixing bugs, clearing out some old TODOs that don't fit with current architecture
• started, completed, and merged in the second string refactor branch, a large-scale function name cleanup
• a few fixes on the POD parser, mostly handed it off to kj
• updated the Parrot Ubuntu packages on the PPA with instructions for using it
• speaking about Parrot at IBM tomorrow

Jerry:

• enjoyed the lively discussion with mtnviewmark, timtoady, and others about metaops
• really like where that ended up
• Jonathan should be making some rakudo commits shortly to clean up the cross metaop spectest failures
• thanks to Larry's comments, i'm polishing S19
• haven't had much uninterrupted time lately, so progress has been slower than I like
• keep thinking that's going to clear up soon... but that hasn't been
the case for weeks now

Larry:

• completely revamped STD.pm's metatoken parsing
• treating generated metaoperators as longest tokens did not scale
• chasing the implications of that through
• LTM occurs on the metaoperator itself separately from the base operator
• (most of the base operators are infixes)
• what it modifies gets parses as a separate token
• has implications on the order you check things
• != isn't a separate token
• has to be parsed with the combination of the ! metaoperator
• cuts the number of tokens way down
• the lexer runs a fair bit faster
• invented a new notation for disambiguating infix operators
• they may now all be put in [], the same form as reduce operators
• internal to a metasequence (need a new name for that) sometimes need to identify which operator to apply in which order
• &[] now refers to the infix function itself
• makes it easy to pass binary operators into various functional programming primitives
• reduce operator still stays out in front with square brackets
• just generalized the notation
• made more comments on S19
• moving away from the notion of a Prelude
• provisionally now called a Setting
• it implies things before and after the text of a program
• the -n and -p loops put a scope around the scope of your file
• invented several new pseudo-package name prefixes to refer to the Setting
• PERL is the outermost setting as well as LANG for the sublanguage the file is actually parsed in (such as from -n or -p)
• just changed the Prelude to the Setting option in S19
• chopped out the old, non-trie lexer since we're not using it
• thought it might clean up the looks of the LTM for anyone interested in implementing his own
• not that mine is a parallel matcher yet....
• still something I'd like to do
• maybe I won't have to
• working with Mark on revisions to the new metaops (old minus metaops from last week)
• many binary operators could use a metaop which reverses their arguments
• would have the same effect on comparison operators as reversing the sense of the test
• mutated into a R operator
• and the X operator lost its second X to work the same
• started a trend: adding more metaoperators with a prefix capital letter is a good approach
• might end up with Z for a zip with operation
• working on moving some of the preludey/settings stuff out of STD.pm into a separate file
- working over the various symbol tables to do lexical scoping more sanely - this is prep work for parsing a real Setting file and dumping out the symbols such that they can be slurped in for the user's file compilation

Will:

• minor station keeping on Parrot
• no major progress on cleaning up deprecated stuff
• hope to release a copy of This Week in Parrot
• trying to resurrect that
• lots of questions in IRC about why things are happening
• we don't have a good way to answer that
• will try to post short articles on front page of parrot.org
• hope to do so every Sunday until at least 1.0
• then hope to hand it off to someone else

Patrick:

• moved Rakudo out of the Parrot repository
• its official location is github
• the old Rakudo stuff is still in the Parrot repository
• it'll be gone tomorrow
• decided on Git because there weren't many pros for sticking with Subversion
• getting everyone up to speed on using Git isn't super easy
• haven't run into any major blockers yet either
• usually just reorienting ourself to different commands
• hacked up a new Configure.pl for Rakudo out of the Parrot tree
• making it work with an installed Parrot
• Parrot isn't quite there yet
• if someone doesn't have Parrot but downloads Rakudo, what's the best way to get them an appropriate revision of Parrot to run on?
• looking at options for that
• very pleased to see Larry's changes to metaops and other pieces
• pleased that Prelude is now Setting
• equally pleased that I haven't worked on any of these pieces yet....
• hooray for delayed binding

c:

• Alan Kay is smiling somewhere

Patrick:

• late binding wins again!
• laziness is paying off
• Rakudo is close to having its own Setting written in Perl 6
• part of the repository
• probably won't happen by Frozen Perl this weekend
• have some documentation to write for that
• should have it by the next Rakudo release, sometime in February

c:

• finishing the draft Parrot support policy
• pretty aggressive, but what we discussed at PDS
• will be in the repository today

Nicholas:

• what's the schedule on Rakudo releases?

Patrick:

• I expect that they'll occur timed with Parrot releases
• but not simultaneous
• Parrot releases on the third Tuesday
• Rakudo will release the weekend after that
• this'll give us time to tie it to that specific release
• we'll continue the monthly release cycle
• for most of 2009, I expect many people won't want to play with the released version of Rakudo
• they'll want to play with the head version
• all of the cool features and bugfixes
• it'll track Parrot, but not necessarily Parrot's head
• it'll make sure you have a sufficient version of Parrot, but not necessarily the latest one
• January has shown that when Rakudo and Parrot are separated, changes to Parrot trunk can easily break Rakudo
• we update a file in the repository now
• we'll update that whenever something happens in Parrot that we need to update it with
• that version will always include at least a Parrot monthly release

Nicholas:

• Parrot's starting to get to the point of Perl 5
• changes can break CPAN modules

c:

• we need to be more aggressive about adding those tests to the core test suite

Since I had no time for Rakudo day last week, this week I'm doing two of them, in order to get caught up a bit! So, here's what I got done today. Some of the tickets I dealt with today weren't just a clear-cut case of "this is wrong", but often needed careful checking against the spec, or clarifications to it; I even rejected one bug report, since it expected something to work that went against the spec (plus there was a failing test for it, which I now removed).

Anyway, here's a list of the things that got fixed.

• Fixed the 'is copy' trait on parameters when applied to array and hash parameters; now it does the Right Thing on them. Wrote some tests.
• Fixed a couple of Null PMC accesses that could show up when you had empty blocks (which took some subtle placement, given that if you get it wrong hash composers break...) Anyway, "my $x = -> {}; my $y = $x()" and "my $x = do given 5 {}" will just give you an undef in $x now, not a Null PMC access.
• Fixed :x($n) in .subst - both the string and regex variants - to handle the case where $n is greater than the number of matches that can be done (it should hand back the original string; before it would do as many as it could and then hand back the string).
• Fixed a couple of different bugs when using the Whatever (*) when doing smart-matches with arrays, plus added them to the test suite.
• where clauses on parameters in routines would not allow you to just write something to smart-match against, but instead required a block. This is now fixed, so you can write "sub foo($bar where /baz/) { }" and such things.
• Subtypes (anonymous and otherwise) did not enforce that parameters to blocks were read-only by default. This is now fixed and tested.
• Implemented the prompt built-in.
• When you use a role as a class, it should "pun" a class that does that role and nothing else. This is still something we're working on, but today I fixed a missing case: you can now pun a role into a class and use the punned class to inherit from.
• Implemented the special smart-match form for a method call on the RHS, which calls the method on the RHS on the thingy on the LHS and gives a boolean result. This is nice for doing predicate tests on an object using given/when.
  class A {
    has $.x;
    method nothing { $.x == 0 }
    method large { $.x > 100 }
};
for 0,100,200 -> $x {
    say "Considering $x";
    given A.new(:$x) {
        when .nothing { say "nothing" }
        when .large { say "large" }
    }
}
Output:
  Considering 0
nothing
Considering 100
Considering 200
large

I have also reviewed quite a lot of tickets, and closed quite a few that related to issue that had now been fixed (either as a side-effect of fixing other tickets, or thanks to ongoing development). Between the fixes on this Rakudo Day and the one earlier in the week (along of course with the input and help of others) we have gone from over 270 tickets on Wednesday morning to 245 tickets now. So while we've some way from the more manageable size of queue that I'd like - one that I can review fairly quickly and mostly keep in my head - we're some steps closer. :-) Thanks very much to Vienna.pm for funding this work.

So, back to the regular-ish Rakudo days! The RT queue for Rakudo has swelled enormously, so I'm planning to use my Rakudo days in the near future to focus on resolving tickets there rather than doing any cool new features. While I doubt this will make as exciting reading, Rakudo will be less buggy as a result, which I hope will make up for it by giving everyone a nicer product to play with.

The first bug I fixed today was one related to proto-object definedness. They are meant to be undefined, something which I accidentally broke in dispatcher changes a while back and that, oddly, was untested. So I fixed that and added tests to make sure we don't regress on it again.

Next up was a more subtle one.

class A {
    has $.b;
};
while shift [A.new( :b(0) )] -> $a {
    say $a.b;
    my $x = $a.clone( :b($a.b + 1) );
    say $a.b;
    last;
}


Clearly, the output should have been two zeroes, but it was giving zero and one - the original object was being modified. The cause was that we were getting some (internal) reference chains, and clone was only dereferencing one level, so we didn't properly clone the object after all. I ripped out the custom dereference code and put in a call to !DEREF, which does the Right Thing. So now clone is shorter and also fixed, at least in this aspect. I added something very much like the above as a test case.

In fact, we have some wider issues with clone in Rakudo. We can directly call Parrot's clone v-table method on things, but since it doesn't know about ObjectRef chains, that doesn't always work out so well. This example:

sub foo($a) { say $a .. 5 }
foo(5)


Gave a warning and no output before I fixed it; the fix was making postfix:<++> (and I did postfix:<--> while I was at it) call .clone() rather than directly using Parrot's clone v-table. I found that this fix also fixed one of our integration tests.

I then moved onto some work to incorporate various bits of STD.pm error checking to make things that should die at parse time do so. Therefore, the following, all of which actually parsed and did weird things at runtime, no longer do so:

my DoesNotExist $x;
regex Foo {}; # Empty regex not allowed
regex Foo = { bar }; # Should barf on the = sign, now we do


Dealing with these allowed to to resolve a few RT tickets.

I then moved onto a couple of tickets about grammars. One of them reported a hang instead of an error message if you invoked .parse on a grammar that had no TOP rule. I corrected that, and also implemented .parsefile, which slurps the passed filename and invokes the TOP rule of the grammar on that. Since pmichaud++ had already done .parse, that was pretty trivial. I added some tests for both methods, including one covering the hang, to the spectests too.

We've been getting readonly variables right for a while mostly, but we still didn't catch modifications through ++ and --. I put in a fix for that, and then realized that some tests for <-> (the lambda that makes things rw by default) were now failing, since they had worked "by accident" before rather than through <-> being implemented. So I implemented that also, and - after pmichaud++ pointed out a corner case - tweaked it to handle those and added more tests.

There were a couple of bug reports relating to optional parameters which had type constraints. When the parameter was not passed, they failed the type check. One was about positional parameters, another about named. Happily, there was a single fix that dealt with both cases. I unfudged a few existing tests for the positional optional typed case, and wrote some for the named case since I didn't find any existing spectests for that.

We've been adding lots of compile-time detection of bad stuff of late. One that was missing was checking that if you "is also" a class that doesn't exist, it should complain. So I fixed that and added a test.

Throughout the day, I've also closed up a few other tickets that were already dealt with, but hadn't been closed. So, a busy day. Thanks go to Vienna.pm for funding all of this. Happy hacking, and keep those bug reports coming in! :-)

Sorry for not writing for so long. This time I've been burying myself not just in code, but also in conference preparation, travel and attendance. Anyway, I'm going to try and catch up a little bit here - more detailed posts about individual features to follow.

First, I'd like to thank Vienna.pm. After funding a great deal of my Rakudo hacking last year, they have kindly agreed to continue the arrangement into this year, which is funding me a day a week. I expect that due to attending workshops there will be some weeks I miss and others that I will do two days, but it should come out to a day a week on average. So, a big thanks to Vienna.pm! I plan to use some of the initial days to go through RT and try to cut down the number of tickets there, most of which are bug reports. It will be a nice complement to my Hague Grant, which is mostly focused on feature development (though implementing those has of course been closing existing tickets, and I've been doing other bits on RT tickets too - but having time I can dedicate to fixes and improving the quality of Rakudo in a wider sense is great).

I am writing this from Sophia in Bulgaria, where I've been attending the first Bulgarian Perl Workshop. It's been a wonderful time for many reasons - not least because of the hospitality of the Bulgarians I have met during my time here. Marian has been especially wonderful, taking me out the day after the conference to see the beautiful mountains and a stunning monastery. I enjoyed speaking too - the room was full and there were lots of good questions, which always makes it much more fun and helps me feel that people are getting it. So, to all who organized the conference and to all those I met and talked with - thanks for making this a great time.

My Hague Grant is going well. In the world of parametric roles, I've done a range of enhancements to make sure you can use type parameters as types in the places you'd expect, and written more tests, plus have done some minor fixes that were needed to let us have roles parameterized on parameterized roles, so you will be able to build nested typed data structures. I also implemented the "of" keyword, and made roles pun to classes if you tried to instantiate them. This actually puts me well ahead of schedule so far as parametric roles go. My remaining work for during February mostly consists of using it to make typed arrays and hashes work. I plan to implement these directly in Perl 6 so far as I can, with some bits of inline PIR here and there. They won't be usable until we have the Perl 6 prelude, but that is high on pmichaud++'s schedule to work on too, so I hope we'll have that before February is out.

The other big-ticket item I've now done is junction auto-threading. For now, we still have the custom code in place to thread operators, and auto-threading for built-in functions and methods does not yet work, but again once we get the Perl 6 prelude in place they will also Just Work based upon what I've already done and the custom code can go away. I've done various tests to make sure auto-threading with multis works out nicely, that we auto-thread over the invocant during method calls (to methods not defined in Junction). I will write a more detailed post on junction auto-threading shortly.

I've done a few other bits, too. The handles trait verb will now accept pairs, a class or role name, a regex or the Whatever *. (In fact, the rule is that it's just a smartmatch with the method name as the topic, so regex and * fall out of the same code, and I'm sure folks will come up with other delightful things to write there). I am missing the s/// variant, since we don't parse that yet in Rakudo and I'm not quite sure yet how to do it. But otherwise, that's the handles work about done too, I think.

I also did some improvements to let us write nested classes, though there's some way to go on that. Further, declaring a proto makes all subs of the same name not marked multi be multis, and this works for upgrading methods to multi-methods too when composing roles if there is a proto in the class (in fact, composing roles that had multis in them before didn't work properly, since a Parrot addition was needed; I did that as part of this work, and added Parrot tests for it as well as then doing the Rakudo bits on top). Finally, I've added redeclaration of types detection at compile time, plus a better error if you try do inherit from a non-existent class or do a non-existent role.

So, quite a list of things that have got done, which is nice. :-) And lightens the load a bit for me in February, so I'm optimistic that I am going to be able to complete my Hague grant on schedule. Well, hopefully I didn't just jinx it...

The Perl 6 design team met by phone on 28 January 2009. Larry, Allison, Patrick, Will, Jerry, and chromatic attended.

Larry:

• still working through various operator questions
• thinking about a better model for preventing unfortunate backtrackings
• current commit model is too simple-minded
• otherwise thinking about various questions people are asking me
• they're on the backburner, mulling in my subconscious
• heading up to Seattle this next week

Allison:

• cleaned up library loading
• generated Debian packages
• trying to get Parrot 1.0 into Debian
• working on documentation, especially make html
• decided to try a POD parser in Parrot
• in PGE, a full POD parser takes about four hours
• wrote the PCT AST nodes, but didn't write the action rules
• seems like kjs took a stab at those
• merged in half of the STRING refactors
• resolved a handful of tickets last night

Will:

• finished the release manager document through May
• ripping out a lot of deprecated items again
• seems like our list gets longer every release
• we really need to make a push to rip it out in February
• working on Tcl, on and off
• all of the details are on the partcl blog

Patrick:

• mostly making plans to move Rakudo to its new repository
• should have that all decided in the next few hours
• will make that happen, so that Parrot will make its move tomorrow with no trouble
• suspect that we'll work on that over the next couple of days
• responding to some questions
• have a new laptop, which is 33% faster at building Parrot than my desktop
• getting ready for Frozen Perl next week

Jerry:

• added portable NaN/Inf support to Parrot and Rakudo
• the 60 or so spectests which failed on non-Linux platforms now pass
• all tests pass for Parrot and Rakudo on Windows
• working a little on the infrastructure
• trying to get e-mail to Trac installed
• opened 14 tickets yesterday
• challenging others to join in on closing them
• S19 could use Larry's review

c:

• working on the support, release, and deprecation policies
• will submit that draft for review this afternoon
• have a couple of bugs to fix
• will close as many tickets as possible

Jerry:

• the Open Group's C99 model suggests inf or INF
• should we follow that or the dynamic language model of Inf?

Larry:

• I like the latter

Patrick:

• the Open Group says when you parse a string, it's case insensitive
• when you produce a string, it's all uppercase or lowercase

Larry:

• Perl 6 doesn't care what other standards groups say

Patrick:

• if we stringify infinity, we get Inf
• if we numify inf, do we get zero or infinity?

Larry:

• infinity