Parrot: October 2008 Archives

The Perl 6 design team met by phone on 29 October 2008. Larry, Allison, Patrick, Jerry, Will, Jesse, Nicholas, and chromatic attended.

Allison:

• slowed down on MMD bug fixing
• cleaning up the last bugs now
• started the IO branch
• getting the code in much cleaner working order
• eliminated the old pseudo-object system we had
• changing that to use the current, existing object system
• it looks like it will work cleanly
• mostly can keep most of the old code
• just change it to use objects, not just function pointers
• the initial FileHandle PMC
• moved the Unix, Windows, and Standard IO functions into a single set of functions loaded by Configure
• moving the UTF-8 stuff into utility functions, instead of layering it on top in a difficult-to-maintain fashion
• looks like we won't use roles at all for IO
• I thought we might need them to emulate layers
• it turns out that most layers weren't composed behavior at all
• you need this set of functions on one OS and this set on another
• roles don't really solve that

Jerry:

• delegation, not composition

Allison:

• it's really a lower level than that
• operating system utility libraries
• an interface to the OS
• you don't want to interact with the OS directly
• behind the interface, you compile it to do a different thing on a different OS

Nicholas:

• Daniel Ruoso asked me on IRC about adding pointers to every SV
• he's looking at that for SMOP
• hope to have the release out on the weekend

Larry:

• still in the middle of doing the doctor thing on my shoulder
• going in for an MRI this afternoon
• making arrangements for my December trip to Australia
• maybe or maybe not Amsterdam, depending if we can work it out
• most of my work has been responding to various things that the SMOP folks and the syntax highlighter folks have found difficult in the STD parser
• refactoring some of that
• parameter parsing was not right in various ways
• the parsing for things like POD comments was suboptimal
• tended to attach it to the previous block where it ought to be independent
• parsed better now
• there were difficulties parsing assignment operators
• tended to parse a <== as less-than equal turned into an assignment operator
• nipping and tucking on trying to cut down memory usage and the amount of cursor construction that has to go own
• figuring out which cursors can be treated as mutable
• and when we have to do copy-on-write
• in general thinking about how I can make the parser faster
• hopefully get rid of TRE over the next month or so

Jerry:

• things are going well
• just returned from the Google Summer of Code mentor summit
• Jonathan Leto and I represented TPF
• had a good time
• I plan to write something about it, when I have the time
• mostly devoting tuits to the Parrot Developer's Summit, coming up in the middle of November
• expect that to be the case for the forseeable future
• unfortunately lost the tuits I had to do namespace importation in Rakudo
• it's very close to being done

Patrick:

• worked more on fixing lexicals in Parrot
• getting them to work the way we need them in Rakudo
• down to 16 failing core tests
• most of those have to do with relying on autoclosing called subroutines
• working on a design and implementation for that
• shouldn't take too long
• it's going cleaner than I thought, assuming I haven't made gross errors
• have worked on cleaning up exception handling from the PIR perspective
• trying to get us caught up to date on that
• published a lengthy note to the list last night describing how we should write exception handlers
• otherwise, responding to the Rakudo ticket queue
• either applying patches or describing why they won't work
• getting feedback to people submitting patches

c:

• I've been fixing a lot of bugs
• wrote up how I fixed autoboxing primitives in MMD
• fixed at least one segfault for Will, have more to look at
• also fixed a couple of memory leaks
• the default PIR programs now should never leak any memory
• but IMCC still leaks memory in some cases
• will continue to fix any segfault reproduced as nicely as Will has done

Will:

• the lead-in to that
• spent most of this week trying to reduce several thousand lines of Tcl into a small PIR program so I could get past a segfault

Jerry:

• one nit in the exception handler API
• when you mark it as not having handled something, you set it to zero

Allison:

• you rethrow it

Patrick:

• that's going away
• at the moment, because exception handlers are automatically disabled, you re-enable it by setting it to zero

Allison:

• that was a hack for backwards compatibility
• it's going away

Jerry:

• I hate magic numbers

Patrick:

• I hate the fact that 0 re-enables it
• I'm about to run the tests that turn off the auto-disable feature
• I have everything that the pops are where they should be
• I'll see what breaks

Allison:

• any failures should be updated pretty quickly

The Perl 6 design team met by phone on 22 October 2008. Larry, Patrick, Allison, Will, Jerry, Jesse, Nicholas, and chromatic attended.

Patrick:

• mostly answering questions about Rakudo design
• continuing to work on namespace handling there and in PCT
• continued to work in my head on the lexicals implementation
• have a little code there too

Larry:

• looking into shoulder surgery in the next couple of weeks
• decided that I didn't like the infix:where operator, so I changed it to also to better indicate the leftness to rightness of it
• working on cutting down the size of the lexer cache in the world of STD and gimme
• they used to take up 3 GB and now they're down to 450 MB
• that's by not caching anonymous grammars which they throw away at the end of the current process anyway
• there's still a long way to go on that
• if you do a bzipped tar of the lex directory, it ends up with a 200:1 compression
• still lots of room for removing redundency
• while I'm looking at lexer caching, I'm trying to figure out the best strategy for getting rid of TRE
• half of the code just works around its bugs
• it'd be simpler and probably faster with something more straightforward
• otherwise little bugs and answering questions

Patrick:

• you also added usage information to viv
• thank you

Larry:

• I abandoned my goal last week of getting it to spit out Perl 6
• sidetracked into hacking on the lexer
• other people started to use it
• but the way they're trying to use it, it runs slowly
• I'm looking at ways of speeding that up

c:

• fixed a lot of bugs last week
• in particular, some of the MMD fallout
• blockers for mod_parrot
• we may be able to get under 600 open bugs by the end of the month
• hopefully under 500 by the end of the year
• otherwise looking forward to lots of Parrot cleanups

Nicholas:

• getting close to a RC for Perl 5.8.9

Allison:

• just flew to SF yesterday and back today

c:

• and boy are her arms tired

Allison:

• spent a lot of time on release bugfixing
• answering questions
• closing tickets
• made the significant TGE change we talked about last week
• updating the calling conventions branch
• then creating the IO branch
• those are my tasks for today

Jesse:

• talked to Richard a little bit about grants

Jerry:

• just released 0.8.0 yesterday
• we fixed a lot of bugs on the way to the release
• but we released it with more than I would have liked
• can't say why that is
• two big branch merges though, the MMD and HLL magic branches
• but we need to address our automated test environment and smokers
• we'll concentrate on that at the Developer's Summit in November
• sent out a survey to the mailing list
• starting to get responses
• going to the GSoC mentor summit on Friday night, come back Sunday evening
• at Mountain View

The Perl 6 design team met by phone on 15 October 2008. Larry, Allison, Will, Jerry, Patrick, Nicholas, and chromatic attended.

Patrick:

• presented at the Pittsburgh Perl Workshop
• talked into being the keynote speaker about Perl 6
• gave a more direct talk on contributing to Rakudo and the test suite
• I don't think I even presented a single line of PIR at the conference

Allison:

• I consider that a great success

c:

• depends on the line of PIR

Patrick:

• I took the MMD failures out of the regression suite temporarily
• didn't want to go into the weekend with test failures
• we're still waiting for a fix there, but there's no particular rush
• 4409 tests passing now, up slightly from last week
• up slightly from even yesterday (30 more)
• worked with Jerry on getting is export to work in Rakudo
• he's done all the work
• I just give him guidelines
• worked with Stephen Weeks to refactor PGE and PCT to use hierarchical names instead of string names
• we merged that
• but it broke TGE and Pheme and Tcl and another language
• PGE expects to use strings to match the match object classes

Allison:

• I'm pretty sure I know exactly which three lines of code to patch

c:

• I sent in a patch

Patrick:

• I'll fix PCT to look in the right place for those
• if you don't hear that I've done it, I haven't beaten you to it
• cleaned up a lot of namespace issues in Rakduo
• fixed some mis-implementations
• trying to get nested namespaces to work so that Perl 6 can use them
• a lot of "two steps forward, one step back"
• updated the progress graph program
• now it shows the total size of the official test suite
• the graphs themselves will change a little bit
• I'll make an explanation of that
• still really want to get to lexical variables or Rakudo's container value model
• I'll do that this weekend, when I have uninterrupted time

Allison:

• cleaning up MMD problems from after the merge
• started a branch for the calling conventions
• Andrew Whitworth is working on that
• wrote a merging and branching guide based on my good and bad experiences
• thinking about things we can do to make merging cleaner
• maybe a checklist of people to talk to before you do a merge
• Patrick, that TODO test you added for the Complex PMC?
• I have it not segfaulting anymore
• it's always returning 0 now
• it creates a new PMC that hasn't been set to the right values
• I think it's a bug in the old proxy code
• we probably need to remove that old code and it'll fix it

Will:

• working on Partcl still
• have another test file parsing
• hoping to use the upcoming Parrot release as a new baseline
• the TGE issue is the blocker for that
• trying to close all of the easy tickets I can find in Parrot

Jerry:

• went to PPW last weekend
• it was very productive and too short
• spoke at Bug Labs about porting Parrot and Rakudo to the Bug
• they ported Perl 5 just before the conference
• some people were playing with it
• it's an ARM processor which runs OpenEmbedded Linux
• if it gets ported, we can run on other devices that size

Allison:

• that's definitely in our target range of uses

Jerry:

• spoke to someone about NVIDIA about using their CUDA GPU API for hyperop support
• that looks promising
• also made the is export trait work
• includes tag sets
• committed failing tests for import()
• working on passing those, may have that done by the end of the call
• discussed branching and merging philosophy with Allison
• put up a Wiki for the Parrot Developers Conference
• has questions which need answering
• I'll mail the list, requesting answers
• that'll give us an idea of logistical questions, including budgeting

Larry:

• on specifications, various simplifications
• expressions in conditionals are not special with respect to blocks that might appear in argument lists
• if you say for map { ... }, it parses the way the user expects
• the block that the for expects must occur where an infix is expected
• that simplification seems more DWIMmy
• I nailed down the operator modifying adverbs
• now applied on the basis of precedence
• no longer a special mechanism

Patrick:

• yay

Larry:

• there will be a few FAQs from that
• by and large it's a win though
• going back and forth on the subject of unpacking arrays and hashes
• someone noticed that the standard parser didn't actually handle those
• I went to add them
• discovered that the hash unpacking syntax using curlies was terribly ambiguous, especially inside a pointy block (arrow lambda)
• tried to rescue that, but in the long run decided that hashes have to be unpacked with signatures, rather than curly notation
• left in the square bracked notation for unpacking lists
• it contributes to clarity
• decided that enumeration types ought to define a .pick operator
• you can do Bool.pick and it'll return true or false
• thinking a lot about IRC and on the mailing lists
• how do you deal nicely about serialized token lists in the order that the tokens occurred in the original?
• how do those bind to signatures usefully?
• given that we want match objects to behave like captures
• did a lot of hacking on STD.pm
• somewhat driven by adding options to viv
• it has several options now
• a match option attaches the old match objects to a position
• a positional option
• the main option is a p6 option which attempts to spit out the exact Perl 6 as originally parsed
• discovered various places where I was throwing away information
• that leads to trying to keep track of tokens and match interstices
• also several other people are trying to use the STD output to try to produce other things
• they're discovering missing information that I've been hacking in
• it threw away all of the information on elsifs and elses
• that made it hard
• removing cyclic references which prevent analysis of the tree as a DAG
• discovered a fundamental bug in the operator precedence parser
• reducing if there were any terms left
• it should have reduced if there were any operators left
• it didn't reduce any unary operators at the end as it should have

c:

• I fixed some bugs
• started looking at the DAG, but couldn't find much documentation
• do you have a getting started guide, or do you say "Look at this code and think really hard?"

Larry:

• it's the latter right now
• but I could at least put a help message on it

c:

• a list of node types and their attributes should suffice

Larry:

• that's the kind of documentation that quickly becomes a lie
• right now, it generates classes on the fly as it sees nodes
• it could at least tell you what it's doing as it's doing it

c:

• looked like it could be fun to play with for a weekend

Larry:

• in my copious free time

Patrick:

• right now, I can look up a namespace by an array, by a key, or by a string
• I can create a class by a namespace, a key, or a string

Allison:

• but not an array?
• I thought we fixed that

Patrick:

• we work around that in P6Object
• get look up the namespace by the array
• and then we look up the class by the namespace

Allison:

• we should be symmetrical about this
• your workaround should move into the Class PMC

Patrick:

• you have to do that for subclass, isa

c:

• get_class

Patrick:

• that specifically does not work
• that's what Stephen Weeks and I tried

Allison:

• submit a ticket
• there's no reason they shouldn't take arrays

Patrick:

• it's not a high priority at all
• just a discontinuity that stunned me when I ran into it

Jerry:

• there's a list of tagsets without commas in is export
• STD.pm allows it
• but it seems to open shift-reduce problems with multiple terms in a row

Larry:

• two adverbs in a row are always considered one term
• just a sequence of adverbs
• whenever you parse one adverb, you look for one more

Patrick:

• they're not adverbs; they're terms

Larry:

• it works for terms too
• whenever you see a colon pair

Patrick:

• if I write 3 + :a :b
• the :a :b is a term
• what kind of thing is that?

Larry:

• a list of pairs
• or a mapping maybe

Patrick:

• that's the same as saying 3 + 2
• the :b would not be an adverb on the plus
• inside an argument list, :a :b should act like two arguments
• not a list of two pairs

Larry:

• if it's coming in as a list of named parameters, it doesn't care
• if it's in a listy area, it gets interpolated as a list
• if you bring them in as a scalar parameter... you can't

Patrick:

• is it the came as :a, :b?

Larry:

• in list context, yes
• when you're looking for named args, yes

Patrick:

• if I had a subroutine that took a positional parameter and a named parameter of a, and I call it with :a :b

Larry:

• it acts like there's a comma between them

Patrick:

• is there a way to see them as a single term?

Larry:

• if you put them in parentheses

Patrick:

• they solve everything
• if I see :a :b in an argument list, I have to translate as though they were separate arguments

Larry:

• I guess
• that's the way it works currently

Patrick:

• if we have a term that's a list of colon pairs
• we have to do some special unpacking stuff when we turn that into an argument list

Jerry:

• you call .list on it?

Patrick:

• this is at compile time
• they have to be interpolated into the capture properly

Larry:

• now that we've changed how adverbs work, they stack up nicely in any case without any special rule
• if we removed the special rule, they'd work with the comma

Patrick:

• getting rid of the quantifiers with the colon pair in noun?

Larry:

• 1 + :a :b would apply the :b to the +

Patrick:

• it gets weird when dealing with it inside the argument work
• we can make it work, but...

Larry:

• the phrase "make it work" means there's something suboptimal
• let me think about it a while longer
• the way the adverbials parse now would apply them to the same operator
• that was the main motivation for stacking them like that, to have multiple of them in the adverbial slot

Patrick:

• I thought part of that was for the filetest operators

Larry:

• that's another one
• let me think about that one

Patrick:

• we'll assume commas there for the time being

Larry:

• sounds good

Jerry:

• I changed the tests to use commas

The Perl 6 design team met by phone on 08 October 2008. Larry, Allison, Patrick, Will, Jerry, Jesse, Nicholas, and chromatic attended.

Larry:

• things are proceeding apace
• spent most of the week either in Illinois or traveling to and from it
• a really good conference, ACM Reflections/Projections
• University of Illinois at Urbana-Champaign
• 500 attendees, a wide variety of interesting speakers
• gave two talks, good attendance
• good hobnobbing with Al Aho too
• think Awk, Dragon Book, regular expressions
• he had some very good questions
• had a great time naming the ten most influential programming languages over lunch
• despite that, I also did a fair amount of hacking
• wrote a little parse tree to AST transformer
• thought it would be complicated
• did it all pretty much with an AUTOLOAD
• basically comes out with a typed parse tree
• I noticed some infelicities in the AST
• some artificial nodes, particularly in nodes
• just a leftover from the mechanism
• I stole an idea from Haskell then
• the ~ macro knows about parsing something between two delimiters
• hacked that into the specs and STD.pm
• it seems to work now
• cleans things up considerably, including the AST
• allows for better error messages
• in the talk on a derivative of APL that Morgan-Stanley uses, Q, I noticed the ability to do something missing in Perl 6
• the ... infix operator iteratively applies a function to the tail of a list as it's generated
• now it's trivial to implement Fibonacci
• lots of hanging out on the mailing lists and IRC
• keeping people feeling like they know what's going on
• a bunch of bug fixes
• more accurate parsing of goal-oriented constructs, such as the end of argument processing

Allison:

• merged in the MMD branch this week
• has been a big thing for the past longer-than-it-should-have-taken
• a couple of issues in some languages after the merge
• most of them will be resolved by dispatch system cleanup
• there are two dispatch systems, PCCINVOKE and the old runops_from_c system
• a few strange issues like results registers not existing when you try to access them
• caused some segfaults
• Andrew and I will try to rush through the first two items and merge that back as soon as possible
• should resolve some issues in Rakudo
• seems to be what's left after we fixed the other problems
• also starting the IO milestone branch
• talked to Simon Peyton-Jones in the UK about some of the stuff we're doing with concurrency and encapsulated dispatch CPS
• didn't plan to meet with him, but my advisor called him over

Patrick:

• two focuses this week
• getting HLL type mapping to work in Parrot and PCT
• language translators need to run in their own namespace
• Stephen Weeks has helped out in a big way
• should open up quite a few avenues for development and make other things easier
• a longstanding issue was moving some of the compiler tools to Parrot's model of namespace handling
• until now, they used a flat namespace
• we're almost ready to merge that branch back in to trunk
• then we can put Rakudo and Cardinal into their own HLL namespaces
• then we can do HLL typemapping
• an integer PMC in Parrot will map to a Perl6Int rather than an Integer PMC

Jesse:

• I thought a discussion in Copenhagen had Allison saying that that automapping shouldn't happen

Patrick:

• it's not cross-language type conversion
• inside of Parrot, in PIR, if I have an Integer constant promoted into an object, what kind of object should it become?
• that depends on the active HLL

Allison:

• it's not really a coersion
• "in this context, Integers autobox to this type of PMC"

Patrick:

• each HLL gets to decide how to autobox low level types

Will:

• is this in PCT?

Patrick:

• yes
• but in the process, we find places where Parrot doesn't do what we want
• I file tickets for those
• because of the MMD merge, Rakudo is failing 74 new tests, most of them due to the Complex PMC
• a little regression there
• I'm sure the problem is the IM and RE macros

Allison:

• I'm not sure
• in theory, it's actually dispatching to the proxy complex object within the Perl 6 complex object
• it may be there's a problem still with Object
• it still uses the old dispatch for subroutines within C
• I've manually updated them to the new approach and it fixes them

Patrick:

• I traced it in gdb
• I get to the appropriate sub and the RE macro doesn't get the right value

Allison:

• I'll try updating Complex to remove those and see if that does it

Jerry:

• on my way to Portland to give a hands-on Rakudo presentation tonight
• I may be able to get them passing by the time I arrive
• but I can explain them away
• we'd like to get them passing before PPW
• it's probably time to make the spectests in Rakudo the default make test target
• spent my week trying to get is export to work
• found a few rabbitholes along the way
• added the apostrophe and other identifiers to the grammar
• added some magical $* variables based on a contribution
• ran into a problem where namespaces aren't working properly
• Patrick's looking at that
• Jeff Horwitz found something similar when working on mod_perl6
• glad to see the MMD branch merge
• but some of the failures make it harder to see if I broke anything
• would like to see that cleaned up in the next couple of days
• or we'll mark those tests as TODO for PPW so that we have clean results
• I have the next release on the 18th
• one of my targets is to get 4500 Perl 6 tests working
• we're at 4300 now, with the failures
• it's a reasonable target
• will be attending PPW

Patrick:

• my focus for PPW is Rakudo
• trying to get people to contribute code or test suites
• I'll conduct sessions to walk people through the steps they need to contribute to either or both

Will:

• moved Partcl out of the Parrot repository into its own repository
• sent out URLs and notifications
• that'll give me a way to target a very specific version of Parrot
• rather than working against what's currently in HEAD
• should give me the ability to make more progress
• have made some, but not as much as the infrastructure changes

Jesse:

• probably won't be here next week
• what do people need?
• okay, that was quiet

Patrick:

• Larry, you mentioned that the .match method sets its caller's $/, is that correct?

Larry:

• yes
• it seems consistent
• essentially an implementation routine for if you use it directly

Patrick:

• Allison, on the calling conventions page, the :lookahead option isn't there
• is that an oversight?

Allison:

• it's there in a sense of "What's this going to do?"
• do you have a ticket for what we talked about in Copenhagen?

Patrick:

• I thought the notion of using a Capture-like object for the calling conventions is a big plus

Allison:

• that's done
• but not pervasive throughout the system yet

Patrick:

• can I put a note in there "don't forget about :lookahead"?

Allison:

• there are no details about what it should do or how
• add to it or create and link to a ticket

Patrick:

• the design seems okay to me
• I'm looking more for an implementation than anything else

Nicholas:

• p6l has gone off about rationals and floating point numbers
• seems to have lost all basis of rationality of implementation
• not sure that any implementor has commented on it
• was that intentional?

Larry:

• on my part, yes

Patrick:

• I'll describe what Rakudo does and what the spec says, but I avoid discussions as to what Perl 6 should be doing

Larry:

• notice the spec hasn't changed

Patrick:

• that said, I agree with the notion that Parrot's Integer should have BigInt semantics without having to change its type
• not that it's always a BigInt, but it could seamlessly change from the default Integer type without having to change its type

Allison:

• that might solve our horrible problems with the overflow code

Patrick:

• I know that Allison mentioned on IRC that morphing maybe should go away
• it has problems with subclassing

Larry:

• Perl 6's view is that it's always a BigInt
• it can cheat on a faster storage format if it can

Patrick:

• Rakudo has a similar view
• the morphing issue...
• Rakudo may end up with its own PMCs for everything

Allison:

• I expected that from the start
• one of the ways an HLL customizes the virtual machine is by providing its own PMCs
• we ended up with a lot of PMCs for some reason

Patrick:

• it would be nice if Parrot supported that natively

c:

• it'd be nice to have a vtable swapping technique that let us get rid of all of those switch statements
• it's just a table full of function pointers
• we can avoid morphing in some cases
• take the Key PMC for example

Allison:

• getting rid of morphing would be nice
• some MMD default variants that set native numbers morph Integers into Floats transparently
• the Integer overflow code is broken
• mostly the problem is that it's doing direct MMD rather than a vtable call
• a vtable call would let BigInt override some of those vtable functions
• would resolve a lot of problems there

Nicholas:

• signed integer overflow is undefined in C
• 64-bit Irix bit me
• unsigned overflow is well-specified

Patrick:

• are any of the n_* opcodes sticking around?

Allison:

• a few
• the ones that didn't have a funky alternate dispatch system
• I didn't modify them because they caused test failures
• assume they're going away
• not and abs

Patrick:

• and neg, or one other one too

Jerry:

• do we need to deprecate them?

Allison:

• we'll do that in the calling conventions cleanup
• when I deprecated them, I think I said "all n_* ops"

The Perl 6 design team met by phone on 01 October 2008. Larry, Patrick, Jerry, Jesse, Nicholas, and chromatic attended.

Larry:

• busy as usual
• did some hacking and a bit of spec work
• defining how assignment operators behave when the left side is a protoobject
• took out the curly stars at the end of rules a few weeks ago
• put in the correct semantics
• calls a reduction with the current tag now
• don't need an action now
• broke macro parsing
• you can't define and parse factorial now there
• going to the University of Illinois tomorrow
• an ACM conference: Reflection/Projection
• giving a keynote on Friday night
• a workshop on Saturday
• that's more or less my standard parser talk
• my keynote is the Studies in the Ballistic Arts talk
• slightly more general interest

Jesse:

• did you say that recent changes should make it easier to parse more Perl 6 in Perl 5?

Larry:

• it should make it easier to translate more of gimme5 with a Perl 6 parser not a custom parser
• should make it easier to parse more Perl 6 into the corresponding Perl 5
• then I can rewrite cursor from Perl 5 into Perl 6, and backtranslate it to whatever engine

Patrick:

• not a lot of coding
• lots of helping others code
• getting a lot of things done
• fixed some Parrot issues out of the way
• chromatic fixed floating point constants
• lots of spec tests passing
• close to 800 new ones in the past week
• 20% in just one week

Jesse:

• how much more spec are they covering?
• how much is it checking addition of pairs of numbers?

Patrick:

• some broken features now work
• many of them are that Moritz and friends have found passing tests in the whole suite and included them in our suite
• some are new features
• it's a pretty good mix
• there's little addition of tests for existing features
• lots of tests for features we didn't test yet
• Stephen Weeks implemented loop control exceptions in PCT
• also for Cardinal
• that was this morning
• we're looking at those exceptions in general
• we want to handle them in a sane manner in Parrot
• otherwise, just asking design questions
• answering questions on IRC
• lots of thinking about fixing reference and value types in Rakduo
• trying to get momentum in my head to do that quickly
• will require refactoring plenty of things in the existing code
• I'll have to review a lot of Jonathan's code in the process

Jerry:

• things are going well
• have a lot more time lately for Perl 6 and Parrot
• currently looking at S11
• the documentation for exportation
• want to get namespace exporting in Rakudo
• that's another step toward writing Rakudo in Perl 6

Patrick:

• we should talk about that
• I'll tell you how to do it
• I have some pretty specific ideas

Jerry:

• submitted a grant proposal to TPF for S19
• the command-line implementation
• lots of IRC conversations
• lots of good questions
• it's hard to give Moritz enough praise for the work he's been doing on the spec tests
• it's amazing progress

c:

• haven't had much spare time
• willing to concentrate on blockers
• may be able to fix one or two crazy bugs a week

Patrick:

• we found a workaround for the weird problem we had last night

c:

• you had a lopsided exception handler

Patrick:

• I saw that
• switched the order of a few statements
• then saw the lopsided exception handler
• I'll file my example code as a bug so we can fix it

Jesse:

• I talked to Robert about how to get a prominent download link for Rakudo on perl.org
• if it's doable to get a nice target page for how to build and use Rakudo, that's easy to get up
• need a page to describe the four steps you need to get /usr/bin/perl6

Jerry:

• and maybe a link for binary distributions
• Debian for example

Jesse:

• that's not terribly common in the Perl world
• Windows maybe
• if the package downloads exist, that's a good idea
• I don't recommend delaying things for that though

Patrick:

• one problem with those distributions is that things go so quickly out of date

Jesse:

• the last release was only a couple of weeks ago

Patrick:

• the difference from week to week can be huge
• list assignment immediately starts working, for example

Nicholas:

• the idea is to shut up naysayers who say "It's not there" or "It's not easy"
• people who say "I tried it, but this didn't work" you can point to something current

Jesse:

• is there a page which describes how to download a tarball and make it go?
• versus a Subversion version?
• is there a "How to build Rakudo" page?

Patrick:

• I'll put one up on the wiki
• someone can help keep it up to date

Jesse:

• there's no reason not to have the canonical version on the wiki
• when someone complains, it can get updated
• mostly I'm just working and running a small business in a slow economy

Patrick:

• do you envision the global flag on regexs part of the regex or the caller?

Larry:

• I've thought of it as part of the thing that calls the regex
• I could see it going the other way too
• no good arguments in my head one way or the other

Patrick:

• tradtitionally I would have preferred it as the thing that calls the regex
• I like it if regexes do one match, and whatever's driving it does it next

Larry:

• if you want .*? semantics inside the regex, you ought to use it

Patrick:

• when we parse :g, it's an adverb to the operator, not the regex

Larry:

• inside we have submatches with ~~
• people will want adverbs on those
• we need to think about whether that's reasonable or possible

Patrick:

• I'll start with the thing that calls it

Larry:

• we can rethink that later if we need to
• we can recast it as rewritten rules of various sorts

Patrick:

• it affects the grammar engine as to what it produces and returns
• I could do it
• but I'm about to refactor PGE and want to get my strategies in place
• Mitchell wants to tag match objects with the rule that generated them
• what do you think?

Larry:

• if we can think of a reasonable name that makes sense downstream
• not everything has one
• apart from a reference to an object
• maybe that's good enough
• it's a bit problematic in a Perl 5 version of the matcher
• there really aren't code objects

Patrick:

• maybe, but not likely?

Larry:

• I see no major conceptual problem in the Perl 6 context
• just an implementation problem

Patrick:

• how much might S05 change?

Larry:

• it has potential ramifications in terms of memory use and GC use
• other than that, it's a vague uneasy feeling
• I can't prove anything

Patrick:

• the .keys, .values, .kv, .pairs methods
• when used as functions, are they named unaries?

Larry:

• I'd like to avoid making more named unaries than are culturally acceptable in mathematical notation
• they tend to confuse people
• not as confusing as zero-or-one, which we've tried to get rid of
• like rand

Patrick:

• just looking for a quick answer

Larry:

• I'm prejudiced against them

Patrick:

• I'm prejudiced against named unaries as well
• if something starts to look non-Perl 5-ish, I get prejudiced

Larry:

• you force people to use parentheses
• but mathemeticians don't want to say sign of something and use parens

Patrick:

• plenty of tests in the suite test that the values of colon pairs are booleans

Larry:

• any boolean used in numeric context turns into zero or one
• if we stringify booleans, we have to turn them into zero or one as strings
• none of this null string as in Perl 5

Patrick:

• they do smart match against bool

Larry:

• preserve the type information
• make it as usefully context-dependent downstream
• the issue is how they stringify
• similar issue as to how proto-objects stringify
• you want to know whether you want to use them as their name or their value
• we really don't make that distinction right now
• except .perl really wants to know the name of things
• if stringify Object, does it print Object or the null string?
• if it's our new undef, it should print the null string
• when you print values of things, it's not useful to print a null string where they were expecting a value
• which is why I wanted to distinguish between stringification and printification
• it's probably too subtle of a distinction
• if you use a protoobject and stringify it, it returns null
• but by default it should produce a warning
• the warning presumably says what the name of the protoobject was

Patrick:

• it might be too subtle or wrong a distinction to be useful
• a scalar initialized to a protoobject effectively holds a reference to the object
• when you stringify the reference, you want that to be the null string
• if you print the object, you don't have a reference
• you're sending the protoobject itself
• I see all sorts of nasty flags floating around that
• that distinction will be there from the Parrot side
• whether it's there conceptually is another question

Larry:

• in general, if I don't answer immediately it's because I'm scratching my head

The Perl 6 design team met by phone on 24 September 2008. Larry, Patrick, Jerry, Nicholas, and chromatic attended.

Larry:

• spent most of the week hacking
• neither golf nor computers
• haven't had much brain to think about anything other than work
• made a few comments on IRC
• thinking that the direction I need to head now is turning the standard grammar into a Cfront for Perl 5
• get rid of gimme5 by taking the standard parse tree
• use that instead of the faked up parse
• spit out the same thing
• then I can translate some subset of Perl 6 back to Perl 5
• then I can think about rewriting other things
• Cursor.pm in Perl 6
• getting ready to go to a conference in Urbana in a week and a half
• ACM-ish thing called Reflections Projection
• giving two talks there
• most of my discretionary time is getting my video output suitable for running Firefox and OpenOffice.org Impress

Patrick:

• not a lot of actual code
• lots of design work and shepherding other code
• getting container design right in Rakudo
• Jonathan and I talked about it
• won't require major Parrot rework
• plan to start implementing that in the next week or so
• will solve some problems with array, hash, and object references
• Moritz has been working like crazy on spec tests
• finding things that Rakudo will pass
• we pass well over 3700 tests
• that's 400 more than last week
• over 200 yesterday alone
• few of these tests are really changes to Rakudo itself
• have about five new contributors to Rakudo in the past two weeks
• spent Monday rethinking how lexicals should work in Parrot
• that's blocking a lot of things in Rakudo
• re-read S04 and various threads on the Parrot lists
• wrote a draft of how I think things ought to work
• awaiting feedback on that
• the more I think about it, the more I think it's the way things ought to go
• ought to simplify things and get rid of a lot of ugly code in Parrot
• unless I'm overlooking things
• might try implementing it myself in a branch unless someone (hint, chromatic) wants to do it instead
• not a lot of changes to PIR itself
• seems to work pretty well in all of the mental cases I've thrown at it thus far

Jerry:

• setting travel plans for fall conferences and meetings
• patched a few HLLs today
• someone added SMOP to the #perl6 eval bot

Nicholas:

• Jesse's going to the Beijing Perl Workshop to talk about Perl 6
• they found a sponsor for him

c:

• fixed Parrot's float output precision
• makes test output different in some cases, so it's a big patch there
• want to get testing on Cygwin, Darwin, and MSVC
• it's a well-understood and well-supported part of sprintf
• also looking at Patrick's lexicals proposal
• had some clarifications and questions
• will get that out tonight
• one question is about an enclosing lexical scope for multiple globally-visible subs
• initialization time is tricky

Patrick:

• I explicitly disclaimed that for now
• I'll look at S04 and take examples from there

c:

• also had a question about recursion
• that can get tricky

Patrick:

• I had that thought too
• I think it works

c:

• I'll definitely send my thoughts
• want to make sure they get in the archives

Nicholas:

• is a Cfront to STD.pm another bootstrap to get Perl 6 running on Perl 5?
• or does it only bootstrap the parser, not the runtime?

Larry:

• you have to have a runtime, or you can't bootstrap
• it just uses Perl 5 for the runtime
• that set of semantics which are easy to do in Perl 5
• or at least possible
• it won't easily do all of Perl 6 semantics
• including lazy lists and such
• not without internal hacking to the regex engine

Nicholas:

• the first thing I thought about was continuations, which Perl 5 won't be doing

c:

• that's not too hard

Larry:

• I'd settle for lazy lists

Patrick:

• I'm looking forward to a Cursor rewritten in Perl 6
• Perl 6 implementations make it easier to bootstrap on whatever implementation we have

The Perl 6 design team met by phone on 17 September 2008. Larry, Allison, Jerry, Jesse, and Nicholas attended.

Jerry:

• gave a presentation last night on Perl 6 and Parrot to the Seattle Perl User Group. It was well received.
• what is the ownership of Perl test suite?

Jesse:

• ask about the Linux kernel for an analogous situation

Jerry:

• what's the license?

Jesse:

• the licence was fixed from the beginning. Audrey did it

Allison:

• it's Artistic 2, early draft.

Jesse:

• I thought it was an early draft.

Patrick:

• Artistic 2 plus MIT

Patrick:

• yesterday I got the September release of Parrot out.

Allison:

• thank you very much.

Patrick:

• It went so smoothly. The documentation is good. Karma points to everyone who put that together. It's working out really well.
• I've been answering questions, guiding other people to do stuff.
• Fine by me, but not done much core hacking. Guiding is good.
• Statement modifiers after listops now work.
• C now a named unary, with the precedence correct.
• I'm answering questions even off the Perl list and subscribed to the November wiki list
• I chime in, see where they are running into Problems.
• I'm giving them advice in designing wikis [http://www.pmwiki.org/]
• Jerry joined it too

Jerry:

• Yesterday put a FAQ on November Wiki about how to subscribe to the list

Patrick:

• I've been talking with Jonathan about design issues.
• We still have problems with lexicals - putting ideas forward.
• Putting ideas forwards on perl6-language about how lexicals and others should work. Got feedback, which is good.
• There's an impedance mismatch between Perl 6 and Parrot about how things should work.
• I have a design, where we don't need scalar PMCs for every aggregate type. I [will?] put it on the web.
• Jonathan and I are working on it tomorrow.
• Good not to have all those GC-able PMCs around.
• It's not a pretty design, but it should be good enough to work.
• This week I will be working on lexicals, and getting list interpolations stored correctly. These are ongoing forms for those using Rakudo.
• This weekend going to restart "this week...", but it might be every 10 days. 2 weeks is too slow, 1 week is to frequent.

Jesse:

• Are the milestones still good?

Patrick:

• I might have knocked one off. I'm doing it by RT ticket. The next to be knocked off is list assignment. We may get lazy lists as part of that as well, but still I'm still thinking.

Larry:

• I have a fabulous cold. My brain is in sideways.
• "Tweaking things here and there as people point out things."
• Tweaking STD.pm as people do things in tests.
• For example, people check in tests and don't syntax check them very well, especially if as TODO tests. For example, unbalanced parentheses.
• STD.pm tends to catch those.
• What people have been saying, eg how lists work.
• My thinking is of a more general nature, how much we expose the indeterminacy of the world, how to balance out eager vs lazy; expectations of how assignment will work.
• Combining paradigms, and how it requires us to set up things like we did with the regexps.
• Some are deterministic, but naturally the non-deterministics.
• Whether they work the same way, or assignment works the same way, eager or lazy, and whether the users' control knob can deliver
• Totally eager isn't going to fly, but "as eager as it can be until it arrives at some Inf", or whether list policy as general says "eager on easy, lazy on hard" - haven't decided that yet.
• POD parsing -- defined POD such that we can say =begin end
• This can go to end of the file if there is no corresponding =end end
• Is that a general policy, for unbalancing?
• Or should end be a special case; =begin end, or =begin data

Jesse:

• =begin data feels more like Perl 5

Larry:

• should we have a different keyword other than BEGIN?

Patrick:

• An adverb, "begin", go all the way to the end, make it explicit.

Jesse:

• anything more, on canonicalising the core test stuff?

Larry:

• the test functions? nothing has been done, but no reason not to.

Patrick:

• I've been spreading the word "this may be coming"

Larry:

• any feedback, let me know

Patrick:

• so far, uniformly positive. People are "oh, that would be nice"
• One thought
• How eager is assignment?

Larry:

• Some thoughts, assignment is naturally eager. It falls out from copy semantics. One can't copy unless source is eager or immutable.
• Else you can't copy unless you snapshot the iterator.
• Eager might fall out of the copy semantics.

Allison:

• 5 failing tests on Macs.
• Almost done, taken 2 weeks longer than I hoped.
• As I make my way through the subsystems, cleaning the problems, each has fewer problems for the surrounding code.
• The lexical is system on my list, after we've found the last few problems.

Jerry:

• I hope to have time for Perl 6 development for the remainder of the
year, starting in October.

The Perl 6 design team met by phone on 10 September 2008. Larry, Allison, Jerry, Patrick, Nicholas, and chromatic attended.

Allison:

• debugging the MMD branch
• we've dgone from 900 failing tests to 47 in the past two days
• working my way through failing tests has had a compounding effect
• chromatic has been helping
• I shifted the nastiest failure from a termination in the middle of nasty code with no warnings to a segfault
• and he fixed that
• we're very close to merging the branch

c:

• probably after the next release

Allison:

• yes, probably

Jerry:

• a segfault is better?

c:

• much easier to fix

Allison:

• this was a PGE problem
• it pretended it was successful
• it output nothing
• it was so much easier to debug then

Nicholas:

• die early, die often

Patrick:

• I can't tell you how many times I had PGE output nothing
• zero?!

Allison:

• we're having a Parrot developer day in November in Moutain View

Jerry:

• finally had time to do a little development this week
• then I had hardware trouble, and couldn't commit anything

Larry:

• between my day job and my grandson visiting over the last week, not a lot of spare time
• took several hundred Parrot-specific tests that Moritz put into t/spec
• fudged them all
• went through several permutations of how to write the tests
• realized that there was a missing operator
• no way to do a pattern object for smart matching that's equivalent to a junctional and but guarantees the order of evaluation
• that's almost exactly what a where infix operator would do
• a type on the left and a constraint on the right of it
• you want to test against a subtype name and a constraint in order
• lots of places want that sort of pattern
• we had no easy way to express that short of defining a subset type, which is hard to use for one op
• or 700 ops in that file
• there's now a where operator that parses just like the junctional &, but in guaranteed order
• otherwise just random spec cleanups
• answering questions
• lots of people thinking about the nature of lists and iterations now
• did a little catchup on email on p6l
• woefully behind on my own personal email
• doubtless there are bugs in that
• oh, ended up adding a Perl 5 grammar to STD
• copied and pasted the Perl 6 regex rules and cut out two thirds of them
• maybe just need to borrow a test from the Perl 5 suite

Patrick:

• PGE uses one
• just uses the test file directly

Larry:

• not fudgeable

Patrick:

• it doesn't use the perl5: flag
• PGE doesn't understand that flag
• it'll fail miserably

Patrick:

• added code to Rakudo to go to standalone PIR
• generates PIR that you can run directly from Parrot

c:

• that should help you help me

Patrick:

• that should help a lot of people help you help me
• updated the Failure type in Parrot
• warns if you use it while undefined
• originally I put it in to die fatally
• that broke the test suite
• a few people are going through the test suite to fix the evaluation of undef Failures
• created a couple of functions in Rakudo to make it easier
• finished my final report for the Mozilla Foundation
• Richard and Jesse are happy with that
• I'll publish that later today or early tomorrow
• that grant is over now
• trying to figure out list interpolation and how things fit together
• Rakudo has some nasty bugs there
• I feel like I'm missing something there
• it seems like there's another type in there

c:

• added a new feature to Rakudo last week
• fixed a couple of bugs to get a lot more tests passing
• keeping up with the incoming patch and bug queue
• Christoph is good at helping
• also trying to fix bugs and help the MMD branch along