Parrot: May 2008 Archives

Moritz Lenz contributed a new "make spectest_regression" target for Rakudo Perl yesterday, and we're already seeing great benefits from it. But rather than me write the details, here they are in Moritz' own words. Thanks Moritz!

---

By Moritz Lenz.

Rakudo's testing had the problem that make test only test basic features, while make spectest produces too much output to be really useful (mainly stack traces from parse failures).

So far the proposed route was to run fudge over them comment out all parse failures and mark them as TODO. Unfortunately that turned out to be very much work, and everybody prefers hacking over tedious, half-mechanical work.

The solution is quite easy: create a whitelist of all spec tests that are known to pass, and invent a new make target for them. After wrestling a bit with t/harness I got the following to work:

$ make spectest_regression
[...]
t/spec/S29-str/sprintf...........ok
t/spec/S29-str/ucfirst...........ok                                          
All tests successful.
Files=32, Tests=569, 85 wallclock secs (84.01 cusr +  1.17 csys = 85.18 CPU)

This roughly triples the number of regression tests for rakudo that are easily accessible.

Currently the new test target has only been tested under Windows and Linux, reports from other platforms are most welcome.

Implementing the changes wasn't very complicated, and required no particular knowledge of parrot or PIR. Thanks to Jonathan for immediate feedback and testing as well as for applying the patches.