=begin pod :kind("Language") :subkind("Language") :category("reference")

=TITLE Testing

=SUBTITLE Writing and running tests in Raku


Testing code is an integral part of software development. Tests provide
automated, repeatable verifications of code behavior, and ensures your
code works as expected.

In Raku, the L<Test|/type/Test> module provides a testing framework, used also
by Raku's official spectest suite.

The testing functions emit output conforming to the
L<Test Anything Protocol|https://testanything.org>. In general, they are used
in sink context:

=for code :preamble<use Test; my ($meta,$relaxed-name); sub check-name($meta,:$relaxed-name){}>
ok check-name($meta, :$relaxed-name), "name has a hyphen rather than '::'"

but all functions also return as a Boolean if the test has been successful or
not, which can be used to print a message if the test fails:

=begin code :preamble<use Test; my ($meta,$relaxed-name); sub check-name($meta,:$relaxed-name){}>
ok check-name($meta, :$relaxed-name), "name has a hyphen rather than '::'" \
  or diag "\nTo use hyphen in name, pass :relaxed-name to check-name\n";
=end code

=head1 Writing tests

Although it is possible to organize your tests differently, the typical
Raku convention is for tests to live under the C<t> directory in the
project's base directory.

A typical test file looks something like this:

=begin code :solo :preamble<my $num-tests = 0>
use Test;      # a Standard module included with Rakudo
use lib 'lib';

plan $num-tests;

# .... tests

done-testing;  # optional with 'plan'
=end code

We load the builtin C<Test> module and specify where our other libraries are.
We then specify how
many tests we I<plan> to run (such that the testing framework can tell us if
more or fewer tests were run than we expected) and when finished with the
tests, we use I<done-testing> to tell the framework we are done.

=head2 Thread safety

Note that routines in C<Test> module are I<not> thread-safe. This means you
should not attempt to use the testing routines in multiple threads
simultaneously, as the L<TAP|https://testanything.org/> output might come out of
order and confuse the program interpreting it.

There are no current plans to make it thread safe. If threaded-testing is
crucial to you, you may find some suitable
L<ecosystem modules|https://modules.raku.org/search/?q=Test> to use
instead of C<Test> for your testing needs.

=head1 Running tests

Tests can be run individually by specifying the test filename on the
command line:

=for code :lang<shell>
$ raku t/test-filename.t

To run all tests in the directory recursively,
L<prove6|https://modules.raku.org/dist/App::Prove6> application can
be used.

You have to install it before using with zef:

=for code :lang<shell>
$ zef install App::Prove6

You can run C<prove6> in a distribution directory this way:

=for code :lang<shell>
$ prove6 --lib t/

The C<t/> argument specified directory that contains tests and the
C<--lib> option is passed to include C<lib> directory into Raku
distribution path, it is an equivalent of C<-Ilib> argument of
C<raku> command.

For more documentation regarding C<prove6> usage refer to
L<its page|https://modules.raku.org/dist/App::Prove6>.

To abort the test suite upon first failure, set the
C<RAKU_TEST_DIE_ON_FAIL> environmental variable:

=for code :lang<shell>
$ RAKU_TEST_DIE_ON_FAIL=1 raku t/test-filename.t

The same variable can be used within the test file. Set it before loading
the C<Test> module:

    BEGIN %*ENV<RAKU_TEST_DIE_ON_FAIL> = 1;
    use Test;
    ...

B<Note:> Before Rakudo version 2020.05 the environment variable
C<PERL6_TEST_DIE_ON_FAIL> was used to enable this feature, it is still
supported but deprecated.

Test timing in microseconds can be emitted by setting the
C<RAKU_TEST_TIMES> environmental variable:

=for code :lang<shell>
$ env RAKU_TEST_TIMES=1 raku -e 'use Test; plan 1; pass sleep(1);'
1..1
# between two timestamps 0 microseconds
ok 1 -
# t=1000721

The same variable can be used within the test file. Set it before loading
the C<Test> module:

=for code
BEGIN %*ENV<RAKU_TEST_TIMES> = 1;
use Test;
...

B<Note:> Before Rakudo version 2020.05 the environment variable
C<PERL6_TEST_TIMES> was used to enable this feature, it is still
supported but deprecated.

=head1 Test plans

Tests plans use L<C<plan>|/type/Test#sub_plan> for declaring how many plans are
going to be done or, as might be the case, skipped. If no plan is declared,
L<C<done-testing>|/type/Test#sub_done-testing> is used to declare the end of the
tests.

=head1 Testing return values

The C<Test> module exports various functions that check the return value of a
given expression and produce standardized test output.

In practice, the expression will often be a call to a function or method that
you want to unit-test. L<C<ok>|/type/Test#sub_ok> and L<C<nok>|/type/Test#sub_nok> will
match C<True> and C<False>. However, where possible it's better to use one of
the specialized comparison test functions below, because they can print more
helpful diagnostics output in case the comparison fails.

=head2 By string comparison

L<C<is>|/type/Test#sub_is> and L<C<isnt>|/type/Test#sub_isnt> test for equality using the
proper operator, depending on the object (or class) it's handled.

=head2 By approximate numeric comparison

L<C<is-approx>|/type/Test#sub_is-approx> compares numbers with a certain precision,
which can be absolute or relative. It can be useful for numeric values whose
precision will depend on the internal representation.

=head2 By structural comparison

Structures can be also compared using L<C<is-deeply>|/type/Test#sub_is-deeply>,
which will check that internal structures of the objects compared is the same.

=head2 By arbitrary comparison

You can use any kind of comparison with L<C<cmp-ok>|/type/Test#sub_cmp-ok>, which
takes as an argument the function or operator that you want to be used for
comparing.

=head2 By object type

L<C<isa-ok>|/type/Test#sub_isa-ok> tests whether an object is of a certain type.


=head2 By method name

L<C<can-ok>|/type/Test#sub_can-ok> is used on objects to check whether they have
that particular method.

=head2 By role

=item X<does-ok($variable, $role, $description?)|does-ok>

L<C<does-ok>|/type/Test#sub_does-ok> checks whether the given variable can do a
certain L<Role|/language/objects#Roles>.

=head2 By regex

L<C<like>|/type/Test#sub_like> and L<C<unlike>|/type/Test#sub_unlike> check using
regular expressions; in the first case passes if a match exists, in the second
case when it does not.

=head1 Testing modules

Modules are tentatively loaded with L<C<use-ok>|/type/Test#sub_use-ok>, which fails
if they fail to load.

=head1 Testing exceptions

L<C<dies-ok>|/type/Test#sub_dies-ok> and L<C<lives-ok>|/type/Test#sub_lives-ok> are
opposites ways of testing code; the first checks that it throws an exception,
the second that it does not; L<C<throws-like>|/type/Test#sub_throws-like> checks
that the code throws the specific exception it gets handed as an argument;
L<C<fails-like>|/type/Test#sub_fails-like>, similarly, checks if the code returns a
specific type of L<Failure|/type/Failure>. L<C<eval-dies-ok>|/type/Test#sub_eval-dies-ok> and
L<C<eval-lives-ok>|/type/Test#sub_eval-lives-ok> work similarly on strings that are
evaluated prior to testing.

=head1 Grouping tests

The result of a group of subtests is only C<ok> if all subtests are C<ok>; they
are grouped using L<C<subtest>|/type/Test#sub_subtest>.


=head1 Skipping tests

Sometimes tests just aren't ready to be run, for instance a feature might not
yet be implemented, in which case tests can be marked as
L<C<todo>|/type/Test#sub_todo>. Or it could be the case that a given feature only
works on a particular platform - in which case one would
L<C<skip>|/type/Test#sub_skip> the test on other platforms;
L<C<skip-rest>|/type/Test#sub_skip-rest> will skip the remaining tests instead of a
particular number given as argument; L<C<bail-out>|/type/Test#sub_bail-out> will
simply exit the tests with a message.

=head1 Manual control

If the convenience functionality documented above does not suit your needs, you
can use the following functions to manually direct the test harness output;
L<C<pass>|/type/Test#sub_pass> will say a test has passed, and
L<C<diag>|/type/Test#sub_diag> will print a (possibly) informative message.

=end pod

# vim: expandtab softtabstop=4 shiftwidth=4 ft=raku
