Blocking Exceptions

This behavior can be disabled totally using "dont_catch_exceptions". Exceptions can also be trapped as usual using "eval" in perlfunc or otherwise from inside a group, in which case the test code of course has full control on what to do next (this is how one should test error management, by the way).

When Test::Group is set to block errors (the default setting, see also "catch_exceptions"), the error messages are displayed as part of the test name, which some may not find very readable. Therefore, one can use a "logfile" instead.

Skipping Groups

Reflexivity

    use Test::Builder;
    use Test::Group;

    sub foobar_ok {
        my ($text, $name) = @_;
        $name ||= "foobar_ok";
        local $Test::Builder::Level = $Test::Builder::Level + 1;
        test $name => sub {
           local $Test::Group::InPredicate = 1;
           like($text, qr/foo/, "foo ok");
           like($text, qr/bar/, "bar ok");
        };
    }

defines a new test predicate foobar_ok that will DWIM regardless of the caller's testing style: for "classical" Test::Simple or Test::More users, foobar_ok will act as just another *_ok predicate (in particular, it always counts for a single test, honors "TODO: BLOCK" in Test::More constructs, etc); and of course, users of Test::Group can freely call foobar_ok from within a group.

Adding 1 to $Test::Builder::Level causes the location of the call to foobar_ok() to be shown if a test fails, see Test::Builder.

Setting $Test::Group::InPredicate to a true value prevents the location of individual failing tests within test groups from being shown.

TODO Tests

   test "something" => sub {
       local $TODO = "this test does not work yet";
       pass;                                         # GOTCHA!
       fail;
   };

Here "pass" is an unexpected success, and therefore the whole test group will report a TODO success despite the test not actually being a success (that is, it would also be defective if one were to comment out the "local $TODO" line). This semantics, on the other hand, DWIMs for marking a portion of the test group as TODO:

   test "something" => sub {
       pass;
       {
          local $TODO = "this part does not work yet";
          fail;
       }
   };

Finally, there is a subtle gotcha to be aware of when setting $TODO outside a test group (that's the second one, so maybe you should not do that to begin with). In this case, the value of $TODO is set to undef inside the group. In other words, this test (similar to the one to be found in "SYNOPSIS") will succeed as expected:

    {
      local $TODO = "not quite done yet";
      test "foo" => sub {
          fail;
          pass;              # NOT an unexpected success, as
                             # this is simply a subtest of the whole
                             # test "foo", which will fail.
      };
    }

OUTPUT FORMAT

However, if you enable the experimental "use_subtest" feature then Test::Group will use the same underlying mechanism as "subtest" in Test::More and produce very similar output.

PLUGIN INTERFACE

The following extensions are distributed with Test::Group:

Test::Group::Plan, Test::Group::NoWarnings

FUNCTIONS

test ($name, $groupsub)

Executes $groupsub, which must be a reference to a subroutine, in a controlled environment and groups the results of all Test::Builder-style subtests launched inside into a single call to "ok" in Test::Builder, regardless of their number. If the test group is to be skipped (as discussed in "Skipping Groups"), calls "skip" in Test::Builder once instead.

In case the test group is not skipped, the first parameter to "ok" in Test::Builder and the value of the TODO string during same (see "TODO: BLOCK" in Test::More) are determined according to the following algorithm:

1.

if the test group terminates by throwing an exception, or terminates normally but without calling any subtest, it fails.

2.

otherwise, if any subtest failed outside of a TODO block, the group fails.

3.

otherwise, if any subtest succeeds inside of a TODO block, the group is flagged as an unexpected success.

4.

otherwise, if any subtest fails inside of a TODO block, the group results in a TODO (excused) failure.

5.

otherwise, the test group managed to avert all hazards and is a straight success (tada!!).

If any sub-tests failed in $groupsub, diagnostics will be propagated using "diag" in Test::Builder as usual.

The return value of test is 1 if the test group is a success (including a TODO unexpected success), 0 if it is a failure (including a TODO excused failure), and undef if the test group was skipped.

skip_next_tests ($number)

skip_next_tests ($number, $reason)

Skips the $number following groups of tests with reason $reason. Dies if we are currently skipping tests already.

skip_next_test ()

skip_next_test ($reason)

Equivalent to:

    skip_next_tests 1;
    skip_next_tests 1, $reason;

begin_skipping_tests ()

    begin_skipping_tests;
    begin_skipping_tests "reason";

Skips all subsequent groups of tests until blocked by "end_skipping_tests".

end_skipping_tests ()

Cancels the effect of "begin_skipping_tests". Has no effect if we are not currently skipping tests.

test_only ()

    test_only "bla()", "reason";
    test_only qr/^bla/;
    test_only sub { /bla/ };

Skip all groups of tests whose name does not match the criteria. The criteria can be a plain string, a regular expression or a function.

    test_only;

Resets to normal behavior.

PLUGIN FUNCTIONS

next_test_plugin ($plugin)

Installs a plugin for the next test group. $plugin must be a subroutine reference.

verbose ($level)

At level 1 and above there is a diagnostic line for the start of each test group.

At level 2 there is a diagnostic line showing the result of each subtest within top-level test groups. At level 3, the subtests of test groups nested within top level test groups also get diagnostic lines, and so on.

The default verbosity level is 0, or the value of the "PERL_TEST_GROUP_VERBOSE" environment variable if it is set.

catch_exceptions ()

Note that catch_exceptions only deals with exceptions arising inside test blocks; those thrown by surrounding code (if any) still cause the test script to terminate as usual unless other appropriate steps are taken.

dont_catch_exceptions ()

logfile ($classstate_logfile)

use_subtest ()

Causes Test::Group to use Test::Builder's subtest() feature as the test aggregation method, rather than doing black magic behind the scenes.

It is a fatal error to call use_subtest() if Test::Builder is too old to support subtests. To use subtests if they are available but fall back to normal operation if they are not, you can do:

  eval { Test::Group->use_subtest };

Test::Group's exception handling mechanism is bypassed under use_subtest(), since Test::Builder::subtest() has its own exception handling system.

no_subtest ()

Similar modules on CPAN