NAME

VERSION

SYNOPSIS

    use Test::Aggregate;

    my $tests = Test::Aggregate->new( {
        dirs => $aggregate_test_dir,
    } );
    $tests->run;

    ok $some_data, 'Test::Aggregate also re-exports Test::More functions';

DESCRIPTION

A common problem with many test suites is that they can take a long time to run. The longer they run, the less likely you are to run the tests. This module borrows a trick from "Apache::Registry" to load up your tests at once, create a separate package for each test and wraps each package in a method named "run_the_tests". This allows us to load perl only once and related modules only once. If you have modules which are expensive to load, this can dramatically speed up a test suite.

DEPRECATION

 BEGIN {
     use_ok 'My::Module' or die;
 }

Instead, just "use" the module and put the "use_ok" tests in a t/load.t file or something similar and don't aggregate it. See the following for more information: <http://use.perl.org/~Ovid/journal/38974>.

USAGE

 use Test::Aggregate;
 my $other_test_dir = 'aggregate_tests';
 my $tests = Test::Aggregate->new( {
    dirs => $other_test_dir
 });
 $tests->run;

 ok $some_data, 'Test::Aggregate also re-exports Test::More functions';

Take your simplest tests and move them, one by one, into the new test directory and keep running the "Test::Aggregate" program. You'll find some tests will not run in a shared environment like this. You can either fix the tests or simply leave them in your regular test directory. See how this distribution's tests are organized for an example.

Note that "Test::Aggregate" also exports all exported functions from "Test::More", allowing you to run other tests after the aggregated tests have run.

 use Test::Aggregate;
 my $other_test_dir = 'aggregate_tests';
 my $tests = Test::Aggregate->new( {
    dirs => $other_test_dir
 });
 $tests->run;
 ok !(-f 't/data/tmp.txt'), '... and our temp file should be deleted';

Some tests cannot run in an aggregate environment. These may include test for this with the $ENV{TEST_AGGREGATE} variable:

 package Some::Package;

 BEGIN {
     die __PACKAGE__ ." cannot run in aggregated tests"
       if $ENV{TEST_AGGREGATE};
 }

METHODS

 my $tests = Test::Aggregate->new(
     {
         dirs            => 'aggtests',
         verbose         => 1,            # optional, but recommended
         dump            => 'dump.t',     # optional
         shuffle         => 1,            # optional
         matching        => qr/customer/, # optional
         set_filenames   => 0,            # optional and not recommended
         tidy            => 1,            # optional and experimental
         test_nowarnings => 0,            # optional and experimental
     }
 );

 $tests->run;

SETUP/TEARDOWN

 #!/usr/bin/perl
 
 use strict;
 use warnings;
 
 use lib 'lib', 't/lib';
 use Test::Aggregate;
 use Test::More;
 
 my $dump = 'dump.t';
 
 my ( $startup, $shutdown ) = ( 0, 0 );
 my ( $setup,   $teardown ) = ( 0, 0 );
 my $tests = Test::Aggregate->new(
     {
         dirs     => 'aggtests',
         dump     => $dump,
         startup  => sub { $startup++ },
         shutdown => sub { $shutdown++ },
         setup    => sub { $setup++ },
         teardown => sub { $teardown++ },
     }
 );
 $tests->run;
 is $startup,  1, 'Startup should be called once';
 is $shutdown, 1, '... as should shutdown';
 is $setup,    4, 'Setup should be called once for each test program';
 is $teardown, 4, '... as should teardown';

Note that you can still dump these to a dump file. This will only work if "Data::Dump::Streamer" 1.11 or later is installed.

There are four attributes which can be passed to the constructor, each of which expects a code reference:

• "startup"

   startup => \&connect_to_database,
      

  This function will be called before any of the tests are run. It is not run in a BEGIN block.

• "shutdown"

   shutdown => \&clean_up_temp_files,
      

  This function will be called after all of the tests are run. It will not be called in an END block.

• "setup"

   setup => sub { 
      my $filename = shift;
      # this gets run before each test program.
   },
      

  The setup function will be run before every test program. The name of the test file will be passed as the first argument.

• "teardown"

   teardown => sub {
      my $filename = shift;
      # this gets run after every test program.
   }
      

  The teardown function gets run after every test program. The name of the test file will be passed as the first argument.

GLOBAL VARIABLES

    local %ENV = %ENV;

The following global variables are localized for you. Any others must be localized manually per test.

• @INC
• %ENV
• %SIG
• $/
• $_
• $|

CAVEATS

• "exit"

  Don't call exit() in your aggregated tests. We now warn very verbosely if this is done, but we still exit on the assumption that further tests cannot run.

• "__END__" and "__DATA__" tokens.

  These won't work and the tests will call BAIL_OUT() if these tokens are seen. However, this limitation does not apply to Test::Aggregate::Nested.

• "BEGIN" and "END" blocks.

  Since all of the tests are aggregated together, "BEGIN" and "END" blocks will be for the scope of the entire set of aggregated tests. If you need setup/teardown facilities, see "TEARDOWN" in SETUP.

• Syntax errors

  Any syntax errors encountered will cause this program to BAIL_OUT(). This is why it's recommended that you move your tests into your new directory one at a time: it makes it easier to figure out which one has caused the problem.

• "no_plan"

  Unfortunately, due to how this works, the plan is always "no_plan". <http://groups.google.com/group/perl.qa/browse_thread/thread/d58c49db734844f4/cd18996391acc601?#cd18996391acc601> for more information.

• "Test::NoWarnings"

  Great module. It loves to break aggregated tests since some might have warnings when others will not. You can disable it like this:

   my $tests = Test::Aggregate->new(
       dirs    => 'aggtests/',
       startup => sub { $INC{'Test/NoWarnings.pm'} = 1 },
   );
      

  As an alternative, you can also disable it with:

   my $tests = Test::Aggregate->new({
      dirs            => 'aggtests',
      test_nowarnings => 0,
   });
      

  We do work internally to subtract the extra test added by "Test::NoWarnings". It's painful and experimental. Good luck.

• "Variable "$x" will not stay shared at (eval ..."

  Because each test is wrapped in a method call, any of your subs which access a variable in an outer scope will likely throw the above warning. Pass in arguments explicitly to suppress this.

  Instead of:

   my $x = 17;
   sub foo {
       my $y = shift;
       return $y + $x;
   }
      

  Write this:

   my $x = 17;
   sub foo {
       my ( $y, $x ) = @_;
       return $y + $x;
   }
      

  However, consider Test::Aggregate::Nested. This warning does not apply with that module.

• Singletons

  Be very careful of code which loads singletons. Oftimes those singletons in test suites may be altered for testing purposes, but later attempts to use those singletons can fail dramatically as they're not expecting the alterations. (Your author has painfully learned this lesson with database connections).

DEBUGGING AGGREGATE TESTS

Debugging aggregated tests which fail is a multi-step process. Let's say the following fails:

 my $tests = Test::Aggregate->new(
     {
         dump    => 'dump.t',
         shuffle => 1,
         dirs    => 'aggtests',
     }
 );
 $tests->run;

 prove -r aggtests/

 prove -r $dumpfile

 if ( __FILE__ eq 'dump.t' ) {
     Test::More::diag("******** running tests for aggtests/boilerplate.t ********")
        if $ENV{TEST_VERBOSE};
     aggtestsboilerplatet->run_the_tests;

     Test::More::diag("******** running tests for aggtests/subs.t ********")
        if $ENV{TEST_VERBOSE};
     aggtestssubst->run_the_tests;

     Test::More::diag("******** running tests for aggtests/00-load.t ********")
        if $ENV{TEST_VERBOSE};
     aggtests00loadt->run_the_tests;

     Test::More::diag("******** running tests for aggtests/slow_load.t ********")
        if $ENV{TEST_VERBOSE};
     aggtestsslow_loadt->run_the_tests;
 }

COMMON PITFALLS

  $server->ip_address('apple');

 DESTROY {
    local $@ = $@;  # localize but preserve the value
    my $self = shift;
    # do whatever you want
 }

 $INC{'Test/NoWarnings.pm'} = 1;

AUTHOR

BUGS

SUPPORT

    perldoc Test::Aggregate

You can also find information oneline:

<http://metacpan.org/release/Test-Aggregate>

ACKNOWLEDGEMENTS

Thanks to Johan Lindström for pointing me to Apache::Registry.

COPYRIGHT & LICENSE

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.