NAME

Benchmark::Dumb - Benchmark.pm compatibility layer for Dumbbench

SYNOPSIS

  use Benchmark::Dumb qw(:all);
  cmpthese(
    0.05, # 5% precision
    {
      fast => 'fast code',
      slow => 'slow code',
    }
  );
  # etc

DESCRIPTION

This module implements an interface that is similar to the functional interface of the Benchmark module. This module, however, uses the Dumbbench benchmarking tool under the hood. For various reasons, the interface and the output of the benchmark runs are not exactly the same. Among other reasons, you would lose out on some of "Dumbbench"'s advantages.

Understanding this documentation requires some familiarity of how "Benchmark.pm" works since it mostly explains how this module is different.

Please read the following section carefully to understand the most important differences:

Differences to Benchmark.pm

This is a list of differences to the interface and behaviour of the "Benchmark" module. It may not be complete. If so, please let me know.

The $count parameter is interpreted very differently!
With "Benchmark.pm", specifying a positive integer meant that the benchmark should be run exactly $count times. A negative value indicated that the code should be run until $count seconds of cumulated run-time have elapsed.

With "Benchmark::Dumb", we can do better. A positive integer specifies the minimum number of iterations. "Dumbbench" may choose to run more iterations to arrive at the necessary precision.

Specifying a certain target run-time (via a negative number for $count) may seem like a tempting idea, but if you care at all about the precision of your result, it's quite useless. This usage is not supported by "Benchmark::Dumb"!

Instead, if you pass a positive floating point number as $count, the fractional part of the number willbe interpreted as the target relative precision that you expect from the result.

Finally, supplying a 0 as $count means that "Dumbbench" will be invoked with the default settings. This is good enough for most cases.
There are no exported functions by default!
The ":hireswallclock" option is ignored. We always use the hi-res wallclock! While on the topic: We also only use wallclock times.
The cache-related functions aren't implemented because we don't use a cache.
The original "Benchmark.pm" implementation provides a rudimentary object-oriented interface. We do not faithfully copy that. See "METHODS" below.
The benchmark code will be run in a special package. It will not be run in the caller package (at this time). If you need access to previously-set-up package variables, you will need to include a "package" statement in your code.
The "debug" method is not implemented and neither is the "countit" function.
Some things that were previously considered functions are now considered primarily methods (see "METHODS" below). But they are all importable and callable as functions.

FUNCTIONS

These functions work mostly (see the $count gotcha above) like the equivalent functions in the "Benchmark" module, but the textual output is different in that it contains estimates of the uncertainties. Some of the style and format options of the original functions are ignored for the time being.

I'm quoting the "Benchmark" documentation liberally.

timethis(COUNT, CODE, [TITLE, [STYLE]])

Time COUNT iterations of CODE. CODE may be a string to eval or a code reference. Unlike with the original "Benchmark", the code will not run in the caller's package. Results will be printed to "STDOUT" as TITLE followed by the "timestr()". TITLE defaults to "timethis COUNT" if none is provided.

STYLE determines the format of the output, as described for "timestr()" below.

Please read the section on "Differences to Benchmark.pm" for a discussion of how the COUNT parameter is interpreted.

Returns a "Benchmark::Dumb" object.

timeit(COUNT, CODE)

Arguments: COUNT is the number of times to run the loop (see discussion above), and CODE is the code to run. CODE may be either a code reference or a string to be eval'd. Unlike with the original "Benchmark", the code will not run in the caller's package.

Returns a "Benchmark::Dumb" object.

timethese(COUNT, CODEHASHREF, [STYLE])

The CODEHASHREF is a reference to a hash containing names as keys and either a string to eval or a code reference for each value. For each (KEY, VALUE) pair in the CODEHASHREF, this routine will call

  timethis(COUNT, VALUE, KEY, STYLE)

The routines are called in string comparison order of KEY.

The COUNT must be positive or zero. See discussion above.

Returns a hash reference of "Benchmark::Dumb" objects, keyed by name.

cmpthese(COUNT, CODEHASHREF, [STYLE]) cmpthese(RESULTSHASHREF, [STYLE])

Optionally calls "timethese()", then outputs a comparison chart. This:

  cmpthese( 500.01, { a => "++\$i", b => "\$i *= 2" } ) ;

outputs a chart like:

                 Rate/s Precision/s       a       b
        a -3.59e+07       6e+06      -- -535.3%
        b  8.24e+06      220000 -123.0%      --

This chart is sorted from slowest to fastest, and shows the percent speed difference between each pair of tests as well as the uncertainties on the rates and the relative speed difference. The uncertainty on a speed difference may be omitted if it is below one tenth of a percent.

c<cmpthese> can also be passed the data structure that "timethese()" returns:

  my $results = timethese( 100.01, { a => "++\$i", b => "\$i *= 2" } ) ;
  cmpthese( $results );

in case you want to see both sets of results. If the first argument is an unblessed hash reference, that is "RESULTSHASHREF"; otherwise that is "COUNT".

Returns a reference to an ARRAY of rows, each row is an ARRAY of cells from the above chart, including labels. This:

  my $rows = cmpthese( 500.01, { a => '++$i', b => '$i *= 2' }, "none" );

returns a data structure like:

        [
          [ '', 'Rate/s', 'Precision/s', 'a', 'b' ],
          [ 'a', '-4.43e+06', '120000', '--', '93+-6.9%' ],
          [ 'b', '-2.294e+06', '52000', '-48.2%', '--' ]
        ];

METHODS

Please note that while the original "Benchmark" objects practically asked for manual introspection since the API didn't provide convenient access to all information, that practice is frowned upon with "Benchmark::Dumb" objects. You have been warned. If there's a piece of API missing, let me know.

There's no public constructor for "Benchmark::Dumb" objects because it doesn't do what the "Benchmark" constructor did: It's not running "time()" for you.

name()

Returns the name of the benchmark result if any. (Not in "Benchmark".)

iters()

Returns the number of samples taken.

timesum($other_benchmark)

Returns a new "Benchmark::Dumb" object that represents the sum of this benchmark and $other_benchmark.

"timesum($b1, $b2)" was a function in the original "Benchmark" module and may be called as a function on two "Benchmark::Dumb" objects as well. It is available for importing into your namespace.

timediff($other_benchmark)

Returns a new "Benchmark::Dumb" object that represents the difference between this benchmark and $other_benchmark ("$this-$other").

"timediff($b1, $b2)" was a function in the original "Benchmark" module and may be called as a function on two "Benchmark::Dumb" objects as well. It is available for importing into your namespace.

timestr()

Returns a textual representation of this benchmark.

"timestr($b)" was a function in the original "Benchmark" module and may be called as a function on a "Benchmark::Dumb" object as well. It is available for importing into your namespace.

AUTHOR

Steffen Mueller, <smueller@cpan.org>

COPYRIGHT AND LICENSE

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.1 or, at your option, any later version of Perl 5 you may have available.