add(LIST)

On the one hand, this may indicate that the patterns you are trying to feed the "R::A" object are too complex. Simpler patterns might allow the algorithm to work more effectively and perform more reductions in the resulting pattern.

On the other hand, you can supply your own pattern to perform the lexing if you need. The test suite contains an example of a lexer pattern that will match one level of nested parentheses.

Note that there is an internal optimisation that will bypass a much of the lexing process. If a string contains no "\" (backslash), "[" (open square bracket), "(" (open paren), "?" (question mark), "+" (plus), "*" (star) or "{" (open curly), a character split will be performed directly.

A list of strings may be supplied, thus you can pass it a file handle of a file opened for reading:

    $re->add( '\d+-\d+-\d+-\d+\.example\.com' );
    $re->add( <IN> );

If the file is very large, it may be more efficient to use a "while" loop, to read the file line-by-line:

    $re->add($_) while <IN>;

The "add" method will chomp the lines automatically. If you do not want this to occur (you want to keep the record separator), then disable "chomp"ing.

    $re->chomp(0);
    $re->add($_) while <IN>;

This method is chainable.

add_file(FILENAME [...])

  $r->add_file( 'file.1', 'file.2' );

If a file cannot be opened, the method will croak. If you cannot afford to let this happen then you should wrap the call in a "eval" block.

Chomping happens automatically unless you the chomp(0) method to disable it. By default, input lines are read according to the value of the "input_record_separator" attribute (if defined), and will otherwise fall back to the current setting of the system $/ variable. The record separator may also be specified on each call to "add_file". Internally, the routine "local"ises the value of $/ to whatever is required, for the duration of the call.

An alternate calling mechanism using a hash reference is available. The recognised keys are:

file

Reference to a list of file names, or the name of a single file.

  $r->add_file({file => ['file.1', 'file.2', 'file.3']});
  $r->add_file({file => 'file.n'});
    

input_record_separator

If present, indicates what constitutes a line

  $r->add_file({file => 'data.txt', input_record_separator => ':' });
    

rs

An alias for input_record_separator (mnemonic: same as the English variable names).

  $r->add_file( {
    file => [ 'pattern.txt', 'more.txt' ],
    input_record_separator  => "\r\n",
  });

clone()

If the Storable module is installed, its dclone method will be used, otherwise the cloning will be performed using a pure perl approach.

You can use this method to take a snapshot of the patterns that have been added so far to an object, and generate an assembly from the clone. Additional patterns may to be added to the original object afterwards.

  my $re = $main->clone->re();
  $main->add( 'another-pattern-\\d+' );

insert(LIST)

This method is chainable, e.g.:

  my $ra = Regexp::Assemble->new
    ->insert( qw[ a b+ c? d* e ] )
    ->insert( qw[ a c+ d+ e* f ] );

Lexing complex patterns with metacharacters and so on can consume a significant proportion of the overall time to build an assembly. If you have the information available in a tokenised form, calling "insert" directly can be a big win.

lexstr

  my @token = $re->lexstr($pattern);
  my $new_pattern = join( '', @token );

If the original pattern contains unnecessary backslashes, or "\x4b" escapes, or quotemeta escapes ("\Q"..."\E") the resulting pattern may not be identical.

Call "lexstr" does not add the pattern to the object, it is merely for exploratory purposes. It will, however, update various statistical counters.

pre_filter(CODE)

A filter to strip out trailing comments (marked by #):

  $re->pre_filter( sub { $_[0] =~ s/\s*#.*$//; 1 } );

A filter to ignore blank lines:

  $re->pre_filter( sub { length(shift) } );

If you want to remove the filter, pass "undef" as a parameter.

  $ra->pre_filter(undef);

This method is chainable.

filter(CODE)

If you know that all patterns you expect to assemble contain a restricted set of of tokens (e.g. no spaces), you could do the following:

  $ra->filter(sub { not grep { / / } @_ });

or

  sub only_spaces_and_digits {
    not grep { ![\d ] } @_
  }
  $ra->filter( \&only_spaces_and_digits );

These two examples will silently ignore faulty patterns, If you want the user to be made aware of the problem you should raise an error (via "warn" or "die"), log an error message, whatever is best. If you want to remove a filter, pass "undef" as a parameter.

  $ra->filter(undef);

This method is chainable.

as_string

indent, the number of spaces used to indent nested grouping of a pattern. Use this to produce a pretty-printed pattern (for some definition of "pretty"). The resulting output is rather verbose. The reason is to ensure that the metacharacters "(?:" and ")" always occur on otherwise empty lines. This allows you grep the result for an even more synthetic view of the pattern:

  egrep -v '^ *[()]' <regexp.file>

The result of the above is quite readable. Remember to backslash the spaces appearing in your own patterns if you wish to use an indented pattern in an "m/.../x" construct. Indenting is ignored if tracking is enabled.

The indent argument takes precedence over the "indent" method/attribute of the object.

Calling this method will drain the internal data structure. Large numbers of patterns can eat a significant amount of memory, and this lets perl recover the memory used for other purposes.

If you want to reduce the pattern and continue to add new patterns, clone the object and reduce the clone, leaving the original object intact.

re

As with "as_string", calling this method will reset the internal data structures to free the memory used in assembling the RE.

The indent attribute, documented in the "as_string" method, can be used here (it will be ignored if tracking is enabled).

With method chaining, it is possible to produce a RE without having a temporary "Regexp::Assemble" object lying around, e.g.:

  my $re = Regexp::Assemble->new
    ->add( q[ab+cd+e] )
    ->add( q[ac\\d+e] )
    ->add( q[c\\d+e] )
    ->re;

The $re variable now contains a Regexp object that can be used directly:

  while( <> ) {
    /$re/ and print "Something in [$_] matched\n";
  )

The "re" method is called when the object is used in string context (hence, within an "m//" operator), so by and large you do not even need to save the RE in a separate variable. The following will work as expected:

  my $re = Regexp::Assemble->new->add( qw[ fee fie foe fum ] );
  while( <IN> ) {
    if( /($re)/ ) {
      print "Here be giants: $1\n";
    }
  }

This approach does not work with tracked patterns. The "match" and "matched" methods must be used instead, see below.

match(SCALAR)

If pattern tracking is in use, you must "use re 'eval'" in order to make things work correctly. At a minimum, this will make your code look like this:

    my $did_match = do { use re 'eval'; $target =~ /$ra/ }
    if( $did_match ) {
        print "matched ", $ra->matched, "\n";
    }

(The main reason is that the $^R variable is currently broken and an ugly workaround that runs some Perl code during the match is required, in order to simulate what $^R should be doing. See Perl bug #32840 for more information if you are curious. The README also contains more information). This bug has been fixed in 5.10.

The important thing to note is that with "use re 'eval'", THERE ARE SECURITY IMPLICATIONS WHICH YOU IGNORE AT YOUR PERIL. The problem is this: if you do not have strict control over the patterns being fed to "Regexp::Assemble" when tracking is enabled, and someone slips you a pattern such as "/^(?{system 'rm -rf /'})/" and you attempt to match a string against the resulting pattern, you will know Fear and Loathing.

What is more, the $^R workaround means that that tracking does not work if you perform a bare "/$re/" pattern match as shown above. You have to instead call the "match" method, in order to supply the necessary context to take care of the tracking housekeeping details.

   if( defined( my $match = $ra->match($_)) ) {
       print "  $_ matched by $match\n";
   }

In the case of a successful match, the original matched pattern is returned directly. The matched pattern will also be available through the "matched" method.

(Except that the above is not true for 5.6.0: the "match" method returns true or undef, and the "matched" method always returns undef).

If you are capturing parts of the pattern e.g. "foo(bar)rat" you will want to get at the captures. See the "mbegin", "mend", "mvar" and "capture" methods. If you are not using captures then you may safely ignore this section.

In 5.10, since the bug concerning $^R has been resolved, there is no need to use "re 'eval'" and the assembled pattern does not require any Perl code to be executed during the match.

new()

anchor_*, a family of optional attributes that allow anchors ("^", "\b", "\Z"...) to be added to the resulting pattern.

flags, sets the "imsx" flags to add to the assembled regular expression. Warning: no error checking is done, you should ensure that the flags you pass are understood by the version of Perl you are using. modifiers exists as an alias, for users familiar with Regexp::List.

chomp, controls whether the pattern should be chomped before being lexed. Handy if you are reading patterns from a file. By default, "chomp"ing is performed (this behaviour changed as of version 0.24, prior versions did not chomp automatically). See also the "file" attribute and the "add_file" method.

file, slurp the contents of the specified file and add them to the assembly. Multiple files may be processed by using a list.

  my $r = Regexp::Assemble->new(file => 're.list');

  my $r = Regexp::Assemble->new(file => ['re.1', 're.2']);

If you really don't want chomping to occur, you will have to set the "chomp" attribute to 0 (zero). You may also want to look at the "input_record_separator" attribute, as well.

input_record_separator, controls what constitutes a record separator when using the "file" attribute or the "add_file" method. May be abbreviated to rs. See the $/ variable in perlvar.

lookahead, controls whether the pattern should contain zero-width lookahead assertions (For instance: (?=[abc])(?:bob|alice|charles). This is not activated by default, because in many circumstances the cost of processing the assertion itself outweighs the benefit of its faculty for short-circuiting a match that will fail. This is sensitive to the probability of a match succeeding, so if you're worried about performance you'll have to benchmark a sample population of targets to see which way the benefits lie.

track, controls whether you want know which of the initial patterns was the one that matched. See the "matched" method for more details. Note for version 5.8 of Perl and below, in this mode of operation YOU SHOULD BE AWARE OF THE SECURITY IMPLICATIONS that this entails. Perl 5.10 does not suffer from any such restriction.

indent, the number of spaces used to indent nested grouping of a pattern. Use this to produce a pretty-printed pattern. See the "as_string" method for a more detailed explanation.

pre_filter, allows you to add a callback to enable sanity checks on the pattern being loaded. This callback is triggered before the pattern is split apart by the lexer. In other words, it operates on the entire pattern. If you are loading patterns from a file, this would be an appropriate place to remove comments.

filter, allows you to add a callback to enable sanity checks on the pattern being loaded. This callback is triggered after the pattern has been split apart by the lexer.

unroll_plus, controls whether to unroll, for example, "x+" into "x", "x*", which may allow additional reductions in the resulting assembled pattern.

reduce, controls whether tail reduction occurs or not. If set, patterns like "a(?:bc+d|ec+d)" will be reduced to "a[be]c+d". That is, the end of the pattern in each part of the b... and d... alternations is identical, and hence is hoisted out of the alternation and placed after it. On by default. Turn it off if you're really pressed for short assembly times.

lex, specifies the pattern used to lex the input lines into tokens. You could replace the default pattern by a more sophisticated version that matches arbitrarily nested parentheses, for example.

debug, controls whether copious amounts of output is produced during the loading stage or the reducing stage of assembly.

  my $ra = Regexp::Assemble->new;
  my $rb = Regexp::Assemble->new( chomp => 1, debug => 3 );

mutable, controls whether new patterns can be added to the object after the assembled pattern is generated. DEPRECATED.

This method/attribute will be removed in a future release. It doesn't really serve any purpose, and may be more effectively replaced by cloning an existing "Regexp::Assemble" object and spinning out a pattern from that instead.

source()

If no successful match has been performed, or the object is not in tracked mode, this method returns "undef".

  my $r = Regexp::Assemble->new->track(1)->add(qw(foo? bar{2} [Rr]at));

  for my $w (qw(this food is rather barren)) {
    if ($w =~ /$r/) {
      print "$w matched by ", $r->source($^R), $/;
    }
    else {
      print "$w no match\n";
    }
  }

mbegin()

mend()

mvar(NUMBER)

If called without a parameter, "mvar" will return a reference to an array containing all captures.

capture

If no captures were found in the match, an empty array is returned, rather than "undef". You are therefore guaranteed to be able to use "for my $c ($re->capture) { ..." without have to check whether anything was captured.

matched()

See below in the NOTES section for additional subtleties of which you should be aware of when tracking patterns.

Note that this method is not available in 5.6.0, due to limitations in the implementation of "(?{...})" at the time.

Statistics/Reporting routines

stats_add

stats_dup

stats_raw()

stats_cooked()

stats_length()

dup_warn(NUMBER|CODEREF)

  $r->dup_warn();

The method may also be passed a code block. In this case the code will be executed and it will receive a reference to the object in question, and the lexed pattern.

  $r->dup_warn(
    sub {
      my $self = shift;
      print $self->stats_add, " patterns added at line $.\n",
          join( '', @_ ), " added previously\n";
    }
  )

Anchor routines

  $r->add(qw(^this ^that ^them))->as_string;

  $r->add(qw(this that them))->anchor_line_begin->as_string;

  # both techniques will produce ^th(?:at|em|is)

All anchors are possible word ("\b") boundaries, line boundaries ("^" and "$") and string boundaries ("\A" and "\Z" (or "\z" if you absolutely need it)).

The shortcut "anchor_mumble" implies both "anchor_mumble_begin" "anchor_mumble_end" is also available. If different anchors are specified the most specific anchor wins. For instance, if both "anchor_word_begin" and "anchor_line_begin" are specified, "anchor_word_begin" takes precedence.

All the anchor methods are chainable.

anchor_word_begin

  $r->add('pre')->anchor_word_begin->as_string;
  # produces '\bpre'

anchor_word_end

  $r->add(qw(ing tion))
    ->anchor_word_end
    ->as_string; # produces '(?:tion|ing)\b'

anchor_word

  $r->add(qw(cat carrot)
    ->anchor_word(1)
    ->as_string; # produces '\bca(?:rro)t\b'

anchor_line_begin

  $r->anchor_line_begin;
  # or
  $r->anchor_line_begin(1);

anchor_line_end

  # turn it off
  $r->anchor_line_end(0);

anchor_line

  $r->add(qw(cat carrot)
    ->anchor_line
    ->as_string; # produces '^ca(?:rro)t$'

anchor_string_begin

  $r->anchor_string_begin(1);

anchor_string_end

  # disable the string boundary end anchor
  $r->anchor_string_end(0);

anchor_string_end_absolute

  # disable the string boundary absolute end anchor
  $r->anchor_string_end_absolute(0);

If you don't understand the difference between "\Z" and "\z", the former will probably do what you want.

anchor_string

  $r->add(qw(cat carrot)
    ->anchor_string
    ->as_string; # produces '\Aca(?:rro)t\Z'

anchor_string_absolute

  $r->add(qw(cat carrot)
    ->anchor_string_absolute
    ->as_string; # produces '\Aca(?:rro)t\z'

debug(NUMBER)

• Off. Turns off all debugging output.
• 1

  Add. Trace the addition of patterns.

• 2

  Reduce. Trace the process of reduction and assembly.

• 4

  Lex. Trace the lexing of the input patterns into its constituent tokens.

• 8

  Time. Print to STDOUT the time taken to load all the patterns. This is nothing more than the difference between the time the object was instantiated and the time reduction was initiated.

    # load=<num>
      

  Any lengthy computation performed in the client code will be reflected in this value. Another line will be printed after reduction is complete.

    # reduce=<num>
      

  The above output lines will be changed to "load-epoch" and "reduce-epoch" if the internal state of the object is corrupted and the initial timestamp is lost.

  The code attempts to load Time::HiRes in order to report fractional seconds. If this is not successful, the elapsed time is displayed in whole seconds.

Values can be added (or or'ed together) to trace everything

  $r->debug(7)->add( '\\d+abc' );

Calling "debug" with no arguments turns debugging off.

dump()

  print $r->dump;

chomp(0|1)

IMPORTANT: As of version 0.24, chomping is now on by default as it makes "add_file" Just Work. The only time you may run into trouble is with "add("\\$/")". So don't do that, or else explicitly turn off chomping.

To avoid incorporating (spurious) record separators (such as "\n" on Unix) when reading from a file, "add()" "chomp"s its input. If you don't want this to happen, call "chomp" with a false value.

  $re->chomp(0); # really want the record separators
  $re->add(<DATA>);

fold_meta_pairs(NUMBER)

Call this method with a false value to prevent this behaviour (which is only a problem when dealing with "\n" if the "/s" expression modifier is also set).

  $re->add( '\\w', '\\W' );
  my $clone = $re->clone;

  $clone->fold_meta_pairs(0);
  print $clone->as_string; # prints '.'
  print $re->as_string;    # print '[\W\w]'

indent(NUMBER)

  $re->indent( 4 );
  print $re->as_string;

lookahead(0|1)

flags(STRING)

modifiers(STRING)

track(0|1)

  $re->track( 1 );
  if( $target =~ /$re/ ) {
    print "$target matched by ", $re->matched, "\n";
  }

Note that when this functionality is enabled, no reduction is performed and no character classes are generated. In other words, "brag|tag" is not reduced down to "(?:br|t)ag" and "dig|dim" is not reduced to "di[gm]".

unroll_plus(0|1)

lex(SCALAR)

reduce(0|1)

mutable(0|1)

reset()

Default_Lexer

The "Default_Lexer" method lets you replace the default pattern used for all subsequently created "Regexp::Assemble" objects. It will not have any effect on existing objects. (It is also possible to override the lexer pattern used on a per-object basis).

The parameter should be an ordinary scalar, not a compiled pattern. If the pattern fails to match all parts of the string, the missing parts will be returned as single chunks. Therefore the following pattern is legal (albeit rather cork-brained):

    Regexp::Assemble::Default_Lexer( '\\d' );

The above pattern will split up input strings digit by digit, and all non-digit characters as single chunks.