"new($source)"

  my $parser = HTML::TokeParser::Simple->new($filename);
  # or
  my $parser = HTML::TokeParser::Simple->new($filehandle);
  # or
  my $parser = HTML::TokeParser::Simple->new(\$html_string);

"new($source_type, $source)"

  my $parser = HTML::TokeParser::Simple->new(file   => $filename);
  # or
  my $parser = HTML::TokeParser::Simple->new(handle => $filehandle);
  # or
  my $parser = HTML::TokeParser::Simple->new(string => $html_string);

Note that you do not have to provide a reference for the string if using the string constructor.

As a convenience, you can also attempt to fetch the HTML directly from a URL.

  my $parser = HTML::TokeParser::Simple->new(url => 'http://some.url');

This method relies on "LWP::Simple". If this module is not found or the page cannot be fetched, the constructor will "croak()".

get_token

get_tag

peek

This is useful when you're trying to debug where you are in a document.

 warn $parser->peek(3); # show the next 3 tokens

Boolean Accessors

• "is_tag([$tag])"

  Use this to determine if you have any tag. An optional "tag type" may be passed. This will allow you to match if it's a particular tag. The supplied tag is case-insensitive.

   if ( $token->is_tag ) { ... }
      

  Optionally, you may pass a regular expression as an argument.

• "is_start_tag([$tag])"

  Use this to determine if you have a start tag. An optional "tag type" may be passed. This will allow you to match if it's a particular start tag. The supplied tag is case-insensitive.

   if ( $token->is_start_tag ) { ... }
   if ( $token->is_start_tag( 'font' ) ) { ... }
      

  Optionally, you may pass a regular expression as an argument. To match all header (h1, h2, ... h6) tags:

   if ( $token->is_start_tag( qr/^h[123456]$/ ) ) { ... }
      

• "is_end_tag([$tag])"

  Use this to determine if you have an end tag. An optional "tag type" may be passed. This will allow you to match if it's a particular end tag. The supplied tag is case-insensitive.

  When testing for an end tag, the forward slash on the tag is optional.

   while ( $token = $p->get_token ) {
     if ( $token->is_end_tag( 'form' ) ) { ... }
   }
      

  Or:

   while ( $token = $p->get_token ) {
     if ( $token->is_end_tag( '/form' ) ) { ... }
   }
      

  Optionally, you may pass a regular expression as an argument.

• "is_text()"

  Use this to determine if you have text. Note that this is not to be confused with the "return_text" (deprecated) method described below! "is_text" will identify text that the user typically sees display in the Web browser.

• "is_comment()"

  Are you still reading this? Nobody reads POD. Don't you know you're supposed to go to CLPM, ask a question that's answered in the POD and get flamed? It's a rite of passage.

  Really.

  "is_comment" is used to identify comments. See the HTML::Parser documentation for more information about comments. There's more than you might think.

• "is_declaration()"

  This will match the DTD at the top of your HTML. (You do use DTD's, don't you?)

• "is_process_instruction()"

  Process Instructions are from XML. This is very handy if you need to parse out PHP and similar things with a parser.

  Currently, there appear to be some problems with process instructions. You can override "HTML::TokeParser::Simple::Token::ProcessInstruction" if you need to.

• "is_pi()"

  This is a shorthand for "is_process_instruction()".

Data Accessors

• "get_tag()"

  Do you have a start tag or end tag? This will return the type (lower case). Note that this is not the same as the "get_tag()" method on the actual parser object.

• "get_attr([$attribute])"

  If you have a start tag, this will return a hash ref with the attribute names as keys and the values as the values.

  If you pass in an attribute name, it will return the value for just that attribute.

  Returns false if the token is not a start tag.

• "get_attrseq()"

  For a start tag, this is an array reference with the sequence of the attributes, if any.

  Returns false if the token is not a start tag.

• "return_text()"

  This method has been heavily deprecated (for a couple of years) in favor of "as_is". Programmers were getting confused over the difference between "is_text", "return_text", and some parser methods such as "HTML::TokeParser::get_text" and friends.

  Using this method still succeeds, but will now carp and will be removed in the next major release of this module.

• "as_is()"

  This is the exact text of whatever the token is representing.

• "get_token0()"

  For processing instructions, this will return the token found immediately after the opening tag. Example: For <?php, "php" will be the start of the returned string.

  Note that process instruction handling appears to be incomplete in "HTML::TokeParser".

  Returns false if the token is not a process instruction.

Finding comments

 use strict;
 use HTML::TokeParser::Simple;

 my @html_docs = glob( "*.html" );

 open PHB, "> phbreport.txt" or die "Cannot open phbreport for writing: $!";

 foreach my $doc ( @html_docs ) {
     print "Processing $doc\n";
     my $p = HTML::TokeParser::Simple->new( file => $doc );
     while ( my $token = $p->get_token ) {
         next unless $token->is_comment;
         print PHB $token->as_is, "\n";
     }
 }

 close PHB;

Stripping Comments

 use strict;
 use HTML::TokeParser::Simple;

 my $new_folder = 'no_comment/';
 my @html_docs  = glob( "*.html" );

 foreach my $doc ( @html_docs ) {
     print "Processing $doc\n";
     my $new_file = "$new_folder$doc";

     open PHB, "> $new_file" or die "Cannot open $new_file for writing: $!";

     my $p = HTML::TokeParser::Simple->new( $file => doc );
     while ( my $token = $p->get_token ) {
         next if $token->is_comment;
         print PHB $token->as_is;
     }
     close PHB;
 }

Changing form tags

 use strict;
 use HTML::TokeParser::Simple;

 my $new_folder = 'new_html/';
 my @html_docs  = glob( "*.html" );

 foreach my $doc ( @html_docs ) {
     print "Processing $doc\n";
     my $new_file = "$new_folder$doc";

     open FILE, "> $new_file" or die "Cannot open $new_file for writing: $!";

     my $p = HTML::TokeParser::Simple->new( file => $doc );
     while ( my $token = $p->get_token ) {
         if ( $token->is_start_tag('form') ) {
             my $action = $token->get_attr(action);
             $action =~ s/www\.foo\.com/www.bar.com/;
             $token->set_attr('action', $action);
         }
         print FILE $token->as_is;
     }
     close FILE;
 }