Tag declaration by subclasses

  # declare a tag declaration package
  package PerlPoint::Tags::New;

  # declare base "class"
  use base qw(PerlPoint::Tags);

Now the tags can be declared. Tag declarations are expected in a global hash named %tags. Each key is the name of a tag, while descriptions are nested structures stored as values.

  # pragmata
  use strict;
  use vars qw(%tags %sets);

  # tag declarations
  %tags=(
         EMPHASIZE => {
                       # options
                       options => TAGS_OPTIONAL,

                       # don't miss the body!
                       body    => TAGS_MANDATORY,
                      },

         COLORIZE => {...},

         FONTIFY  => {...},

         ...
        );

This looks complicated but is easy to understand. Each option is decribed by a hash. The body slot just expresses if the body is obsolete, optional or mandatory. This is done by using constants provided by PerlPoint::Constants. Obsolete bodies will not be recognized by the parser.

The body slot may be omitted. This means the body is optional.

There are the same choices for options in general: they may be obsolete, optional or mandatory. If the slot is omitted this means that the tag does not need any options. The parser will not accept a tag option part in this case.

To sum it up, options and body of a tag can be declared as mandatory by "TAGS_MANDATORY", optional by "TAGS_OPTIONAL", or obsolete by "TAGS_DISABLED".

If you need further checks you can hook into the parser by using the "hook" key:

  %tags=(
         EMPHASIZE => {
                       # options
                       options => TAGS_OPTIONAL,

                       # perform special checks
                       hook => sub {
                                    # get parameters
                                    my ($tagname, $options, $body, $anchor)=@_;

                                    # checks
                                    $rc=...

                                    reply results
                                    $rc;
                                   }
                      },

         COLORIZE => {...},

         FONTIFY  => {...},

         ...
        );

An option hook function receives the tag name, a reference to a hash of option name / value pairs to check, a body array reference and an anchor object. Using the option hash reference, the hook can modify the options. The passed body array is a copy of the body part of the stream. The hook therefore cannot modify the body part on parsers side. The anchor object can be used to store new anchors or query anchors already known, see \"PerlPoint::Anchors" for details of this objects interface.

The following return codes are defined:

PARSING_COMPLETED

Parsing will be stopped successfully.

PARSING_ERASE

The parser will throw away the tag and all its content.

PARSING_ERROR

A semantic error occurred. This error will be counted, but parsing will be continued to possibly detect even more errors.

PARSING_FAILED

A syntactic error occured. Parsing will be stopped immediately.

PARSING_IGNORE

The parser will ignore the tag, but stream the body. The result is similar to a source omitting the tag.

PARSING_OK

The checked object is declared to be OK, parsing will be continued.

Hooks are an interesting way to extend document parsing, but please take into consideration that tag hooks might be called quite often. So, if checks have to be performed, users will be glad if they are performed quickly.

And there is another hook interface. It might happen that several operations need parsing to be completed before they can start, like checking an referenced anchor which might be defined after the referencing tag. To handle such situations, a subroutine can be declared as value of key "finish". The parser will invoke this code when parsing is done and the tag was parsed successfully. (So if a "hook" function returned an error code, the "finish" hook will be ignored.)

Here is an example (from an implementation of the basic tag \REF):

  # afterburner
  finish =>  sub
              {
               # declare and init variable
               my $ok=PARSING_OK;

               # take parameters
               my ($options, $anchors)=@_;

               # check link for being valid
               unless (my $anchor=$anchors->query($options->{name}))
                 {
                  $ok=PARSING_FAILED,
                  warn qq(\n\n[Error] Unknown link address "$options->{name}."\n);
                 }
               else
                 {
                  # link ok, get value (there is only one key/value pair
                  # in the received hash)
                  ($options->{__value__})=(values %$anchor);
                 }

               # supply status
               $ok;
              },

Because several informations are no longer available after parsing, finish hooks have a different interface. They receive options and anchors like parsing hooks, but no line number and no body information.

Options can be modified as well as in parsing hooks. Return codes are the same, but are evaluated slightly different according to the invokation time:

PARSING_COMPLETED

All right. This code is accepted for reasons of convenience, it is recommended to use "PARSING_OK" instead.

PARSING_ERASE

The backend will ignore the tag and all its contents (which means its body).

PARSING_ERROR

A semantic error occurred. This error will be counted.

PARSING_FAILED

An error occured. Because parsing is already finished, this will be counted as an sematic error.

This code is accepted for reasons of convenience, it is recommended to use "PARSING_ERROR" instead.

PARSING_IGNORE

The backend will ignore the tag, but process its body. This simply means that the tag takes no effect.

PARSING_OK

All right.

The order of finish hook invokation can differ from the order of tag usage. Do not depend on it.

A finish hook is not invoked unless the tag was processed and streamed successfully at parsing time. This simply means if the parsing hook returned "PARSING_OK", or if there was no parsing hook at all.

Marking tags that can act standalone

The standalone attribute instructs the parser to strip off the wrapping paragraph from a handle that is used as its only content. If there is more content in the paragraph the paragraph wrapper will not be removed.

The flag should be set to a true value to activate the occasional paragraph stripping.

Example:

  standalone => 1,

Using other tag definitions

 $rc=PerlPoint::Tags::call('TAG', 'hook', @_);

Valid hook types are "hook" and "finish" (currently). If the tag is not registered, or has no hook of the specified type, an undefined value is supplied, otherwise the return code of the invoked function.

It is not checked if you call a "hook" function from a "finish" hook or vice versa. Take care!

This feature is made available to interchange hooks between several tag definition modules. If you want to share hook functions between tags declared by the same tag module, it is recommended to use common Perl techniques.

Tag activation

  # declare all the tags to recognize
  use PerlPoint::Tags::New;

This updates a structure in the PerlPoint::Tags namespace. The parser knows about this structure and will automatically evaluate it.

Several declaration modules can be loaded subsequently. Each new tag is added to the internal structure, while predeclared tags are overwritten by new declarations.

  # declare all the tags to recognize
  use PerlPoint::Tags::Basic;
  use PerlPoint::Tags::HTML;
  use PerlPoint::Tags::SDF;
  use PerlPoint::Tags::New;

Activating certain tags

  # declare all the tags to recognize
  use PerlPoint::Tags::New qw(COLORIZE);

This does only declare the "COLORIZE" tag, but ignores "EMPHASIZE" and "FONTIFY".

Tag sets

  %sets=(
         pointto => [qw(EMPHASIZE COLORIZE)],
        );

This allows a translator autor to activate "EMPHASIZE" and "COLORIZE" at once:

  # declare all the tags to recognize
  use PerlPoint::Tags::New qw(:pointto);

The syntax is borrowed from the usual import mechanism.

Tag sets can overlap:

  %sets=(
         pointto => [qw(EMPHASIZE COLORIZE)],
         set2    => [qw(COLORIZE FONTIFY)],
        );

And of course they can be nested:

  %sets=(
         pointto => [qw(EMPHASIZE COLORIZE)],
         all     => [(':pointto', qw(FONTIFY))],
        );

Allowing translator users to import foreign tag declarations

The problem could be solved if there would be a way to inform the parser about the tags initially used. That's why this module provides addTagSets(), a method that imports foreign declarations at run time. Suppose a translator provides an option "-tagset" to let a user specify which tag sets the document was initially written for. Then the following code makes them known to the parser, addtionally to the declarations the translator itself already made as usual (see above):

  # load module to access the function
  use PerlPoint::Tags;

  # get options
  ...

  # import foreign tags
  PerlPoint::Tags::addTagSets(@{$options{tagset}})
    if exists $options{tagset};

(Note: this example is based on the Getopt::Long option access interface. Other interfaces might need adaptations.)

Tags imported via "addTagSets()" do not overwrite original definitions.

A "tag set", in this context, is the set of tag declarations a certain translator makes. So, the attributes to addTagSets() are expected to be target languages corresponding to the translators name, making usage easy for the user. So, pp2sdf is expected to provide a "tag set" declaration module PerlPoint::Tags::SDF, pp2html PerlPoint::Tags::HTML, pp2xml PerlPoint::Tags::XML and so on.

If all translators provide this same interface, usage should be easy. A user who wrote a document with "pp2html" in mind, passing it to "pp2sdf" which provides significantly less tags, only has to add the option "-tagset HTML" to the "pp2sdf" call to make his document pass the PerlPoint parser.

addTagSets()

call()