The "feature" pragma

Currently the following new features are available: "switch" (adds a switch statement), "say" (adds a "say" built-in function), and "state" (adds a "state" keyword for declaring "static" variables). Those features are described in their own sections of this document.

The "feature" pragma is also implicitly loaded when you require a minimal perl version (with the "use VERSION" construct) greater than, or equal to, 5.9.5. See feature for details.

New -E command-line switch

Defined-or operator

    $a // $b

is merely equivalent to

   defined $a ? $a : $b

and the statement

   $c //= $d;

can now be used instead of

   $c = $d unless defined $c;

The "//" operator has the same precedence and associativity as "||". Special care has been taken to ensure that this operator Do What You Mean while not breaking old code, but some edge cases involving the empty regular expression may now parse differently. See perlop for details.

Switch and Smart Match operator

    given ($foo) {
        when (/^abc/) { $abc = 1; }
        when (/^def/) { $def = 1; }
        when (/^xyz/) { $xyz = 1; }
        default { $nothing = 1; }
    }

A more complete description of how Perl matches the switch variable against the "when" conditions is given in "Switch statements" in perlsyn.

This kind of match is called smart match, and it's also possible to use it outside of switch statements, via the new "~~" operator. See "Smart matching in detail" in perlsyn.

This feature was contributed by Robin Houston.

Regular expressions

Recursive Patterns

It is now possible to write recursive patterns without using the "(??{})" construct. This new way is more efficient, and in many cases easier to read.

Each capturing parenthesis can now be treated as an independent pattern that can be entered by using the "(?PARNO)" syntax ("PARNO" standing for "parenthesis number"). For example, the following pattern will match nested balanced angle brackets:

    /
     ^                      # start of line
     (                      # start capture buffer 1
        <                   #   match an opening angle bracket
        (?:                 #   match one of:
            (?>             #     don't backtrack over the inside of this group
                [^<>]+      #       one or more non angle brackets
            )               #     end non backtracking group
        |                   #     ... or ...
            (?1)            #     recurse to bracket 1 and try it again
        )*                  #   0 or more times.
        >                   #   match a closing angle bracket
     )                      # end capture buffer one
     $                      # end of line
    /x
    

PCRE users should note that Perl's recursive regex feature allows backtracking into a recursed pattern, whereas in PCRE the recursion is atomic or "possessive" in nature. As in the example above, you can add (?>) to control this selectively. (Yves Orton)

Named Capture Buffers

It is now possible to name capturing parenthesis in a pattern and refer to the captured contents by name. The naming syntax is "(?<NAME>....)". It's possible to backreference to a named buffer with the "\k<NAME>" syntax. In code, the new magical hashes "%+" and "%-" can be used to access the contents of the capture buffers.

Thus, to replace all doubled chars with a single copy, one could write

    s/(?<letter>.)\k<letter>/$+{letter}/g
    

Only buffers with defined contents will be "visible" in the "%+" hash, so it's possible to do something like

    foreach my $name (keys %+) {
        print "content of buffer '$name' is $+{$name}\n";
    }
    

The "%-" hash is a bit more complete, since it will contain array refs holding values from all capture buffers similarly named, if there should be many of them.

"%+" and "%-" are implemented as tied hashes through the new module "Tie::Hash::NamedCapture".

Users exposed to the .NET regex engine will find that the perl implementation differs in that the numerical ordering of the buffers is sequential, and not "unnamed first, then named". Thus in the pattern

   /(A)(?<B>B)(C)(?<D>D)/
    

$1 will be 'A', $2 will be 'B', $3 will be 'C' and $4 will be 'D' and not $1 is 'A', $2 is 'C' and $3 is 'B' and $4 is 'D' that a .NET programmer would expect. This is considered a feature. :-) (Yves Orton)

Possessive Quantifiers

Perl now supports the "possessive quantifier" syntax of the "atomic match" pattern. Basically a possessive quantifier matches as much as it can and never gives any back. Thus it can be used to control backtracking. The syntax is similar to non-greedy matching, except instead of using a '?' as the modifier the '+' is used. Thus "?+", "*+", "++", "{min,max}+" are now legal quantifiers. (Yves Orton)

Backtracking control verbs

The regex engine now supports a number of special-purpose backtrack control verbs: (*THEN), (*PRUNE), (*MARK), (*SKIP), (*COMMIT), (*FAIL) and (*ACCEPT). See perlre for their descriptions. (Yves Orton)

Relative backreferences

A new syntax "\g{N}" or "\gN" where "N" is a decimal integer allows a safer form of back-reference notation as well as allowing relative backreferences. This should make it easier to generate and embed patterns that contain backreferences. See "Capture buffers" in perlre. (Yves Orton)

"\K" escape

The functionality of Jeff Pinyan's module Regexp::Keep has been added to the core. In regular expressions you can now use the special escape "\K" as a way to do something like floating length positive lookbehind. It is also useful in substitutions like:

  s/(foo)bar/$1/g
    

that can now be converted to

  s/foo\Kbar//g
    

which is much more efficient. (Yves Orton)

Vertical and horizontal whitespace, and linebreak

Regular expressions now recognize the "\v" and "\h" escapes that match vertical and horizontal whitespace, respectively. "\V" and "\H" logically match their complements.

"\R" matches a generic linebreak, that is, vertical whitespace, plus the multi-character sequence "\x0D\x0A".

Optional pre-match and post-match captures with the /p flag

There is a new flag "/p" for regular expressions. Using this makes the engine preserve a copy of the part of the matched string before the matching substring to the new special variable "${^PREMATCH}", the part after the matching substring to "${^POSTMATCH}", and the matched substring itself to "${^MATCH}".

Perl is still able to store these substrings to the special variables "$`", "$'", $&, but using these variables anywhere in the program adds a penalty to all regular expression matches, whereas if you use the "/p" flag and the new special variables instead, you pay only for the regular expressions where the flag is used.

For more detail on the new variables, see perlvar; for the use of the regular expression flag, see perlop and perlre.

"say()"

Lexical $_

    my $_;

The operations that default on $_ will use the lexically-scoped version of $_ when it exists, instead of the global $_.

In a "map" or a "grep" block, if $_ was previously my'ed, then the $_ inside the block is lexical as well (and scoped to the block).

In a scope where $_ has been lexicalized, you can still have access to the global version of $_ by using $::_, or, more simply, by overriding the lexical declaration with "our $_". (Rafael Garcia-Suarez)

The "_" prototype

This has a small incompatible consequence: the prototype() function has been adjusted to return "_" for some built-ins in appropriate cases (for example, "prototype('CORE::rmdir')"). (Rafael Garcia-Suarez)

UNITCHECK blocks

"CHECK" and "INIT" blocks, while useful for some specialized purposes, are always executed at the transition between the compilation and the execution of the main program, and thus are useless whenever code is loaded at runtime. On the other hand, "UNITCHECK" blocks are executed just after the unit which defined them has been compiled. See perlmod for more information. (Alex Gough)

New Pragma, "mro"

Note that, due to changes in the implementation of class hierarchy search, code that used to undef the *ISA glob will most probably break. Anyway, undef'ing *ISA had the side-effect of removing the magic on the @ISA array and should not have been done in the first place. Also, the cache *::ISA::CACHE:: no longer exists; to force reset the @ISA cache, you now need to use the "mro" API, or more simply to assign to @ISA (e.g. with "@ISA = @ISA").

readdir() may return a "short filename" on Windows

Similarly, $^X, @INC, and $ENV{PATH} are preprocessed at startup to make sure all paths are valid in the ANSI codepage (if possible).

The Win32::GetLongPathName() function now returns the UTF-8 encoded correct long file name instead of using replacement characters to force the name into the ANSI codepage. The new Win32::GetANSIPathName() function can be used to turn a long pathname into a short one only if the long one cannot be represented in the ANSI codepage.

Many other functions in the "Win32" module have been improved to accept UTF-8 encoded arguments. Please see Win32 for details.

readpipe() is now overridable

Default argument for readline()

state() variables

To use state variables, one needs to enable them by using

    use feature 'state';

or by using the "-E" command-line switch in one-liners. See "Persistent Private Variables" in perlsub.

Stacked filetest operators

UNIVERSAL::DOES()

See "$obj->DOES( ROLE )" in UNIVERSAL.

Formats

Byte-order modifiers for pack() and unpack()

"no VERSION"

"chdir", "chmod" and "chown" on filehandles

OS groups

Recursive sort subs

Exceptions in constant folding

Source filters in @INC

New internal variables

"${^RE_DEBUG_FLAGS}"

This variable controls what debug flags are in effect for the regular expression engine when running under "use re "debug"". See re for details.

"${^CHILD_ERROR_NATIVE}"

This variable gives the native status returned by the last pipe close, backtick command, successful call to wait() or waitpid(), or from the system() operator. See perlvar for details. (Contributed by Gisle Aas.)

"${^RE_TRIE_MAXBUF}"

See "Trie optimisation of literal string alternations".

"${^WIN32_SLOPPY_STAT}"

See "Sloppy stat on Windows".

Miscellaneous

"mkdir()" without arguments now defaults to $_.

The internal dump output has been improved, so that non-printable characters such as newline and backspace are output in "\x" notation, rather than octal.

The -C option can no longer be used on the "#!" line. It wasn't working there anyway, since the standard streams are already set up at this point in the execution of the perl interpreter. You can use binmode() instead to get the desired behaviour.

UCD 5.0.0

MAD

kill() on Windows

Packing and UTF-8 strings

To be consistent with pack(), the "C0" in unpack() templates indicates that the data is to be processed in character mode, i.e. character by character; on the contrary, "U0" in unpack() indicates UTF-8 mode, where the packed string is processed in its UTF-8-encoded Unicode form on a byte by byte basis. This is reversed with regard to perl 5.8.X, but now consistent between pack() and unpack().

Moreover, "C0" and "U0" can also be used in pack() templates to specify respectively character and byte modes.

"C0" and "U0" in the middle of a pack or unpack format now switch to the specified encoding mode, honoring parens grouping. Previously, parens were ignored.

Also, there is a new pack() character format, "W", which is intended to replace the old "C". "C" is kept for unsigned chars coded as bytes in the strings internal representation. "W" represents unsigned (logical) character values, which can be greater than 255. It is therefore more robust when dealing with potentially UTF-8-encoded data (as "C" will wrap values outside the range 0..255, and not respect the string encoding).

In practice, that means that pack formats are now encoding-neutral, except "C".

For consistency, "A" in unpack() format now trims all Unicode whitespace from the end of the string. Before perl 5.9.2, it used to strip only the classical ASCII space characters.

Byte/character count feature in unpack()

The $* and $# variables have been removed

The deprecated $# variable (output format for numbers) has been removed.

Two new severe warnings, "$#/$* is no longer supported", have been added.

substr() lvalues are no longer fixed-length

Parsing of "-f _"

":unique"

Effect of pragmas in eval

As a side-effect of this, the overloaded-ness of constants now propagates into eval("").

chdir FOO

Handling of .pmc files

Previously, .pmc files were loaded only if more recent than the matching .pm file. Starting with 5.9.4, they'll be always loaded if they exist.

$^V is now a "version" object instead of a v-string

@- and @+ in patterns

$AUTOLOAD can now be tainted

Tainting and printf

undef and signal handlers

strictures and dereferencing in defined()

    use strict 'refs';
    my $x = 'foo';
    if (defined $$x) {...}

This now correctly produces the run-time error "Can't use string as a SCALAR ref while "strict refs" in use".

"defined @$foo" and "defined %$bar" are now also subject to "strict 'refs'" (that is, $foo and $bar shall be proper references there.) ("defined(@foo)" and "defined(%bar)" are discouraged constructs anyway.) (Nicholas Clark)

"(?p{})" has been removed

Pseudo-hashes have been removed

Removal of the bytecode compiler and of perlcc

However the B compiler framework stays supported in the perl core, as with the more useful modules it has permitted (among others, B::Deparse and B::Concise).

Removal of the JPL

Recursive inheritance detected earlier

Previously, the exception would not occur until Perl attempted to make use of the recursive inheritance while resolving a method or doing a "$foo->isa($bar)" lookup.

warnings::enabled and warnings::warnif changed to favor users of modules

Assume the following code:

  main calls Foo::Bar::baz()
  Foo::Bar inherits from Foo::Base
  Foo::Bar::baz() calls Foo::Base::_bazbaz()
  Foo::Base::_bazbaz() calls: warnings::warnif('substr', 'some warning 
message');

On 5.8.x, the code warns when Foo::Bar contains "use warnings;" It does not matter if Foo::Base or main have warnings enabled to disable the warning one has to modify Foo::Bar.

On 5.10.0 and newer, the code warns when main contains "use warnings;" It does not matter if Foo::Base or Foo::Bar have warnings enabled to disable the warning one has to modify main.

Upgrading individual core modules

Pragmata Changes

"feature"

The new pragma "feature" is used to enable new features that might break old code. See "The "feature" pragma" above.

"mro"

This new pragma enables to change the algorithm used to resolve inherited methods. See "New Pragma, "mro"" above.

Scoping of the "sort" pragma

The "sort" pragma is now lexically scoped. Its effect used to be global.

Scoping of "bignum", "bigint", "bigrat"

The three numeric pragmas "bignum", "bigint" and "bigrat" are now lexically scoped. (Tels)

"base"

The "base" pragma now warns if a class tries to inherit from itself. (Curtis "Ovid" Poe)

"strict" and "warnings"

"strict" and "warnings" will now complain loudly if they are loaded via incorrect casing (as in "use Strict;"). (Johan Vromans)

"version"

The "version" module provides support for version objects.

"warnings"

The "warnings" pragma doesn't load "Carp" anymore. That means that code that used "Carp" routines without having loaded it at compile time might need to be adjusted; typically, the following (faulty) code won't work anymore, and will require parentheses to be added after the function name:

    use warnings;
    require Carp;
    Carp::confess 'argh';
    

"less"

"less" now does something useful (or at least it tries to). In fact, it has been turned into a lexical pragma. So, in your modules, you can now test whether your users have requested to use less CPU, or less memory, less magic, or maybe even less fat. See less for more. (Joshua ben Jore)

New modules

• "encoding::warnings", by Audrey Tang, is a module to emit warnings whenever an ASCII character string containing high-bit bytes is implicitly converted into UTF-8. It's a lexical pragma since Perl 5.9.4; on older perls, its effect is global.
• "Module::CoreList", by Richard Clamp, is a small handy module that tells you what versions of core modules ship with any versions of Perl 5. It comes with a command-line frontend, "corelist".
• "Math::BigInt::FastCalc" is an XS-enabled, and thus faster, version of "Math::BigInt::Calc".
• "Compress::Zlib" is an interface to the zlib compression library. It comes with a bundled version of zlib, so having a working zlib is not a prerequisite to install it. It's used by "Archive::Tar" (see below).
• "IO::Zlib" is an "IO::"-style interface to "Compress::Zlib".
• "Archive::Tar" is a module to manipulate "tar" archives.
• "Digest::SHA" is a module used to calculate many types of SHA digests, has been included for SHA support in the CPAN module.
• "ExtUtils::CBuilder" and "ExtUtils::ParseXS" have been added.
• "Hash::Util::FieldHash", by Anno Siegel, has been added. This module provides support for field hashes: hashes that maintain an association of a reference with a value, in a thread-safe garbage-collected way. Such hashes are useful to implement inside-out objects.
• "Module::Build", by Ken Williams, has been added. It's an alternative to "ExtUtils::MakeMaker" to build and install perl modules.
• "Module::Load", by Jos Boumans, has been added. It provides a single interface to load Perl modules and .pl files.
• "Module::Loaded", by Jos Boumans, has been added. It's used to mark modules as loaded or unloaded.
• "Package::Constants", by Jos Boumans, has been added. It's a simple helper to list all constants declared in a given package.
• "Win32API::File", by Tye McQueen, has been added (for Windows builds). This module provides low-level access to Win32 system API calls for files/dirs.
• "Locale::Maketext::Simple", needed by CPANPLUS, is a simple wrapper around "Locale::Maketext::Lexicon". Note that "Locale::Maketext::Lexicon" isn't included in the perl core; the behaviour of "Locale::Maketext::Simple" gracefully degrades when the later isn't present.
• "Params::Check" implements a generic input parsing/checking mechanism. It is used by CPANPLUS.
• "Term::UI" simplifies the task to ask questions at a terminal prompt.
• "Object::Accessor" provides an interface to create per-object accessors.
• "Module::Pluggable" is a simple framework to create modules that accept pluggable sub-modules.
• "Module::Load::Conditional" provides simple ways to query and possibly load installed modules.
• "Time::Piece" provides an object oriented interface to time functions, overriding the built-ins localtime() and gmtime().
• "IPC::Cmd" helps to find and run external commands, possibly interactively.
• "File::Fetch" provide a simple generic file fetching mechanism.
• "Log::Message" and "Log::Message::Simple" are used by the log facility of "CPANPLUS".
• "Archive::Extract" is a generic archive extraction mechanism for .tar (plain, gzipped or bzipped) or .zip files.
• "CPANPLUS" provides an API and a command-line tool to access the CPAN mirrors.
• "Pod::Escapes" provides utilities that are useful in decoding Pod E<...> sequences.
• "Pod::Simple" is now the backend for several of the Pod-related modules included with Perl.

Selected Changes to Core Modules

"Attribute::Handlers"

"Attribute::Handlers" can now report the caller's file and line number. (David Feldman)

All interpreted attributes are now passed as array references. (Damian Conway)

"B::Lint"

"B::Lint" is now based on "Module::Pluggable", and so can be extended with plugins. (Joshua ben Jore)

"B"

It's now possible to access the lexical pragma hints ("%^H") by using the method B::COP::hints_hash(). It returns a "B::RHE" object, which in turn can be used to get a hash reference via the method B::RHE::HASH(). (Joshua ben Jore)

"Thread"

As the old 5005thread threading model has been removed, in favor of the ithreads scheme, the "Thread" module is now a compatibility wrapper, to be used in old code only. It has been removed from the default list of dynamic extensions.

In-place sorting

Likewise, "reverse sort ..." is now optimized to sort in reverse, avoiding the generation of a temporary intermediate list.

Lexical array access

XS-assisted SWASHGET

Constant subroutines

Several of the core modules have been converted to use this feature for their system dependent constants - as a result "use POSIX;" now takes about 200K less memory.

"PERL_DONT_CREATE_GVSV"

Weak references are cheaper

sort() enhancements

Memory optimisations

UTF-8 cache optimisation

Sloppy stat on Windows

Regular expressions optimisations

Engine de-recursivised

The regular expression engine is no longer recursive, meaning that patterns that used to overflow the stack will either die with useful explanations, or run to completion, which, since they were able to blow the stack before, will likely take a very long time to happen. If you were experiencing the occasional stack overflow (or segfault) and upgrade to discover that now perl apparently hangs instead, look for a degenerate regex. (Dave Mitchell)

Single char char-classes treated as literals

Classes of a single character are now treated the same as if the character had been used as a literal, meaning that code that uses char-classes as an escaping mechanism will see a speedup. (Yves Orton)

Trie optimisation of literal string alternations

Alternations, where possible, are optimised into more efficient matching structures. String literal alternations are merged into a trie and are matched simultaneously. This means that instead of O(N) time for matching N alternations at a given point, the new code performs in O(1) time. A new special variable, ${^RE_TRIE_MAXBUF}, has been added to fine-tune this optimization. (Yves Orton)

Note: Much code exists that works around perl's historic poor performance on alternations. Often the tricks used to do so will disable the new optimisations. Hopefully the utility modules used for this purpose will be educated about these new optimisations.

Aho-Corasick start-point optimisation

When a pattern starts with a trie-able alternation and there aren't better optimisations available, the regex engine will use Aho-Corasick matching to find the start point. (Yves Orton)

Configuration improvements

"-Dusesitecustomize"

Run-time customization of @INC can be enabled by passing the "-Dusesitecustomize" flag to Configure. When enabled, this will make perl run $sitelibexp/sitecustomize.pl before anything else. This script can then be set up to add additional entries to @INC.

Relocatable installations

There is now Configure support for creating a relocatable perl tree. If you Configure with "-Duserelocatableinc", then the paths in @INC (and everything else in %Config) can be optionally located via the path of the perl executable.

That means that, if the string ".../" is found at the start of any path, it's substituted with the directory of $^X. So, the relocation can be configured on a per-directory basis, although the default with "-Duserelocatableinc" is that everything is relocated. The initial install is done to the original configured prefix.

strlcat() and strlcpy()

The configuration process now detects whether strlcat() and strlcpy() are available. When they are not available, perl's own version is used (from Russ Allbery's public domain implementation). Various places in the perl interpreter now use them. (Steve Peters)

"d_pseudofork" and "d_printf_format_null"

A new configuration variable, available as $Config{d_pseudofork} in the Config module, has been added, to distinguish real fork() support from fake pseudofork used on Windows platforms.

A new configuration variable, "d_printf_format_null", has been added, to see if printf-like formats are allowed to be NULL.

Configure help

"Configure -h" has been extended with the most commonly used options.

Compilation improvements

Parallel build

Parallel makes should work properly now, although there may still be problems if "make test" is instructed to run in parallel.

Borland's compilers support

Building with Borland's compilers on Win32 should work more smoothly. In particular Steve Hay has worked to side step many warnings emitted by their compilers and at least one C compiler internal error.

Static build on Windows

Perl extensions on Windows now can be statically built into the Perl DLL.

Also, it's now possible to build a "perl-static.exe" that doesn't depend on the Perl DLL on Win32. See the Win32 makefiles for details. (Vadim Konovalov)

ppport.h files

All ppport.h files in the XS modules bundled with perl are now autogenerated at build time. (Marcus Holland-Moritz)

C++ compatibility

Efforts have been made to make perl and the core XS modules compilable with various C++ compilers (although the situation is not perfect with some of the compilers on some of the platforms tested.)

Support for Microsoft 64-bit compiler

Support for building perl with Microsoft's 64-bit compiler has been improved. (ActiveState)

Visual C++

Perl can now be compiled with Microsoft Visual C++ 2005 (and 2008 Beta 2).

Win32 builds

All win32 builds (MS-Win, WinCE) have been merged and cleaned up.

Installation improvements

Module auxiliary files

README files and changelogs for CPAN modules bundled with perl are no longer installed.

New Or Improved Platforms

Many improvements have been made towards making Perl work correctly on z/OS.

Perl has been reported to work on DragonFlyBSD and MidnightBSD.

Perl has also been reported to work on NexentaOS ( http://www.gnusolaris.org/ ).

The VMS port has been improved. See perlvms.

Support for Cray XT4 Catamount/Qk has been added. See hints/catamount.sh in the source code distribution for more information.

Vendor patches have been merged for RedHat and Gentoo.

DynaLoader::dl_unload_file() now works on Windows.

Reordering of SVt_* constants

Elimination of SVt_PVBM

New type SVt_BIND

Removal of CPP symbols

Less space is used by ops

New parser

Also, Dave Mitchell improved the lexer debugging output under "-DT".

Use of "const"

Mathoms

"AvFLAGS" has been removed

"av_*" changes

$^H and %^H

B:: modules inheritance changed

Anonymous hash and array constructors

UTF-8 problems