On Deploying Perl Applications

 % sshnuke.pl 10.2.2.2 -rootpw="Z1ON0101"
 Perl v5.6.1 required--this is only v5.6.0, stopped at sshnuke.pl line 1.
 BEGIN failed--compilation aborted at sshnuke.pl line 1.

• Q: "Help! I can't run your program!"
• A1: Install Perl & "perl -MCPAN -e'install(...)'"

•

A2: Install Perl & "tar zxf my_perllib.tgz"

•

A3: Use the executable generated by "perlcc sshnuke.pl"

PAR, the Perl Archive Toolkit

•

Do what JAR (Java Archive) does for Perl

•

PAR files can be packed into self-contained scripts

Simple Packaging

• PAR files are just Zip files with modules in it
• Any Zip tools can generate them:

   % zip foo.par Hello.pm World.pm        # pack two modules
   % zip -r bar.par lib/          # grab all modules in lib/
      

• To load modules from PAR files:

   use PAR;
   use lib "foo.par";             # the .par part is optional
   use Hello;
      

• This also works:

   use PAR "/home/mylibs/*.par";  # put all of them into @INC
   use Hello;
      

PAR Loaders

• Use "par.pl" to run files inside a PAR archive:

   % par.pl foo.par               # looks for 'main.pl' by default
   % par.pl foo.par test.pl       # runs script/test.pl in foo.par
      

• Same thing, with the stand-alone "parl" or "parl.exe":

   % parl foo.par                 # no perl or PAR.pm needed!
   % parl foo.par test.pl         # ditto
      

• The PAR loader can prepend itself to a PAR file:

Dependency Scanning

• Recursively scan dependencies with "scandeps.pl":

   % scandeps.pl sshnuke.pl
   # Legend: [C]ore [X]ternal [S]ubmodule [?]NotOnCPAN
   'Crypt::SSLeay'       => '0', #  X   #
   'Net::HTTP'           => '0', #      #
   'Crypt::SSLeay::X509' => '0', # S    # Crypt::SSLeay
   'Net::HTTP::Methods'  => '0', # S    # Net::HTTP
   'Compress::Zlib'      => '0', #  X   # Net::HTTP::Methods
      

• Scan an one-liner, list all involved files:

   % scandeps.pl -V -e "use Dynaloader;"
   ...
   # auto/DynaLoader/dl_findfile.al [autoload]
   # auto/DynaLoader/extralibs.ld [autoload]
   # auto/File/Glob/Glob.bs [data]
   # auto/File/Glob/Glob.so [shared]
   ...
      

Perl Packager: "pp"

• Combines scanning, zipping and loader-embedding:

   % pp -o out.exe src.pl         # self-contained .exe
   % out.exe                      # runs anywhere on the same OS
      

• Bundle additional modules:

   % pp -o out.exe -M CGI src.pl  # pack CGI + its dependencies, too
      

• Pack one-liners:

   % pp -o out.exe -e 'print "Hi!"'   # turns one-liner into executable
      

• Generate PAR files instead of executables:

   % pp -p src.pl                 # makes 'source.par'
   % pp -B -p src.pl              # include core modules
      

How it works

•

Command-line options are almost identical to "perlcc"'s

• Small initial overhead; no runtime overhead
• Dependencies are POD-stripped before packing
• Loads modules directly into memory on demand
• Shared libraries (DLLs) are extracted with File::Temp
• Works on Perl 5.6.0 or above
• Tested on Win32 (VC++ and MinGW), FreeBSD, NetBSD, Linux, MacOSX, Cygwin, AIX, Solaris, HP-UX, Tru64...

Aggregating multiple programs

• A common question:

   > I have used pp to make several standalone applications which work
   > great, the only problem is that for each executable that I make, I am
   > assuming the parl.exe is somehow bundled into the resulting exe.
      

• The obvious workaround:

   You can ship parl.exe by itself, along with .par files built
   by "pp -p", and run those PAR files by associating them to parl.exe.
      

• On platforms that have "ln", there is a better solution:

   % pp --output=a.out a.pl b.pl  # two scripts in one!
   % ln a.out b.out               # symlink also works
   % ./a.out                      # runs a.pl
   % ./b.out                      # runs b.pl
      

Cross-platform Packages

• Of course, there is no cross-platform binary format
• Pure-perl PAR packages are cross-platform by default

• Suppose we need "out.par" on both Win32 and Finix:

   C:\> pp --multiarch --output=out.par src.pl
   ...copy src.pl and out.par to a Finix machine...
   % pp --multiarch --output=out.par src.pl
      

• Now it works on both platforms:

   % parl out.par                 # runs src.pl
   % perl -MPAR=out.par -e '...'  # uses modules inside out.par
      

The Anatomy of a PAR file

• Modules can reside in several directories:

   /                      # casual packaging only
   /lib/                  # standard location
   /arch/                 # for creating from blib/ 
   /i386-freebsd/         # i.e. $Config{archname}
   /5.8.0/                # i.e. Perl version number
   /5.8.0/i386-freebsd/   # combination of the two above
      

• Scripts are stored in one of the two locations:

   /                      # casual packaging only
   /script/               # standard location
      

• Shared libraries may be architecture- or perl-version-specific:

   /shlib/(5.8.0/)?(i386-freebsd/)?
      

• PAR files may recursively contain other PAR files:

   /par/(5.8.0/)?(i386-freebsd/)?
      

Special files

•

MANIFEST

•

META.yml

•

SIGNATURE

Advantages over perlcc, PerlApp and Perl2exe

•

This is not meant to be a flame

•

perlcc

•

PerlApp / Perl2exe

MANIFEST: Best viewed with Mozilla

• The URL of "MANIFEST" inside "/home/autrijus/foo.par":

   jar:file:///home/autrijus/foo.par!/MANIFEST
      

• Open it in a Gecko browser (e.g. Netscape 6+) with Javascript enabled:
• No needed to unzip anything; just click on files to view them

META.yml: Metadata galore

•

Static, machine-readable distribution metadata

• A typical "pp"-generated "META.yml" looks like this:

   build_requires: {}
   conflicts: {}
   dist_name: out.par
   distribution_type: par
   dynamic_config: 0
   generated_by: 'Perl Packager version 0.03'
   license: unknown
   par:
     clean: 0
     signature: ''
     verbatim: 0
     version: 0.68
      

• The "par:" settings controls its runtime behavior

SIGNATURE: Signing and verifying packages

•

OpenPGP clear-signed manifest with SHA1 digests

• A typical "SIGNATURE" looks like this:

   -----BEGIN PGP SIGNED MESSAGE-----
   Hash: SHA1
  
   SHA1 8a014cd6d0f6775552a01d1e6354a69eb6826046 AUTHORS
   ...
   -----BEGIN PGP SIGNATURE-----
   ...
   -----END PGP SIGNATURE-----
      

• Use "pp" and "cpansign" to work with signatures:

   % pp -s -o foo.par bar.pl      # make and sign foo.par from bar.pl
   % cpansign -s foo.par  # sign this PAR file
   % cpansign -v foo.par  # verify this PAR file
      

Perl Servlets with Apache::PAR

•

Framework for self-contained Web applications

•

A complete web application inside a ".par" file

•

One additional special file: "web.conf"

 Alias /myapp/cgi-perl/ ##PARFILE##/
 <Location /myapp/cgi-perl>
     Options +ExecCGI
     SetHandler perl-script
     PerlHandler Apache::PAR::Registry
 </Location>
    

Hon Dah, A-par-che!

• First, make a "hondah.par" from an one-liner:

   # use the "web.conf" from the previous slide
   % pp -p -o hondah.par -e 'print "Hon Dah!\n"' \
        --add web.conf
   % chmod a+x hondah.par
      

• Add this to "httpd.conf", then restart apache:

   <IfDefine MODPERL2>
   PerlModule Apache2
   </IfDefine>
   PerlAddVar PARInclude /home/autrijus/hondah.par
   PerlModule Apache::PAR
      

• Test it out:

   % GET http://localhost/myapp/cgi-perl/main.pl
   Hon Dah!
      

• Instant one-liner web application that works!

On-demand library fetching

• With LWP installed, your can use remote PAR files:

   use PAR;
   use lib 'http://aut.dyndns.org/par/DBI-latest.par';
   use DBI;    # always up to date!
      

• Modules are now cached under $ENV{PAR_GLOBAL_TEMP}
• Auto-updates with "LWP::Simple::mirror"

•

Makes large-scale deployment a breeze

Code Obfuscation

•

Also known as source-hiding techniques

•

Usually easy to defeat

•

PAR now supports pluggable input filters with "pp -f"

•

Alternatively, just keep core logic in your server and use RPC

Accessing packed files

• To get the host archive from a packed program:

   my $zip = PAR::par_handle($0); # an Archive::Zip object
   my $content = $zip->contents('MANIFEST');
      

• Same thing, but with "read_file()":

   my $content = PAR::read_file('MANIFEST');
      

• Loaded PAR files are stored in %PAR::LibCache:

   use PAR '/home/mylibs/*.par';
   while (my ($filename, $zip) = each %PAR::LibCache) {
       print "[$filename - MANIFEST]\n";
       print $zip->contents('MANIFEST');
   }
      

Packing GUI applications

• GUI toolkits often need to link with shared libraries:

   # search for libncurses under library paths and pack it
   % pp -l ncurses curses_app.pl  # same for Tk, Wx, Gtk, Qt...
      

• Use "pp --gui" on Win32 to eliminate the console window:

   # pack 'src.pl' into a console-less 'out.exe' (Win32 only)
   % pp --gui -o out.exe src.pl
      

• "Can't locate Foo/Widget/Bar.pm in @INC"?

Precompiled CPAN distributions

•

Installing XS extensions from CPAN was difficult

• ...but not anymore -- "Module::Install" to the rescue!

   # same old Makefile.PL, with a few changes
   use inc::Module::Install;      # was "use ExtUtils::MakeMaker;"
   WriteMakefile( ... );          # same as the original
   check_nmake();                 # make sure the user have nmake
   par_base('AUTRIJUS');          # your CPAN ID or a URL
   fetch_par() unless can_cc();   # use precompiled PAR only if necessary
      

• Users will not notice anything, except now it works

Thank you!

•

Additional resources

•

Any questions?

Overview of PAR.pm's Implementation

•

Here begins the scary part

•

PAR invokes four areas of Perl arcana:

•

The first two only works on 5.6 or later

Code References in @INC

•

On 1999-07-19, Ken Fox submitted a patch to P5P

• Coderefs in @INC may return a fh, or undef to 'pass':

   push @INC, sub {
       my ($coderef, $filename) = @_;  # $coderef is \&my_sub
       open my $fh, "wget ftp://example.com/$filename |";
       return $fh;        # using remote modules, indeed!
   };
      

• Perl 5.8 let you open a file handle to a string, so we just use that:

          open my $fh, '<', \($zip->memberNamed($filename)->contents);
          return $fh;
      

• But Perl 5.6 does not have that, and I don't want to use temp files...

Source Filtering without Filter::* Modules

•

... Undocumented features to the rescue!

•

This is how "Acme::use::strict::with::pride" works:

 # Force all modules used to use strict and warnings
 open my $fh, "<", $filename or return;
 my @lines = ("use strict; use warnings;\n", "#line 1 \"$full\"\n");
 return ($fh, sub {
     return 0 unless @lines;    
     push @lines, $_; $_ = shift @lines; return length $_;
 });
    

Source Filtering without Filter::* Modules (cont.)

• But we don't really have a filehandle for anything
• Another undocumented feature saves the day!
• We can actually omit the first return value altogether:

   # Return all contents line-by-line from the file inside PAR
   my @lines = split(
       /(?<=\n)/,
       $zip->memberNamed($filename)->contents
   );
   return (sub {
       $_ = shift(@lines);
       return length $_;
   });
      

Overriding DynaLoader::bootstrap

•

XS modules have dynamically loaded libraries

•

Module names are passed to "bootstrap" for XS loading

•

Our "_bootstrap" just checks if the library is in PARs

Anatomy of a Self-Contained PAR executable

•

The par script ($0) itself

•

Any number of embedded files

•

One PAR file

•

Ending section

Self-Bootstrapping Tricks

•

All we can expect is a working perl interpreter

•

Answer: bundle all modules + libraries used by PAR.pm

•

Minimizing the amount of temporary files

Thank you (again)!

•

Any questions, please?