GSP
Quick Navigator
Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages
pmapfilter(3) SiLK Tool Suite pmapfilter(3)

pmapfilter - User-defined labels for IPs and protocol/port pairs 

  rwfilter --pmap-file=[MAPNAME:]FILENAME
        [--pmap-file=[MAPNAME:]FILENAME ...]
        [--pmap-src-MAPNAME=LABELS] [--pmap-dst-MAPNAME=LABELS]
        [--pmap-any-MAPNAME=LABELS] ...

  rwcut --pmap-file=[MAPNAME:]FILENAME
        [--pmap-file=[MAPNAME:]FILENAME ...]
        --fields=FIELDS [--pmap-column-width=NUM]

  rwgroup --pmap-file=[MAPNAME:]FILENAME
        [--pmap-file=[MAPNAME:]FILENAME ...]
        --id-fields=FIELDS

  rwsort --pmap-file=[MAPNAME:]FILENAME
        [--pmap-file=[MAPNAME:]FILENAME ...]
        --fields=FIELDS

  rwstats --pmap-file=[MAPNAME:]FILENAME
        [--pmap-file=[MAPNAME:]FILENAME ...]
        --fields=FIELDS [--pmap-column-width=NUM]

  rwuniq --pmap-file=[MAPNAME:]FILENAME
        [--pmap-file=[MAPNAME:]FILENAME ...]
        --fields=FIELDS [--pmap-column-width=NUM]

Prefix maps provide a mapping from values on a SiLK Flow record to string labels. The binary prefix map file is created from textual input with rwpmapbuild. See the rwpmapbuild(1) manual page for the syntax of input file. This manual page describes how to use a prefix map file to augment the features of some commonly used SiLK applications.

A prefix map file maps either an IP address or a protocol/port pair to a label. The mode statement in the input to rwpmapbuild determines whether the prefix map file is a mapping for IPs or for protocol/port pairs. To see the mode of an existing prefix map, use rwpmapcat(1) and specify --output-type=type.

When using the prefix map file as described in this manual page, one typically uses the prefix map's map-name. The map-name statement in the input to rwpmapbuild allows one to assign the map-name when creating the prefix map. To see the map-name of an existing prefix map, use rwpmapcat --output-type=mapname. To assign a map-name when loading a prefix map file, use the --pmap-file switch and specify the map-name you want to use, a colon, and the file name. A map-name provided to the --pmap-file switch overrides the map-name in the file (if one exists).

When using a prefix map in rwfilter(1), the map-name is combined with the prefix --pmap-src-, --pmap-dst-, or --pmap-any- to create the partitioning switches. When using the prefix map to create fields in rwcut(1), rwgroup(1), rwsort(1), rwstats(1), and rwuniq(1), the map-name must be combined with the prefix "src-" or "dst-" to create the field names.

The applications support using multiple prefix map files in a single invocation. When using multiple prefix map files, each file must have a unique map-name (or be assigned a unique map-name on the command line).

When a prefix map file does not contain a map-name and no map-name is provided on the command line, SiLK processes the prefix map in legacy mode. When in legacy mode, only one prefix map file may be used. See the "LEGACY" section for details.

Three types of prefix map files are currently implemented:

proto-port
Maps a protocol/port pair to a label.
IPv4-address
Maps an IPv4 address to a label. When used with IPv6 addresses, an IPv6 address in the ::ffff:0:0/96 prefix is converted to IPv4 and mapped to the label. Any other IPv6 address is mapped to the label "UNKNOWN".
IPv6-address
Maps an IPv6 address to a label. When used with an IPv4 address, the IPv4 address is converted to IPv6, mapping the IPv4 address into the ::ffff:0:0/96 prefix.

For more information on constructing prefix map files, see the rwpmapbuild(1) documentation. To view the contents, type, or map-name of a prefix map file, use rwpmapcat(1). To map textual input to the labels in a prefix map, use rwpmaplookup(1).

The --pmap-file switch is used to load the prefix map into the application. Use of the prefix map varies by application.

To use a prefix map within a supported application, one or more --pmap-file switches are required. Multiple --pmap-file switches are allowed as long as each prefix map is associated with a unique map-name. The switch has two forms:

--pmap-file=MAPNAME:FILENAME
FILENAME refers to a prefix map file generated using rwpmapbuild. MAPNAME is a name that may be used to refer to the fields or options specific to that prefix map. Specify FILENAME as "-" or "stdin" to read the prefix map from the standard input.
--pmap-file=FILENAME
When a MAPNAME is not specified explicitly as part of the argument, the prefix map file is checked to determine if a map-name was set when the prefix map was created (see rwpmapbuild). If so, that map-name is used. If not, the prefix map is processed in legacy mode for backward compatibility. See "LEGACY" below for more information. A prefix map's map-name is printed by the rwfileinfo(1) command or by specifying --output-types=mapname to rwpmapcat.

When using the prefix map in rwfilter(1), the map-name is combined with the prefix --pmap-src-, --pmap-dst-, or --pmap-any- to create the partitioning switches; that is, the switch name depends in part on the map-name of the prefix map.
--pmap-src-map-name=LABELS
If the prefix map associated with map-name is an IP prefix map, this matches records with a source address that maps to a label contained in the list of labels in LABELS.

If the prefix map associated with map-name is a proto-port prefix map, this matches records with a protocol and source port combination that maps to a label contained in the list of labels in LABELS.

--pmap-dst-map-name=LABELS
Similar to --pmap-src-map-name, but uses the destination IP or the protocol and destination port.
--pmap-any-map-name=LABELS
If the prefix map associated with map-name is an IP prefix map, this matches records with a source or destination address that maps to a label contained in the list of labels in LABELS.

If the prefix map associated with map-name is a proto-port prefix map, this matches records with a protocol and a source or destination port combination that maps to a label contained in the list of labels in LABELS.

When using the prefix map to create fields in rwcut(1), rwgroup(1), rwsort(1), rwstats(1), and rwuniq(1), the map-name must be combined with the prefix "src-" or "dst-" to create the field names. The field names depend in part on the map-name of the prefix map.
--fields=FIELDS
FIELDS refers to a list of fields to use for the operation. Each prefix map associated with map-name creates two additional fields, "src-map-name" and "dst-map-name", available for display, sorting, and counting using the rwcut, rwgroup, rwsort, rwstats, and rwuniq tools.
src-map-name
The value for the source from the prefix map file associated with map-name. For an IP-based prefix map file, this corresponds to the source IP. For a proto-port prefix map, it is the protocol/source-port.
dst-map-name
As src-map-name for the destination IP address or protocol/destination-port. It is possible to encode type and code in a proto-port prefix map, but it will only work when used for the protocol/destination-port.
--pmap-column-width=NUM
Set the maximum number of characters to use when displaying the textual value of any prefix map field in rwcut, rwstats, and rwuniq to NUM. This switch must precede the --fields switch. This switch is useful for prefix map files that have very long dictionary values.

When a prefix map file does not contain a map-name and no map-name is specified in the --pmap-file argument, SiLK processes the prefix map as it did prior to SiLK 2.0, which is called legacy mode. When in legacy mode, only one prefix map file may be used by the application. Legacy mode is deprecated, but it is maintained for backwards compatibility.

When a prefix map is loaded into rwfilter in legacy mode, the following switches are defined:
--pmap-saddress=LABELS
Match records with a source IP address that maps to a label contained in the list of labels in LABELS. Only works with IP prefix maps.
--pmap-daddress=LABELS
As --pmap-saddress for the destination IP.
--pmap-any-address=LABELS
Match records with a source or destination IP address that maps to a label contained in the list of labels in LABELS. Only works with IP prefix maps.
--pmap-sport-proto=LABELS
Match records with a protocol and source port combination that maps to a label contained in the list of labels in LABELS. Only works with proto-port prefix maps.
--pmap-dport-proto=LABELS
As --pmap-saddress for the protocol and destination port.
--pmap-any-port-proto=LABELS
Match records with a protocol and a source or destination port combination that maps to a label contained in the list of labels in LABELS. Only works with proto-port prefix maps.

When a prefix map is loaded into rwcut, rwgroup, rwsort, rwstats, or rwuniq in legacy mode, the following fields are made available to the --fields switch:
sval
The value from the prefix map file for the source. For an IP-based prefix map file, this corresponds to the source IP. For a proto-port prefix map, it is the protocol/source-port.
dval
As sval for the destination IP address or protocol/destination-port.

In the following examples, the dollar sign ("$") represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash ("\") is used to indicate a wrapped line.

The following examples explicitly specify the map name on the command line, ensuring the examples work any prefix map file. The examples use two prefix map files:

carnegiemellon.pmap
Maps the internal IP space of Carnegie Mellon to labels specifying the department that has been assigned that IP space. (An IPv4 prefix map provides a label for every IPv4 address; in this case, any IP outside of Carnegie Mellon's IP space is given the label "external".)
service.pmap
Maps protocol/ports pairs to well-known services associated with those pairs (e.g., based the file /etc/protocols and /etc/services). For example, 80/tcp is labeled "TCP/HTTP", 25/tcp is "TCP/SMTP", ephemeral ports in protocol 6 are "TCP", protocol 1 is "ICMP", etc.

To find today's incoming flow records going to "FineArts":

 $ rwfilter --type=in,inweb --pmap-file=CMU:carnegiemellon.pmap     \
        --pmap-dst-CMU="FineArts" --pass=fine-arts-in.rw

To find today's outgoing flow records coming from "ChemE":

 $ rwfilter --type=out,outweb --pmap-file=CMU:carnegiemellon.pmap   \
        --pmap-src-CMU="ChemE" --pass=cheme-out.rw

To find today's internal traffic from "FineArts" to "ChemE":

 $ rwfilter --type=int2int --pmap-file=CMU:carnegiemellon.pmap      \
        --pmap-src-CMU="FineArts" --pmap-dst-CMU="ChemE"            \
        --pass=finearts-to-cheme.rw

To find the reverse traffic:

 $ rwfilter --type=int2int --pmap-file=CMU:carnegiemellon.pmap      \
        --pmap-src-CMU="ChemE" --pmap-dst-CMU="FineArts"            \
        --pass=cheme-to-finearts.rw

To find today's internal traffic that started or ended at "FineArts" and "ChemE" (this will find traffic between them, as well as traffic they had with any other university department):

 $ rwfilter --type=int2int --pmap-file=CMU:carnegiemellon.pmap      \
        --pmap-any-CMU="ChemE,FineArts"                             \
        --pass=cheme-finearts.rw

Using the service.pmap file with rwcut to print the label for the protocol/port pairs:

 $ rwcut --pmap-file=service:service.pmap                           \
        --fields=protocol,dport,dst-service,sport,src-service       \
        flow-records.rw
 pro|dPort|dst-service|sPort|src-service|
  17|   53|    UDP/DNS|29617|        UDP|
  17|29617|        UDP|   53|    UDP/DNS|
   6|   22|    TCP/SSH|29618|        TCP|
   6|29618|        TCP|   22|    TCP/SSH|
   1|  771|       ICMP|    0|       ICMP|
  17|   67|   UDP/DHCP|   68|   UDP/DHCP|
   6|  443|  TCP/HTTPS|28816|        TCP|
   6|29897|        TCP|   25|   TCP/SMTP|
   6|29222|        TCP|   80|   TCP/HTTP|
  17|29361|        UDP|   53|    UDP/DNS|

Using the service.pmap file with rwuniq:

 $ rwuniq --pmap-file=serv:service.pmap --fields=dst-serv           \
        --values=bytes flow-records.rw
    dst-serv|               Bytes|
     TCP/SSH|          3443906999|
    TCP/SMTP|           780000305|
         TCP|        114397570896|
   TCP/HTTPS|           387741258|
    TCP/HTTP|          1526975653|
     UDP/NTP|             1176632|
         UDP|            14404581|
    UDP/DHCP|             5121392|
     UDP/DNS|             3797474|
        ICMP|            10695328|

Using the service.pmap file with rwstats:

 $ rwstats --pmap-file=srvc:service.pmap --fields=dst-srvc  \
        --values=bytes --count=5 flow-records.rw
 INPUT: 501876 Records for 10 Bins and 120571390518 Total Bytes
 OUTPUT: Top 5 Bins by Bytes
    dst-srvc|               Bytes|    %Bytes|   cumul_%|
         TCP|        114397570896| 94.879532| 94.879532|
     TCP/SSH|          3443906999|  2.856322| 97.735854|
    TCP/HTTP|          1526975653|  1.266449| 99.002303|
    TCP/SMTP|           780000305|  0.646920| 99.649223|
   TCP/HTTPS|           387741258|  0.321586| 99.970809|

Using rwsort with two prefix maps, where the records are first sorted by the originating department and then by the service they are requesting:

 $ rwsort --pmap-file=service:service.pmap                  \
        --pmap-file=cmu:carnegiemellon.pmap                 \
        --fields=src-cmu,dst-service flow-records.rw

To see the partitioning switches that a prefix map adds to rwfilter, load the prefix map file prior to specifying the --help switch.

 $ rwfilter --pmap-file=carnegiemellon.pmap --help          \
   | sed -n '/^--pmap-/p'

To see the fields that a prefix map file adds to rwcut, rwgroup, rwsort, rwstats, or rwuniq, load the prefix map file prior to specifying --help, and then view the description of the --fields switch.

 $ rwsort --pmap-file=service.pmap --help                   \
   | sed -n '/^--fields/,/^--/p'

rwcut(1), rwfilter(1), rwgroup(1), rwpmapbuild(1), rwpmapcat(1), rwpmaplookup(1), rwsort(1), rwstats(1), rwuniq(1), rwfileinfo(1), silk(7)
2022-04-12 SiLK 3.19.1
Search for    or go to Top of page |  Section 3 |  Main Index

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with ManDoc.