NAME

rwrandomizeip - Randomize the IP addresses in a SiLK Flow file

SYNOPSIS

  rwrandomizeip [--seed=NUMBER] [--only-change-set=CHANGE_IPSET]
        [--dont-change-set=KEEP_IPSET]
        [--consistent] [--save-table=FILE] [--load-table=FILE]
        [--site-config-file=FILENAME]
        [INPUT_FILE [OUTPUT_FILE]]

  rwrandomizeip --help

  rwrandomizeip --version

DESCRIPTION

Read SiLK Flow records from INPUT_FILE, substitute a pseudo-random IP address for the source and destination IP addresses, and write the result to OUTPUT_FILE.

rwrandomizeip reads its input from the standard input either when no non-switch arguments are given or when INPUT_FILE is the string "stdin" or "-". rwrandomizeip writes its output to the standard output either when the number of non-switch arguments is less than two or when OUTPUT_FILE is the string "stdout" or "-". Since rwrandomizeip processes binary data, it exits with an error if either INPUT_FILE or OUTPUT_FILE refer to a terminal. rwrandomizeip is able to read and write files that have been compressed with gzip(1) when the file name ends with ".gz".

To only change a subset of the IP addresses, the optional switches --only-change-set or --dont-change-set may be used; each switch takes an IPset file as its required argument. When the --only-change-set=CHANGE_IPSET switch is given, rwrandomizeip modifies only the IP addresses listed in the CHANGE_IPSET file. To change all addresses except a specified set, use rwsetbuild(1) to create an IPset file containing those IPs and pass the name of the file to the --dont-change-set switch. An address listed in both the only-change-set and the dont-change-set is not modified. When the same IPset is passed to the --only-change-set and --dont-change-set switches, the output is identical to the input for all records.

The --seed switch may be used to initialize the pseudo-random number generator to a known state.

Use of the --consistent, --load-table, or --save-table switches causes rwrandomizeip to operate in consistent mode. When none of the switches are specified, it operates in inconsistent mode.

Consistent Mode

In consistent mode, the octets of an IPv4 address are modified such that structural information of the data is maintained, and multiple instances of an input IP address are mapped to the same randomized output address. Unfortunately, this comes at a cost of less randomness in the output. Specifically, rwrandomizeip creates four internal tables with each table having 256 entries containing the values 0--255 that have been randomly shuffled. When an IP address is read, each table is used to map the values for a specific octet of that IP address. For example, when modifying the IP address 10.10.10.10, the value at position 10 from each table is substituted into the IP.

Inconsistent Mode

In this mode, rwrandomizeip uses a pseudo-random address for each source and destination IP address it processes. Each record is handled individually, and an IP address that appears multiple times in the input file is mapped to a different output address each time. Thus, no structural information in the input is maintained. rwrandomizeip changes each IP address to a non-routable address from the CIDR blocks 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--seed=NUMBER: Use NUMBER to seed the pseudo-random number generator. This may be used to put the random number generator into a known state, which is useful for testing.
--only-change-set=CHANGE_IPSET: Only modify the source or destination IP address if it appears in the given IPset file CHANGE_IPSET. The rwsetbuild command may be used to create an IPset file. When the --dont-change-set=KEEP_IPSET switch is also given, the IPs it contains override those in the CHANGE_IPSET file.
--dont-change-set=KEEP_IPSET: Do not modify the source or destination IP address if the address appears in the given IPset file KEEP_IPSET. The rwsetbuild command may be used to create an IPset file. The interaction of this switch with the --only-change-set switch is described immediately above.
--consistent: Randomize the IP addresses consistently, so that an input IP address is always mapped to the same value. The default behavior is to use a random IP address for each IP, even if the IP has been seen before.
--save-table=FILE: Randomize the IP addresses consistently and save this run's randomization table for future use. The table is written to the specified FILE, which must not not exist. This switch is incompatible with the --load-table switch.
--load-table=FILE: Randomize the IP addresses consistently using the randomization table contained in FILE that was created by a previous invocation of rwrandomizeip. This switch is incompatible with the --save-table switch.
--site-config-file=FILENAME: Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, rwrandomizeip searches for the site configuration file in the locations specified in the "FILES" section.
--help: Print the available options and exit.
--version: Print the version number and information about how SiLK was configured, then exit the application.

ENVIRONMENT

SILK_CLOBBER: The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction.
SILK_CONFIG_FILE: This environment variable is used as the value for the --site-config-file when that switch is not provided.
SILK_DATA_ROOTDIR: This variable gives the root of the directory tree where the data store of SiLK Flow files is maintained, overriding the location that is compiled into the tools (/data). rwswapbytes may search for the site configuration file, silk.conf, in this directory. See the "FILES" section for details.
SILK_PATH: This environment variable gives the root of the directory tree where the tools are installed. As part of its search for the site configuration file, rwrandomizeip may use this variable. See the "FILES" section for details.

FILES

${SILK_CONFIG_FILE}
${SILK_DATA_ROOTDIR}/silk.conf
/data/silk.conf
${SILK_PATH}/share/silk/silk.conf
${SILK_PATH}/share/silk.conf
/usr/local/share/silk/silk.conf
/usr/local/share/silk.conf: Possible locations for the SiLK site configuration file.

BUGS

rwrandomizeip does not support IPv6 flow records. When an input file contains IPv6 records, rwrandomizeip converts records that contain addresses in the ::ffff:0:0/96 prefix to IPv4 and processes them. rwrandomizeip silently ignores IPv6 records containing addresses outside of that prefix.

Only the source and destination IP fields are modified; additional fields in the SiLK Flow records may leak sensitive information.

Prior to SiLK 3.16.0, rwrandomizeip required explicit arguments for the input file and the output file.