NAME

ipaggmanip - manipulate aggregate statistics, often of network traffic

SYNOPSIS

ipaggmanip [transformation options...] [file] > output

DESCRIPTION

The ipaggmanip program reads an aggregate file summarizing IP trace data (or other data), transforms that file or calculates one of its statistics, and writes the result to standard output. Aggregate files use the format produced by ipaggcreate(1), and can be text or binary.

An aggregate file consists of pairs of labels and counts. The resulting dataset is modeled either as a partial function mapping labels to counts, or as a multiset of labels. Elements of the multiset are called "packets" for convenience, although counts need not correspond to network packets. For example, the --sample option is described as sampling "packets"; this means that the aggregate file is treated as a multiset of labels, and each element of the multiset is dropped with a uniform probability. When an option refers explicitly to a "label", the partial function representation is assumed. Thus, the --num-labels option reports the number of labels. This is the same as the number of labels whose count is greater than 0, since labels with 0 counts are left out of the partial function.

Several options refer to "p-aggregates". A p-aggregate consists of a set of labels whose high-order p bits are the same. Labels are 32 bits long, so a 32-aggregate is the same as a label. Consider an aggregate file with the following data:

  0 3
  1 2
  2 1

This file contains two 31-aggregates, as follows:

  0 5    # combines labels 0 and 1
  2 1    # only label 2

A p-aggregate label has the lower 32-p bits set to zero.

OPTIONS

Transformations

Transformation options change an aggregate file into another aggregate file. You may supply several transformation options; they are applied in order. Thus, for example, `ipaggmanip --prefix 16 --posterize' will read an aggregate file, aggregate to prefix level 16, and then replace all nonzero counts with 1. This has the same effect as `ipaggmanip --prefix 16 | ipaggmanip --posterize'.

--prefix p, -p p

Aggregates to prefix level p. That is, replaces each label with its p-aggregate's label and outputs the result. The counts from any labels in the same aggregate are combined.

--posterize, -P

Replaces each label's (nonzero) count with 1.

--sample n

Sample packets with uniform random probability 1/n. The output will have roughly n times fewer packets compared with the input.

--cull n

Reduce the input to at most n packets by randomly sampling packets. Input aggregates with no more than n packets are output unchanged.

--cull-labels n

Reduce the input to at most n labels by randomly sampling labels. Input aggregates with n or fewer labels are output unchanged. If a label is included in the output, then its count is the same as in the input.

--cull-labels-by-packets n

Reduce the input to at most n labels by randomly sampling packets. That is, throw away packets one at a time until the aggregate has at most n labels. Input aggregates with n or fewer labels are output unchanged. Labels included in the output may have lower counts than in the input because of the packet sampling.

--cut-smaller n

Drop labels with count less than n. Other labels are left unchanged.

--cut-larger n

Drop labels with count greater than or equal to n.

--cut-smaller-aggregates p,n

Drop all labels whose containing p-aggregates contain less than n packets. For instance, given this input:

the `--cut-smaller-aggregates 31,3' option would produce:

  0 1
  1 3

Labels 2 and 3 have been dropped because their shared 31-aggregate contains only 2 packets.

--cut-larger-aggregates p,n

Drop all labels whose containing p-aggregates contain greater than or equal to n packets.

--cut-smaller-label-aggregates p,n

Drop all labels whose containing p-aggregates contain greater than or equal to n labels. For instance, given this input:

the `--cut-smaller-label-aggregates 30,3' option would produce:

  0 1
  1 1
  2 1

Labels 4 and 5 have been dropped because their shared 30-aggregate contains only 2 labels.

--cut-larger-label-aggregates p,n

Drop all labels whose containing p-aggregates contain greater than or equal to n labels.

--fake-by-discriminating-prefixes[=type]

--fake-by-branching-counts

--fake-by-dirichlet

--remap-prefixes arg

Actions

Action options calculate a statistic from an aggregate file and output that statistic. Each ipaggmanip run can contain at most one action. Unless otherwise noted, statistics containing multiple numbers are output on one line, separated by spaces.

--num-labels, -n: Output the number of labels.
--num-in-prefixes: Output the number of active p-aggregates for each p, 0<=p<=32. The result is 33 space-separated numbers; the first is the number of active 0-aggregates, the last the number of active 32-aggregates (that is, the number of labels). The pth number is at most 2^p.
--num-in-left-prefixes: Output the number of active left-hand p-aggregates for each p, 0<=p<=32. A left-hand p-aggregate has its lowest-order bit equal to 0. Consider label 6, which identifies a 32-aggregate and a 31-aggregate. (It does not identify a 30-aggregate or above.) Label 6 is a left-hand 32-aggregate, since bit 31 is 0, but it is right-hand 31-aggregate, since bit 30 is 1. As a special case, the single 0-aggregate is considered left-hand. The result is 33 space-separated numbers, as in --num-in-prefixes.
--discriminating-prefix-counts: Output the number of labels that have discriminating prefix p for each p, 0<=p<=32. The discriminating prefix for a label L is the smallest p so that the p-aggregate containing L contains no other label. The result is 33 space-separated numbers whose sum will equal --num-labels.
--all-discriminating-prefix-counts: The output is 33 lines, numbered 0 through 32. Line number p equals the result of `--prefix p --discriminating-prefix-counts' for the input data: that is, the discriminating prefixes for the data's p-aggregates. The discriminating prefix for a p-aggregate must be less than or equal to p, so line p contains p+1 space-separated numbers.
--counts: Output the count for each active label, sorted in label order. The result is --num-labels space-separated numbers.
--sorted-counts: Output the count for each active label, sorted in descending order by count. The result is --num-labels space-separated numbers.
--count-counts: For each count c, calculate n, the number of labels that have count c. Output a series of lines containing "c n", sorted in ascending order by c.
--container-counts p: Calculate the number of packets in each label's containing p-aggregate. Output each active label's value, sorted in label order. The result is --num-labels space-separated numbers.
--balance p
--average-and-variance
--average-and-variance-by-prefix
--haar-wavelet-energy
--balance n
--balance-histogram n,nbuckets
--branching-counts p,step
--all-branching-counts step
--conditional-split-counts p
--correlation-size-container-addresses p

Multiple files

--each, -e
--or, -|
--and, -&
--minus
--xor, -^
--and-list
--assign-counts

Other options

--read file, -r file
--output file, -o file
--binary, -b
--text
--ip
--help, -h: Print a help message to the standard output, then exit.
--version, -v: Print version number and license information to the standard output, then exit.

AUTHOR

Eddie Kohler <kohler@cs.ucla.edu>, based on the Click modular router.