|
NAMEnfdump - netflow display and analyze programSYNOPSISnfdump [options] [filter] DESCRIPTIONnfdump is the netflow display and analyzing program of the nfdump tool set. It reads the netflow data from files stored by nfcapd and processes the flows according the options given. The filter syntax is comparable to tcpdump and extended for netflow data. Nfdump can also display many different top N flow and flow element statistics.OPTIONS
/any/dir Read recursively all files in directory
dir.
/dir/file Read all files beginning with file. /dir/file1:file2 Read all files from file1 to file2. When using in combination with a sub hierarchy: /dir/sub1/sub2/file1:sub3/sub4/file2 Read all files from sub1/sub2/file1 sub3/sub4/file2 iterating over all required hierarchy levels. Note: files are read in alphabetical sequence.
flows Sort according the number of flows
packets Sort according to (in)packets ipkg Same as packets opkg Sort according to output packets bytes Sort according to (in)bytes ibyte Same as bytes obyte Sort according to output bytes pps Sort according to (in)packets per second ipps Same as ipps opps Sort according to out packets per second bps Sort according to (in)bytes per second ibps Same as bps obps Sort according to output bytes per second bpp Sort according to (in)bytes per packet ibpp Same as bpp obpp Sort according to output packets tstart Sort according to start time of flow - former -m tend Sort according to end time of flows
proto IP protocol
srcip Source IP address dstip Destination IP address srcip4/net IPv4 source IP address with applied netmask srcip6/net IPv6 source IP address with applied netmask dstip4/net IPv4 destination IP address with applied netmask dstip6/net IPv6 destination IP address with applied netmask srcnet Apply netmask srcmask in netflow record for source IP dstnet Apply netmask dstmask in netflow record for dest IP srcport Source port dstport Destination port srcmask Source mask dstmask Destination mask srcvlan Source vlan label dstvlan Destination vlan label srcas Source AS number dstas Destination AS number nextas BGP Next AS prevas BGP Previous AS inif SNMP input interface number outif SNMP output interface number next IP next hop bgpnext BGP next hop insrcmac In source MAC address outdstmac out destination MAC address indstmac In destintation MAC address outsrcmac Out source MAC address tos Source type of service srctos Source type of Service dsttos Destination type of Service mpls1 MPLS label 1 mpls2 MPLS label 2 mpls3 MPLS label 3 mpls4 MPLS label 4 mpls5 MPLS label 5 mpls6 MPLS label 6 mpls7 MPLS label 7 mpls8 MPLS label 8 mpls9 MPLS label 9 mpls10 MPLS label 10 router Exporting router IP xsrcip X-late source IP address, if compiled with NSEL support xdstip X-late destination IP address, if compiled with NSEL support xsrcport X-late source port, if compiled with NSEL support xdstport X-late destination port, if compiled with NSEL support nfdump automatically compiles an appropriate output format for the selected aggregation unless an explicit output format is given. The automatic output format is identical to -o 'fmt:%ts %td <fields> %pkt %byt %bps %bpp %fl' where <fields> represents the selected aggregation tags. Example: -A proto,srcip,dstport
-A srcas,dstas
record Statistic about arregated netflow records.
srcip Statistic about source IP addresses dstip Statistic about destination IP addresses ip Statistic about any (source or destination) IP addresses nhip Statistic about next hop IP addresses nhbip Statistic about BGP next hop IP addresses router Statistic about exporting router IP address srcport Statistic about source ports dstport Statistic about destination ports port Statistic about any (source or destination) ports tos Statistic about type of service - default src srctos Statistic about src type of service dsttos Statistic about dst type of service dir Statistic about flow directions ingress/egress srcas Statistic about source AS numbers dstas Statistic about destination AS numbers as Statistic about any (source or destination) AS numbers inif Statistic about input interface outif Statistic about output interface if Statistic about any interface srcmask Statistic about src mask dstmask Statistic about dst mask srcvlan Statistic about src vlan label dstvlan Statistic about dst vlan label vlan Statistic about any vlan label insrcmac Statistic about input src MAC address outdstmac Statistic about output dst MAC address indstmac Statistic about input dst MAC address outsrcmac Statistic about output src MAC address srcmac Statistic about any src MAC address dstmac Statistic about any dst MAC address inmac Statistic about any input MAC address outmac Statistic about any output MAC address mask Statistic about any mask proto Statistic about IP protocols mpls1 Statistic about MPLS label 1 mpls2 Statistic about MPLS label 2 mpls3 Statistic about MPLS label 3 mpls4 Statistic about MPLS label 4 mpls5 Statistic about MPLS label 5 mpls6 Statistic about MPLS label 6 mpls7 Statistic about MPLS label 7 mpls8 Statistic about MPLS label 8 mpls9 Statistic about MPLS label 9 mpls10 Statistic about MPLS label 10 sysid Internal SysID of exporter NSEL/ASA stats
NAT stats
By adding :p to the statistic name, the resulting statistic is split up into transport layer protocols. Default is transport protocol independent statistics. orderby is optional and specifies the order by which the statistics is ordered and can be flows, packets, bytes, pps, bps or bpp. You may specify more than one orderby which results in the same statistic but ordered differently. If no orderby is given, statistics are ordered by flows. You can specify as many -s flow element statistics on the command line for the same run. Example: -s srcip -s ip/flows -s dstport/pps/packets/bytes -s
record/bytes
raw Print full flow record on multiple lines.
line Print each flow on one line. Default format. long Print each flow on one line with more details biline Same as line, but for bidir flows bilong Same as long, but for bidir flows extended Print each flow on one line with even more details. nsel Print each NSEL event on one line. Default if NSEL/NAT nel Print each NAT event on one line. csv Comma separated output for machine readable processing. json Print full record as separate json object pipe Legacy machine readable format: fields '|' separated. fmt:format User defined output format. For each defined output format except -o
fmt:<format> an IPv6 long output format exists. line6, long6 and
extended6. See output formats below for more information.
RETURN VALUEReturns0 No error.
255 Initialization failed. 254 Error in filter syntax. 250 Internal error. OUTPUT FORMATSThe output format raw prints each flow record on multiple lines, including all information available in the record. This is the most detailed view on a flow.Other output formats print each flow on a single line. Predefined output formats are line, long and extended The output format line is the default output format when no format is specified. It limits the imformation to the connection details as well as number of packets, bytes and flows. The output format long is identical to the format line, and includes additional information such as TCP flags and Type of Service. The output format extended is identical to the format long, and includes additional computed information such as pps, bps and bpp. Fields: Date flow start: Start time flow first seen. ISO
8601 format including milliseconds.
Duration: Duration of the flow in seconds and milliseconds. If flows are aggregated, duration is the time span over the entire periode of time from first seen to last seen. Proto: Protocol used in the connection. Src IP Addr:Port: Source IP address and source port. Dst IP Addr:Port: Destination IP address and destination port. In case of ICMP, port is decodes as type.code. Flags: TCP flags ORed of the connection. Tos: Type of service. Packets: The number of packets in this flow. If flows are aggregated, the packets are summed up. Bytes: The number of bytes in this flow. If flows are aggregated, the bytes are summed up. pps: The calculated packets per second: number of packets / duration. If flows are aggregated this results in the average pps during this periode of time. bps: The calculated bits per second: 8 * number of bytes / duration. If flows are aggregated this results in the average bps during this periode of time. Bpp: The calculated bytes per packet: number of bytes / number of packets. If flows are aggregated this results in the average bpp during this periode of time. Flows: Number of flows. If flows are listed only, this number is always 1. If flows are aggregated, this shows the number of aggregated flows to one record. Numbers larger than 1'000'000 (1000*1000), are scaled to 4 digits and one decimal digit including the scaling factor M, G or T for cleaner output, e.g. 923.4 M To make the output more readable, IPv6 addresses are shrinked down to 16 characters. The seven most and seven least digits connected with two dots '..' are displayed in any normal output formats. To display the full IPv6 address, use the appropriate long format, which is the format name followed by a 6. Example: -o line displays an IPv6 address as 2001:23..80:d01e where as the format -o line6 displays the IPv6 address in full length 2001:234:aabb::211:24ff:fe80:d01e. The combination of -o line -6 is equivalent to -o line6. The output format fmt:<format> allows you to define your own output format. A format description format consists of a single line containing arbitrary strings and format specifier as described below %<format> Inserts the predefined
format at this position. e.g. %line
%ff flow record flags in hex. %ts Start Time - first seen %tsr Start Time, but in fractional seconds since the epoch (1970-01-01) %te End Time - last seen %ter End Time, in fractional seconds %tr Time the flow was received by the collector %trr Time the flow was received, in fractional seconds %td Duration %pr Protocol %exp Exporter ID %eng Engine Type/ID %lbl Flowlabel %sa Source Address %da Destination Address %sap Source Address:Port %dap Destination Address:Port %sp Source Port %dp Destination Port %sn Source Network, mask applied %dn Destination Network, mask applied %nh Next-hop IP Address %nhb BGP Next-hop IP Address %ra Router IP Address %sas Source AS %das Destination AS %nas Next AS %pas Previous AS %in Input Interface num %out Output Interface num %pkt Packets - default input %ipkt Input Packets %opkt Output Packets %byt Bytes - default input %ibyt Input Bytes %obyt Output Bytes %fl Flows %flg TCP Flags %tos Tos - default src %stos Src Tos %dtos Dst Tos %dir Direction: ingress, egress %smk Src mask %dmk Dst mask %fwd Forwarding Status %svln Src vlan label %dvln Dst vlan label %ismc Input Src Mac Addr %odmc Output Dst Mac Addr %idmc Input Dst Mac Addr %osmc Output Src Mac Addr %mpls1 MPLS label 1 %mpls2 MPLS label 2 %mpls3 MPLS label 3 %mpls4 MPLS label 4 %mpls5 MPLS label 5 %mpls6 MPLS label 6 %mpls7 MPLS label 7 %mpls8 MPLS label 8 %mpls9 MPLS label 9 %mpls10 MPLS label 10 %mpls MPLS labels 1-10 %bps bps - bits per second %pps pps - packets per second %bpp bps - Bytes per package NSEL specific formats
NEL/NAT specific formats
Nprobe formats
The "flow flags" format (%ff) prints the internal record flags as a single hexadecimal number, consisting of any of these flag values or-ed together: 1 Record contains IPv6 addresses 2 Packet counters are 64-bit 4 Byte counters are 64-bit 8 IP next hop is an IPv6 address 16 BGP next hop is an IPv6 address 32 Exporting router is an IPv6 address 64 Record is an EVENT record 128 Record is sampled Example: the standard output format long can be created as -o "fmt:%ts %td %pr %sap -> %dap %flg %tos
%pkt %byt %fl"
You may also define your own output format and have it compiled into nfdump. See nfdump.c section Output Formats for more details. The csv output format is intended to be read by another program for further processing. As an example, see the parse_csv.pl Perl program. The cvs output format consists of one or more output blocks and one summary block. Each output block starts with a cvs index line followed by the cvs record lines. The index lines describes the order, how each following record is composed. Example: Index line: ts,te,td,sa,da,sp,dp,pr,...
Record line: 2004-07-11 10:30:00,2004-07-11 10:30:10,10.010,... All records are in ASCII readable form. Numbers are not scaled, so each line can easily be parsed. Indices used in nfdump 1.6: ts,te,td time records: t-start, t-end, duration
sa,da src dst address sp,dp src, dst port pr protocol PF_INET or PF_INET6 flg TCP Flags: 000001 FIN. 000010 SYN 000100 RESET 001000 PUSH 010000 ACK 100000 URGENT e.g. 6 => SYN + RESET fwd forwarding status stos src tos ipkt,ibyt input packets/bytes opkt,obyt output packets, bytes in,out input/output interface SNMP number sas,das src, dst AS smk,dmk src, dst mask dtos dst tos dir direction nh,nhb nethop IP address, bgp next hop IP svln,dvln src, dst vlan id ismc,odmc input src, output dst MAC idmc,osmc input dst, output src MAC mpls1,mpls2 MPLS label 1-10 mpls3,mpls4 mpls5,mpls6 mpls7,mpls8 mpls9,mpls10 ra router IP eng router engine type/id See parse_csv.pl for more details. FILTERThe filter syntax is similar to the well known pcap library used by tcpdump. The filter can be either specified on the command line after all options or in a separate file. It can span several lines. Anything after a '#' is treated as a comment and ignored to the end of the line. There is virtually no limit in the length of the filter expression. All keywords are case independent.Any filter consists of one or more expressions expr. Any number of expr can be linked together: expr and expr, expr or expr, not expr and ( expr ). Expr can be one of the following filter primitives:
[src|dst] ip <ipaddr>
[src|dst] host <ipaddr> with <ipaddr> as any valid IPv4, IPv6 address, or a full qualified hostname. In case of a hostname, the IP address is looked up in DNS. If more than a single IP address is found, all IP addresses are chained together. (ip1 or ip2 or ip3 ... ) To check if an IP address is in a known IP list, use
[src|dst] port [comp] <num>
with <num> as any valid port number. If comp is omitted, '=' is assumed. comp is explained more detailed below. [src|dst] port in [ <portlist> ] A port can be compared against a know list, where <portlist> is a space separated list of individual port numbers.
icmp-type <num>
icmp-code <num> with <num> as a valid icmp type/code. This automatically implies proto icmp.
engine-type <num>
engine-id <num> sysid <num> with <num> as a valid router engine type/id or exporter ID(0..255).
A ACK.
The ordering of the flags is not relevant. Flags not mentioned are treated as
don't care. In order to get those flows with only the SYN flag set, use the
syntax 'flags S and not flags AFRPU'.
S SYN. F FIN. R Reset. P Push. U Urgent. X All flags on.
NSEL/ASA Event asa event <ignore|create|term|delete|deny>
asa event [comp] <number> select NSEL/ASA event by name or number. If given as number it can be compared with a number
asa event denied
<ingress|egress|interface|nosyn>
Select a NSEL/ASA denied event by type
asa xevent [comp] <num>
Select an extended NSELL ASA event by number, or optionally compared by a number.
[src|dst] xip <ip>
Select the translated IP address [src|dst] xnet <net>/<num>
[src|dst] xport <port>
ingress <ACL|ACE|XACE> [comp] number
Select/compare an ingress ACL egress ACL [comp] <number>
NAT Event nat event <add|delete>
nat event [comp] <number> select NEL NAT event by name or number. If given as number it can be compared with a number
[src|dst] nip <ip>
Select the NAT IP address [src|dst] nport <port>
FlowlabelOne or more specific filter expressions can be assigned a flowlabel in order to identify the flow in the output according to the label. A flowlabel has the form %LabelName and is appended or prepended to a filter expression in braces. It may have up to 16 characters. Example: (ip 8.8.8.8) %GoogleDNS. If a filter matches, with a labeled expressions, and that expression is in the matching filter patch, the label can be printed in the output, using the %%lbl format token. See OUTPUT FORMATS. Example: Add flowlabel to end of 'line' format:./nfdump -r <file> -o 'fmt:%line %lbl" .. Note: A filter may have multiple matching paths - for example proto tcp or ip 8.8.8.8 The shortest path which evaluates successfully, wins. Other paths are skipped, which means that flowlabels are not printed in not evaluated filter paths. A filter may contain multiple flowlabels. The flowlabel of the last matching expression in the winning path is printed. Flowlabels are most useful in large and complex filters stored in one or multiple files, to better read the flow output list. Example: (ip in [172.16.1.0/24]) %ISP_1 or (ip in [172.16.16.0/24]) %IPS_2 or %GoogleDNS((proto udp or proto tcp) and ip 8.8.8.8) EXAMPLESnfdump -r /and/dir/nfcapd.201107110845 -c 100 'proto tcp and ( src ip 172.16.17.18 or dst ip 172.16.17.19 )' Dumps the first 100 netflow records which match the given filter:nfdump -r /and/dir/nfcapd.201107110845 -B Map matching flows as bin-directional single flow. nfdump -R /and/dir/nfcapd.201107110845:nfcapd.200407110945 'host 192.168.1.2' Dumps all netflow records of host 192.168.1.2 from July 11 08:45 - 09:45 nfdump -M /to/and/dir1:dir2 -R nfcapd.200407110845:nfcapd.200407110945 -s record -n 20 Generates the Top 20 statistics from 08:45 to 09:45 from 3 sources nfdump -r /and/dir/nfcapd.201107110845 -s record -n 20 -o extended Generates the Top 20 statistics, extended output format nfdump -r /and/dir/nfcapd.201107110845 -s record -n 20 'in if 5 and bps > 10k' Generates the Top 20 statistics from flows coming from interface 5 nfdump -r /and/dir/nfcapd.201107110845 'inet6 and proto tcp and ( src port > 1024 and dst port 80 ) Dumps all port 80 IPv6 connections to any web server. NOTESGenerating the statistics for data files of a few hundred MB is no problem. However be careful if you want to create statistics of several GB of data. This may consume a lot of memory and can take a while. Flow anonymization has moved into nfanon.SEE ALSOnfcapd(1), nfanon(1), nfprofile(1), nfreplay(1)BUGSThere is still the famous last bug. Please report them - all the last bugs - back to me.
Visit the GSP FreeBSD Man Page Interface. |