CONTROL DESCRIPTIONS

AnalyzeBlockSize: [1-1048576]

Applies to dig, mad, and map.

AnalyzeBlockSize is optional. It instructs the analysis engine to use the specified block size (in bytes) when reading and processing file data. The default value for this control is 32768 (32 KB).

AnalyzeByteCount: [0-18446744073709551615]

Applies to dig, mad, and map.

AnalyzeByteCount is optional. It instructs the analysis engine to stop processing once the specified amount of data (in bytes) has been read/analyzed. The default value for this control is zero, which means do not impose a byte count limit.

Note: When this control is set, map mode hash values and XMagic file types may be different from those computed during a traditional snapshot (i.e., one where all bytes in each file are processed). Other than the information recorded in the log file, there is no convenient way to distinguish between traditional snapshots and those where a byte count limit. This could lead to unexpected compare mode results.

Note: Support for this control was added in version 3.9.0.

AnalyzeCarrySize: [1-1048576]

Applies to dig, and mad.

AnalyzeCarrySize is optional. It instructs the analysis engine to use the specified block size (in bytes) when saving (or carrying) data from one dig operation to the next. The default value for this control is 1024 (1 KB).

Note: The value for this control must not exceed AnalyzeBlockSize, and it must be equal to or larger than the maximum string length for normal and case insenstive strings. If either condition is not met, the program will abort.

AnalyzeDeviceFiles: [Y|N]

Applies to dig, mad, and map.

AnalyzeDeviceFiles is optional. When enabled ('Y' or 'y'), it instructs FTimes to analyze block/character device files that have been specifically included by name on the command line or through an Include (e.g., Include=/dev/ad0). Device files that reside in an included directory (e.g., Include=/dev) are not analyzed simply because their parent was included -- you must specifically call them out. Also, any device files that were specifically included will be pruned if their parent or any higher level directory was included too. The default value is 'N'.

Note: Analyzing block/character device files can take a very long time or forever (e.g., /dev/zero).

AnalyzeMaxDps: [0-2097152]

Applies to dig, mad, and map.

AnalyzeMaxDps is optional. It instructs the analysis engine to throttle the Data Processing Speed (DPS) to the specified number of KB per second. This capability is useful for systems that are sensitive to the load imposed by running this utility. The default value is zero, which means do not throttle the DPS.

Note: This is a best-effort control. In other words, there is no guarantee that the final DPS will precisely match the specified limit.

Note: This control has no influence over the priority of the executing process.

Note: The upper limit is fixed according to following relation: 2^31/1024, which guarantees that a signed integer can represent the maximum DPS as either KB/s or B/s.

Note: Support for this control was added in version 3.9.0.

AnalyzeRemoteFiles: [Y|N]

Applies to dig, mad, and map.

AnalyzeRemoteFiles is optional. When enabled ('Y' or 'y'), it instructs FTimes to ignore the fact that a given Include does not reside on the local system. The result is that FTimes will attempt to analyze remote files. The default value is 'N'.

Note: Analyzing remote file systems can create large amounts of network traffic. Just remember that you may be mapping an entire disk.

AnalyzeStartOffset: [0-18446744073709551615]

Applies to dig, mad, and map.

AnalyzeStartOffset is optional. It instructs the analysis engine to seek to the specified offset (in bytes) before processing any data. The default value for this control is zero.

Note: To map or dig through a supported block/character device file, AnalyzeDeviceFiles must be enabled and the device file must be explicitly included on the command line or in the config file. FTimes refuses to read device files by default since that could result in hung process.

Note: Device files may require you to specify an offset that is aligned to a block boundary. Your mileage may vary.

Note: When this control is set, map mode hash values and XMagic file types will be different from those computed during a traditional snapshot (i.e., one where the start offset was zero). Other than the information recorded in the log file, there is no convenient way to distinguish between traditional snapshots and those where an offset was set. This could lead to unexpected compare mode results.

Note: Support for this control was added in version 3.9.0.

AnalyzeStepSize: [1-1048576]

Applies to dig, and mad.

AnalyzeStepSize is optional. It instructs the analysis engine to step through the search buffer in increments of the specified size. Currently, this control may only be used in conjunction with the DigStringXMagic control. Internally, AnalyzeStepSize is capped to prevent its value from exceeding the value for AnalyzeBlockSize. The default value for this control is 32768 (32 KB).

Note: This control is only available if XMagic support was compiled into the binary.

BaseName: <name|->

Applies to dig, get, mad, and map.

BaseName is required. It specifies the name prefix that will be attached to the various output files. It also serves as the CLIENTID parameter in GET/PING/PUT requests. The recommended name format is one that matches the following regular expression:

    ^[.0-9A-Za-z_-]{1,64}$
    

This is because nph-ftimes.cgi uses that expression to validate the CLIENTID parameter in GET/PING/PUT requests. Typically, BaseName and URLUsername will be the same when basic authentication is enabled, but this is not a requirement.

A good naming convention would be to use the hostname of the system being baselined. Also, you may specify a BaseName value of '-'. This causes FTimes to write its output to stdout/stderr.

BaseNameSuffix: [datetime|none|pid]

Applies to dig, mad, and map.

BaseNameSuffix is optional. It specifies the type of suffix that will be attached to the BaseName. If BaseNameSuffix is set to datetime, a suffix of the following format will be appended to the BaseName: YYYYMMDDHHMMSS. If it is set to none, no suffix will be appended, and if it is set to pid, the value of the current process ID will be appended. The default value is none.

Compress: [Y|N]

Applies to mad and map.

Compress is optional. When enabled ('Y' or 'y'), it activates a form of lossless ASCII compression. This yields a compression ratio that can be as good as three to one. The default value is 'N'.

As a side note, compressing compressed snapshots with a program like gzip(1) yields better compression than if gzip(1) was used alone on the same data in its uncompressed form.

DigString: <string> [<tag>]

Applies to digauto, dig, and mad.

DigString is an alias for DigStringNormal, and it is being phased out. Please use DigStringNormal instead.

DigStringNoCase: <string> [<tag>]

Applies to digauto, dig, and mad.

DigStringNoCase is conditionally required. It specifies a case insensitive search string. This string must be URL encoded in the same manner as a normal dig string -- refer to the DigStringNormal control description for the details. Internally, all alpha characters (i.e., [A-Za-z]) are converted to lower case.

An optional tag may specified. Tags can be used to identify a given dig string or set of strings -- the same tag may be assigned to multiple strings. This can make the process of analyzing dig output much easier. However, tags must not contain embedded white space, and they must be less than 64 bytes long.

DigStringNormal: <string> [<tag>]

Applies to digauto, dig, and mad.

DigStringNormal is conditionally required. It specifies a search string. This string must be URL encoded if it contains '%', '+', ' ', or any non-printable characters. When in doubt about whether or not a value should be encoded, encode it. To encode a character, convert its hex value according to the format %HH where H is a hex digit. Spaces may alternatively be encoded as '+'.

An optional tag may specified. Tags can be used to identify a given dig string or set of strings -- the same tag may be assigned to multiple strings. This can make the process of analyzing dig output much easier. However, tags must not contain embedded white space, and they must be less than 64 bytes long.

DigStringRegExp: <regexp> [<tag>]

Applies to digauto, dig, and mad.

DigStringRegExp is conditionally required. It specifies a Perl compatible regular expression. Unlike the strings specified in the DigStringNormal and DigStringNoCase controls, this string must not be URL encoded. With DigStringRegExp patterns, you must specify no more than nine (9) capturing '()' subpatterns. You can use '(?:)' if you require additional parentheses for grouping purposes. If you do not specify a capturing subpattern, the entire match will be captured and used as the output value. Otherwise, the output value is taken from \1 -- (i.e., the first capturing subpattern).

An optional tag may specified. Tags can be used to identify a given dig string or set of strings -- the same tag may be assigned to multiple strings. This can make the process of analyzing dig output much easier. However, tags must not contain embedded white space, and they must be less than 64 bytes long.

Note: This control is only available if PCRE support was compiled into the binary. As of version 3.5.0, PCRE support is enabled by default.

Note: Embedded white space is not currently allowed in the regular expression. Use '\x20' as a substitute for a single space.

DigStringXMagic: <file> [<tag>]

Applies to digauto, dig, and mad.

DigStringXMagic is conditionally required. It specifies the name of a file that contains one or more XMagic incantations. The XMagic file may be specified as a relative path.

An optional tag may specified. Tags can be used to identify a given dig string or set of strings -- the same tag may be assigned to multiple strings. This can make the process of analyzing dig output much easier. However, tags must not contain embedded white space, and they must be less than 64 bytes long.

Note: This control is only available if XMagic support was compiled into the binary.

EnableRecursion: [Y|N]

Applies to dig, mad, and map.

EnableRecursion is optional. When enabled ('Y' or 'y'), it instructs FTimes to recursively process directories. The default value is 'Y'.

Exclude: [directory|file|link]

Applies to dig, mad, and map.

Exclude controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one Exclude control/value pair per line. It is not necessary to explicitly exclude special file systems such as PROCFS as FTimes will detect their presence and automatically exclude them. Exclude values must be specified as a fully qualified path (see Include control). If ExcludesMustExist is enabled, then each Exclude must reference an existing file, directory, or symbolic link. Otherwise, FTimes will abort.

If you need to exclude paths that contain special or non-printable characters, you must specify the path in the form of a file URL. Special characters are defined (in square brackets) as "[`'"|%+#]" or any non-printable character except for the NULL byte (%00), which is not allowed for excludes. For example, the path '/tmp/foo#bar' would need to be specified as 'file:///tmp/foo%23bar'.

Note: Symbolic links are not supported in WIN32-based file systems.

Note: The exlude mechanism works on an exact match basis, but it can be used to produce a recursive effect. For example, if you include '/' and exclude '/etc', then '/etc' and anything below it will not be processed. However, if you include '/etc/hosts' and exclude '/etc', then '/etc/hosts' will be processed because the recursive effect would not be in play, and there is no Exclude that exactly matches it.

ExcludeFilter: <regexp>

Applies to dig, mad, and map.

ExcludeFilter controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one ExcludeFilter control/value pair per line.

Each ExcludeFilter must be specified as a Perl compatible regular expression with the caveat that no more than nine capturing '()' subpatterns for backreferences are allowed. Use '(?:)' if you require parentheses for grouping purposes.

Any path matched by an ExcludeFilter will be rejected. All other paths will be mapped subject to include filter processing. If no exclude filters are specified, exclude filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

ExcludeFilterMd5: <regexp>

Applies to mad, and map.

ExcludeFilterMd5 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one ExcludeFilterMd5 control/value pair per line.

Each ExcludeFilterMd5 value must be specified as a Perl compatible regular expression with the exception that capturing '()' subpatterns and back references are not allowed. You can use '(?:)' if you require parentheses for grouping purposes.

Any record containing an MD5 that is matched by an ExcludeFilterMd5 value will be rejected. All other records will be written to the output stream subject to the rules of include filter processing. If no filters are specified, exclude filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

Note: This control is incompatible with the Compress and HashDirectories controls. If either of those controls are set when attribute filtering is in play, the program will abort.

Note: Support for this control was added in version 3.9.0.

ExcludeFilterSha1: <regexp>

Applies to mad, and map.

ExcludeFilterSha1 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one ExcludeFilterSha1 control/value pair per line.

Each ExcludeFilterSha1 value must be specified as a Perl compatible regular expression with the exception that capturing '()' subpatterns and back references are not allowed. You can use '(?:)' if you require parentheses for grouping purposes.

Any record containing an SHA1 that is matched by an ExcludeFilterSha1 value will be rejected. All other records will be written to the output stream subject to the rules of include filter processing. If no filters are specified, exclude filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

Note: This control is incompatible with the Compress and HashDirectories controls. If either of those controls are set when attribute filtering is in play, the program will abort.

Note: Support for this control was added in version 3.9.0.

ExcludeFilterSha256: <regexp>

Applies to mad, and map.

ExcludeFilterSha256 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one ExcludeFilterSha256 control/value pair per line.

Each ExcludeFilterSha256 value must be specified as a Perl compatible regular expression with the exception that capturing '()' subpatterns and back references are not allowed. You can use '(?:)' if you require parentheses for grouping purposes.

Any record containing an SHA256 that is matched by an ExcludeFilterSha256 value will be rejected. All other records will be written to the output stream subject to the rules of include filter processing. If no filters are specified, exclude filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

Note: This control is incompatible with the Compress and HashDirectories controls. If either of those controls are set when attribute filtering is in play, the program will abort.

Note: Support for this control was added in version 3.9.0.

ExcludesMustExist: [Y|N]

Applies to dig, mad, and map.

ExcludesMustExist is optional. When enabled ('Y' or 'y'), it instructs FTimes to check that every Exclude exists prior to mapping or digging. If this control is enabled and any Exclude does not exist, FTimes will abort. The default value is 'N'.

FieldMask: <mask>

Applies to compare, mapauto, mad, and map.

FieldMask is required in compare. Its value indicates what fields are to be compared for change.

FieldMask is required in mapauto, mad, and map. Its value dictates what attributes get collected or derived during a scan.

There can be no embedded white space in a given mask specification, and it must comply with the following case insensitive syntax:

    ALL[<+|-><field>...]
    

or

    NONE<+|-><field>[<+|-><field>...]
    

The following fields may be specified on WINX platforms with two caveats: (1) chtime is only available on Windows NT or higher and (2) altstreams is only available if the target file system is NTFS.

    volume     - Volume serial number
    findex     - File serial number
    attributes - File attributes
    atime      - Time of last file access
    mtime      - Time of last file modification
    ctime      - Creation time
    chtime     - Change time (undocumented)
    size       - File size in bytes
    altstreams - Number of alternate or named streams
    md5        - MD5 digest of the file's data stream
    sha1       - SHA1 digest of the file's data stream
    sha256     - SHA256 digest of the file's data stream
    magic      - File type
    osid       - Owner SID
    gsid       - Group SID
    dacl       - Discretionary access control list
    

The following fields may be specified on UNIX platforms:

    dev        - Device identification number
    inode      - File identification number
    mode       - File attributes and permissions
    nlink      - Number of hard links
    uid        - User identification number
    gid        - Group identification number
    rdev       - Device type (contains major/minor numbers)
    atime      - Time of last file access
    mtime      - Time of last file modification
    ctime      - Time of last file status change
    size       - File size in bytes
    md5        - MD5 digest of the file's data stream
    sha1       - SHA1 digest of the file's data stream
    sha256     - SHA256 digest of the file's data stream
    magic      - File type
    

In addition to the standard fields, the following group fields may be specified:

    hashes     - Compute all supported hash values
    times      - Collect all supported time values
    

For example, the "hashes" field is equivalent to "md5+sha1+sha256". Similarly, the "times" field is equivalent to "atime+mtime+ctime" and "atime+mtime+ctime+chtime" on UNIX and WINX platforms, respectively.

The "attributes" field is a collection of flags stored as a bitmap. The breakdown for these flags is as follows:

    ------------------------
    bit  0     - 0x00001 Read Only
    bit  1     - 0x00002 Hidden
    bit  2     - 0x00004 System
    bit  3     - 0x00008 Volume Label (?)
    bit  4     - 0x00010 Directory
    bit  5     - 0x00020 Archive
    bit  6     - 0x00040 Device
    bit  7     - 0x00080 Normal
    ------------------------
    bit  8     - 0x00100 Temporary
    bit  9     - 0x00200 Sparse
    bit 10     - 0x00400 Reparse Point
    bit 11     - 0x00800 Compressed
    bit 12     - 0x01000 Offline
    bit 13     - 0x02000 Not Content Indexed
    bit 14     - 0x04000 Encrypted
    bit 15     - 0x08000 (?)
    ------------------------
    bit 16     - 0x10000 Virtual
    ------------------------
    

Additional file attribute details are documented here:

    msdn.microsoft.com/en-us/library/ee332330(VS.85).aspx
    

The "mode" field is a collection of flags stored as a bitmap. The breakdown for these flags is as follows:

    ------------------------
    bit  0     - X for other
    bit  1     - W for other
    bit  2     - R for other
    ------------------------
    bit  3     - X for group
    bit  4     - W for group
    bit  5     - R for group
    ------------------------
    bit  6     - X for owner
    bit  7     - W for owner
    bit  8     - R for owner
    ------------------------
    bit  9     - sticky bit
    bit 10     - set group ID on execution
    bit 11     - set owner ID on execution
    ------------------------
    bits 12-15 - file type
    ------------------------
    

To isolate the file type portion of the mode, AND the mode with octal 0170000. The breakdown for file types is as follows:

    0010000    - named pipe (fifo)
    0020000    - character special
    0040000    - directory
    0060000    - block special
    0100000    - regular
    0120000    - symbolic link
    0140000    - socket
    0160000    - whiteout
    

FileHook: <klel-expression>

Applies to dig, mad, and map.

FileHook controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one FileHook control/value pair per line. Each hook must be specified as a special type of Kl-El expression known as a guarded command.

Any path matched by a hook will cause the specified command to be executed. All digging/mapping operations will cease until this command completes.

Note: This control is only available if file hooks were enabled at compile time.

FileSizeLimit: <integer>

Applies to dig, mad, and map.

FileSizeLimit is optional. It instructs the analysis engine to skip files that are larger than the specified size limit. The default value is zero, which means do not impose a limit.

GetAndExec: [Y|N]

Applies to get.

GetAndExec is optional. When enabled ('Y' or 'y'), it causes FTimes to start a new snapshot once the download is complete. This is accomplished through an exec() call. GetAndExec depends on GetFileName. The default value is 'N'.

Note: Take care when specifying GetFileName. If you choose a location that is writeable by other processes, FTimes may not read the config file you intended it to. That is to say, some other process may have modified or replaced the origianl file.

GetFileName: <file>

Applies to get.

GetFileName is required if GetAndExec is enabled. Its value is the name of the file in which the downloaded configuration information is to be stored. GetFileName may be specified as a relative path.

HashDirectories: [Y|N]

Applies to mad, and map.

HashDirectories is optional. When enabled ('Y' or 'y'), it instructs FTimes to compute digests for directories. This is done by hashing the digests of all files and directories contained in a given directory in the order they are encountered. Thus, if a directory has the following structure where D{1|2} and F{1|2} represent directories and files respectively,

  D1
   |
   - F1
   + D2
      |
      - F2
    

then, assuming that F1 is mapped before D2, D1 and D2 have the following hashes:

  Hash(D2) = H(H(F2))
  Hash(D1) = H(H(F1), Hash(D2))
    

where H represents the hash algorithm (e.g., MD5, SHA1, etc.).

If an entry within the directory is a special file (e.g., a device) or cannot be opened/hashed, a 16 byte string consisting of all zeros is used for the computation. The default value is 'N'.

HashSymbolicLinks: [Y|N]

Applies to mad, and map.

HashSymbolicLinks is optional. When enabled ('Y' or 'y'), it instructs FTimes to compute digests for symbolic links. This is done by hashing the data returned by readlink(). The default value is 'Y'.

Import: <file>

Applies to dig, mad, map, and get.

Import is optional. When specified, the directives contained in the file referenced by this control are included in the current configuration. Multiple instances of this control are allowed per file, and recursion is permitted up to three levels deep. Imports may be specified using a relative path.

Include: [directory|file|link]

Applies to dig, mad, and map.

Include controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one Include control/value pair per line. If no Include controls are specified, FTimes will attempt to map the entire system. If IncludesMustExist is enabled, then each Include must reference an existing file, directory, or symbolic link. Otherwise, FTimes will abort.

Include values must be a regular file, directory, or symbolic link specified as a fully qualified path. For WIN32 file systems, this means that each Include must begin with a drive designator ([A-Za-z]:) and be followed by the target path (e.g., 'c:\temp'). For UNIX file systems, each Include must begin with a forward slash (e.g., '/tmp').

If you need to include paths that contain special or non-printable characters, you must specify the path in the form of a file URL. Special characters are defined (in square brackets) as "[`'"|%+#]" or any non-printable character except for the NULL byte (%00), which is not allowed for includes. For example, the path '/tmp/foo#bar' would need to be specified as 'file:///tmp/foo%23bar'.

Note: Symbolic links are not supported in WIN32-based file systems.

Note: Take care when including file systems that reside on remote shares because FTimes may attempt to map them. To prevent this from happening, you can either exclude the remote file system or disable AnalyzeRemoteFiles.

Note: Directory hashing only applies to directories. This means that each file Include is treated as an isolated object apart from any specific tree. The exception to this is any Include that gets automatically pruned because it's part of a larger branch.

IncludeFilter: <regexp>

Applies to dig, mad, and map.

IncludeFilter controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one IncludeFilter control/value pair per line.

Each IncludeFilter must be specified as a Perl compatible regular expression with the caveat that no more than nine capturing '()' subpatterns for backreferences are allowed. Use '(?:)' if you require parentheses for grouping purposes.

Any path matched by an IncludeFilter will be mapped. Any other paths will be rejected. Also note that exclude filter processing occurs before include filter processing. If no include filters are specified, include filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

Warning: Due to their nature, include filters require mandatory directory traversal. This, in turn, will potentially alter directory access times. For example, suppose that you want to process all files that end with the string: '.pdf'. The IncludeFilter would be specified as follows:

    IncludeFilter=\.pdf$
    

Generally speaking, there is no way to know, apriori, where all these files are located. Therefore, it is necessary to traverse all directories applying the filter to each path as it's encountered.

IncludeFilterMd5: <regexp>

Applies to mad, and map.

IncludeFilterMd5 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one IncludeFilterMd5 control/value pair per line.

Each IncludeFilterMd5 value must be specified as a Perl compatible regular expression with the exception that capturing '()' subpatterns and back references are not allowed. You can use '(?:)' if you require parentheses for grouping purposes.

Any record containing an MD5 that is matched by an IncludeFilterMd5 value will be written to the output stream. All other records will be rejected. If no filters are specified, include filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

Note: This control is incompatible with the Compress and HashDirectories controls. If either of those controls are set when attribute filtering is in play, the program will abort.

Note: Support for this control was added in version 3.9.0.

IncludeFilterSha1: <regexp>

Applies to mad, and map.

IncludeFilterSha1 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one IncludeFilterSha1 control/value pair per line.

Each IncludeFilterSha1 value must be specified as a Perl compatible regular expression with the exception that capturing '()' subpatterns and back references are not allowed. You can use '(?:)' if you require parentheses for grouping purposes.

Any record containing an SHA1 that is matched by an IncludeFilterSha1 value will be written to the output stream. All other records will be rejected. If no filters are specified, include filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

Note: This control is incompatible with the Compress and HashDirectories controls. If either of those controls are set when attribute filtering is in play, the program will abort.

Note: Support for this control was added in version 3.9.0.

IncludeFilterSha256: <regexp>

Applies to mad, and map.

IncludeFilterSha256 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one IncludeFilterSha256 control/value pair per line.

Each IncludeFilterSha256 value must be specified as a Perl compatible regular expression with the exception that capturing '()' subpatterns and back references are not allowed. You can use '(?:)' if you require parentheses for grouping purposes.

Any record containing an SHA256 that is matched by an IncludeFilterSha256 value will be written to the output stream. All other records will be rejected. If no filters are specified, include filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

Note: This control is incompatible with the Compress and HashDirectories controls. If either of those controls are set when attribute filtering is in play, the program will abort.

Note: Support for this control was added in version 3.9.0.

IncludesMustExist: [Y|N]

Applies to dig, mad, and map.

IncludesMustExist is optional. When enabled ('Y' or 'y'), it instructs FTimes to check that every Include exists prior to mapping or digging. If this control is enabled and any Include does not exist, FTimes will abort. The default value is 'N'.

LogDigStrings: [Y|N]

Applies to dig, and mad.

LogDigStrings is optional. When enabled ('Y' or 'y'), it instructs FTimes to record each dig string in the log file. Preserving this information along side the dig output is often convenient and useful, but in some cases (e.g., searches for sensitive data), it's better to suppress it. The default value is 'N'.

LogDir: <directory>

Applies to dig, mad, and map.

LogDir is optional. It tells FTimes where to write log data. If not specified, the value of OutDir will be used as the value for LogDir. LogDir may be specified as a relative path.

MagicFile: <file>

Applies to mad, and map.

MagicFile is optional. If the magic field in FieldMask is set and MagicFile references a valid XMagic file, FTimes attempts to determine the type of each file it maps. If the magic field is not set, this control is ignored. MagicFile may be specified as a relative path. Additional information regaring XMagic can be found here:

  http://ftimes.sourceforge.net/FTimes/XMagic.shtml
    

Note: XMagic is not built into FTimes by default. If your version of FTimes does not have this support and the magic field is requested, FTimes will produce null fields.

Note: XMagic is automatically disabled if compression is enabled.

MapRemoteFiles: [Y|N]

Applies to dig, mad, and map.

MapRemoteFiles is an alias for AnalyzeRemoteFiles, and it is being phased out. Please use AnalyzeRemoteFiles instead.

MatchLimit: <integer>

Applies to dig, and mad.

MatchLimit is optional. It instructs the search engine to stop searching for a particular pattern within a file once the specified match limit has been reached. The default value is zero, which means do not impose a limit.

Note: Searching for a string such as 'A' with a MatchLimit of zero may produce a tremendous amount of output.

NewLine: [LF|CRLF]

Applies to dig, mad, and map.

NewLine is optional. When set to the value CRLF, it generates WINX style line feeds (i.e., carriage return and linefeed). When NewLine is set to the value LF, it generates UNIX style line feeds (i.e., linefeed). This control is useful if you review/analyze data sets from different platforms on one particular computer. The default value is the native value for the operating system running the program.

OutDir: <directory>

Applies to dig, mad, and map.

OutDir is required. It tells FTimes where to write output data. If not specified, the program will abort. OutDir may be specified as a relative path.

Priority: [low|below_normal|normal|above_normal|high]

Applies to compare, digauto, dig, mad, mapauto, and map.

Priority is optional. Use it to set the priority of the running program. The environment variable, FTIMES_PRIORITY, trumps the value of this control when set. No priority is set by default.

On UNIX platforms, the priority values listed above correspond to the following priority levels:

    low .............. 20
    below_normal ..... 10
    normal ...........  0
    above_normal .....-10
    high .............-20
    

On WINX platforms, the priority values listed above correspond to the following priority classes:

    low .............. IDLE_PRIORITY_CLASS
    below_normal ..... BELOW_NORMAL_PRIORITY_CLASS
    normal ........... NORMAL_PRIORITY_CLASS
    above_normal ..... ABOVE_NORMAL_PRIORITY_CLASS
    high ............. HIGH_PRIORITY_CLASS
    

Note: If the specified priority can't be set, the program will abort.

Note: If an environment variable is defined but exceeds the maximum allowed length, it will be rejected.

Note: The priority must be set via the environment for the compare and mapauto modes of operation.

Note: Support for this control was added in version 3.9.0.

RequirePrivilege: [Y|N]

Applies to dig, mad, and map.

RequirePrivilege is optional. When enabled ('Y' or 'y'), it indicates that the operator wants to ensure that the snapshot is run from a privileged account. On UNIX systems this means that FTimes must be run from an account that has a real user id of zero (i.e., root). On NT/2K systems this means that it must be run from an account that has the backup and restore user rights. The default value is 'N'.

Note: FTimes will work without privilege, but it's likely to generate more errors due to permission problems.

RunType: [baseline|linktest|snapshot]

Applies to dig, mad, and map.

RunType is optional. This control sets a corresponding flag in the log file that classifies output data as baseline, snapshot, or linktest. The value of this control does not affect the format or content of the output. It simply classifies the data so that automated analysis applications can process it accordingly. The default value is baseline.

SSLBundledCAsFile: <file>

Applies to dig, mad, map, and get.

SSLBundledCAsFile is required when SSLVerifyPeerCert is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded file that contains a bundled set of Certificate Authority (CA) certificates. Any validated peer certificate that is signed by one of these certificate authorities will be accepted provided that the SSLMaxChainLength and SSLExpectedPeerCN checks are also satisfied. SSLBundledCAsFile may be specified as a relative path.

SSLExpectedPeerCN: <name>

Applies to dig, mad, map, and get.

SSLExpectedPeerCN is required when SSLVerifyPeerCert is enabled. The value of this control represents the peer's expected Common Name (CN). Conventionally, CNs are specified as fully qualified domain names. This control eliminates the need to perform a DNS lookup at the time of certificate validation. This, in turn, may help to prevent attacks involving DNS spoofing.

SSLMaxChainLength: [1-10]

Applies to dig, mad, map, and get.

SSLMaxChainLength is optional when SSLVerifyPeerCert is enabled. The value of this control determines how deep a certificate chain may be before it is considered invalid. The default value is one.

SSLPassPhrase: <passphrase>

Applies to dig, mad, map, and get.

SSLPassPhrase is optional when SSLUseCertificate is enabled. Its value, if specified, is used to decrypt the contents of the client's private key file (see SSLPrivateKeyFile).

SSLPrivateKeyFile: <file>

Applies to dig, mad, map, and get.

SSLPrivateKeyFile is required when SSLUseCertificate is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded key file that can be used to sign SSL certificates. SSLPrivateKeyFile may be specified as a relative path.

SSLPublicCertFile: <file>

Applies to dig, mad, map, and get.

SSLPublicCertFile is required when SSLUseCertificate is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded certificate that will be provided during SSL handshakes. SSLPublicCertFile may be specified as a relative path.

SSLUseCertificate: [Y|N]

Applies to dig, mad, map, and get.

SSLUseCertificate is optional. When enabled ('Y' or 'y'), it instructs the application to provide client side certificate authentication, if necessary. The default value is 'N'.

SSLVerifyPeerCert: [Y|N]

Applies to dig, mad, map, and get.

SSLVerifyPeerCert is optional. When enabled ('Y' or 'y'), it instructs the application to verify the credentials of the peer server. The default value is 'N'.

StrictControls: [Y|N]

Applies to dig, mad, map, and get.

StrictControls is optional. When enabled ('Y' or 'y'), any unknown or unsupported configuration controls will cause the program to abort. When disabled ('N' or 'n'), any unknown or unsupported controls will be silently ignored. The default value is 'N'.

Note: If you want to enforce strict controls within a single config file (or chain of config files), StrictControls must be the first control processed. You may also enable/disable this control multiple times. This allows you to control the strictness of certain sections or starting at a particular point in the entire configuration. Once set, this control takes effect immediately.

Note: For versions 3.8.0 and below, this control was not supported, and the default behavior was to abort on any unknown or unsupported configuration control.

URLAuthType: [basic|none]

Applies to dig, mad, map, and get.

URLAuthType is optional. It identifies what authentication scheme to use when issuing HTTP/HTTPS requests. The value specified by this control applies to any requests involving URLGetURL or URLPutURL. When URLAuthType is set to basic, user credentials are base 64 encoded and incorporated into the request header. User credentials specified in the URL take precedence over credentials specified in the URLUsername and URLPassword controls. The default value is none.

URLGetRequest: [DigConfig|MapConfig]

Applies to get.

URLGetRequest is required. This control specifies what kind of config file the client is requesting when it issues a GET request. It also determines the next runmode when GetAndExec is enabled. Thus, values of DigConfig will cause FTimes to restart in dig mode, and values of MapConfig will cause FTimes to restart in map mode.

URLGetURL: <url>

Applies to get.

URLGetURL is required. It defines the scheme, user credentials, host address, port, and CGI application to be used when making requests. If a username/password pair is specified in the URL, that pair takes precedence over the values specified by URLUsername/URLPassword, if any. URLs must use a scheme of http or https and satisfy the following regular expression:

scheme://(user(:pass)?@)?host(:port)?/(path(\?query)?)?

URLPassword: <password>

Applies to dig, mad, map, and get.

URLPassword is optional. It identifies the password to use when accessing an Integrity Server. The value specified by this control is used in conjunction with URLGetURL and URLPutURL unless those controls supply their own username/password pair.

URLPutSnapshot: [Y|N]

Applies to dig, mad, and map.

URLPutSnapshot is optional. When enabled ('Y' or 'y'), FTimes attempts to post the snapshot to an Integrity Server. Prior to starting its scan, FTimes will transmit an application layer PING to the server to verify that it is accessible and functional. If the remote CGI application is functioning properly, the extended HTTP response code 250 (Ping Received) is returned. URLPutSnapshot depends on URLPutURL. If basic authentication is required, the controls URLUsername, URLPassword, and URLAuthType may need to be specified as well. The default value is 'N'.

URLPutURL: <url>

Applies to dig, mad, and map.

URLPutURL is optional. It defines the scheme, user credentials, host address, port, and CGI application to be used when making PUT requests. If a username/password pair is specified in the URL, that pair takes precedence over the values specified by URLUsername/URLPassword, if any. In any event, user credentials are only sent when basic authentication has been requested (See URLAuthType). URLPutURL uses the same syntax as URLGetURL.

URLUnlinkOutput: [Y|N]

Applies to dig, mad, and map.

URLUnlinkOutput is optional when URLPutSnapshot is enabled. When enabled ('Y' or 'y'), any output files are overwritten and unlinked before the program exits. The default value is 'N'.

URLUsername: <username>

Applies to dig, mad, map, and get.

URLUsername is optional. It identifies the username to use when accessing an Integrity Server. The value specified by this control is used in conjunction with URLGetURL and URLPutURL unless those controls supply their own username/password pair.

Example 1. Using map mode to baseline and upload

The first thing that needs to be done is to obtain the necessary upload information. Assume that an Integrity Server has already been configured to receive snapshots. Also, assume that the following information has been provided:

    URL = https://192.168.1.50:443/cgi-client/nph-ftimes.cgi
    Authentication Type = Basic
    Username/ClientID = client_1
    Password = password
    AllowedFieldMask = ALL-magic
    AllowedDataType = map
    Server Validates Client Certificates = N

Observe that remote server speaks HTTPS. Therefore, an SSL enabled version of FTimes is needed. To determine if FTimes has SSL support, run the following command:

    ftimes --version

If FTimes has SSL support, you will see the string 'ssl' in the output.

The next item to tackle is the creation of a suitable config file. The following file contains the necessary directives to complete this task.

    --- example1.cfg ---
    BaseName=client_1
    OutDir=.
    RunType=baseline
    FieldMask=ALL-magic
    URLPutSnapshot=Y
    URLPutURL=https://192.168.1.50:443/cgi-client/nph-ftimes.cgi
    URLAuthType=basic
    URLUsername=client_1
    URLPassword=password
    Compress=Y
    --- example1.cfg ---

Note that there are no Include directives. This omission causes FTimes to map everything (i.e., the entire system).

Also, note that OutDir has been set to the current directory (i.e., '.'). This, while entirely legal, assumes that (1) '.' is writeable and (2) there is enough disk space to hold all generated output.

Compression was enabled to expedite data transfer.

The last step is to run FTimes and review the corresponding log output to ensure that snapshot was uploaded successfully.

    ftimes --map example1.cfg

Example 2. Using dig mode to search for strings

Given that the target file system is "/disk1", and the list of strings to to be sought are:

    String1 = /dev/ptyqa
    String2 = 0xda 0xbe
    String3 = 0x00 A 0x00 A 0x00 A 0x00 A
    String4 = A date of the form <YYYY-MM-DD HH:MM:SS>
    String5 = Line oriented IP addresses of the form <BOL|tab><192.168.1.*><tab|EOL>

where

    BOL = Beginning Of Line
    EOL = End Of Line

Strings 1-3 are expressed as (ignore whitespace) ASCII, HEX, and HEX/ASCII, respectively. Strings 4 and 5, on the other hand, are textual descriptions that must be translated into regular expressions. Assume that the required MatchLimit is three. That is to say, no more than three matches per file of any single string should be recorded.

The following config file contains those directives required to search all files in /disk1.

    --- example2.cfg ---
    BaseName=digger
    OutDir=.
    MatchLimit=3
    DigStringNormal=/dev/ptyqa
    DigStringNormal=%da%be
    DigStringNoCase=%00A%00A%00A%00A
    DigStringRegExp=\d{4}-\d{2}-\d{2}\x20\d{2}:\d{2}:\d{2}
    DigStringRegExp=(?m)(?:^|\t)(192\.168\.1\.\d{1,3})(?=\t|$)
    Include=/disk1
    --- example2.cfg ---

Observe that strings two and three are URL encoded and that string three will be processed in a case insensitive manner. Read the DigStringNormal control description for more details on URL encoding. Also notice that the fourth and fifth strings correspond to Perl compatible regular expressions -- these types of dig strings are available if you compiled FTimes with PCRE support. With DigStringRegExp patterns, however, the number of capturing '()' subpatterns is limited (see the description of that control for details). You can use '(?:)' if you require additional parentheses for grouping purposes. If you do not specify a capturing subpattern, the entire match will be captured and used as the output value. Otherwise, the output value is taken from \1 -- (i.e., the first capturing subpattern). In this particular case, only the IP address will be captured. Also note the use of '\x20' to represent a single space. This translation is necessary because dig strings may not contain embedded white space.

At this point FTimes may be run as follows:

    ftimes --dig example2.cfg

If spontaneity is desired over strict configuration, switch to the digauto mode of operation. In this mode elaborate config files aren't necessary, and output is written to stdout. Also, no match limit is imposed. The following example shows how to search an image file (e.g., example2.image) for a set of specific strings. The output, if any, can be piped into other tools that take their input on stdin.

    --- strings.cfg ---
    DigStringNormal=This+box+is+0wn3d    attack_program
    DigStringNormal=3l33t                attack_program
    DigStringNormal=175.20.1.7           attack_ip
    DigStringNormal=hacklist@foo.bar.com
    --- strings.cfg ---

    ftimes --digauto strings.cfg example2.image | some-other-tool

Just remember that dig strings must be URL encoded with no embedded white space. That's why the spaces in the first string have been replaced with '+'. Also, note that first three dig strings are tagged. In other words, each string has been assigned a value that gives it a special meaning or identity. Because the tag for first two dig strings is the same, it can be used to classify both strings as belonging to the same group.

Example 3. Change analysis

Given that the following files are valid, uncompressed snapshots:

    Baseline = daily_20001230203000.map
    Snapshot = daily_20001231203000.map

Compare MD5 digests to determine (C)hanged, (M)issing, and (N)ew files.

The only critical observation needed here is that daily_20001230203000.map is considered to be the baseline. In other words, a baseline is a snapshot taken at an arbitrary point in time to which subsequent snapshots are compared.

This comparison is carried out with the following command:

    ftimes --compare none+md5 daily_20001230203000.map daily_20001231203000.map

To compare multiple attributes at the same time, simply specify the additional fields in the FieldMask. For example, the following mask would compare MD5, SHA1, and size attributes:

    none+md5+sha1+size

To compare all attributes at the same time, use a FieldMask of 'all'.

Example 4. Using get mode to download a config file

The first thing that needs to be done is to obtain the necessary download information. Assume that an Integrity Server has already been configured to serve config files. Also, assume that the following information has been provided:

    URL = https://www.integrity.net:443/cgi-client/nph-ftimes.cgi
    Authentication Type = Basic
    Username/ClientID = client_1
    Password = password
    Server Validates Client Certificates = Y
    Server Common Name = www.integrity.net
    Maximum Certificate Chain Length = 2

Observe that remote server requires certificates. This means that you'll need three additional PEM encoded files: Public Certificate, Private Key, and Bundled Certificate Authorities. Assume that these files are located on the target system as follows:

    /usr/local/ftimes/etc/PublicCert.pem
    /usr/local/ftimes/etc/PrivateKey.pem
    /usr/local/ftimes/etc/BundledCAs.pem

Armed with that information, the following config file may be constructed.

    --- example5.cfg ---
    BaseName=client_1
    URLGetURL=https://www.integrity.net:443/cgi-client/nph-ftimes.cgi
    URLGetRequest=MapConfig
    GetAndExec=N
    URLAuthType=basic
    URLUsername=client_1
    URLPassword=password
    SSLUseCertificate=Y
    SSLPublicCertFile=/usr/local/ftimes/etc/PublicCert.pem
    SSLPrivateKeyFile=/usr/local/ftimes/etc/PrivateKey.pem
    SSLPassPhrase=passphrase
    SSLVerifyPeerCert=Y
    SSLBundledCAsFile=/usr/local/ftimes/etc/BundledCAs.pem
    SSLExpectedPeerCN=www.integrity.net
    SSLMaxChainLength=2
    --- example5.cfg ---

The following command will attempt to download a map config file from the specified Integrity Server. If successful, the contents of the config file will be written to stdout.

    ftimes --get example5.cfg

If you want to download directly to a file, you can redirect the output to a file or add the GetFileName control to your config file. Then, if you wanted to download a config file and take a snapshot in one operation, simply enable GetAndExec. This causes FTimes to restart in --map mode. If URLGetRequest was set to DigConfig, then FTimes would request a dig config file and subsequently restart in --dig mode.

Another way to achieve the same effect is to use the original config file and construct the following pipeline:

    ftimes --get example5.cfg -l 6 | ftimes --map -

This has the benefit that potentially sensitive configuration information is not specifically written to a file on disk.

Finally, note that the LogLevel for the first command was set to its highest value. This was done simply to reduce log output from that process.