The FlowTraq Command Line Interface (CLI) provides an easy way for custom scripts and third party applications to query FlowTraq Server for flow information.

The CLI tools are installed with FlowTraq Server in the /path/to/flowtraq/clitools directory.

[Tip]Tip

The CLI tools, like the client, connect to FlowTraq Server via 9640/tcp. You don't have to run the CLI tools from the host on which you installed FlowTraq Server.

There are eight command line tools.

ftsq

FlowTraq Session Query Retrieval Tool. The ftsq command allows you to retrieve bi-directional session data assembled from the unidirectional flow data. This command accepts as parameters a report type, a timeframe, and an optional filter string to narrow the scope of the report. It presents its results as CSV or an ASCII table.

ftstat

FlowTraq Statistical Query Retrieval Tool. Use ftstat to retrieve the kinds of statistical reports you can retrieve in a FlowTraq Workspace, such as "Hosts Ranked by Bytes Sent" or "Applications ranked by Sessions Received". Like the ftsq command, ftstat accepts a timeframe and filter string. It presents tabular results as either CSV or an ASCII table.

ftname

FlowTraq Friendly Names. The ftname command allows users to set userfriendly names for objects such as IP addresses, exporters, interfaces, traffic groups, and autonomous systems.

ftum

FlowTraq User Management. The ftum command allows administrative users to create and delete users, reset passwords, and grant administrative privileges. The use of this tool is covered in the Administration Section of this manual.

fttg

FlowTraq Traffic Groups. The fttg command allows administrative users to create traffic groups for tagging flows upon ingress. The use of this tool is covered in the Administration Section of this manual.

ftgroup

FlowTraq Partition Management The ftgroup command allows administrative users to create and manage partitions. The use of this tool is covered in the Administration Section of this manual.

ftnbi

FlowTraq Alert and Policy Management The ftnbi command allows administrative users to create and manage policies, and allows all users to retrieve alerts. The use of this tool is covered in the Alerting section of this manual.

ftplugin

FlowTraq Plugin Management The ftplugin command allows administrative users to view and manage plugins.

To retrieve raw session records, use the ftsq command.

For example, the following invocation of ftsq returns all records in the last hour to HTTP servers with a client address that is outside the 123.45.67.89 class-C block, in CSV format with a header line:


The ftsq commands accepts a wide range of parameters. Some are optional and some are required.

You should always specify a FlowTraq Server to log in to (or accept the default, localhost), supply a username and password, and select a timeframe over which to perform your query (or accept the default, which is the last 15 minutes).

Optionally, you may supply a filter string to further narrow your query, and you may specify a preference for how you would like the command's output formatted.

Most of the parameters are self-explanatory, but timeframe specification and the filter string syntax are described in depth in Section 3.4.3, “Time Navigation” and Section 3.4.4, “Filter String Syntax”. First, however, please review the complete list of parameters:





[Important]Important

Note that the -snd and -rcv parameters are not applicable to the ftsq command, since rankings are not generated when returning raw session records. Use these parameters in conjunction with ftstat, as described below.


[Important]Important

Note that the -w parameter is not applicable to the ftsq command, since there is no accompanying time series for raw session records. Use this parameter in conjunction with ftstat, as described below.

[Important]Important

Note that the -g, -gx, and -gy parameters are not applicable to the ftsq command, since there is no accompanying stack graph for raw session records. Use these parameters in conjunction with ftstat, as described below.

All data retrieval commands accept an optional filter string. The filter string is used to select which sessions to include in the retrieval.

Filter strings consist of statements, such as SRVIP==123.45.67.89 ("server IP address is 123.45.67.89") and CLNPKTS>=100 ("number of client-transmitted packets is at least 100"), which may be combined using logical operators.

You can combine statements using the following logical operators:

For example: SRVIP==123.45.67.89 && CLNIP==89.67.45.123 .

[Tip]Tip

You can build compound statements and specify precedence by using parentheses. For example, you might use:

(SRVPORT==22 && SRVIP==123.45.67.89) || (SRVIP==89.67.45.123)

to specify all sessions which are either connections on port 22 to 123.45.67.89, or connections on ANY port to 89.67.45.123.

Statements are formed by combining field names (such as SRVIP) with comparators (such as ==) and values to compare them to. The following lists the available comparators and field names.

Table 3.6. Filter String Fields

FieldDescriptionValid Comparators
SRVIP server IP (or CIDR), IPv4: 123.45.67.89/32, or IPv6: fed9::c0:ffee/128 ==, !=
CLNIP client IP (or CIDR), same as SRVIP ==, !=
ADDR IP or CIDR block ==, !=
SRVPORT server port, integer number ==, !=, >=, <=, >, <
CLNPORT client port, integer number ==, !=, >=, <=, >, <
PORT port, integer number ==, !=, >=, <=, >, <
PROTO protocol, one of TCP/UDP/ICMP, or integer number ==, !=, >=, <=, >, <
CLNPKTS number of client transmitted packets, integer number ==, !=, >=, <=, >, <
SRVPKTS number of server transmitted packets, integer number ==, !=, >=, <=, >, <
PACKETS match either of the packet fields (server or client), integer number ==, !=, >=, <=, >, <
TOTPKTS total packets (server plus client), integer number ==, !=, >=, <=, >, <
CLNBYTS number of client transmitted bytes, integer number ==, !=, >=, <=, >, <
SRVBYTS number of server transmitted bytes, integer number ==, !=, >=, <=, >, <
BYTES match either of the bytes fields (server or client), integer number ==, !=, >=, <=, >, <
TOTBYTS total bytes (server plus client), integer number ==, !=, >=, <=, >, <
TTIME total time of session, floating point, in seconds: e.g. 2.5 ==, !=, >=, <=, >, <
STRTIME start time, time notation: MM/DD/YY hh:mm:ss.fracsec ==, !=, >=, <=, >, <
UPDTIME time of last update, time notation: MM/DD/YY hh:mm:ss.fracsec ==, !=, >=, <=, >, <
TOS ToS, QoS, DiffServ, integer number 0-256 ==, !=, >=, <=, >, <
CLNCC client country code, two characters: 'US', 'NL' ==, !=
SRVCC server country code, same as client country code ==, !=
INIF inbound interface, integer number 0-65536 ==, !=, >=, <=, >, <
OUTIF outbound interface, integer number 0-65536 ==, !=, >=, <=, >, <
IFACE match either of the interface fields (inbound or outbound), integer number 0-65536 ==, !=, >=, <=, >, <
INVLAN inbound VLAN, integer number 0-4096 ==, !=, >=, <=, >, <
OUTVLAN outbound VLAN, integer number 0-4096 ==, !=, >=, <=, >, <
VLAN match either of the VLAN fields (inbound or outbound), integer number 0-4096 ==, !=, >=, <=, >, <
CLNAS client autonomous system number, integer number ==, !=, >=, <=, >, <
SRVAS server autonomous system number, integer number ==, !=, >=, <=, >, <
ASN match either of the autonomous system number fields (server or client), integer number ==, !=, >=, <=, >, <
ASAEVT ASA event code, integer number ==, !=, >=, <=, >, <
ASAEXTEVT ASA extended event code, integer number ==, !=, >=, <=, >, <
FLAGS TCP flags in session, one of: 'FSYN' (syn), 'FACK' (ack), 'FRST' (reset), 'FFIN' (fin), 'FPSH' (push), 'FECN' (ECN-echo), 'FCWR' (congestion window reduced), 'FURG' (urgent) ==, !=
EXPIP IP of the device that exported the record ==, !=
EXPV flow version, use: 1, 5, 7, 9 (NetFlow v1/5/7/9), 18, 20, 21 (sFlow v2/4/5) ==, !=

The FlowTraq Statistical Query Retrieval command ftstat creates tables and graphs of grouped items that are ranked by some criterion. For example, you can retrieve the list of hosts that sent the most packets during a given timeframe or the list of hosts that received the most packets during the same. You can also find out which port/application accounted for the most bytes on your network, find which host pair exchanged the most bytes, and more.

It is also possible to score by more complex criteria. For instance, it is possible to find the list of hosts that contacted the largest number of unique hosts, or the list of countries that contacted your servers on the largest number of unique server ports.

As with the ftsq command, you must specify a FlowTraq Server to connect to, supply login details, select a timeframe and (optionally) specify a filter. And like ftsq, the results are returned in a formatted table, by default, or in CSV format (use either the -c option for CSV without a header, or the -c+ option for CSV with a header line). Please refer to the complete list of parameters in Section 3.4.2, “Retrieving Raw Session Data from the Command Line with ftsq, Section 3.4.4, “Filter String Syntax” for more information on the filter language syntax, and Section 3.4.3, “Time Navigation” for information on timeframe specifications.

Where the usage of ftstat differs from that of ftsq is in specifying the desired statistic to calculate. Specify the statistic by using the -grp and -cnt paramaters:


[Tip]ftstat Example 1

To retrieve the top 25 hosts by bytes sent in the last week, use the following command:

[Tip]ftstat Example 2

To retrieve the five host-pairs that communicated over the largest number of ports during last five hours, use the following command:

[Tip]Tip

You may use the -g parameter to request the accompanying stack graph, and the -gx and -gy parameters to specify the size of the graph you would like.

[Tip]Tip

You may use the -w parameter to request a timeseries for each row of the table.

The ftsq command is also able to export flow records in IPFIX format on demand, using the switch:

		-ipfix DESTINATION PORT
		

This can be appended to any other ftsq command that results in returned flow records. For example, to export the last 35 minutes of TCP/22 flows to the IPFIX destination 192.168.1.101:4739 (credentials omitted for clarity):

		/opt/flowtraq/clitools/ftsq -tn -35m -q "(SRVPORT==22)&&(PROTO==TCP)" -ipfix 192.168.1.101 4739
		

The IPFIX output is not rate-limited; ftsq will generate flow records as quickly as it can. For very large numbers of flows, this may result in dropped packets or filled buffers on the receiving end, so it may be necessary to limit the number of flow records sent in this way.

It is also possible to use ftsq to fabricate flow records, for example from parsed log files or to test specific functionality. With the -read flag, ftsq will read lines from stdin, parse them, and output flow records. By default, it will export them to the command line; with the -ipfix switch, it will send parsed records to the selected destination. This is often useful when importing data into FlowTraq from a flow-like format, or to test security policies without actually simulating dangerous conditions (such denial of service attacks).

Parsed flow is accepted as a CSV-formatted line with the following columns:


As an example, the following file describes two sessions:

            192.168.2.100, 12208, 8.8.8.8, 53, UDP, 1, 72, 1, 72, 5/31/2016 14:52:22.000000, 5/31/2016 14:52:23.426324, 192.168.2.1, dns
            192.168.2.100, 12209, 8.8.8.8, 53, UDP, 1, 73, 1, 133, 5/31/2016 14:52:23.123456, 5/31/2016 14:52:24.000000, 192.168.2.1, dns
        

This file can be read into ftsq -read, to view the resulting output:

           # cat flows.txt | /opt/flowtraq/clitools/ftsq -read
        

Or with the following command to export the file's parsed records to 192.168.1.101:

           # cat flows.txt | /opt/flowtraq/clitools/ftsq -read -ipfix 192.168.1.101 4739
        

The session key reauthentication mechanism allows for FlowTraq's command line tools to be easily integrated with third-party applications and applications hosted on other systems. The use of session keys allows automated scripts and script-based interfaces such as web GUIs to call additional command line tools without the need to store the username and password in a client-side cookie. Since the session key automatically expires, and is only valid from the originating IP address, it is unnecessary to perform an explicit "log out."

[Important]Default Timeout 30 Minutes

The FlowTraq server daemon times session keys out automatically after they have not been used for 30 minutes. Additionally, session keys will only be accepted from the IP address that generated the session key. To change this default behavior, please refer to the Administration Section of this manual.

To create and use a session key, a command line tool must first provide a valid user's credentials to log into a session, and provide the -us parameter to request that a session key be created. Any command will work, but ftum is convenient because it doesn't need to interact with session data, so we use it in our example:

			ftum -un USERNAME -up PASSWORD -us
		

If the credentials provided are valid, the stderr output of the command will be a session key; for example:

			91389bd1127bce0a2615d390be08f696
		

The session key may subsequently used with the -us argument instead of a username/password combination to re-login to the same FlowTraq Server from the same IP address. Continuing our example:

			ftstat -us 91389bd1127bce0a2615d390be08f696 [...]
		
[Tip]Tip

Each time the session key is used, the timer is reset. The session key will eventually expire on the server side after the period of time specified in the userdata/maxsessionkeyage configuration parameter.

The ftum command, in addition to managing individual user accounts, can be used to query the FlowTraq service for health information using the -info switch. The resulting output is a set of key/value pairs, such as this:

@local build_no="17022" release="Q1/16" mfgdate="1455886836" tsnow="1461332794.212263" tsboot="1460574802.312135" utcoff="-14400" nodename="flowtraq" ostype="Linux 3.2.0-4-amd64" architecture="x86_64 (64-bit)" numcpu="1" numslices="1" systemram="641372160" ramuse_cur="614200426" ramuse_max="481029120" ramfrac_base="86.048" ramfrac_cache="9.190" ramfrac_iomap="13.951" conntrack_size="16384" conntrack_use="15974" conntrack_active="8327" conntrack_fidelity="46852" cache_size="131072" cache_use="131072" cache_fidelity="410869" db_size="331942580" db_use="366030" db_fidelity="758070" db_earliest="1460487851" archive_size="0" archive_use="0" archive_fidelity="0" archive_earliest="0" input_flowrate_fps="1" lru_droprate_sps="0" max_queuefill="0.127%" max_cycletime_ms="0" max_archive_queuefill="0.000%" max_archive_cycletime_ms="0" max_inputfill="0.000%" max_writerate_sps="0" max_readrate_sps="0" target_readrate_sps="0" license_type="DEMO" license_end="1385823660" license_hash="0x9020e1b16080b1e"
		

Any individual items to be selected (instead of all items) can be requested in a comma-separated list as an argument to -info:

#root@flowtraq:/opt/flowtraq/clitools# ./ftum -un admin -info tsnow,tsboot
Password for admin login:
@local tsnow="1461332927.643921" tsboot="1460574802.312135"
		

The elements are broken down into four main catagories: install status, system profile, resource use, and flow rate.

The FlowTraq Install Status items detail the current FlowTraq parameters, including version information and license status:

The System Profile items detail the host on which this FlowTraq install is running, such as operating system, uptime, and other parameters:

The cache items detail the state of this host's session caches on RAM and disk. This includes long-term settings and short-term values:

The Flow Rate items provide details on the current flow rate, and on how this FlowTraq install is keeping pace with the incoming flows: