Indicadores de desempeño (KPI)

4.2

Analyzing Packet Data with rwptoflow and rwpmatch

Analysts use therwptoflow and rwpmatch tools to generate single-packet flow records from packet content (i.e., PCAP) data, analyze and filter those flow records using the SiLK tools, and subsequently filter the packet data based on that analysis. Third-party tools, such as ngrep (http://ngrep.sourceforge.net) may also filter packet content data based on regular expressions.

Another option for processing packets is to aggregate the packets into true flow records. Analysts can do that through the rwp2yaf2silk tool that uses the features of rwtuc and the yaf and yafscii tools (the latter two are available fromhttp://tools.netsa.cert.org/yaf/). Once converted to flow records, all the SiLK tools can process them as if they were from the repository, but it is difficult to re-identify packets with processed flow records. For analyses that involve both packet and flow analysis,rwptoflow and rwpmatch are preferred.

4.2.1

Creating Flows from Packets Using rwptoflow

Therwptoflow tool generates a single-packet flow record for every IP packet in a packet capture file. The packet formats do not contain routing information, which is available in some flow record formats. The values for routing-information flow record fields may be set for the generated flows using the parameters --set-sensorid, --set-inputindex, --set-outputindex, and --set-nexthopip. For example, it is pos- sible to set the sensor ID manually for a packet content source so that network flow data combined from several sensors can be filtered or sorted by the sensor value later. rwptoflow is summarized in Figure 4.7. rwptoflow with --active-time can be used to specify the generation of flows only for a specific time interval of interest. During this time interval,--packet-pass-output and --packet-reject-output can be used to produce packet files that were either converted to flows or not converted to flows. Finally, the--plugin parameter can be used to incorporate plug-ins for additional functionality in packet conversion, analogous torwfilter plug-ins.17

A packet might not be converted to a flow record for these reasons:

• The packet is not for an Internet protocol (IP or its encapsulated protocols). LAN-based protocols (such as the Address Resolution Protocol (ARP) are not Internet protocols. As such, there isn’t enough information in the packet to build a flow record for it. Other tools, such as tcpdump or Wireshark® can be used to examine and analyze these packets.

• The packet is erroneous, and the information used to build a flow record is inconsistent in a way that prevents record generation. This may happen because of transmission problems with the packet or because the capture file may have been corrupted.

• The packet capture snapshot length isn’t large enough to capture all the needed fields. If a very short snapshot length is used, not all of the header may be captured. Therefore, the captured packet may not contain enough information to build a flow record for it.

• The IP packets are encapsulated in IEEE 802.1Q Virtual LAN (VLAN) frames.

Any of these will cause the packet to be rejected. Example4.9shows a simple conversion of thepackets.pcap capture file into the mypkts.rw1 flow record file, restricting the conversion to a specific time period and producing dumps of packets converted (mypkts.dmp) and rejected (mypkts-bad.dmp).

17_{In the current version, PySiLK plug-ins are not implemented forrwptoflow. All plug-ins currently must use the C language} API.

Figure 4.7: Summary of rwptoflow

rwptoflow

Description Reads a packet capture file and generates a SiLK flow record for every packet

Call rwptoflow packets.pcap --flow-output=single_pkt_flows.rw1 Parameters --active-time Sets the time interval of interest

--packet-pass-output Specifies a path for valid packets in the time interval of interest

--packet-reject-output Specifies a path for invalid packets in the time inter- val of interest

--plugin Specifies a plug-in to be used in the conversion

--flow-output Writes the generated SiLK flow records to the specified path or stdout if the path is not given

--reject-all-fragments Does not generate a SiLK flow record for any frag- mented packets

--reject-nonzero-fragments Does not generate a SiLK flow record for any packet with a non-zero fragment offset

--reject-incomplete Does not generate a SiLK flow record for any zero- fragment or unfragmented packets when the record cannot be completely filled (missing ICMP type & code, TCP/UDP ports, TCP flags)

--set-sensorid Sets the sensor ID for all flows (0–65,534). When not specified, the ID is set to 65,535.

--set-inputindex Sets the input SNMP index for all flows, 0–65,535. --set-outputindex Sets the output SNMP index for all flows, 0–65,535. --set-nexthopip Sets the next-hop IP address for all flows.

--print-statistics Prints the count of packets read, packets processed, and bad packets to standard error

For additional parameters, see Table4.4on page 134 and Table3.14on page 72.

Example 4.9: rwptoflow for Simple Packet Conversion

<1>$ rwptoflow packets . pcap --active - time =2013/8/25 T05 :27 -2013/8/25 T05 :45 \ --packet -pass - output = mypkts . dmp --packet - reject - output = mypkts - bad . dmp \ --flow - output = mypkts . rw1

4.2. ANALYZING PACKET DATA WITH RWPTOFLOW AND RWPMATCH 95

4.2.2

Matching Flow Records with Packet Data Using rwpmatch

rwpmatch takes a packet capture input file and filters it based on flow records from a SiLK flow record file. It is designed to allow flow records fromrwptoflow (that are filtered or processed) to be matched with the packet content data that produced them. The resulting packet capture file is output on standard output. The flow record file input torwpmatch should contain single-packet flow records (e.g., those originally derived from a packet capture file usingrwptoflow). If a flow record is found that does not represent a corresponding packet record,rwpmatch will return an error. Both the packet capture and the flow record file inputs must be time-ordered. The syntax of rwpmatch is summarized in Figure 4.8. By default,rwpmatch will consider only the source address, destination address, and the time to the second. By using the--ports-compare pa- rameter, the source and destination port can also be considered in the match. By using the--msec-compare parameter, time will be compared to the millisecond.

Figure 4.8: Summary ofrwpmatch

rwpmatch

Description Matches a packet capture file against a SiLK flow record file that has a flow for every packet, producing a new packet capture file on standard output

Call rwpmatch packets.pcap --flow-file=selected.rw1 >selected.pcap Parameters --flow-file Specifies the flow record file to be used in the match (required)

--ports-compare Uses port information in the match --msec-compare Uses milliseconds in the match

For additional parameters, see Table4.4on page 134 and Table3.14on page 72.

rwpmatch is I/O intensive. The tool works by reading an entire packet capture file and the entire flow record file. It may be worthwhile to optimize an analysis process to avoid usingrwpmatch until payload filtering is necessary. Saving the output fromrwpmatch as a partial-results file and comparing that file to files generated by later steps in the analysis (rather than comparing the later results against the original packet capture file) can also provide significant performance gains.

The packet-analysis tools are typically used in combination with payload-filtering tools, like ngrep, that allow an analyst to partition traffic based on payload signatures prior to using the SiLK tools for analysis or, conversely, to identify a traffic phenomenon (e.g., worm propagation) through flow analysis and then filter the packets that correspond to the flow records that make up that traffic.

In Example4.10, thedata.pcap packet capture file is filtered by the sip.set IP-set file by converting it to a SiLK flow record file, filtering the flows by the source IP addresses found in the set, and then matching the original packet capture file against the filtered flow file.

Example 4.10: rwptoflow and rwpmatch for Filtering Packets Using an IP Set

<1>$ rwptoflow data . pcap --flow - output = data . rw1

<2>$ rwfilter data . rw1 -- sipset = sip . set --pass = filtered . rw1 <3>$ rwpmatch data . pcap --flow - file = filtered . rw1 >filtered . pcap