A "Distributed Pcap" for
Remote Monitoring LANs & WANs
(Design Notes for the SITA ACN device)
SITA INC Canada, Inc.
Revised: October 2, 2007
The ACN provides a customized/distributed version of this library that alows SMPs to interact with the various IOPs within the site providing a standard mechanism to capture LAN and WAN message traffic.
|SMP||The Supervisory Management Processor where Wireshark (or equivalent) runs in conjuction with a libpcap front-end.|
|IOP||I/O Processors where the monitored ports exist in conjunction with a custom device driver/libpcap back-end.|
Each IOP will be capable of supporting multiple connections from an SMP enabling monitoring of more than one interface at a time, each through its own seperate connection. The IOP is responsible to ensure and report an error if any attempt is made to monitor the same interface more than once.
There are three applications that will be supported by the ACN version of libpcap. They each use a slightly different mode for looping/capturing and termination as summarized in the following table:
|wireshark||pcap_dispatch(all packets in one buffer of capture only)||pcap_breakloop()|
|tshark||pcap_dispatch(one buffer of capture only)||Since a CTRL-C was used to terminate the application, pcap_breakloop() is never called.|
|tcpdump||pcap_loop(all packets in the next buffer, and loop forever)||pcap_breakloop()|
Note: In all cases, the termination of capturing is always (apparently) followed by pcap_close(). Pcap_breakloop() is only used to stop/suspend looping/processing, and upon close interpretation of the function definitions, it is possible to resume capturing following a pcap_breakloop() without any re-initialization.
|pcap_open_live()||Used to obtain a packet capture descriptor to look at packets on the network.|
|pcap_findalldevs()||It constructs a list of network devices that can be opened with pcap_open_live().|
|pcap_freealldevs()||Used to free a list allocated by pcap_findalldevs().|
|pcap_dispatch()||Used to collect and process packets.|
|pcap_loop()||Is similar to pcap_dispatch() except it keeps reading packets until the requested number of packets are processed or an error occurs.|
|pcap_next()||It reads the next packet (by calling pcap_dispatch() with a count of 1) and returns a pointer to the data in that packet.|
|pcap_next_ex()||Reads the next packet and returns a success/failure indication.|
|pcap_setfilter()||Used to specify a filter program.|
|pcap_stats()||Fills in a pcap_stat struct with packet statistics.|
|pcap_close()||Closes the file and deallocates resources.|
Communications between an SMP and an IOP consists of a TCP session between an ephemeral port on the SMP and the well known port of 49152 (which is the first available port in the 'dynamic and/or private port' range) on an IOP.
Following a TCP open operation the IOP receives a null terminated 'interface ID' string to determine the type of operation that follows:
Every command received by an IOP implies a 'stop trace/stop forwarding' operation must occur before executing the received command.
A session is closed when the SMP closes the TCP session with the IOP. Obviously monitoring and forwarding is also stopped at that time. Note: All multi-octet entities are sent in network neutral order.
|pcap_findalldevs()||SMP -> IOP||Open socket (to each IOP), and sends:
|IOP -> SMP||Send its (possibly empty) NULL terminated error response string.|
|SMP -> IOP||Sends the 'interface query request':
|IOP -> SMP||The IOP returns a list of sequences of information as
defined by the return parameter of this function call (as shown in the following table).
Elements are specified by providing an unsigned byte preceeding the actual data that contains length information.
|SMP -> IOP||Close the socket.|
|IOP -> SMP||Close the socket.|
|pcap_open_live()||SMP -> IOP||Open socket, and sends:
|IOP -> SMP||Send its NULL terminated error response string.|
|SMP -> IOP||On the first invocation following a pcap_open_live() or pcap_breakloop() additional information is sent:
|IOP -> SMP||Sends captured packets.|
|pcap_setfilter()||SMP -> IOP||At any time, the SMP can issue a set filter command which contains
an indicator, a count of the number of statements in the filter,
followed by the sequence of filter commands represented as a sequence
of C-style structures.
|IOP -> SMP||In return the IOP will send its (possibly empty) NULL terminated error response string.|
|pcap_stats()||SMP -> IOP||At any time, the SMP can issue a 'retrieve statistics' command which contains:
|IOP -> SMP||In return the IOP will send:
|pcap_close()||SMP -> IOP||At any time, the SMP can close the TCP session with the IOP.|
Although a libpcap file begins with a global header followed by zero or more records for each captured packet, trace data sent to the SMP does NOT begin with a global header. A trace sequence looks like this:
|[Packet Header]||[Packet Data]||[Packet Header]||[Packet Data]||[Packet Header]||[Packet Data]||...|
uint32 tv_sec; /* timestamp seconds */ uint32 tv_usec; /* timestamp microseconds */ uint32 caplen; /* number of octets in the following packet */ uint32 len; /* original length of packet on the wire */
|tv_sec||The date and time when this packet was captured. This value is in seconds since January 1, 1970 00:00:00 GMT; this is also known as a UN*X time_t. You can use the ANSI C time() function from time.h to get this value, but you might use a more optimized way to get this timestamp value. If this timestamp isn't based on GMT (UTC), use thiszone from the global header for adjustments.|
|tv_usec||The microseconds when this packet was captured, as an offset to ts_sec. Beware: this value must never reach 1 second (1,000,000), in this case ts_sec must be increased instead!|
|caplen||The number of bytes actually provided in the capture record. This value should never become larger than len or the snaplen value specified during the capture.|
|len||The length of the packet "on the wire" when it was captured. If caplen and len differ, the actually saved packet size was limited by the value of snaplen specified during one of the capture directives such as pcap_dispatch().|
Although each packet follows the standard libpcap format, since there are two types of interfaces that can be monitored, the format of the data packet varies slightly.
|0||Control / Status||xxxxxxx0||Transmitted by capture device||(see 'Errors' octets)|
|xxxxxxx1||Received by capture device|
|1xxxxxxx||No buffer was available during capture of previous packet.|
|xxxx1xxx||Re-Tx Limit Reached||Long Frame|
|xxxx1xxx||Undefined||Digital PLL Error|
|xx1xxxxx||Undefined||Frame Length Violation|
Note 1: Ethernet and Token Ring frames will never be sent as DLT_SITA (with the 5 octet header), but will be sent as their corresponding DLT types instead.