wireshark/wiretap

$Id$

Wiretap is a library that is being developed as a future replacement for
libpcap, the current standard Unix library for packet capturing. Libpcap
is great in that it is very platform independent and has a wonderful
BPF optimizing engine. But it has some shortcomings as well. These
shortcomings came to a head during the development of Wireshark
(http://www.wireshark.org/), a packet analyzer. As such, I began developing
wiretap so that:

1. The library can easily be amended with new packet filtering objects.
Libpcap is very TCP/IP-oriented. I want to filter on IPX objects, SNA objects,
etc. I also want any decent programmer to be able to add new filters to the
library.

2. The library can read file formats from many packet-capturing utilities.
Libpcap only reads Libpcap files.

3. The library can capture on more than one network interface at a time, and
save this trace in one file.

4. Network names can be resolved immediately after a trace and saved in the
trace file. That way, I can ship a trace of my firewall-protected network to a
colleague, and he'll see the proper hostnames for the IP addresses in the
packet capture, even though he doesn't have access to the DNS server behind my
LAN's firewall.

5. I want to look into the possibility of compressing packet data when saved
to a file, like Sniffer.

6. The packet-filter can be optimized for the host OS. Not all OSes have BPF;
SunOS has NIT and Solaris has DLPI, which both use the CMU/Stanford
packet-filter pseudomachine. RMON has another type of packet-filter syntax
which we could support.

Wiretap is very good at reading many file formats, as per #2
above. Wiretap has no filter capability at present; it currently doesn't
support packet capture, so it wouldn't be useful there, and filtering
when reading a capture file is done by Wireshark, using a more powerful
filtering mechanism than that provided by BPF.


File Formats
============

Libpcap
-------
The "libpcap" file format was determined by reading the "libpcap" code;
wiretap reads the "libpcap" file format with its own code, rather than
using the "libpcap" library's code to read it.

Sniffer (compressed and uncompressed)
-------
The uncompressed Sniffer format is documented in the Sniffer manual.
Unfortunately, Sniffer manuals tend to document only the format for
the Sniffer model they document. Token-Ring and ethernet seems to work
well, though.  If you have an ATM Sniffer file, both Guy and Gilbert
would be *very* interested in receiving a sample. (see 'AUTHORS' file
for our e-mail addresses).

LANalyzer
---------
The LANalyzer format is available from http://www.novell.com. Search
their knowledge base for "Trace File Format".

Network Monitor
---------------
Microsoft's Network Monitor file format is supported, at least under
Ethernet and token-ring. If you have capture files of other datalink
types, please send them to Guy.

"snoop"
-------
The Solaris 2.x "snoop" program's format is documented in RFC 1761.

"iptrace"
---------
This is the capture program that comes with AIX 3.x and 4.x.  AIX 3 uses
the iptrace 1.0 file format, while AIX4 uses iptrace 2.0.  iptrace has
an undocumented, yet very simple, file format.  The interesting thing
about iptrace is that it will record packets coming in from all network
interfaces; a single iptrace file can contain multiple datalink types.

Sniffer Basic (NetXRay)/Windows Sniffer Pro
-------------------------------------------
Network Associates' Sniffer Basic (formerly NetXRay from Cinco Networks)
file format is now supported, at least for Ethernet and token-ring. 
Network Associates' Windows Sniffer Pro appears to use a variant of that
format; it's supported to the same extent.

RADCOM WAN/LAN Analyzers
------------------------
Olivier Abad has added code to read Ethernet and LAPB captures from
RADCOM WAN/LAN Analyzers (see http://www.radcom-inc.com/).

Lucent/Ascend access products
-----------------------------
Gerald

HP-UX nettl
-----------
nettl is used on HP-UX to trace various streams based subsystems.  Wiretap
can read nettl files containing IP frames (NS_LS_IP subsystem) and LAPB
frames (SX25L2 subsystem). It has been tested with files generated on
HP-UX 9.04 and 10.20.
Use the following commands to generate a trace :
# IP capture. 0x30000000 means PDU in and PDU out :
nettl -tn 0x30000000 -e NS_LS_IP -f tracefile
# X25 capture. You must specify an interface :
nettl -tn 0x30000000 -e SX25l2 -d /dev/x25_0 -f tracefile
# stop capture. subsystem is NS_LS_IP or SX25L2 :
nettl -tf -e subsystem

One may be able to specify "-tn pduin pduout" rather than
"-tn 0x30000000"; the nettl man page for HP-UX 10.30 implies that it
should work.

There is also basic support for nettl files containing NS_LS_DRIVER, 
NS_LS_TCP, NS_LS_UDP, NS_LS_LOOPBACK, unknown type 0xb9, and NS_LS_ICMP.
However, NS_LS_ICMP will not be decoded since WTAP lacks a raw ICMP 
encapsulation type.


Toshiba ISDN Router
-------------------
An under-documented command that the router supports in a telnet session
is "snoop" (not related to the Solaris "snoop" command). If you give
it the "dump" option (either by letting "snoop" query you for its next
argument, or typing "snoop dump" on the command line), you'll get a hex
dump of all packets across the router (except of your own telnet session
-- good thinking Toshiba!). You can select a certain channel to sniff
(LAN, B1, B2, D), but the default is all channels.  You save this hex
dump to disk with 'script' or by 'telnet | tee'. Wiretap will read the
ASCII hex dump and convert it to binary data.

ISDN4BSD "i4btrace" utility
---------------------------
Bert Driehuis

Cisco Secure Intrustion Detection System iplogging facility
-----------------------------------------------------------
Mike Hall

pppd logs (pppdump-format files)
--------------------------------
Gilbert

VMS TCPTRACE
------------
Compaq VMS's TCPIPTRACE format is supported.  This is the capture program
that comes with TCP/IP or UCX as supplied by Compaq or Digital Equipment
Corporation.

Under UCX 4.x, it is invoked as TCPIPTRACE.  Under TCPIP 5.x, it is invoked
as TCPTRACE.

TCPTRACE produces an ascii text based format, that has changed slightly over
time.

DBS Etherwatch (text format)
----------------------------
Text output from DBS Etherwatch is supported.  DBS Etherwatch is available
from: http://www.users.bigpond.com/dbsneddon/software.htm.

Catapult DCT2000 (.out files)
-----------------------------
DCT2000 test systems produce ascii text-based .out files for ports
that have logging enabled. When being read, the data part of the message is
prefixed with a short header that provides some context (context+port,
direction, original timestamp, etc).

You can choose to suppress the reading of non-standard protocols
(i.e. messages between layers rather than the well-known link-level protocols
usually found on board ports).


Gilbert Ramirez <gram@alumni.rice.edu>
Guy Harris <guy@alum.mit.edu>