WTAP_ENCAP_UNKNOWN is used for two different cases:
1. Encapsulation type values that are unsupported by libwiretap or
bogus values (and thus "unknown" to libwiretap).
2. An initial state where the encapsulation type is "not yet" known
for a file type like pcapng without a single encapsulation type in the
header, before any packets or interfaces that set the encapsulation type
have been read. (If the file has no packets, this may be the value after
the file is entirely read in.) This can be the value when an output file
is written out simultaneously with reading an input file, rather than
reading the entire input file first, and, e.g., there is a custom block
before any IDBs.
The first case can never be handled when writing out a file, but the
second case can possibly be handled, so long as (for pcapng) IDBs
are available to write when they become necessary, or (for file
types like pcap with a single link-layer type in the header) the
writer waits until a link-layer type is seen to create the output
header. (It is possible, of course, that writing would fail in the
middle if an unsupported encapsulation type appears, or if the
encapsulation becomes per-packet for file types that don't support that,
but that is an unavoidable risk when writing without reading the entire
input file(s).)
Introduce WTAP_ENCAP_NONE for the second case, and use it for pcapng,
where we guarantee that any necessary IDBs will be passed along.
Continue to use WTAP_ENCAP_UNKNOWN for the first case.
Allow pcapng files to open a file for writing with WTAP_ENCAP_NONE.
There are some other file types that support per-packet link-types,
and could also use WTAP_ENCAP_NONE, but they require more work to
generate IDBs. (Note that all of them currently are impossible to
write to pcapng when they have multiple encapsulations, even if
the encapsulations are all supported by pcapng, because they don't
properly generate IDBs.)
Remove the workaround in ef43fd48b4
for tshark writing to pcapng when the source file is WTAP_ENCAP_UNKNOWN,
since now such files will be WTAP_ENCAP_NONE and work properly (and
also work in editcap, mergcap, etc.)
Along with 8cddc32d35, fix#18449.
Use a pointer to the growing array of NRBs from the source
file, as with DSBs, so as to handle reading NRBs in the middle
of a file in one-pass mode.
Write NRBs when reading a file with editcap, or in tshark when
not dissecting packets and writing our own NRB. Continue not
to write the NRB if we're supplying our own list of address info
instead.
If we have already read the entire source file in (such as in
two-pass tshark), move all the NRBs to the beginning of the file
before packets, as done with DSBs.
When merging files with mergecap, write both sets of NRBs. (There
is no attempt to merge the NRBs by looking for common entries.)
Check for name resolution data in the middle of dumping a file,
not just at the end, and check for DSBs at the end of a file,
after all the packets. This means that Wireshark no longer writes
the NRB at the very end of the file after all the packets (which
is worse for future one-pass reads), and DSBs after all packets
are preserved.
Ping #15502
Keep name resolution information as mandatory elements for
NRBs, and when the ipv4 or ipv6 callback is set, have name
resolution entries from already read NRBs sent to the callback.
rescan_packets can use this when redissecting to reobtain the
name resolution entries from the NRB, similar to what is done
with Decryption Secrets Blocks. (This can also later be used
if we read NRBs and DSBs in pcapng_open before the first packet,
and before the callbacks are set.)
This doesn't yet make the changes to wtap_dumper to write them out,
but is a step towards that too. (It's not clear in cases where we
dissect packets whether we want to copy the entire NRB, or only
write out actually used addresses as done now. For copying without
reading a file, like with editcap, we presumably do want to copy them.)
Fix#13425. Ping #15502
This is a port of SiliconDust ALP decoder provided in public repository
https://github.com/silicondust/wireshark
Original Author: Nick Kelsey <nickk@silicondust.com>
Libpcap assumes that packet length is greater or equal to captured data
length. However, due to a bug in libpcap, it was possible for libpcap to
generate isochronous URB packets (WTAP_ENCAP_USB_LINUX_MMAPPED) with
captured data length greater than packet length. The discrepancy comes
from slightly different semantics in Linux kernel.
Linux kernel usbmon packet documentation mentions:
unsigned int length; /* 32: Length of data (submitted or actual) */
unsigned int len_cap; /* 36: Delivered length */
Wireshark shows usbmon packet length as URB length (usb.urb_len) and
len_cap as Data length (usb.data_len). For usbmon isochronous IN packets
containing data (URB complete), usbmon length is "actual". Actual length
is the sum of payload packets length received from device. Delivered
length refers to the amount of data associated with usbmon packet, that
is the isochronous descriptors and actual isochronous data. There can be
multiple isochronous descriptors in single URB and the actual payload in
special cases can be noncontiguous (there can be gaps).
Libpcap when reading usbmon capture calculates packet length based on
usbmon packet structure size (64), "actual length" and number of
isochronous descriptors. This gives expected packet length as long as
there are no gaps between isochronous data. If there are gaps, the
calculated packet length will be smaller than delivered length.
Wireshark should show the frame length and captured length as provided
by the capture engine, even if the capture length is greater than frame
length. Silently limiting captured length essentially hides the issue
from the user and allows misbehaving capture engine to go unnoticed.
Passing unmodified Frame Length and Capture Length to dissectors (and
thus complete tvb) allows USB dissector to show all ISO Data fields
captured on Linux usbmon interface using bugged libpcap.
Fixes#18021
USB 2.0/1.1/1.0 devices (or 3.x and newer when connected to hosts that
are not Super-Speed capable) operate at one of three speeds:
* Low-Speed (1.5 Mbps)
* Full-Speed (12 Mbps)
* High-Speed (480 Mbps)
Supporting speed specific linktypes allows speed specific dissection
without the need for user to manually set the speed.
Add a "section number" field to wtap_rec, with a presence flag, and
provide the section number (0-based) for pcapng files.
Display it (1-based) if present.
Without that, you could add a comment to a record in a file format the
reading code for which doesn't allocate blocks, but the comment doesn't
get saved, as there's no block in which to save the comment option.
This simplifies some code paths, as we're either using the record's
modified block or we're using the block as read from the file, there's
no third possibility.
If we attempt to read a record, and we get an error, and a block was
allocated for the record, unreference it, so the individual file readers
don't have to worry about it.
Bug 17478 was caused by `wtap_rec.block` being allocated for each
packet, but not freed when it was done being used -- typically at the
end of a loop.
Rather than requiring each caller of `wtap_read()` to know to free a
member of `rec`, I added a new function `wtap_rec_reset()` for a
slightly cleaner API. Added calls to it everywhere that seemed to make
sense.
Fixes#17478
Spell out "DESCRIPTION" for the IDB description option, as it's spelled
out in the pcapng spec.
Put the #defines for various options in the same order as the block
types for them are in the pcapng spec.
"User" sounds as if the blocks belong to the user; at most, the current
user might have modified them directly, but they might also have, for
example, run a Lua script that, unknown to them, modified comments.
Also, a file might have "user comments" added by a previous user, who
them wrote the file and and provided it to the current user.
"Modified" seems a bit clearer than "changed".
Mostly functioning proof of concept for #14329. This work is intended to
allow Wireshark to support multiple packet comments per packet.
Uses and expands upon the `wtap_block` API in `wiretap/wtap_opttypes.h`.
It attaches a `wtap_block` structure to `wtap_rec` in place of its
current `opt_comment` and `packet_verdict` members to hold OPT_COMMENT
and OPT_PKT_VERDICT option values.
Replace all instances of "Nordic BLE Sniffer" with
"nRF Sniffer for Bluetooth LE" which is the name used by
nordic semiconductor for the development tool on the homepage.
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
Replace most instances of ws_debug_printf() except in
epan/dissectors and dissector plugins.
Some replacements use printf(), some use ws_debug(), and
some were removed because they were dead or judged to be
temporary.
Instead of a "supports name resolution" Boolean and bitflags for types of
comments supported, provide a list of block types that the file
type/subtype supports, with each block type having a list of options
supported. Indicate whether "supported" means "one instance" or
"multiple instances".
"Supports" doesn't just mean "can be written", it also means "could be
read".
Rename WTAP_BLOCK_IF_DESCRIPTION to WTAP_BLOCK_IF_ID_AND_INFO, to
indicate that it provides, in addition to information about the
interface, an ID (implicitly, in pcapng files, by its ordinal number)
that is associated with every packet in the file. Emphasize that in
comments - just because your capture file format can list the interfaces
on which a capture was done, that doesn't mean it supports this; it
doesn't do so if the file doesn't indicate, for every packet, on which
of those interfaces it was captured (I'm looking at *you*, Microsoft
Network Monitor...).
Use APIs to query that information to do what the "does this file
type/subtype support name resolution information", "does this file
type/subtype support all of these comment types", and "does this file
type/subtype support - and require - interface IDs" APIs did.
Provide backwards compatibility for Lua.
This allows us to eliminate the WTAP_FILE_TYPE_SUBTYPE_ values for IBM's
iptrace; do so.
Save a copy of the pathname used to open a file in the wtap structure.
This allows the BER file reader to put a pointer to it in the
pseudo-header; it also would allow file readers to attempt to read
"associated" files that have the same name as the file, but with a
different extension.
Instead of having cf_open() special-case BER files, and calling a
routine in the BER dissector to specify the file name to the dissector,
have separate dissectors for "dissect packet payload as BER" and
"dissect a file as BER", and have the latter get the pathname of the
file from the pseudo-header and determine the ASN.1 syntax from that.
(Side-effect - this means that you can now dissect a BER file, and have
the syntax be determined by the file extension, in TShark as well; the
above cf_open() special-casing was *not* done in TShark, so it didn't
work before. Now the application code doesn't need to do any of that,
so it works in TShark as well as Wireshark.)
Provide a wiretap routine to get an array of all savable file
type/subtypes, sorted with pcap and pcapng at the top, followed by the
other types, sorted either by the name or the description.
Use that routine to list options for the -F flag for various commands
Rename wtap_get_savable_file_types_subtypes() to
wtap_get_savable_file_types_subtypes_for_file(), to indicate that it
provides an array of all file type/subtypes in which a given file can be
saved. Have it sort all types, other than the default type/subtype and,
if there is one, the "other" type (both of which are put at the top), by
the name or the description.
Don't allow wtap_register_file_type_subtypes() to override any existing
registrations; have them always register a new type. In that routine,
if there are any emply slots in the table, due to an entry being
unregistered, use it rather than allocating a new slot.
Don't allow unregistration of built-in types.
Rename the "dump open table" to the "file type/subtype table", as it has
entries for all types/subtypes, even if we can't write them.
Initialize that table in a routine that pre-allocates the GArray before
filling it with built-in types/subtypes, so it doesn't keep getting
reallocated.
Get rid of wtap_num_file_types_subtypes - it's just a copy of the size
of the GArray.
Don't have wtap_file_type_subtype_description() crash if handed an
file type/subtype that isn't a valid array index - just return NULL, as
we do with wtap_file_type_subtype_name().
In wtap_name_to_file_type_subtype(), don't use WTAP_FILE_TYPE_SUBTYPE_
names for the backwards-compatibility names - map those names to the
current names, and then look them up. This reduces the number of
uses of hardwired WTAP_FILE_TYPE_SUBTYPE_ values.
Clean up the type of wtap_module_count - it has no need to be a gulong.
Have built-in wiretap file handlers register names to be used for their
file type/subtypes, rather than building the table in init.lua.
Add a new Lua C function get_wtap_filetypes() to construct the
wtap_filetypes table, based on the registered names, and use it in
init.lua.
Add a #define WSLUA_INTERNAL_FUNCTION to register functions intended
only for internal use in init.lua, so they can be made available from
Lua without being documented.
Get rid of WTAP_NUM_FILE_TYPES_SUBTYPES - most code has no need to use
it, as it can just request arrays of types, and the space of
type/subtype codes can be sparse due to registration in any case, so
code has to be careful using it.
wtap_get_num_file_types_subtypes() is no longer used, so remove it. It
returns the number of elements in the file type/subtype array, which is
not necessarily the name of known file type/subtypes, as there may have
been some deregistered types, and those types do *not* get removed from
the array, they just get cleared so that they're available for future
allocation (we don't want the indices of any registered types to changes
if another type is deregistered, as those indicates are the type/subtype
values, so we can't shrink the array).
Clean up white space and remove some comments that shouldn't have been
added.
Remove NG from the names - it adds nothing.
Don't use the abbreviations for pcapng block names, spell out what the
block does (e.g. "WTAP_BLOCK_DECRYPTION_SECRETS" rather than
"WTAP_BLOCK_DSB"), to make it more obvious what the block does.
Spell out some other abbreviations.
Add WTAP_BLOCK_PACKET for future use for packet blocks; there's no need
to distinguish between the Enhanced Packet Block, the Simple Packet
Block, and the deprecated Packet Block here.
Currently, only pcapng has one, and it does nothing, but this mechanism
will be used more in the future.
Update comments in epan/dissectors/CMakeLists.txt and ui/taps.h while
we're at it.
Instead *_register_plugin() is turned into a noop (with a warning).
The test suit is failing with ENABLE_PLUGINS=Off (it was already failing
before and this patch didn't affect that).
Closes#17202.
Change the data structure for that option to have a type field,
indicating that it's either a pcap filter string or a BPF program,
followed by a union with a string-pointer member for pcap filter strings
and an instruction-count-and-pointer-to-instructions structure for BPF
programs.
Have routines to add, set, and fetch that option that handle that
structure; discard the "generic structured option" routines. That means
there's more type checking possible at compile time.
Add more code to handle BPF programs.
When writing pcapng files, check, both for that option and for string
options, whether the option length is too big for the data to fit in a
pcapng option, and don't write it if it is. (XXX - truncate the data?
Report an error?)
Instead of grabbing the set of IDBs found at open time, have a loop
using wtap_get_next_interface_description() to read all unread IDBs run
after opening the input file, after reading a packet from the input
file, and after getting an EOF on the input file.
Add a routine wtap_uses_interface_ids() to check whether the file type
and subtype for a dump file uses interface IDs and requires IDBs. If
so, in the aforementioned loop, add the IDBs to the dump stream.
Add a routine wtap_dump_add_idb() to add IDBs to a dump stream. Have it
call a file-format-specific routine to add the IDBs; the only file type
that supports it is pcapng, and it 1) writes out the IDB and 2) adds it
to the set of IDBs for the stream.
Add a wtap_dump_params_init_no_idbs() routine that prevents the IDBs
from the input file from being used to initialize the output file; use
it in cases where we're using the aforementioned loop to copy over IDBs.
Don't require any IDBs to be present when opening a pcapng file for
writing; 1) the simplest pcapng file has just an SHB in it, 2) that
requirement causes dumps that don't provide IDBs at open time to fail,
and 3) the real issue is that we don't want packets with an interface ID
not corresponding to a known IDB, and we already have a check for that.
(There are some hacks here; eventually, when everything processes the
IDBs in such a loop, we may be able to get rid of the "two favors of
dump parameter initialization" hack.)
Fixes#15844.
Addresses the same issue in #15502, but there are other issues there
that also need to be addressed.
In addition, the merge code also needs to be changed to handle this.
It currently wraps wtap_block_create() and wtap_block_copy(); if there
are no remaining use cases for wtap_block_copy() at some point, it can
just *replace* wtap_block_copy().
In a wtap, keep track of the first interface description not yet fetched
with wtap_get_next_interface_description() and, when
wtap_get_next_interface_description() is called, have it return that
description, as a wtap_block_t for its IDB. If there are no
as-yet-unfetched interface descriptions, return NULL; there may, in the
future, be more interface descriptions for the file, so this should be
called:
* after the file is opened;
* after wtap_read() returns TRUE, indicating that it's returned a
record (and *before* you process the record that wtap_read()
returns, as it might be the interface description for the
interface on which the packet in that record arrived);
* after wtap_read() returns FALSE, indicating an EOF or an error
return (as there might have been interfaces at the end of the
file or before the error point).
At each of those points, the caller should loop until
wtap_get_next_interface_description() returns NULL.
Not used yet (but tested with capinfos, which found a reason why you
have to wait until the end of the file before processing the interface
information - there's now a comment in the code giving that reason).
This will probably be used in the future.
Add a tsprec value to the wtap_dump_params structure, giving the
per-file time stamp precision.
In wtap_dump_init_dumper(), when constructing a dummy IDB for files that
don't have one, fill in the tsprecision and time_units_per_second values
based on the tsprec value in the wtap_dump_params structure.
Change-Id: I3708b144d4d0ac0dfbe32bd1c16768a75c942141
Reviewed-on: https://code.wireshark.org/review/37979
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
It generates a fake IDB for files that don't have interface information
and that have a per-file encapsulation type, snapshot length, and time
stamp precision, and adds it to the file's list of IDBs.
Use it for libpcap.
We will use it later for other file formats, so that code such as the
mergecap code to merge into a pcapng file can handle input files that
don't have interface information.
(We should have a way to indicate whether the IDBs are real or fake, so
that capinfos and Statistics > Capture File Properties don't report
meaningless IDB information and make it look as if it's known that the
capture was done on one interface with the properties in question.)
Change-Id: Iec124bf3c7cbd4c69ec2ac7d0dd776e5287f8576
Reviewed-on: https://code.wireshark.org/review/37982
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Add support to read/write the new EPB options, epb_packetid,
epb_queue and epb_verdict, from/to pcap files.
In addition, it updates the packet-frame dissector to dissect
these new fields.
More details on the options can be found in the PcapNG
specification: https://github.com/pcapng/pcapng
An application using these new fields can be found here:
https://github.com/chaudron/xdp-tools/tree/dev/pcapngII/xdp-dump
Change-Id: I761b8114b437fe573dd2c750e35586ad88494938
Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Reviewed-on: https://code.wireshark.org/review/37412
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Different header, with a different size, an additional field, and with
fields being in a different order.
Distinguish between V1 and V2 by giving the version.
That means we can no longer use the "ethertype" dissector as it stands,
because the packet type field isn't at the end of the header, right
before the payload; pull the "add the type field to the protocol tree"
functionality out of the "ethertype" dissector and leave it up to the
dissector calling it.
Change-Id: I72b8a2483c0a539919fbe5d35fd7e60bff4bf75a
Reviewed-on: https://code.wireshark.org/review/37169
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Make wtap_file_get_shb() take a section number argument, and update code
that called it. In most cases, we convert the code to iterate over
sections; in cases where a big code change would be required, we
temporarily pass it 0 and mark the code as "needs to be updated for
multiple sections".
Eliminate cf_read_section_comment(); in calls outside file.c, other code
directly calls the libwiretap routines it calls and, inside file.c, we
just transplant the code and then fix it not to assume a single SHB.
Change-Id: I85e94d0a4fc878e9d937088759be04cb004e019b
Reviewed-on: https://code.wireshark.org/review/37000
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
John Thacker