WTAP_ENCAP_UNKNOWN is used for two different cases:
1. Encapsulation type values that are unsupported by libwiretap or
bogus values (and thus "unknown" to libwiretap).
2. An initial state where the encapsulation type is "not yet" known
for a file type like pcapng without a single encapsulation type in the
header, before any packets or interfaces that set the encapsulation type
have been read. (If the file has no packets, this may be the value after
the file is entirely read in.) This can be the value when an output file
is written out simultaneously with reading an input file, rather than
reading the entire input file first, and, e.g., there is a custom block
before any IDBs.
The first case can never be handled when writing out a file, but the
second case can possibly be handled, so long as (for pcapng) IDBs
are available to write when they become necessary, or (for file
types like pcap with a single link-layer type in the header) the
writer waits until a link-layer type is seen to create the output
header. (It is possible, of course, that writing would fail in the
middle if an unsupported encapsulation type appears, or if the
encapsulation becomes per-packet for file types that don't support that,
but that is an unavoidable risk when writing without reading the entire
input file(s).)
Introduce WTAP_ENCAP_NONE for the second case, and use it for pcapng,
where we guarantee that any necessary IDBs will be passed along.
Continue to use WTAP_ENCAP_UNKNOWN for the first case.
Allow pcapng files to open a file for writing with WTAP_ENCAP_NONE.
There are some other file types that support per-packet link-types,
and could also use WTAP_ENCAP_NONE, but they require more work to
generate IDBs. (Note that all of them currently are impossible to
write to pcapng when they have multiple encapsulations, even if
the encapsulations are all supported by pcapng, because they don't
properly generate IDBs.)
Remove the workaround in ef43fd48b4
for tshark writing to pcapng when the source file is WTAP_ENCAP_UNKNOWN,
since now such files will be WTAP_ENCAP_NONE and work properly (and
also work in editcap, mergcap, etc.)
Along with 8cddc32d35, fix#18449.
g_utf_8_make_valid() replaces embedded NULs with unicode
replacement characters and this behaviour does not match
the pcapng spec; the first NULL should terminate the string
instead.
Use ws_utf8_make_valid() which provides the correct behaviour.
Just because a file type supports per packet encapsulation does
not mean it can write every encapsulation type.
When determining what file-level encapsulation to write the merged
file, check each file's encapsulation to see if it is writable by
the output file type. If any are not, report _UNKNOWN so that the
merge will fail early with an appropriate error message, instead of
always reporting WTAP_ENCAP_PER_PACKET.
This prevents confusing errors when trying to merge, e.g., a JSON
file and a normal packet file into a pcapng.
Use a pointer to the growing array of NRBs from the source
file, as with DSBs, so as to handle reading NRBs in the middle
of a file in one-pass mode.
Write NRBs when reading a file with editcap, or in tshark when
not dissecting packets and writing our own NRB. Continue not
to write the NRB if we're supplying our own list of address info
instead.
If we have already read the entire source file in (such as in
two-pass tshark), move all the NRBs to the beginning of the file
before packets, as done with DSBs.
When merging files with mergecap, write both sets of NRBs. (There
is no attempt to merge the NRBs by looking for common entries.)
Check for name resolution data in the middle of dumping a file,
not just at the end, and check for DSBs at the end of a file,
after all the packets. This means that Wireshark no longer writes
the NRB at the very end of the file after all the packets (which
is worse for future one-pass reads), and DSBs after all packets
are preserved.
Ping #15502
pcapng allows multiple link-layer types, and allows new link-layer types
in the middle of a file. Many (most) other capture types allow a single
link-layer type, which must be specified in the initial header.
When reading files and writing their contents to another file (which
may be of a different type), many programs using the wiretap API want
want to know the link-layer type upon initially opening the source
file, so that they can check if that encapsulation can be written to
the output file, and so that they can write the output file header.
They should be able to wait until a link-layer type is seen before
creating the output type, but don't. (Wireshark reads the entire file
in intially, so this isn't a problem, but that isn't much of an option
for some command line tools, particularly when operating on a pipe or
FIFO.) Note that regardless, if a new link-layer type is encountered
partway through a file, they would still have to fail in the middle
of reading and writing.
However, to make this a little bit easier for such file types, pcapng
block types that are handled strictly internally and not passed back
to the reader can be processed initially in pcapng_open(). (Note
that for DSBs and NRBs, any blocks processed in pcapng_open() will
automatically be sent to the callbacks when the callbacks are added
later.) Previously we just processed all the IDBs immediately after
the initial SHB, instead of all the internal block types.
Fix#18581. Ping #15502.
Keep name resolution information as mandatory elements for
NRBs, and when the ipv4 or ipv6 callback is set, have name
resolution entries from already read NRBs sent to the callback.
rescan_packets can use this when redissecting to reobtain the
name resolution entries from the NRB, similar to what is done
with Decryption Secrets Blocks. (This can also later be used
if we read NRBs and DSBs in pcapng_open before the first packet,
and before the callbacks are set.)
This doesn't yet make the changes to wtap_dumper to write them out,
but is a step towards that too. (It's not clear in cases where we
dissect packets whether we want to copy the entire NRB, or only
write out actually used addresses as done now. For copying without
reading a file, like with editcap, we presumably do want to copy them.)
Fix#13425. Ping #15502
Develpment headers are a sizeable part of the binary installation
and most users won't ever require them. It's recommended to package
them separately in a devel package or SDK.
Create a CMake installation component for development headers
and add the EXCLUDE_FROM_ALL property.
Headers can be installed using the invocation:
cmake --install <dir> --component Development
This is a port of SiliconDust ALP decoder provided in public repository
https://github.com/silicondust/wireshark
Original Author: Nick Kelsey <nickk@silicondust.com>
Validate UTF-8 encoding for pcapng string options. To
avoid two unnecessary memory allocations for invalid strings and
make the code cleaner a new wtap_block_add_string_option_owned()
function is used.
Add UTF-8 debug check for wiretap API.
Fixes#18703.
Rename flex macros using parenthesis (mostly a style issue):
DIAG_OFF_FLEX -> DIAG_OFF_FLEX()
DIAG_ON_FLEX -> DIAG_ON_FLEX()
Use the same kind of construct with lemon generated code using
DIAG_OFF_LEMON() and DIAG_ON_LEMON(). Use %include and %code
directives to enforce the desired order with generated code
in the middle in between pragmas.
Fix a clang-specific pragma to use DIAG_OFF_CLANG().
DIAG_OFF(unreachable-code) -> DIAG_OFF_CLANG(unreachable-code).
Apparently GCC is ignoring the -Wunreachable flag, that's why
it did not trigger an unknown pragma warning. From [1}:
The -Wunreachable-code has been removed, because it was unstable: it
relied on the optimizer, and so different versions of gcc would warn
about different code. The compiler still accepts and ignores the
command line option so that existing Makefiles are not broken. In some
future release the option will be removed entirely. - Ian
[1] https://gcc.gnu.org/legacy-ml/gcc-help/2011-05/msg00360.html
This fixes a bug in the canfdmessage64 encoding in BLF that leads to
CAN-FD frames being interpreted as Remote Frames instead of correctly
ignoring the SRR flag. Makes canfdmessage encoding more robust as well.
All other functions are declared with two lines:
WS_DLL_PUBLIC <type>
<function_name(<args>);
Declare wtap_block_foreach_option() that way, too, for consistency.
A given protocol's packet format may depend, for example, on which
lower-level protocol is transporting the protocol in question. For
example, protocols that run atop both byte-stream protocols such as TCP
and TLS, and packet-oriented protocols such as UDP or DTLS, might begin
the packet with a length when running atop a byte-stream protocol, to
indicate where this packet ends and the next packet begins in the byte
stream, but not do so when running atop a packet-oriented protocol.
Dissectors can handle this in various ways:
For example, the dissector could attempt to determine the protocol over
which the packet was transported.
Unfortunately, many of those mechanisms do so by fetching data from the
packet_info structure, and many items in that structure act as global
variables, so that, for example, if there are two two PDUs for protocol
A inside a TCP segment, and the first protocol for PDU A contains a PDU
for protocol B, and protocol B's dissector, or a dissector it calls,
modifies the information in the packet_info structure so that it no
longer indicates that the parent protocol is TCP, the second PDU for
protocol A might not be correctly dissected.
Another such mechanism is to query the previous element in the layers
structure of the packet_info structure, which is a list of protocol IDs.
Unfortunately, that is not a list of earlier protocols in the protocol
stack, it's a list of earlier protocols in the dissection, which means
that, in the above example, when the second PDU for protocol A is
dissected, the list is {...,TCP,A,B,...,A}, which means that the
previous element in the list is not TCP, so, again, the second PDU for
protocol A will not be correctly dissected.
An alternative is to have multiple dissectors for the same protocol,
with the part of the protocol that's independent of the protocol
transporting the PDU being dissected by common code. Protocol B might
have an "over a byte-stream transport" dissector and an "over a packet
transport" dissector, with the first dissector being registered for use
over TCP and TLS and the other dissector being registered for use over
packet protocols. This mechanism, unlike the other mechanisms, is not
dependent on information in the packet_info structure that might be
affected by dissectors other than the one for the protocol that
transports protocol B.
Furthermore, in a LINKTYPE_WIRESHARK_UPPER_PDU pcap or pcapng packet for
protocol B, there might not be any information to indicate the protocol
that transports protocol B, so there would have to be separate
dissectors for protocol B, with separate names, so that a tag giving the
protocol name would differ for B-over-byte-stream and B-over-packets.
So:
We rename EXP_PDU_TAG_PROTO_NAME and EXP_PDU_TAG_HEUR_PROTO_NAME to
EXP_PDU_TAG_DISSECTOR_NAME and EXP_PDU_TAG_HEUR_DISSECTOR_NAME, to
emphasize that they are *not* protocol names, they are dissector names
(which has always been the case - if there's a protocol with that name,
but no dissector with that name, Wireshark will not be able to handle
the packet, as it will try to look up a dissector given that name and
fail).
We fix that exported PDU dissector to refer to those tags as dissector
names, not protocol names.
We update documentation to refer to them as DISSECTOR_NAME tags, not
PROTO_NAME tags. (If there is any documentation for this outside the
Wireshark source, it should be updated as well.)
We add comments for calls to dissector_handle_get_dissector_name() where
the dissector name is shown to the user, to indicate that it might be
that the protocol name should be used.
We update the TLS and DTLS dissectors to show the encapsulated protocol
as the string returned by dissector_handle_get_long_name(); as the
default is "Application Data", it appeaers that a descriptive name,
rather than a short API name, should be used. (We continue to use the
dissector name in debugging messages, to indicate which dissector was
called.)
Call the Vector Informatik BLF format "Vector Informatik Binary Logging
Format (BLF) logfile", to give more details to people who don't
necessarily know what "BLF" means (British Lung Foundation?).
Note that in the introductory comments to blf.c and blf.h as well.
Note that we should perhaps report unknown flags/packet timestamp type
values as errors or warnings.
Note that we can, and should, report at least some 802.11 radio
information to our caller.
This patch adds the format 2 and 3 of the ObjectHeader, so that the blf
code does not stop reading the file, when one of these header formats is
present.
Minor refactoring was necessary.
Add support for displaying one or more packet hashes that
have been recorded in EPB options.
A patch to add support for EPB hash option is pending for next
DPDK release.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
It's set from the result of mktime(), which returns a time_t, and it's
assigned to a time_t that's ultimately assigned to the time_t secs
member of an nstime, so no reason for it to be a guint32.
This should squelch Coverity CID 1509354.
When reading memory-mapped Linux capture files, fix up the "real" length
field, in case the file was written by a program doing a capture done
with a version of libpcap with a bug that causes it to incorrectly set
the "real" length for isochronous transfers.
Libpcap assumes that packet length is greater or equal to captured data
length. However, due to a bug in libpcap, it was possible for libpcap to
generate isochronous URB packets (WTAP_ENCAP_USB_LINUX_MMAPPED) with
captured data length greater than packet length. The discrepancy comes
from slightly different semantics in Linux kernel.
Linux kernel usbmon packet documentation mentions:
unsigned int length; /* 32: Length of data (submitted or actual) */
unsigned int len_cap; /* 36: Delivered length */
Wireshark shows usbmon packet length as URB length (usb.urb_len) and
len_cap as Data length (usb.data_len). For usbmon isochronous IN packets
containing data (URB complete), usbmon length is "actual". Actual length
is the sum of payload packets length received from device. Delivered
length refers to the amount of data associated with usbmon packet, that
is the isochronous descriptors and actual isochronous data. There can be
multiple isochronous descriptors in single URB and the actual payload in
special cases can be noncontiguous (there can be gaps).
Libpcap when reading usbmon capture calculates packet length based on
usbmon packet structure size (64), "actual length" and number of
isochronous descriptors. This gives expected packet length as long as
there are no gaps between isochronous data. If there are gaps, the
calculated packet length will be smaller than delivered length.
Wireshark should show the frame length and captured length as provided
by the capture engine, even if the capture length is greater than frame
length. Silently limiting captured length essentially hides the issue
from the user and allows misbehaving capture engine to go unnoticed.
Passing unmodified Frame Length and Capture Length to dissectors (and
thus complete tvb) allows USB dissector to show all ISO Data fields
captured on Linux usbmon interface using bugged libpcap.
Fixes#18021
USB 2.0/1.1/1.0 devices (or 3.x and newer when connected to hosts that
are not Super-Speed capable) operate at one of three speeds:
* Low-Speed (1.5 Mbps)
* Full-Speed (12 Mbps)
* High-Speed (480 Mbps)
Supporting speed specific linktypes allows speed specific dissection
without the need for user to manually set the speed.
Rename the following build targets, similar to the recent macOS target
name changes:
nsis_package_prep to wireshark_nsis_prep
nsis_package to wireshark_nsis
Rename some NSIS files to reflect that they're specific to Wireshark.
Update the documentation and CI configurations.
Support merging files with IDBs in the middle of the file.
Use wtap_get_next_interface_description when doing the initial
list of interfaces so that we can correctly get IDBs later.
Note that while IDB merge modes "any" and "none" work as expected, the
default "all" mode can't really work for IDBs in the middle of the file
without adding a two-pass mode. In "all" mode, if there are any such IDBs,
merge them with duplicates iff the interfaces at the beginning of the
files were merged.
Related to #15502 and #16542.
If we do a save with copy, so that we just copied the binary
file, everything in the wtap structure should be the same except
for the filename and the file descriptors, so just change that
instead of closing wtap and reopening it.
The current behavior of calling wtap_open_offline does not work
for files that have blocks (SHBs, IDBs, NRBs, DSBs, ISBs, etc.) in
the middle of the file instead of at the beginning, but we shouldn't
have to waste time rescanning the entire file either.
In the case where a specific file format reader was manually selected,
this will keep the same file format as selected instead of switching to
the auto-detection when opening the copy, just as SAVE_WITH_MOVE already
does and presumably what the user wants.
Update wtap_fdreopen to change the wtap struct's pathname if
wtap_fdreopen is called with a different filename than currently.
Fix#17472
If merge_files_common() is called with a non NULL value for out_filenamep,
that always indicates tempfile mode, even if the tempdir is not set.
A NULL value for the tempdir is handled by wtap_dump_open_tempfile,
which writes to the OS default temp directory.
Only write to stdout if both out_filename and out_filenamep are NULL.
Fixes a crash introduced by commit 1e0d117eb7
when selecting Merge from the GUI and the new temp_dir option is not set.
Add a "section number" field to wtap_rec, with a presence flag, and
provide the section number (0-based) for pcapng files.
Display it (1-based) if present.
David Perry