WTAP_ENCAP_UNKNOWN is used for two different cases:
1. Encapsulation type values that are unsupported by libwiretap or
bogus values (and thus "unknown" to libwiretap).
2. An initial state where the encapsulation type is "not yet" known
for a file type like pcapng without a single encapsulation type in the
header, before any packets or interfaces that set the encapsulation type
have been read. (If the file has no packets, this may be the value after
the file is entirely read in.) This can be the value when an output file
is written out simultaneously with reading an input file, rather than
reading the entire input file first, and, e.g., there is a custom block
before any IDBs.
The first case can never be handled when writing out a file, but the
second case can possibly be handled, so long as (for pcapng) IDBs
are available to write when they become necessary, or (for file
types like pcap with a single link-layer type in the header) the
writer waits until a link-layer type is seen to create the output
header. (It is possible, of course, that writing would fail in the
middle if an unsupported encapsulation type appears, or if the
encapsulation becomes per-packet for file types that don't support that,
but that is an unavoidable risk when writing without reading the entire
input file(s).)
Introduce WTAP_ENCAP_NONE for the second case, and use it for pcapng,
where we guarantee that any necessary IDBs will be passed along.
Continue to use WTAP_ENCAP_UNKNOWN for the first case.
Allow pcapng files to open a file for writing with WTAP_ENCAP_NONE.
There are some other file types that support per-packet link-types,
and could also use WTAP_ENCAP_NONE, but they require more work to
generate IDBs. (Note that all of them currently are impossible to
write to pcapng when they have multiple encapsulations, even if
the encapsulations are all supported by pcapng, because they don't
properly generate IDBs.)
Remove the workaround in ef43fd48b4
for tshark writing to pcapng when the source file is WTAP_ENCAP_UNKNOWN,
since now such files will be WTAP_ENCAP_NONE and work properly (and
also work in editcap, mergcap, etc.)
Along with 8cddc32d35, fix#18449.
g_utf_8_make_valid() replaces embedded NULs with unicode
replacement characters and this behaviour does not match
the pcapng spec; the first NULL should terminate the string
instead.
Use ws_utf8_make_valid() which provides the correct behaviour.
Use a pointer to the growing array of NRBs from the source
file, as with DSBs, so as to handle reading NRBs in the middle
of a file in one-pass mode.
Write NRBs when reading a file with editcap, or in tshark when
not dissecting packets and writing our own NRB. Continue not
to write the NRB if we're supplying our own list of address info
instead.
If we have already read the entire source file in (such as in
two-pass tshark), move all the NRBs to the beginning of the file
before packets, as done with DSBs.
When merging files with mergecap, write both sets of NRBs. (There
is no attempt to merge the NRBs by looking for common entries.)
Check for name resolution data in the middle of dumping a file,
not just at the end, and check for DSBs at the end of a file,
after all the packets. This means that Wireshark no longer writes
the NRB at the very end of the file after all the packets (which
is worse for future one-pass reads), and DSBs after all packets
are preserved.
Ping #15502
pcapng allows multiple link-layer types, and allows new link-layer types
in the middle of a file. Many (most) other capture types allow a single
link-layer type, which must be specified in the initial header.
When reading files and writing their contents to another file (which
may be of a different type), many programs using the wiretap API want
want to know the link-layer type upon initially opening the source
file, so that they can check if that encapsulation can be written to
the output file, and so that they can write the output file header.
They should be able to wait until a link-layer type is seen before
creating the output type, but don't. (Wireshark reads the entire file
in intially, so this isn't a problem, but that isn't much of an option
for some command line tools, particularly when operating on a pipe or
FIFO.) Note that regardless, if a new link-layer type is encountered
partway through a file, they would still have to fail in the middle
of reading and writing.
However, to make this a little bit easier for such file types, pcapng
block types that are handled strictly internally and not passed back
to the reader can be processed initially in pcapng_open(). (Note
that for DSBs and NRBs, any blocks processed in pcapng_open() will
automatically be sent to the callbacks when the callbacks are added
later.) Previously we just processed all the IDBs immediately after
the initial SHB, instead of all the internal block types.
Fix#18581. Ping #15502.
Keep name resolution information as mandatory elements for
NRBs, and when the ipv4 or ipv6 callback is set, have name
resolution entries from already read NRBs sent to the callback.
rescan_packets can use this when redissecting to reobtain the
name resolution entries from the NRB, similar to what is done
with Decryption Secrets Blocks. (This can also later be used
if we read NRBs and DSBs in pcapng_open before the first packet,
and before the callbacks are set.)
This doesn't yet make the changes to wtap_dumper to write them out,
but is a step towards that too. (It's not clear in cases where we
dissect packets whether we want to copy the entire NRB, or only
write out actually used addresses as done now. For copying without
reading a file, like with editcap, we presumably do want to copy them.)
Fix#13425. Ping #15502
Validate UTF-8 encoding for pcapng string options. To
avoid two unnecessary memory allocations for invalid strings and
make the code cleaner a new wtap_block_add_string_option_owned()
function is used.
Add UTF-8 debug check for wiretap API.
Fixes#18703.
Add support for displaying one or more packet hashes that
have been recorded in EPB options.
A patch to add support for EPB hash option is pending for next
DPDK release.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Add a "section number" field to wtap_rec, with a presence flag, and
provide the section number (0-based) for pcapng files.
Display it (1-based) if present.
Use the same style of message for too-short block errors ("pcapng: total
block length XXX of {a,an} XXX is too small...").
Add an additional check for the "skip" Netflix cutom block, to make sure
it has enough room for the 4-byte "skipped" value.
wblock->internal is not initialized on pcapng_read_custom_block function
pcapng.c:3747:9: warning: Branch condition evaluates to a garbage value [core.uninitialized.Branch]
Use compute_options_size() to get the total size of all the options, and
use write_options() to write out the options for those blocks, as we do
for other blocks.
Get rid of wtap_block_option_get_value_size() and
wtap_block_get_options_size_padded(); they're no longer needed, and
their notion of an option's "size" is "size in a pcapng file", so that
doesn't belong in code that's intended to support all file types.
Move the extraction of the option value from the option content from the
callers of pcapng_process_uint32_option() to
pcapng_process_uint32_option() itself.
That way, add-on modules to handle block types not handled by the core
pcapng code can use pcap_process_options() to process a block's options
and can use the routines to handle the "standard" option value types to
handle particular options.
Also, allow both everything-is-little-endian and
everything-is-big-endian Custom Block types in pcap_process_options().
Spell out "DESCRIPTION" for the IDB description option, as it's spelled
out in the pcapng spec.
Put the #defines for various options in the same order as the block
types for them are in the pcapng spec.
Mostly functioning proof of concept for #14329. This work is intended to
allow Wireshark to support multiple packet comments per packet.
Uses and expands upon the `wtap_block` API in `wiretap/wtap_opttypes.h`.
It attaches a `wtap_block` structure to `wtap_rec` in place of its
current `opt_comment` and `packet_verdict` members to hold OPT_COMMENT
and OPT_PKT_VERDICT option values.
Consistently use pcapng_compute_XXX_option_size() for routines to
compute the size of an option of type XXX and pcapng_write_XXX_option()
for routins to write out an option of type XXX.
Sort the routines by the order in which their option type values are
defined.
Have common routines that iterate over all the options, processing
comment and custom options in common code (as they're defined
independently of particular block types), with callbacks to handle the
options for particular block types.
Have the routine it calls return a Boolean value, with "true" meaning
"keep going" and "false" meaning "stop iterating and return a failure
indication". If the callback routine never returns "false", the routine
returns "true" as a success indication.
The name of the block, in the pcapng specification is the systemd
Journal Export Block; add "export" after "journal" in various
variable/enum/define names.
Now that it's being done in common code, we don't need to do it in the
routines to read sysdig event blocks, systemd journal export blocks, or
unknown blocks.
Add in a comment to match other comments while we're at it.
John Thacker