This allows the "needs to be reloaded" indication to be set in the close
process, as is the case for ERF; having a routine that returns the value
of that indication is not useful if it gets seet in the close process,
as the handle for the wtap_dumper is no longer valid after
wtap_dump_close() finishes.
We also get rid of wtap_dump_get_needs_reload(), as callers should get
that information via the added argument to wtap_dump_close().
Fixes#17989.
Bug 17478 was caused by `wtap_rec.block` being allocated for each
packet, but not freed when it was done being used -- typically at the
end of a loop.
Rather than requiring each caller of `wtap_read()` to know to free a
member of `rec`, I added a new function `wtap_rec_reset()` for a
slightly cleaner API. Added calls to it everywhere that seemed to make
sense.
Fixes#17478
Spell out "DESCRIPTION" for the IDB description option, as it's spelled
out in the pcapng spec.
Put the #defines for various options in the same order as the block
types for them are in the pcapng spec.
Experience has shown that:
1. The current logging methods are not very reliable or practical.
A logging bitmask makes little sense as the user-facing interface (who
would want debug but not crtical messages for example?); it's
computer-friendly and user-unfriendly. More importantly the console
log level preference is initialized too late in the startup process
to be used for the logging subsystem and that fact raises a number
of annoying and hard-to-fix usability issues.
2. Coding around G_MESSAGES_DEBUG to comply with our log level mask
and not clobber the user's settings or not create unexpected log misses
is unworkable and generally follows the principle of most surprise.
The fact that G_MESSAGES_DEBUG="all" can leak to other programs using
GLib is also annoying.
3. The non-structured GLib logging API is very opinionated and lacks
configurability beyond replacing the log handler.
4. Windows GUI has some special code to attach to a console,
but it would be nice to abstract away the rest under a single
interface.
5. Using this logger seems to be noticeably faster.
Deprecate the console log level preference and extend our API to
implement a log handler in wsutil/wslog.h to provide easy-to-use,
flexible and dependable logging during all execution phases.
Log levels have a hierarchy, from most verbose to least verbose
(debug to error). When a given level is set everything above that
is also enabled.
The log level can be set with an environment variable or a command
line option (parsed as soon as possible but still later than the
environment). The default log level is "message".
Dissector logging is not included because it is not clear what log
domain they should use. An explosion to thousands of domains is
not desirable and putting everything in a single domain is probably
too coarse and noisy. For now I think it makes sense to let them do
their own thing using g_log_default_handler() and continue using the
G_MESSAGES_DEBUG mechanism with specific domains for each individual
dissector.
In the future a mechanism may be added to selectively enable these
domains at runtime while trying to avoid the problems introduced
by G_MESSAGES_DEBUG.
Replace most instances of ws_debug_printf() except in
epan/dissectors and dissector plugins.
Some replacements use printf(), some use ws_debug(), and
some were removed because they were dead or judged to be
temporary.
Instead of a "supports name resolution" Boolean and bitflags for types of
comments supported, provide a list of block types that the file
type/subtype supports, with each block type having a list of options
supported. Indicate whether "supported" means "one instance" or
"multiple instances".
"Supports" doesn't just mean "can be written", it also means "could be
read".
Rename WTAP_BLOCK_IF_DESCRIPTION to WTAP_BLOCK_IF_ID_AND_INFO, to
indicate that it provides, in addition to information about the
interface, an ID (implicitly, in pcapng files, by its ordinal number)
that is associated with every packet in the file. Emphasize that in
comments - just because your capture file format can list the interfaces
on which a capture was done, that doesn't mean it supports this; it
doesn't do so if the file doesn't indicate, for every packet, on which
of those interfaces it was captured (I'm looking at *you*, Microsoft
Network Monitor...).
Use APIs to query that information to do what the "does this file
type/subtype support name resolution information", "does this file
type/subtype support all of these comment types", and "does this file
type/subtype support - and require - interface IDs" APIs did.
Provide backwards compatibility for Lua.
This allows us to eliminate the WTAP_FILE_TYPE_SUBTYPE_ values for IBM's
iptrace; do so.
Remove NG from the names - it adds nothing.
Don't use the abbreviations for pcapng block names, spell out what the
block does (e.g. "WTAP_BLOCK_DECRYPTION_SECRETS" rather than
"WTAP_BLOCK_DSB"), to make it more obvious what the block does.
Spell out some other abbreviations.
Add WTAP_BLOCK_PACKET for future use for packet blocks; there's no need
to distinguish between the Enhanced Packet Block, the Simple Packet
Block, and the deprecated Packet Block here.
Currently, the only file types that use them are pcapng and IBM's
iptrace; we don't support writing the latter, so this is mainly of
interest for pcapng.
This makes it a bit more obvious what some "is this pcapng?" tests are
really trying to determine, and allows them to automatically support any
new file types that use them.
(With regard to interface descriptions, tere are three types of file:
1) files that contain no interface information;
2) files that contain "just FYI" interface information but that don't
tie packets or other records to particular interfaces;
3) files that contain interface information and tie all packets (and
possibly other records) to an interface.
This tests for files of type 3.)
generate_merged_idb() can generate multiple IDBs, so rename it to
generate_merged_idbs().
Change-Id: I4c54326f69ff0de16f0a716b7c82beefdda99cbd
Reviewed-on: https://code.wireshark.org/review/38040
Reviewed-by: Guy Harris <gharris@sonic.net>
Change all wireshark.org URLs to use https.
Fix some broken links while we're at it.
Change-Id: I161bf8eeca43b8027605acea666032da86f5ea1c
Reviewed-on: https://code.wireshark.org/review/34089
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Ethernet packets without the CRC are 1514 bytes long, not 1500 bytes
long; using 1514 bytes will avoid a reallocation for a full-sized
Ethernet packet.
Change-Id: Ie8da3f13bf3df07e23e4478b7dcf84f06dec6a9d
Reviewed-on: https://code.wireshark.org/review/32761
Reviewed-by: Guy Harris <guy@alum.mit.edu>
That makes it - and the routines that implement it - work more like the
seek-read routine.
Change-Id: I0cace2d0e4c9ebfc21ac98fd1af1ec70f60a240d
Reviewed-on: https://code.wireshark.org/review/32727
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Support reading and writing pcapng files with DSBs. A DSB may occur
multiple times but should appear before packets that need those
decryption secrets (so it cannot be moved to the end like NRB). The TLS
dissector will be updated in the future to make use of these secrets.
pcapng spec update: https://github.com/pcapng/pcapng/pull/54
As DSBs may be interleaved with packets, do not even try to read it in
pcapng_open (as is done for IDBs). Instead process them during the
sequential read, appending them to the 'wtap::dsbs' array.
Writing is more complicated, secrets may initially not be available when
'wtap_dumper' is created. As they may become available in 'wtap::dsbs'
as more packets are read, allow 'wtap_dumper::dsbs_growing' to reference
this array. This saves every user from checking/dumping DSBs.
If the wtap user needs to insert extra DSBs (while preserving existing
DSBs), they can set the 'wtap_dumper::dsbs_initial' field.
The test file was creating using a patched editcap (future patch) and
combined using mergecap (which required a change to preserve the DSBs).
Change-Id: I74e4ee3171bd852a89ea0f6fbae9e0f65ed6eda9
Ping-Bug: 15252
Reviewed-on: https://code.wireshark.org/review/30692
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The three merge_files routines (filename, tempfile, stdout) have exactly
the same code except for a single wtap_dump_open routine. Reduce code
duplication to ease further improvements to this file.
Change-Id: I4fa890730d54c11b3614e56cf4d3d3da1ae9f5fd
Reviewed-on: https://code.wireshark.org/review/30678
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
This:
1) means that we don't have to flag the compression argument with a
comment to indicate what it means (FALSE doesn't obviously say "not
compressed", WTAP_UNCOMPRESSED does);
2) leaves space in the interfaces in question for additional compression
types.
(No, this is not part 1 of an implementation of additional compression
types, it's just an API cleanup. Implementing additional compression
types involves significant work in libwiretap, as well as UI changes to
replace "compress the file" checkboxes with something to indicate *how*
to compress the file, or to always use some other form of compression).
Change-Id: I1d23dc720be10158e6b34f97baa247ba8a537abf
Reviewed-on: https://code.wireshark.org/review/30660
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Use it for all the per-file information, including the per-file
link-layer type and the per-file snapshot length.
Change-Id: Id75687c7faa6418a2bfcf7f8198206a9f95db629
Reviewed-on: https://code.wireshark.org/review/30616
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Have the routines always take a parameters pointer; pass either null or
a pointer to an initialized-to-nothing structure in cases where we were
calling the non-_ng versions.
Change-Id: I23b779d87f3fbd29306ebe1df568852be113d3b2
Reviewed-on: https://code.wireshark.org/review/30590
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Four variants of wtap_dump_open_ng exists, each of them take the same
three parameters for the SHB, IDB and NRB blocks that has to be written
before packets are even written. Similarly, a lot of tools always create
these arguments based on an existing capture file session (wth).
Address the former duplication by creating a new data structure to hold
the arguments. Address the second issue by creating new helper functions
to initialize the parameters based on a wth. This refactoring should
make it easier to add the new Decryption Secrets Block (DSB).
No functional change intended.
Change-Id: I42c019dc1d48a476773459212ca213de91a55684
Reviewed-on: https://code.wireshark.org/review/30578
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Support for writing it in live captures will come later; this change,
but not that one, will be backported so older versions of Wireshark
won't remove it when writing a file out.
Change-Id: I9fd4067991acfd2d18c03d0a373ce8337a9f3a76
Reviewed-on: https://code.wireshark.org/review/29064
Reviewed-by: Guy Harris <guy@alum.mit.edu>
This allows code to initialize them without having to know the details
of the structure; the initializes should, and will, be changed if the
members of the structure are changed.
Change-Id: I93e6ebfcde9ceca17df696fcba4e8410c5afb175
Reviewed-on: https://code.wireshark.org/review/28501
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Separate the stuff that any record could have from the stuff that only
particular record types have; put the latter into a union, and put all
that into a wtap_rec structure.
Add some record-type checks as necessary.
Change-Id: Id6b3486858f826fce4b096c59231f463e44bfaa2
Reviewed-on: https://code.wireshark.org/review/25696
Reviewed-by: Guy Harris <guy@alum.mit.edu>
The first is deprecated, as per https://spdx.org/licenses/.
Change-Id: I8e21e1d32d09b8b94b93a2dc9fbdde5ffeba6bed
Reviewed-on: https://code.wireshark.org/review/25661
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Petri-Dish: Dario Lombardo <lomato@gmail.com>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Not all do, so test the preference bits for them.
Change-Id: I62976f5d17de3611c4d2f9eb64a0763c0b698c8d
Reviewed-on: https://code.wireshark.org/review/25618
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Not everything wtap_read() returns is a packet.
Change-Id: I3784bbfa308da52f4c55db2a90f9b55f8bfbb2ef
Reviewed-on: https://code.wireshark.org/review/25617
Reviewed-by: Guy Harris <guy@alum.mit.edu>
If merge_open_in_files() is going to fail, free files array to avoid memleak.
Found by clang.
Change-Id: I156c5f1c041cd7779ff0a0095bc2810f50768ab6
Reviewed-on: https://code.wireshark.org/review/25421
Petri-Dish: Jakub Zawadzki <darkjames-ws@darkjames.pl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Existence of in_file has been checked in line 908.
Change-Id: Ida6c06362a1f88caec40701be7f3e42133ce404a
Reviewed-on: https://code.wireshark.org/review/24994
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Use WTAP_MAX_PACKET_SIZE_STANDARD, set to 256KB, for everything except
for D-Bus captures. Use WTAP_MAX_PACKET_SIZE_DBUS, set to 128MB, for
them, because that's the largest possible D-Bus message size. See
https://bugs.freedesktop.org/show_bug.cgi?id=100220
for an example of the problems caused by limiting the snapshot length to
256KB for D-Bus.
Have a snapshot length of 0 in a capture_file structure mean "there is
no snapshot length for the file"; we don't need the has_snap field in
that case, a value of 0 mean "no, we don't have a snapshot length".
In dumpcap, start out with a pipe buffer size of 2KB, and grow it as
necessary. When checking for a too-big packet from a pipe, check
against the appropriate maximum - 128MB for DLT_DBUS, 256KB for
everything else.
Change-Id: Ib2ce7a0cf37b971fbc0318024fd011e18add8b20
Reviewed-on: https://code.wireshark.org/review/21952
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Either 1) it can be determined from the libwiretap encapsulation type,
in which case it's redundant information or 2) there *is* no pcap/pcapng
link-layer header type for that encapsulation type, in which case you
need to check for the attempt to determine it failing and handle that
failure appropriately.
Change-Id: Ie9557b513365c1fc8c6df74b9c8239e29aad46bc
Reviewed-on: https://code.wireshark.org/review/21924
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Have them just return the information needed for the caller to produce
an error message, and have the callers use the new cfile_ routines for
reporting errors.
This requires that the "write failure alert box" routine take the
*input* file name as an argument, so that, on a merge, if the problem is
that a record from a given input file can't be written out to the type
of output file we're generating, the input file name can be given, along
with the record number in that file.
Change-Id: If5a5e00539e7e652008a523dec92c0b359a48e71
Reviewed-on: https://code.wireshark.org/review/21257
Reviewed-by: Guy Harris <guy@alum.mit.edu>
This is similar to what we have for opening a dump file - one API that
uses the file name as specified, one that creates a temporary file and
provides the file name, and one that uses the standard output.
All of those APIs handle closing the output file.
Change-Id: I56beea7be347402773460b9148ab31a8f8bc51e1
Reviewed-on: https://code.wireshark.org/review/19059
Reviewed-by: Guy Harris <guy@alum.mit.edu>
The old information is copied over from one of the input files; if we
don't have information about the OS on the machine writing the *output*
file, just throw the old information away.
(We need a better way to preserve information from the input files;
perhaps this:
http://www.winpcap.org/pipermail/pcap-ng-format/2016-June/000362.html
might work.)
Change-Id: Ia25771736d267173f2b6949a91e81e217ee7d16f
Reviewed-on: https://code.wireshark.org/review/16730
Reviewed-by: Guy Harris <guy@alum.mit.edu>
To remove OPT_SHB_HARDWARE, use wtap_block_remove_option().
To get the string value from a GString, use g_string_free(string,
FALSE), not g_string_free(string, TRUE) - the latter will free the
string value and return NULL.
Change-Id: I0c5a9f818543f6752f455f04fb3c024208e23954
Reviewed-on: https://code.wireshark.org/review/16567
Reviewed-by: Guy Harris <guy@alum.mit.edu>
A string option, if present, always has a value; it might be a null
*string*, but you won't get a null pointer (if the option isn't present,
it simply isn't present).
Fix some comments while we're at it.
Change-Id: I9c1420f56998a7d04de5c5cc2e92631b181f303a
Reviewed-on: https://code.wireshark.org/review/16564
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Fix a typo - compare for unequal snapshot lengths, not equal snapshot
lengths.
Also, move the debug messages about checks right above the checks.
Change-Id: If6f5e125f05f3788b63e9f75d98f55e27830870b
Reviewed-on: https://code.wireshark.org/review/16470
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Guy Harris