epan_dissect_run_* and epan_dissect_reset unreference the packet
block that is part of the record, which frees it if the ref count
drops to zero. However, tshark needs the block later to, e.g.,
copy the options. process_cap_file_[single|second]_pass still
unreference and free the block with wtap_rec_reset() at the end
of each packet loop.
Fix#18693
WTAP_ENCAP_UNKNOWN is used for two different cases:
1. Encapsulation type values that are unsupported by libwiretap or
bogus values (and thus "unknown" to libwiretap).
2. An initial state where the encapsulation type is "not yet" known
for a file type like pcapng without a single encapsulation type in the
header, before any packets or interfaces that set the encapsulation type
have been read. (If the file has no packets, this may be the value after
the file is entirely read in.) This can be the value when an output file
is written out simultaneously with reading an input file, rather than
reading the entire input file first, and, e.g., there is a custom block
before any IDBs.
The first case can never be handled when writing out a file, but the
second case can possibly be handled, so long as (for pcapng) IDBs
are available to write when they become necessary, or (for file
types like pcap with a single link-layer type in the header) the
writer waits until a link-layer type is seen to create the output
header. (It is possible, of course, that writing would fail in the
middle if an unsupported encapsulation type appears, or if the
encapsulation becomes per-packet for file types that don't support that,
but that is an unavoidable risk when writing without reading the entire
input file(s).)
Introduce WTAP_ENCAP_NONE for the second case, and use it for pcapng,
where we guarantee that any necessary IDBs will be passed along.
Continue to use WTAP_ENCAP_UNKNOWN for the first case.
Allow pcapng files to open a file for writing with WTAP_ENCAP_NONE.
There are some other file types that support per-packet link-types,
and could also use WTAP_ENCAP_NONE, but they require more work to
generate IDBs. (Note that all of them currently are impossible to
write to pcapng when they have multiple encapsulations, even if
the encapsulations are all supported by pcapng, because they don't
properly generate IDBs.)
Remove the workaround in ef43fd48b4
for tshark writing to pcapng when the source file is WTAP_ENCAP_UNKNOWN,
since now such files will be WTAP_ENCAP_NONE and work properly (and
also work in editcap, mergcap, etc.)
Along with 8cddc32d35, fix#18449.
Put the "attempt to compile a filter string as a capture filter" code
into a common routine, and, if the attempt succeeds, free up the
generated capture filter code.
Fixes#18837.
Use a pointer to the growing array of NRBs from the source
file, as with DSBs, so as to handle reading NRBs in the middle
of a file in one-pass mode.
Write NRBs when reading a file with editcap, or in tshark when
not dissecting packets and writing our own NRB. Continue not
to write the NRB if we're supplying our own list of address info
instead.
If we have already read the entire source file in (such as in
two-pass tshark), move all the NRBs to the beginning of the file
before packets, as done with DSBs.
When merging files with mergecap, write both sets of NRBs. (There
is no attempt to merge the NRBs by looking for common entries.)
Check for name resolution data in the middle of dumping a file,
not just at the end, and check for DSBs at the end of a file,
after all the packets. This means that Wireshark no longer writes
the NRB at the very end of the file after all the packets (which
is worse for future one-pass reads), and DSBs after all packets
are preserved.
Ping #15502
Dependent frames list order does not matter and thus significantly
faster data structure can be used. Replace the list with hash table to
avoid excessive CPU usage when opening files containing reassembled
packets consisting of large number of fragments.
Save all dependent frames when there are multiple levels
of reassembly.
This is a retry of !6329, combined with the fix in !6509 which
were reverted in !6545.
epan: fix a segfault, introduced in !6329
Return an struct containing error information. This simplifies
the interface to more easily provide richer diagnostics in the future.
Add an error code besides a human-readable error string to allow
checking programmatically for errors in a robust manner. Currently
there is only a generic error code, it is expected to increase
in the future.
Move error location information to the struct. Change callers and
implementation to use the new interface.
The "conversation table" mechanism supports two types of tables, one for
the "Conversations" menu item under "Statistics" and one for the
"Endpoints" menu item under "Statistics". The first of them shows
statistics for conversations at various layers of the networking stack;
the second of them shows statistics for endpoints at various layers of
the networking stack.
The latter is *not* a table of hosts; an endpoint might be a host,
identified by an address at some network level (MAC, IP, etc.), or it
might be a port on a host, identified by an address/port pair.
Some data types, function names, etc. use "host" or "hostlist" or other
terms that imply that an endpoint is a host; change them to speak of
endpoints rather than hosts, using names similar to the corresponding
functions for conversations.
Provide wrapper functions and typedefs for backwards source and binary
compatibility; mark them as deprecated in favor of the new names.
Clean up some comment errors found in the process.
Use the timer polling approach on Windows. GLib timer callbacks execute
in main thread. Remove useless mutex as there is no point in protecting
resources if only can thread can access the resources. Simply wait on
capture child handle instead of periodically checking process state.
On Unix systems, register the pipe fd for polling inside GLib mainloop.
Add a function to get the column text of the nth column, taking
into account whether the column is resolved or unresolved. Use
this function in the GUI, as well as in tshark, when writing
PSML, exporting dissection to PSML, etc., instead of accessing
col_data directly.
This removes the direct accesses of col_data from outside
column.c and column-utils.c
Fix#18168.
If a configuration profile is requested on the command line that
does not exist as a personal profile but does exist as a global
profile, copy it to the personal directory and use it, the same
as when selecting a global profile in the GUI.
Add the same feature to tshark and tfshark as well, where it
is particularly useful.
Add a selected frame option that does pretty much what the name
indicates.
This is not meaningful in the CLI but is useful to simulate the
selected frame action in the GUI for unit testing purposes. The
option is not documented for that reason.
A selected frame is used in display filters with field references.
$ tshark -r ../test/captures/dhcp.pcap
1 0.000000 0.0.0.0 → 255.255.255.255 DHCP 314 DHCP Discover - Transaction ID 0x3d1d
2 0.000295 192.168.0.1 → 192.168.0.10 DHCP 342 DHCP Offer - Transaction ID 0x3d1d
3 0.070031 0.0.0.0 → 255.255.255.255 DHCP 314 DHCP Request - Transaction ID 0x3d1e
4 0.070345 192.168.0.1 → 192.168.0.10 DHCP 342 DHCP ACK - Transaction ID 0x3d1e
$ tshark --selected-frame=2 -2 -Y 'frame.number <= ${frame.number}' -r ../test/captures/dhcp.pcap
1 0.000000 0.0.0.0 → 255.255.255.255 DHCP 314 DHCP Discover - Transaction ID 0x3d1d
2 0.000295 192.168.0.1 → 192.168.0.10 DHCP 342 DHCP Offer - Transaction ID 0x3d1d
If we don't have an offset, don't print anything with underline.
Also it can underline filters using macros correctly now.
$ tshark -Y 'ip and ${private_ipv4:ip.sr}' -r /dev/null
tshark: Left side of "==" expression must be a field or function, not "ip.sr".
ip and ip.sr == 192.168.0.0/16 or ip.sr == 172.16.0.0/12 or ip.sr == 10.0.0.0/8
^~~~~
Rename init_progfile_dir to configuration_init. Add an argument which
specifies our configuration namespace, which can be "Wireshark"
(default) or "Logwolf".
This allows the "needs to be reloaded" indication to be set in the close
process, as is the case for ERF; having a routine that returns the value
of that indication is not useful if it gets seet in the close process,
as the handle for the wtap_dumper is no longer valid after
wtap_dump_close() finishes.
We also get rid of wtap_dump_get_needs_reload(), as callers should get
that information via the added argument to wtap_dump_close().
Fixes#17989.
Add a separate menu for Strip Headers (similar to Export PDU, but exporting
to an encapsulation other than WIRESHARK_UPPER_PDU everything for
that encapsulation). Add to the usage output of tshark for the "-U"
option which encapsulation a export tap will produce.
Add docs/diagnostic-options.adoc, which is a snippet that documents our
various --log-* options. Include it in the dumpcap, rawshark, and tshark
man pages.
Make the ws_log_print_usage output more consistent.
This is a first pass that covers the WSDG, WSUG, man page, a code
comment and a README. Plenty left to do in the Debian files, a few
Lua examples and other misc files.
Besides the obvious limitation of being unavailable on Windows,
the standard is vague about getopt() and getopt_long() has many
non-portable pitfalls and buggy implementations, that increase
the maintainance cost a lot. Also the GNU libc code currently
in the tree is not suited for embedding and is unmaintainable.
Own maintainership for getopt_long() and use the musl implementation
everywhere. This way we don't need to worry if optreset is available,
or if the $OPERATING_SYSTEM version behaves in subtly different ways.
The API is under the Wireshark namespace to avoid conflicts with
system headers.
Side-note, the Mingw-w64 9.0 getopt_long() implementation is buggy
with opterr and known to crash. In my experience it's a headache to
use the embedded getopt implementation if the system provides one.
Wireshark/tshark may be built without Lua support. This patch adds an
error message if the user specifies the `-X lua_script` command-line
argument to a program built without Lua support, so the user is not left
wondering why their script isn't working.
Bug 17478 was caused by `wtap_rec.block` being allocated for each
packet, but not freed when it was done being used -- typically at the
end of a loop.
Rather than requiring each caller of `wtap_read()` to know to free a
member of `rec`, I added a new function `wtap_rec_reset()` for a
slightly cleaner API. Added calls to it everywhere that seemed to make
sense.
Fixes#17478
Move those checks out of #ifdef HAVE_LIBPCAP/#endif, as that option is
supported even if we don't build with pcap - it's also used when reading
one file and writing another.
Don't check for pcapng when deciding whether, when reading from an
existing capture file, we can write it with added file comments; check
whether the specified file type supports file comments and, if it
doesn't, report all file formats that do as part of the error.
Don't store the comments in a capture_options structure, because that's
available only if we're being built with capture support, and
--capture-comment can be used in TShark when reading a capture file and
writing another capture file, with no live capture taking place.
This means we don't handle that option in capture_opts_add_opt(); handle
it in the programs that support it.
Support writing multiple comments in dumpcap when capturing.
These changes also fix builds without pcap, and makes --capture-comment
work in Wireshark when a capture is started from the command line with
-k.
Update the help messages to indicate that --capture-comment adds a
capture comment, it doesn't change any comment (much less "the" comment,
as there isn't necessarily a single comment).
Update the man pages:
- not to presume that only pcapng files support file comments (even if
that's true now, it might not be true in the future);
- to note that multiple instances of --capture-comment are supported,
and that multiple comments will be written, whether capturing or reading
one file and writing another;
- clarify that Wireshark doesn't *discard* SHB comments other than the
first one, even though it only displays the first one;
Allows adding one or more capture comments to a new pcapng file when
tshark is reading from a file. Currently, tshark only allows setting one
capture comment, and that only when doing a live capture.
The use case for this feature is given in bug #15005.
I decided to allow multiple capture comments to match the same ability
in `editcap`.
To allow this change, I changed the function signature of
`process_cap_file()` so it takes a `capture_options` struct instead of
individual parameters that affect the capture.
Mostly functioning proof of concept for #14329. This work is intended to
allow Wireshark to support multiple packet comments per packet.
Uses and expands upon the `wtap_block` API in `wiretap/wtap_opttypes.h`.
It attaches a `wtap_block` structure to `wtap_rec` in place of its
current `opt_comment` and `packet_verdict` members to hold OPT_COMMENT
and OPT_PKT_VERDICT option values.
Version info is an aspect of UI implementation so move it to
a more appropriate place, such as ui/. This also helps declutter
the top-level.
A static library is appropriate to encapsulate the dependencies
as private and it is better supported by CMake than object libraries.
Also version_info.h should not be installed as a public header.
Instead of receiving the program name from GLib, pass it explicitly
to ws_log_init() instead and use that to initialize the GLib program
name.
ws_log_parse_args() will now exit the program when it encounters an
argument error if exit_failure >= 0.
A domain filter can be given in the environment variable
'WS_LOG_DOMAINS' or in a command-line options "--log-domains".
The filter is specified as a comma separated case insensitive list,
for example:
./tshark --log-domains=main,capture
Domain data type switches from an enum to a string. There is no
constaint on adding new domains, neither in code or at runtime.
The string format is arbitrary, only positive matches will produce
output.
Experience has shown that:
1. The current logging methods are not very reliable or practical.
A logging bitmask makes little sense as the user-facing interface (who
would want debug but not crtical messages for example?); it's
computer-friendly and user-unfriendly. More importantly the console
log level preference is initialized too late in the startup process
to be used for the logging subsystem and that fact raises a number
of annoying and hard-to-fix usability issues.
2. Coding around G_MESSAGES_DEBUG to comply with our log level mask
and not clobber the user's settings or not create unexpected log misses
is unworkable and generally follows the principle of most surprise.
The fact that G_MESSAGES_DEBUG="all" can leak to other programs using
GLib is also annoying.
3. The non-structured GLib logging API is very opinionated and lacks
configurability beyond replacing the log handler.
4. Windows GUI has some special code to attach to a console,
but it would be nice to abstract away the rest under a single
interface.
5. Using this logger seems to be noticeably faster.
Deprecate the console log level preference and extend our API to
implement a log handler in wsutil/wslog.h to provide easy-to-use,
flexible and dependable logging during all execution phases.
Log levels have a hierarchy, from most verbose to least verbose
(debug to error). When a given level is set everything above that
is also enabled.
The log level can be set with an environment variable or a command
line option (parsed as soon as possible but still later than the
environment). The default log level is "message".
Dissector logging is not included because it is not clear what log
domain they should use. An explosion to thousands of domains is
not desirable and putting everything in a single domain is probably
too coarse and noisy. For now I think it makes sense to let them do
their own thing using g_log_default_handler() and continue using the
G_MESSAGES_DEBUG mechanism with specific domains for each individual
dissector.
In the future a mechanism may be added to selectively enable these
domains at runtime while trying to avoid the problems introduced
by G_MESSAGES_DEBUG.
Add a generic function to write content to file. Use this on write
TLS session keys from UI and tshark, and for export objects.
Remove the now unused export_object_ui.[ch].
The tshark help and documentation has been incorrect for at least
eight years, claiming that by default all name resolutions are
performed. Fixes#11762
Replace most instances of ws_debug_printf() except in
epan/dissectors and dissector plugins.
Some replacements use printf(), some use ws_debug(), and
some were removed because they were dead or judged to be
temporary.
If the variable `dfilter' always points to malloc-ed memory, it should
be easier to avoid any leaks.
Leak:
```
Direct leak of 46 byte(s) in 1 object(s) allocated from:
#0 0x7fadf5a67bc8 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
#1 0x7fadd7ecbe98 in g_malloc (/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x57e98)
#2 0x5556272dbfd5 in main /home/ivan/svnrepos/wireshark/tshark.c:1594
#3 0x7fadd71ed0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
```
Initialize the exit status before the loop, and just break out of the
loop if something fails, so that the code following the loop can destroy
the console in Wireshark on Windows and then go to the clean exit code.
In dumpcap, if we're being run by TShark or Wireshark, if there are no
link-layer types, just provide an empty list to our caller; let them
construct an empty list of link-layer types when they read our output.
In the code that reads that list, don't report an error if the list is
empty, rely on the caller to do so.
Have capture_opts_print_if_capabilities() do more work, moving some
functions from its callers to it.
Replace the Wireshark code for that with code that matches what TShark
does.
Update a comment in TShark while we're at it.
Fixes#14215.
(Still leaves it popping up the full window, but that's a bigger
change.)
It's not a generic capture option also supported by TShark and dumpcap,
it's Wireshark-specific (dumpcap *always* starts a capture, and TShark
starts one iff it's passed one or more interfaces on which to capture;
only Wireshark needs it to start the capture immediately - that's a
relic of the days when Wireshark *itself* did what dumpcap now does for
Wireshark).
Handle it in commandline_other_options(), rather than in
capture_opts_add_opt().
That lets us get rid of an argument to capture_opts_add_opt(), and dummy
variables in TShark and dumpcap used to work with that extra argument.
"Commonly-used" meaning "used by more than one source file".
Clean up the exit codes, combining some duplicates with different names,
and using some instead of raw numbers in some places.
The distinction between the different kinds of capture utility
may not warrant a special subfolfer for each, and sometimes the
distinction is not be clear or some functions could stradle
multiple "categories" (like capture_ifinfo.[ch]).
Simplify by having only a generic 'capture' subfolder. The
separate CMake libraries are kept as a way to reuse object code
efficiently.
Have routines to report capture-file errors, using libwireshark error
codes and strings, that call through a pointer, so they can pop up
dialogs in GUI apps, print a message to the standard error on
command-line apps, and possibly do something different on server
programs.
Have init_report_message() take a pointer to structure containing those
function pointers, rather than the function pointers themselves, as
arguments.
Make other API changes to make that work.
The export PDU API now allows writing to a different file type. tshark
already has a -F flag for the output file type. If that option is given,
respect it for export PDU. Also, rec.rec_header.packet_header.pkt_encap
expects WTAP encapsulation types, not PCAP encapsulation types, so don't
call wtap_wtap_encap_to_pcap_encap(), or else it won't actually write to
pcap files, only pcapng (using the wrong sort of encap numbers eventually
leads to WTAP_ENCAP_PER_PACKET, which we don't write to non-pcapng.)
Allow "-U ?" as well as an empty argument; an empty argument is a bit
counterintuitive.
Simplify the introductory line of output - asking for a list of taps
isn't an error in which the user failed to supply a tap name, it's a
case where the user suplied a request for a list of tap names.
Just use fprintf() to print the list, and indent the elements of the
list, as we do with other lists of valid arguments.
List the valid arguments if the user specified an invalid argument as
well.
On Windows, a write to a pipe where the read side has been closed
apparently may return the Windows error ERROR_BROKEN_PIPE, which the
Visual Studio C library maps to EPIPE, or may return the Windows error
ERROR_NO_DATA, which the Visual Studio C library maps to EINVAL.
So, on Windows, for errors other than the ones for which we're reporting
a special error message, check for EINVAL with a *Windows* error of
ERROR_NO_DATA and, if that's what we have, don't print an error message;
otherwise, print an error message that reports a message based on the
Windows error (rather than a relatively uninformative "Invalid argument"
error).
This should fix issue #16192.
Clean up indentation while we're at it.
Have dumpcap in child mode return an error message with a primary and
secondary string, instead of using stderr. When writing to the console
log we ignore the second message to prevent flooding the log with
tutorial-like info on permissions.
Register the pcap and pcapng file types/subtypes rather than hardwiring
them into the table.
Call the registration routines for them directly, rather than through a
generated table; they're always supposed to be there, as some code in
Wireshark either writes only one of those formats or defaults to writing
one of those formats. Don't run their source code through the
registration-routine-finder script.
Have the file type/subtype codes for them be directly exported to the
libwiretap core, and provide routines to return each of them, to be used
by the aforementioned code.
When reporting errors with cfile_write_failure_message(), use
wtap_dump_file_type_subtype() to get the file type/subtype value for the
wtap_dumper to which we're writing, rather than hardcoding it.
Have the "export PDU" code capable of supporting arbitrary file
types/subtypes, although we currently only use pcapng.
Get rid of declarations of now-static can_write_encap and
dump_open routines in various headers.
Instead of a "supports name resolution" Boolean and bitflags for types of
comments supported, provide a list of block types that the file
type/subtype supports, with each block type having a list of options
supported. Indicate whether "supported" means "one instance" or
"multiple instances".
"Supports" doesn't just mean "can be written", it also means "could be
read".
Rename WTAP_BLOCK_IF_DESCRIPTION to WTAP_BLOCK_IF_ID_AND_INFO, to
indicate that it provides, in addition to information about the
interface, an ID (implicitly, in pcapng files, by its ordinal number)
that is associated with every packet in the file. Emphasize that in
comments - just because your capture file format can list the interfaces
on which a capture was done, that doesn't mean it supports this; it
doesn't do so if the file doesn't indicate, for every packet, on which
of those interfaces it was captured (I'm looking at *you*, Microsoft
Network Monitor...).
Use APIs to query that information to do what the "does this file
type/subtype support name resolution information", "does this file
type/subtype support all of these comment types", and "does this file
type/subtype support - and require - interface IDs" APIs did.
Provide backwards compatibility for Lua.
This allows us to eliminate the WTAP_FILE_TYPE_SUBTYPE_ values for IBM's
iptrace; do so.
Eliminate WTAP_FILE_TYPE_SUBTYPE_ERF and
WTAP_FILE_TYPE_SUBTYPE_SYSTEMD_JOURNAL - instead, fetch the values by
name, using wtap_name_to_file_type_subtype().
This requires that wtap_init() be called before epan_init(); that's
currently the case, but put in comments to indicate why it must continue
to be the case.
Provide a wiretap routine to get an array of all savable file
type/subtypes, sorted with pcap and pcapng at the top, followed by the
other types, sorted either by the name or the description.
Use that routine to list options for the -F flag for various commands
Rename wtap_get_savable_file_types_subtypes() to
wtap_get_savable_file_types_subtypes_for_file(), to indicate that it
provides an array of all file type/subtypes in which a given file can be
saved. Have it sort all types, other than the default type/subtype and,
if there is one, the "other" type (both of which are put at the top), by
the name or the description.
Don't allow wtap_register_file_type_subtypes() to override any existing
registrations; have them always register a new type. In that routine,
if there are any emply slots in the table, due to an entry being
unregistered, use it rather than allocating a new slot.
Don't allow unregistration of built-in types.
Rename the "dump open table" to the "file type/subtype table", as it has
entries for all types/subtypes, even if we can't write them.
Initialize that table in a routine that pre-allocates the GArray before
filling it with built-in types/subtypes, so it doesn't keep getting
reallocated.
Get rid of wtap_num_file_types_subtypes - it's just a copy of the size
of the GArray.
Don't have wtap_file_type_subtype_description() crash if handed an
file type/subtype that isn't a valid array index - just return NULL, as
we do with wtap_file_type_subtype_name().
In wtap_name_to_file_type_subtype(), don't use WTAP_FILE_TYPE_SUBTYPE_
names for the backwards-compatibility names - map those names to the
current names, and then look them up. This reduces the number of
uses of hardwired WTAP_FILE_TYPE_SUBTYPE_ values.
Clean up the type of wtap_module_count - it has no need to be a gulong.
Have built-in wiretap file handlers register names to be used for their
file type/subtypes, rather than building the table in init.lua.
Add a new Lua C function get_wtap_filetypes() to construct the
wtap_filetypes table, based on the registered names, and use it in
init.lua.
Add a #define WSLUA_INTERNAL_FUNCTION to register functions intended
only for internal use in init.lua, so they can be made available from
Lua without being documented.
Get rid of WTAP_NUM_FILE_TYPES_SUBTYPES - most code has no need to use
it, as it can just request arrays of types, and the space of
type/subtype codes can be sparse due to registration in any case, so
code has to be careful using it.
wtap_get_num_file_types_subtypes() is no longer used, so remove it. It
returns the number of elements in the file type/subtype array, which is
not necessarily the name of known file type/subtypes, as there may have
been some deregistered types, and those types do *not* get removed from
the array, they just get cleared so that they're available for future
allocation (we don't want the indices of any registered types to changes
if another type is deregistered, as those indicates are the type/subtype
values, so we can't shrink the array).
Clean up white space and remove some comments that shouldn't have been
added.
The "short name" is really just the name, used to look it up. The
"name" is really a description intended solely for human consumption.
Rename the fields, and the functions that access them, to match.
The "description" maintained by Lua for file type handlers is used
*only* for one debugging message; we should probably just eliminate it.
Call it an "internal description" for now.
In our first pass through our options, look for ones that might require
extcap. Call extcap_register_preferences() only when that's the case.
Warn about missing extcap preferences only when we've loaded them.
Pull the code to register plugin taps, and the loop to register built-in
taps, into a single register_all_tap_listeners() routine.
This leaves it up to libwireshark, not to the programs using it, to know
how to register them.
extcap_register_preferences is only called with the -G option
(to dump information) and extcap preferences are not loading,
loading it unconditionally avoids this, as it is done in the
GUI startup.
When there is no do_dissection cf is missing some variables
for cf_close() call. Therefore we have to set them explicitly.
Fixes: wireshark/wireshark#17021
João Valverde