In the first pass of two-pass wireshark, where we can do
asynchronous DNS lookups, make sure to actually take the
requests off the queue and process them, instead of waiting
until the end of the first pass.
Use a mutex to protect taking requests off the queue, just in
case.
Related to #19629.
The capture session sometimes calls the error handler with an
empty error message. This is generally on extcap errors, because
extcap errors aren't reported from dumpcap over the sync pipe
but instead gathered from the extcap stderr only after the session
closes. They're reported a bit later in the closed function.
Avoid console messages like:
** [Capture MESSAGE] -- Error message from child: "", ""
and
tshark:
Add a new hfi reference type for when we're printing items,
that supersedes direct reference - in addition to ensuring that
we don't fake an item, it also defaults the item to visible
(doesn't mark it as hidden when the tree isn't visible), so
that the string representation isn't faked either for fields
that have non-default formats.
Use it when fields are specified with -e; instead of setting
the entire tree as visible, only mark visible the items that
we want to print. This speeds up tshark -e output with all the
-T options that support it, sometimes by 2 to 4 times.
Part of #19573
We've been planning on removing -G with no argument for
18 years (2f7fd680e2); start
warning users that it is deprecated.
Single letter options with optional arguments are tricky and
deprecated, see Guideline 12 of the POSIX Utility Syntax Guidelines.
( https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html )
We have special handling for -G as a result which forces it to the
first argument. -G with no argument can't be mixed with other options,
unlike the other reports. Removing this would allow relaxation of that
restriction.
Related to #17924
The -G protocols, -G heuristic-decodes, and -G decodes reports
all print information about the current configuration. The former
two differentiate between dissectors that are currently enabled
and those that are enabled by default. However, at the point that
we parse the -G option, epan_load_settings() has not been called
so the reports always print out the defaults unless epan_load_settings()
is specifically called.
Fix up 44e4ad3bc5
Related to #16789, #17924.
Add a minimum and maximum API level. Backward-compatible changes to
the API only bump the maximum API level. Backward incompatible
changes bump the maximum API level and the mininum, to the
new (maximum) level.
This may allow codec plugins to continue working without recompilation,
possibly with reduced functionality.
The API level is only defined for codecs because it is a small
and easy to define API, and very stable.
Maybe we could do the same for wiretap (file type) plugins. For the
various epan plugin types it seems pointless and futile. I cannot
see a scenario where a new Wireshark minor release does not increase
the minimum API level.
Remove the major.minor version from the plugin path, i.e:
lib/plugins/X.Y/{epan,wiretap,codecs}
and use an unversioned path:
lib/plugins/{epan,wiretap,codecs}
Introduce a new naming policy for plugins that requires
name.so.ABI_VERSION.
This is a simplified filesystem layoutfor plugins some
important benefits such as:
* improves compatibility between Wireshark versions, because
a plugin that wasn't recompiled will be automatically picked
up, but only if it has a compatible ABI version in the file name.
* does not clash with Apple guidelines
* simpler for users to understand and apply
* just overall simpler and easier to maintain, removes a lot
of complexity from CMake code
It does impose more requirements on the plugin naming scheme
but this should be handled completely transparently
by the build system.
It would also be possible to add support for unversioned *.so file
extensions at the same time, although in ths case it is not possible
to support multiple Wireshark ABI versions with only *.so, of course.
This wasn't done here but it may or may not be a useful enhancement
in the future.
Follow-up to 90b16b4092.
Allow epan plugins to push descriptions for each individual
plugin or extension managed by the epan plugin interface.
For example a Lua or Python epan plugin can push
descriptions for each *.lua or *.py script it registers.
The --color option by itself is not reason to do dissection.
If we're not printing anything (packet summaries, PDML, PSML,
packet details including what coloring rule matched) or applying
a filter, then we don't need to do dissection.
Possibly we should warn or error if the --color option is given
and there's no output or filtering, e.g. when capturing and writing
to file without appropriate options.
For tshark, cache the interface list the first time it is retrieved,
instead of launching dumpcap once for each interface specified in
the capture options, and when getting each interfaces descriptive
name at the start of the capture. If the interface list changes
when in the middle of processing options before the capture starts
we have problems anyway.
On Windows, this means not getting multiple UAC pop-ups if
npcap is installed limited to Administrator privileges.
We can probably do this for the GUI as well, since the command
line options for capture are only parsed at startup.
Fix#16625
get_interface_descriptive_name is slow, because it retrieves
the interface list. As called in tshark, it doesn't do anything,
because even though it changes interface_opts->descr, the
display_name is still set, so get_iface_list_string doesn't use it.
If the display_name were not set, then get_iface_list_string would
call get_interface_descriptive_name anyway. So either way it's
pointless.
We don't want get_iface_list_string to do extra interface list
retrievals by calling get_interface_descriptive_name in general,
but we probably do want to check the user-supplied description
(via prefs) and special user-supplied description of stdin (via -X),
which capture_opts fill_in_interface_opts_from_ifinfo doesn't do.
As it is, tshark makes extra retrievals of the interface list
and then doesn't do anything with the information.
Related to #16625
Instead of always calling capture_interface_list, have
get_interface_descriptive_name take the capture_options (which
we always have when calling it) as a parameter, and use the
get_iface_list() member of that. Generally that is the same
as calling capture_interface list, but if we cache the interface
list (e.g., when running tshark) it could be different.
We also probably should be looking through the already retrieved
interfaces in ifaces or all_ifaces most of the time before
doing another call to dumpcap, since we call this on interfaces
that are already set up. Passing in the capture_options allows
that change to be made later.
Related to #16625
Do not read the capture filter list unless needed.
Do not use a static list because the capture filter list can change during program execution
and we want to be able to read multiple copies whenever.
Improve the public API function names.
pcapng allows simple packet blocks (which don't have timestamps),
enhanced packet blocks (which do) and custom blocks (which might
or might not have timestamps, and even if they do have timestamps,
libwiretap might not know about them), and so some records may have
timestamps while others do not.
Do not use frames without timestamps in delta time calculations.
Don't use them as reference frames for time calculations, or for
the previously displayed frame for time calculations, where the
previously displayed frame that actually has a timestamp is used.
Have the various _get_frame_ts functions return null instead of
their ts value (that is currently handled; if records without
timestamps set their abs_ts to the special "unset" value of nstime_t
that could work too, except that isn't currently handled.)
Still allow the GUI to set frames without timestamps as "Time
References", because that does still affect the "Cumulative Bytes"
column, so it's not entirely pointless; unset the reference time
so that the timestamp from the next frame that does have a timestamp
will be used as reference time.
The "previous captured frame" will show a 0 time delta when
the previous frame doeesn't have a timestamp. Perhaps a user
would also want "previous captured frame with a timestamp,"
but we'd have to store that in frame data (adding memory to
that struct.)
Fix#19397
The tshark man page and help claims to support the --read-file,
--read-filter, and --display-filter long options. Make it so.
Programs have to dissect in order to run filters, so add the
filtering options to dissect_opts.h, which Wireshark, tshark,
and rawshark already include, so that they stay consistent
between applications.
Reading a file is a common activity that we might want to have
more (all?) command line tools support with the same syntax
eventually, so add that to clopts_common.h
rawshark doesn't distinguish between read filters and display
filters; to make things easy and consistent, make them synonymous
for rawshark.
Fix#19365
Way back in e4379f0ea1 we added an option to dumpcap to output the name
of the most recently closed ringbuffer file. Expose this option to
tshark, and make tshark correctly pass it to dumpcap.
Avoid calling tzset() with every localtime_r() and
gmtime_r() invocation. It's inneficient because it
reads the system configuration, even if it doesn't
need to do anything with it.
We assume the timezone will not change during the process
lifetime, or if it does it can be safely ignored, with some
time values in the old timezone perhaps.
Call it during startup for log initialization and some processes
that are known to require it.
If the second argument is NULL, it means "the string must just contain a
number", so, if there's anything following the number, it reports an
error. No need to get the pointer to the next character past the number
and check that ourselves.
On at least some Linux distributions, dumpcap is either installed with
elevated privileges sufficient to support traffic capture by default or
can optionally be given those privileges. If it has those privileges,
it's typically made group-executable but not world-executable and owned
by a special group, e.g. "wireshark", so that only users in that group
can use dumpcap to capture traffic.
The user installing the Wireshark package is *not* necessarily put into
that group by default; this means that any attempt by Wireshark or
TShark to run dumpcap will fail with EACCES.
If the exec call in the child process sends text error mesages, intended
for end users, up the message pipe, as we had been doing, then figuring
out *why* the exec failed would require some heuristic parsing to figure
out whether it's a permissions problem or not.
Instead of doing that, just send a message giving the errno for exec
failing.
For now, we just format an error message for that in the parent process,
but this leaves room to do a better job.
While we're at it, fix some cases where an empty error message could be
printed.
The messages "Capture started" and the temporary capture file
are always printed by default. That seems excessive. TShark
should omit or at least be very reserved with diagnostic/status
messages unless requested.
Add timing measurements for tshark when dissecting
and filtering a capture file.
The output is in JSON. The "elapsed" member is the total
elapsed time for all passes. After that there is
an array with an object for each pass. We only
have two passes at most currently. The single
pass dissection (the default) generates
an array with one element.
Currently there is a counter for total time
for the pass, and time elapsed in dissection,
applying the read filter and applying the
display filter. If any of these is not
active the respective entry contains zero.
All values are in milliseconds.
Add a tshark option to print the timing info.
Example:
$ tshark -r ~/captures/http.pcapng --print-timers -2 -Y 'http && frame.len > 1' > /dev/null
{
"version": "v4.1.1rc0-30-gb7e187fe2993",
"path": "/home/jpv/captures/http.pcapng",
"filter": "http && frame.len > 1",
"time_unit": "millisecond",
"elapsed": 22981243,
"dfilter_expand": 3,
"dfilter_compile": 35,
[
{
"elapsed": 11022013,
"dissect": 10580266,
"display_filter": 24567,
"read_filter": 0
},
{
"elapsed": 11959230,
"dissect": 11454227,
"display_filter": 21052,
"read_filter": 0
}
]
}
Do that rather than having that code directly call
capture_interface_list(), with the programs providing their own versions
of capture_interface_list().
Don't do so only if no interfaces are reported; we now append extcaps to
the list even if, due to an error, no interfaces were found, so report
an error in that cas regardless of whether the interface list is empty
or not.
Add additional exit status values for -D errors (and potentially for
errors elsewhere), use them, and allow "pcap error" in the tests
for -D.
Restore the ability in tshark -e to filter based on column titles,
at least for the seven default columns. Internally alias the fields
based on their titles to the column type based fields, e.g.
"_ws.col.Info" to "_ws.col.info"
This may be removed in the future, but is useful to have as there
are tools and scripts that use the old beahvior (and there are
supports versions, 3.6.x and 4.0.x, that only support the old
behavior.)
Putting it there makes it not obvious why the check is being done.
Instead, do the check before calling timestamp_set_precision() - and do
the same for timestamp_set_type(), as, in both cases, the _NOT_SET value
means that no command-line option was specified to set the timestamp
type or precision, so there's nothing to set them *to*.
--disable-all-protocols will mark all protocols as disabled by default,
and then disable them. Certain protocols can then be enabled one by one
by using --enable-protocol.
--only-protocols is a helper option to make it easier to enable only
certain protocols It's equivalent to passing --disable-all-protocols and
then several --enable-protocol options. It accepts a comma separated
list of protocols. First all protocols will be disabled, and then all
protocols included in the list will be enabled one by one.
Side-note, it wouldn't make much sense to enable only "tcp" for example
without enabling the protocols in the lower layers (e.g: eth, sll, ip,
ipv6). In this case, something like --only-protocols eth,sll,ip,ipv6,tcp
will generally be needed in order to make sure that TCP is decoded.
Signed-off-by: Juanma Sanchez <juasanch@redhat.com>
Make the text of each registered column a FT_STRING field that can be
filtered, prefixed with _ws.col - these work in display filters, filters
in taps, coloring rules, Wireshark read filters, and in the -Y, -R, -e,
and -j options to tshark. Use them as the default "Apply as Filter" value
for the columns that aren't handled by anything else currently.
Because only the columns formats that actually correspond to columns
get filled in (invisible columns work), register and deregister the
fields when the columns change.
Use the lower case version of the rest of the COL_* define for each
column as the field name.
This adds a number of conditions to "when are the columns needed",
including when the main display filter or any filter on a tap is
using one of these fields.
Custom columns are currently not implemented. For custom columns, the
tree then has to be further primed with any fields used by the custom
columns as well. (Perhaps that should happen in epan_dissect_run() -
are there any cases where we construct the columns and don't want to
prime with any field that custom columns contains? Possibly in taps
that we know only use build in columns.)
Thus, for performance reasons, you're better off matching an ordinary
field if possible; it takes extra time to generate the columns and many
of them are numeric types. (Note that you can always convert a non-string
field to a string field if you want regex matching, consult the
*wireshark-filter(4)* man page.) It does save a bit on typing (especially
for a multifield custom column) and remembering the column title might
be easier in some cases.
The columns are set before the color filters, which means that you
can have a color filter that depends on a built-in column like Info or
Protocol.
Remove the special handling for the -e option to tshark. Note that
the behavior is a little different now, because fixed field names
are used instead of the titles (using the titles allowed illegal
filter names, because it wasn't going through the filter engine.)
For default names, this means that they're no longer capitalized,
so "_ws.col.info" instead of "_ws.col.Info" - hopefully a small
price in exchange for the filters working everywhere.
The output format for -T fields remains the same; all that special
handling is removed (except for remembering if someone asked for
a column field to know that columns should be constructed.)
They're also set before the postdissectors, so postdissectors can
have access.
Anything that depends on whether a packet and previous packets are
displayed (COL_DELTA_TIME_DIS or COL_CUMULATIVE_BYTES) doesn't work
the way most people expect, so don't register fields for those.
(The same is already true of color filters that use those, along with
color filters that use the color filter fields.)
Fix#16576. Fix#17971. Fix#4684. Fix#13491. Fix#13941.
It's fine to write to a pipe and dissect, as long as we're not
doing live capture. Move the check for writing to a pipe and dissecting
inside the code path where we know we're dissecting.
Make sure it's after starting the statistics taps; currently we check
if we're tapping and writing to a pipe before starting the statistics
taps, so the check doesn't work, leading to some odd errors because
the tshark reads random bytes from the pipe output.
Make sure if we're printing packet info and need a preamble, that we
write it after checking for all the invalid options. We don't want
to print the preamble and then just quit.
Related to #17900
Move the protocolfilter map to being a member of output_fields,
consolidating some of the API. The only functions that accept
one accept both, and callers that always pass either as NULL
always pass both as NULL.
Implements suggestion in issue #18714.
Proposed syntax for setting subsecond precision is "tshark -t adoy.3" for
millisecond accuracy in output. Using a dot separator indicates the precision
of what follows the dot in the output.
The following tshark -t combinations are supported:
1. Specifying just the format with e.g. "-t a" and defaulting the precision.
2. Specifying both format and precision, with "-t ad.2" or "-t ad -t .2".
3. Specifying only the precision with "-t .6" and defaulting format.
4. Use "-t a." or even "-t ." to specify auto precision from trace.
The latter use case is particularly useful with wireshark/logray.
Using a dot like this avoids introducing a new command line option.
Reduce the default update interval for dumpcap to notify its parent
of new packets (or to check if we've met file duration, etc.) from
500 ms to 100 ms, and put in the capture options.
This makes the GUI appear to update more in real time rather than
in visible batches of packets.
This also reduces the amount of ring buffer space needed in cases
where we're doing dissection, and dissection is able to keep up,
but the files can be deleted before tshark gets to them because of
the notification lag. (See #1650.)
Store the field filter strings in a wmem_map pointing to the
field flags for each string. This allows specifying multiple
filter options (-j or -J) on the command line, including some
of both.
Fix#17470
John Thacker