We've been planning on removing -G with no argument for
18 years (2f7fd680e2); start
warning users that it is deprecated.
Single letter options with optional arguments are tricky and
deprecated, see Guideline 12 of the POSIX Utility Syntax Guidelines.
( https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html )
We have special handling for -G as a result which forces it to the
first argument. -G with no argument can't be mixed with other options,
unlike the other reports. Removing this would allow relaxation of that
restriction.
Related to #17924
If the quote character appears in a field value, then escape
it by printing the character twice. When escaping whitespace
with the backslash character, also escape the backslash
character itself.
Add a ws_escape_csv function to wsutil and use it for tshark.
Adopt the existing static escape_string_len function so that
ws_escape_csv can use it while maintaining the same output
for the other ws_escape_ functions.
Fix#10284
Document the help and version option handling, including long option
form, the same for all the command line tools, both in the their
help output and in any manpages. Add version option to randpkt.
Fix#15483
Way back in e4379f0ea1 we added an option to dumpcap to output the name
of the most recently closed ringbuffer file. Expose this option to
tshark, and make tshark correctly pass it to dumpcap.
Add "utf-8" as an output mode to the follow tap for tshark.
This produces the same output that the Qt version does (passing
through all valid UTF-8, including control codes and internal
NULs, substituting illegal UTF-8 sequences with REPLACEMENT CHARACTER,
and not handling UTF-8 sequences split between unreassembled frames),
except for some differences which are common to how the tshark
and Wireshark Follow output differs for ASCII and EBCDIC as well:
Tshark includes additional header information and a line length
before each section of output, and leaves end-of-line terminators
untouched; Wireshark, due to the use of Qt code, automatically
translates end-of-line terminators to a LF (including from, e.g.
HTTP), except in "raw" mode. Neither tshark nor Wireshark write
in text mode, i.e. translate end of line terminators to a platform
specific CRLF on Windows.
Related to #19280
Add a new stats tree window under IPv4 Statistics and
IPv6 Statistics that shows TTLs grouped by source address
(and destination address grouped under TTL/Hop Limit.)
Fix#19321
The messages "Capture started" and the temporary capture file
are always printed by default. That seems excessive. TShark
should omit or at least be very reserved with diagnostic/status
messages unless requested.
Add timing measurements for tshark when dissecting
and filtering a capture file.
The output is in JSON. The "elapsed" member is the total
elapsed time for all passes. After that there is
an array with an object for each pass. We only
have two passes at most currently. The single
pass dissection (the default) generates
an array with one element.
Currently there is a counter for total time
for the pass, and time elapsed in dissection,
applying the read filter and applying the
display filter. If any of these is not
active the respective entry contains zero.
All values are in milliseconds.
Add a tshark option to print the timing info.
Example:
$ tshark -r ~/captures/http.pcapng --print-timers -2 -Y 'http && frame.len > 1' > /dev/null
{
"version": "v4.1.1rc0-30-gb7e187fe2993",
"path": "/home/jpv/captures/http.pcapng",
"filter": "http && frame.len > 1",
"time_unit": "millisecond",
"elapsed": 22981243,
"dfilter_expand": 3,
"dfilter_compile": 35,
[
{
"elapsed": 11022013,
"dissect": 10580266,
"display_filter": 24567,
"read_filter": 0
},
{
"elapsed": 11959230,
"dissect": 11454227,
"display_filter": 21052,
"read_filter": 0
}
]
}
This commit adds new fields to the output of both `-G protocols` and
`-G heuristic-decodes` in `tshark`.
For `-G protocols`, three new fields (4, 5 and 6) have been appened to
the existing ones:
- Field 1: protocol name
- Field 2: protocol short name
- Field 3: protocol filter name
- Field 4 (NEW): protocol enabled (e.g. "T" or "F")
- Field 5 (NEW): protocol enabled by default (e.g. "T" or "F")
- Field 6 (NEW): protocol can toggle (e.g. "T" or "F")
For `-G heuristic-decodes`, similarly three new fields (4, 5 and 6)
have been appended to the existing ones:
- Field 1: underlying dissector (e.g. "tcp")
- Field 2: name of heuristic decoder (e.g. "ucp")
- Field 3: heuristic enabled (e.g. "T" or "F")
- Field 4 (NEW): heuristic enabled by default (e.g. "T" or "F")
- Field 5 (NEW): heuristic short name (e.g. "ucp_tcp")
- Field 6 (NEW): heuristic display name (e.g. "UCP over TCP")
The new fields added to `-G heuristic-decodes` are useful as the short
name argument required for `--enable-heuristic` was not previously
shown in the `-G heuristic-decodes` output.
Make the text of each registered column a FT_STRING field that can be
filtered, prefixed with _ws.col - these work in display filters, filters
in taps, coloring rules, Wireshark read filters, and in the -Y, -R, -e,
and -j options to tshark. Use them as the default "Apply as Filter" value
for the columns that aren't handled by anything else currently.
Because only the columns formats that actually correspond to columns
get filled in (invisible columns work), register and deregister the
fields when the columns change.
Use the lower case version of the rest of the COL_* define for each
column as the field name.
This adds a number of conditions to "when are the columns needed",
including when the main display filter or any filter on a tap is
using one of these fields.
Custom columns are currently not implemented. For custom columns, the
tree then has to be further primed with any fields used by the custom
columns as well. (Perhaps that should happen in epan_dissect_run() -
are there any cases where we construct the columns and don't want to
prime with any field that custom columns contains? Possibly in taps
that we know only use build in columns.)
Thus, for performance reasons, you're better off matching an ordinary
field if possible; it takes extra time to generate the columns and many
of them are numeric types. (Note that you can always convert a non-string
field to a string field if you want regex matching, consult the
*wireshark-filter(4)* man page.) It does save a bit on typing (especially
for a multifield custom column) and remembering the column title might
be easier in some cases.
The columns are set before the color filters, which means that you
can have a color filter that depends on a built-in column like Info or
Protocol.
Remove the special handling for the -e option to tshark. Note that
the behavior is a little different now, because fixed field names
are used instead of the titles (using the titles allowed illegal
filter names, because it wasn't going through the filter engine.)
For default names, this means that they're no longer capitalized,
so "_ws.col.info" instead of "_ws.col.Info" - hopefully a small
price in exchange for the filters working everywhere.
The output format for -T fields remains the same; all that special
handling is removed (except for remembering if someone asked for
a column field to know that columns should be constructed.)
They're also set before the postdissectors, so postdissectors can
have access.
Anything that depends on whether a packet and previous packets are
displayed (COL_DELTA_TIME_DIS or COL_CUMULATIVE_BYTES) doesn't work
the way most people expect, so don't register fields for those.
(The same is already true of color filters that use those, along with
color filters that use the color filter fields.)
Fix#16576. Fix#17971. Fix#4684. Fix#13491. Fix#13941.
Allow the escaping of whitespace characters and backspace with
the -T fields options to be disabled. There may be some use
cases (particular for redirected output instead of viewing at
a terminal) for not escaping, particularly since escaping makes
it difficult to distinguish a literal "\n" from an escaped newline.
Document this option, which also documents the escaping behavior.
Also add vertical tab to the list of escaped characters, for the
same reason as the others.
Fix#15796
Add docs/dissection-options.adoc as a snippet similar to
diagnostic-options.adoc to try to keep the man pages consistent
between dissecting programs and provide some logical separation
to avoid overwhelming a user with the huge list of options.
Use it for tshark and wireshark.
Continue to have more Decode As examples on the tshark page,
but have (in the HTML version) the cross-reference from the
wireshark page to the tshark Decode As examples link to an
anchor to the examples.
Make the name resolution option description accurate.
Reduce the default update interval for dumpcap to notify its parent
of new packets (or to check if we've met file duration, etc.) from
500 ms to 100 ms, and put in the capture options.
This makes the GUI appear to update more in real time rather than
in visible batches of packets.
This also reduces the amount of ring buffer space needed in cases
where we're doing dissection, and dissection is able to keep up,
but the files can be deleted before tshark gets to them because of
the notification lag. (See #1650.)
Update the example typical location for the temporary directory
on Windows in the manpages to something newer than where Windows NT
or Windows 98 might put it.
Fix#18463
Consistently speak of "UNIX-compatible systems" when comparing UN*Xes
and Windows, and, the first time we mention "UNIX-compatible systems" in
a section or a list item, enumerate the not-dead-or-moribund ones.
(HP-UX is deemed moribund given that Itanium processors are no longer
being manufactured and HPE are apparently not porting HP-UX to x86-64,
choosing instead to run HP-UX Itanium applications in a compatibility
environment under Linux on x86-64.)
For the -D option, don't bother mentioning ifconfig -a or ip link show,
as there's no reason not to use -D if you want to know what you can
caputre on - for one thing, -D may list devices *other* than the network
interfaces listed by ifconfig -a or ip link show. In addition, don't
speak of code testing whether the interface can be opened, as recent
versions of libpcap don't check that, and neither do any of the programs
in the Wireshark release. (This was done so that, if there's an
itnerface that shows up in the enumeration but that can't be opened,
it'll be offered to the user, and they'll get a message if they try to
capture on it, indicating either that they need to somehow get the
necessary permissions or should report a bug.)
For the -i option, don't mention ifconfig -a or ip link show, as the
user should, again, use -D.
Give more detail when describing files and directories under the global
or personal preferences directory, calling out macOS specially for the
global preferences directory, as it's in the app bundle, and taking into
account that Wireshark might be installed under /usr rather than
/usr/local (for example, if it's installed from a package that's part of
a Linux distribution).
Replace the "Overrides XXX' description of some environment variables
with a more verbose description similar to what's used for other
environment variables.
Set our ld library path and our data, extcap, and plugin directories.
Document WIRESHARK_EXTCAP_DIR and WIRESHARK_PLUGIN_DIR. Note that we
might want to set our various directories relative to the program path.
The beginning of the tshark manual talks about read filters and
using the -R option. Switch all that to display filters and -Y,
since that's the typical use now, with -R limited to two-pass
analysis.
Add docs/diagnostic-options.adoc, which is a snippet that documents our
various --log-* options. Include it in the dumpcap, rawshark, and tshark
man pages.
Make the ws_log_print_usage output more consistent.
Repeated words were found with:
egrep "(\b[a-zA-Z]+) +\1\b" . -Ir
and then manually reviewed.
Non-displayed strings (e.g., in comments)
were also corrected, to ease future review.
John Thacker