The output of the "values" tshark glossary has over 1.3M lines. Writing
this to stdout with some test failures is problematic in a number of ways.
Also it's not helpful because stderr is written after stdout (not interleaved)
so there is no output context to the error message. The error/warning
message (from stderr, that triggered the test failure) needs to be
sufficient to provide a good understaning of the test failure.
The output is trimmed to first+last N lines. Some lines are kept as
informational and because it may be useful if the program aborts.
Fixes#17203.
Add partial support for decrypting captures with connections
established using FT-EAP. To support deriving keys for FT-EAP
the MSK is needed. This change adds MSK as a valid IEEE 802.11
protocol input key type preference as well.
Note that FT-EAP support comes with the following imitations:
- Keys can only be derived from the FT 4-way handshake messages.
- Roaming is not supported.
Add partial support for decrypting captures with connections
established using FT BSS Transition (IEEE 802.11r).
FT BSS Transition decryption comes with the following limitations:
- Only FT-PSK is supported.
- Keys can only be derived from the FT 4-way handshake messages.
- Roaming is not supported.
Some .proto files contain complex syntax that does not be described in protobuf official site
(https://developers.google.com/protocol-buffers/docs/reference/proto3-spec).
1. Update 'epan/protobuf_lang_parser.lemon' to:
1) Support complex option names format (EBNF):
optionName = ( ident | "(" fullIdent ")" ) { "." ( ident | "(" fullIdent ")" ) }
for example, "option (complex_opt2).(grault) = 654;".
2) Make enum body support 'reserved' section (EBNF):
enumBody = "{" { reserved | option | enumField | emptyStatement } "}"
3) Allow the value of field or enumValue option to be "{ ... }" other than constant:
enumValueOption = optionName "=" ( constant | customOptionValue ) ";"
fieldOption = optionName "=" ( constant | customOptionValue ) ";"
4) Allow 'group' section missing 'label' (for example, in 'oneof' section).
5) Make 'oneof' section support 'option' and 'group' sections (BNF):
oneof = "oneof" oneofName "{" { oneofField | option | group | emptyStatement } "}"
6) Ignore unused 'extend' section.
7) Fix the bug of one string being splitted into multi-lines.
2. Update 'epan/protobuf_lang_tree.c' to:
8) Fix the bug of parsing repeated option.
3. Update 'test/suite_dissection.py' to add test case for parsing complex syntax .proto files:
test/protobuf_lang_files/complex_proto_files/unittest_custom_options.proto
test/protobuf_lang_files/complex_proto_files/complex_syntax.proto
and dependency files:
test/protobuf_lang_files/well_know_types/google/protobuf/any.proto
test/protobuf_lang_files/well_know_types/google/protobuf/descriptor.proto
Refer to issue #17046
My initial fix caused several double-offset errors in TvbRange_raw()
because I was adjusting for the TvbRange's offset too early in the
process. The proper fix is to only adjust for it in the final call to
get the data.
I also simplified some of the bounds checks to be based on the values in
the TvbRange instead of calling `tvb_captured_length()` and the like,
because its bounds are already checked against the backing Tvb when it's
first taken.
Massively expanded the lua test suite to account for every combination
of passing offsets and lengths to a Tvb or TvbRange and to the
subsequent `:raw()` call.
Add case_dissect_protobuf and case_dissect_grpc in test/suite_dissection.py.
Add *.proto into the sub directories of test/protobuf_lang_files/.
Run command like 'pytest --program-path .\run\Debug\ -k "grpc or protobuf"'
in build directory (in windows) to test these cases only.
Set the FI_BITS_OFFSET and FI_BITS_SIZE flags appropriately on [u]int[64]
(and thus chars and booleans) where the bitmask is passed in on the
header_field_info. Also set the flags on bitmask items by ORing the bitmasks
from the constituent fields. These flags are only used right now in the
packet diagram.
This makes the packet diagram display those types of fields correctly without
having to use proto_item_set_bits_offset_len(), so long as the bitmask is
correct and the field width of the type matches the octet length. (If it
doesn't match, that's a dissector bug.)
split bit items are a more complicated case and still not handled correctly.
PEP 394[1] says,
"In cases where the script is expected to be executed outside virtual
environments, developers will need to be aware of the following
discrepancies across platforms and installation methods:
* Older Linux distributions will provide a python command that refers
to Python 2, and will likely not provide a python2 command.
* Some newer Linux distributions will provide a python command that
refers to Python 3.
* Some Linux distributions will not provide a python command at all by
default, but will provide a python3 command by default."
Debian has forced the issue by choosing the third option[2]:
"NOTE: Debian testing (bullseye) has removed the "python" package and
the '/usr/bin/python' symlink due to the deprecation of Python 2."
Switch our shebang from "#!/usr/bin/env python" to "#!/usr/bin/env
python3" in some places. Remove some 2/3 version checks if we know we're
running under Python 3. Remove the "coding: utf-8" in a bunch of places
since that's the default in Python 3.
[1]https://www.python.org/dev/peps/pep-0394/#for-python-script-publishers
[2]https://wiki.debian.org/Python
For WPA security association (SA) entries are created on sucessful
PTK derivation from 4-way handshake frames. WEP though don't use
4-way handshake frames for key derivation and therefore no SA entry
is created. Still WEP decryption implementaton expects to find
an SA otherwise the decryption is skipped.
Fix broken WEP decryption by removing the check for an existing SA
entry and instead form the SA on first successful decryption.
Add also a test for WEP decryption.
Fixes: v3.3.0rc0-1263-g099d241046 ("dot11decrypt: Avoid allocating SA on packet decryption")
As noted in bug #16386, glib's `g_base64_decode_inplace()` aborts
decoding of base64 strings that aren't padded. This addresses that by
adding padding "=" characters if needed to the buffer which will be
decoded.
I added the test case from the bug report to the test suite, though the
location therein may not be ideal.
Closes#16386
Add ui/urls.h to define some URLs on various of our websites. Use the
GitLab URL for the wiki. Add a macro to generate wiki URLs.
Update wiki URLs in comments etc.
Use the #defined URL for the docs page in
WelcomePage::on_helpLabel_clicked; that removes the last user of
topic_online_url(), so get rid of it and swallow it up into
topic_action_url().
Fix some issues discovered by common python linters including:
* switch `None` comparisons to use `is` rather than `==`. Identity !=
equality, and I've spent 40+ hours before tracking down a subtle bug
caused by exactly this issue. Note that this may introduce a problem if
one of the scripts is depending on this behavior, in which case the
comparison should be changed to `True`/`False` rather than `None`.
* Use `except Exception:` as bare `except:` statements have been
discouraged for years. Ideally for some of these we'd examine if there
were specific exceptions that should be caught, but for now I simply
caught all. Again, this could introduce very subtle behavioral changes
under Python 2, but IIUC, that was all fixed in Python 3, so safe to
move to `except Exception:`.
* Use more idiomatic `if not x in y`--> `if x not in y`
* Use more idiomatic 2 blank lines. I only did this at the beginning,
until I realized how overwhelming this was going to be to apply, then I
stopped.
* Add a TODO where an undefined function name is called, so will fail
whenever that code is run.
* Add more idiomatic spacing around `:`. This is also only partially
cleaned up, as I gave up when I saw how `asn2wrs.py` was clearly
infatuated with the construct.
* Various other small cleanups, removed some trailing whitespace and
improper indentation that wasn't a multiple of 4, etc.
There is still _much_ to do, but I haven't been heavily involved with
this project before, so thought this was a sufficient amount to put up
and see what the feedback is.
Linters that I have enabled which highlighted some of these issues
include:
* `pylint`
* `flake8`
* `pycodestyle`
This adds a protocol post-dissector for Community ID support to
Wireshark/tshark: https://github.com/corelight/community-id-spec
The protocol is disabled by default. It establishes one new filter
value, "communityid".
Includes test cases and baselines to verify correct Community ID
strings based on similar testsuites in the existing Zeek and Python
implementations.
Replace bugs.wireshark.org links with their equivalent
gitlab.com/wireshark/wireshark/issues links in the AsciiDoctor buglink
macro and the please_report_bug function. Update the bug URLs in
comments in the tools and test directories.
Add a new top-level view that shows each packet as a series of diagrams
similar to what you'd find in a networking textook or an RFC.
Add proto_item_set_bits_offset_len so that we can display some diagram
fields correctly.
Bugs / to do:
- Make this a separate dialog instead of a main window view?
- Handle bitfields / flags
Change-Id: Iba4897a5bf1dcd73929dde6210d5483cf07f54df
Reviewed-on: https://code.wireshark.org/review/37497
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Add support for TK user input keys. With this Wireshark can
decrypt packet captures where 4WHS frames are missing and
packet captures with non-supported AKMS, for example
802.11r / Fast BSS Transitioning.
Decryption using user TK works as a backup if the normal
decryption flow does not succeed. Having TK decryption keys
added will affect general IEEE 802.11 dissector performance
as each encrypted packet will be tested with every TK.
Worst case scenario is plenty of TKs where none of them
matches encrypted frames.
On successful user TK decryption an SA is formed based on
parameters used to decrypt the frame. This SA is similar to
what is formed when Wireshark detects and derive keys from
4WHS messages. With the SA entry in place the decryption
performance (success case) should be on par with "normal"
decryption flow.
Bug: 16579
Change-Id: I72c2c1e2c6693131d3ba07f8ddb8ff772c1b54a9
Reviewed-on: https://code.wireshark.org/review/37217
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Different header, with a different size, an additional field, and with
fields being in a different order.
Distinguish between V1 and V2 by giving the version.
That means we can no longer use the "ethertype" dissector as it stands,
because the packet type field isn't at the end of the header, right
before the payload; pull the "add the type field to the protocol tree"
functionality out of the "ethertype" dissector and leave it up to the
dissector calling it.
Change-Id: I72b8a2483c0a539919fbe5d35fd7e60bff4bf75a
Reviewed-on: https://code.wireshark.org/review/37169
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
pytest and `pycodestyle test/suite_*.py --select=W605` warned about it.
Change-Id: I015351d1c00d17aa9f04ab17abed00586ee09e89
Reviewed-on: https://code.wireshark.org/review/36771
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
MFP enabled connections with SHA-256 key management (IEEE 802.11w) use
EAPOL key version == 3. This case was missing making decryption of such
connections fail. Allow key version 3 to handle these too.
Change-Id: If9e3fcc5c3bbfb46e82b39dfed5b2a74787a4f16
Reviewed-on: https://code.wireshark.org/review/36534
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Fields such as 'frame.time_delta' have no byte selection, they are added
with offset 0 and length 0, and evidently 'ws_tvb' is NULL. As
tvb_bytes_to_str expects a non-NULL tvb, explicitly check for this and
add a dummy placeholder. This is intended to be a human-readable string,
so prefer `<EMPTY>` over an empty string.
Change-Id: I32efe4cbefc6bcf0fa9fb94fcf25d7bf1628f3a7
Reviewed-on: https://code.wireshark.org/review/36440
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
CCMP-256 decryption support is only available with libgcrypt >= 1.6 so
skip corresponding test if lib is too old.
Fixes: v3.3.0rc0-733-geed31f13be ("ieee80211: Add CCMP-256 decryption support")
Change-Id: I6145f2ec49e19d1356915b9b1d8da54332a4a16a
Reviewed-on: https://code.wireshark.org/review/36414
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Run Wireshark tests on macOS only if SECURITYSESSIONID. If this isn't
reliable enough we might be able to get away with running a small UI
utility, e.g. `pbpaste > /dev/null`.
Change-Id: Ieccb87bcc1312f02c22ec2905a80437bf136d964
Reviewed-on: https://code.wireshark.org/review/36195
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Update existing tests to the new smb2_seskey_list syntax and add new
tests for decrypting using different combinations of provided keys.
Change-Id: I86fda351ff736cae6029ec2321c45a02c1917226
Reviewed-on: https://code.wireshark.org/review/36137
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
When dissecting a capture made in the middle of an existing encrypted
session we cannot decrypt the traffic because we don't know:
* what SMB dialect and encryption algorithm was picked during the
session establishment
* which host is the server and which host is the client
Since we know the decrypted payload always starts with a valid header
we use this as an heuristic and try all possible decryption settings.
Change-Id: I1daa297ced98e62cf361b9022871c668e56f8f4b
Reviewed-on: https://code.wireshark.org/review/36136
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Previously users could only give a session key via
uat:smb2_seskey_list:<id>,<seskey>
which was used to generate the decryption keys, as long as the trace
contained the session establishement.
Users have often asked about how to decrypt traffic captured in the
middle of an existing session but this wasn't possible.
This commit extends uat:smb2_seskey_list with 2 extra columns to store
decryption keys so that traffic can be decrypted at any point of the
session.
This has the side effect of changing the current syntax from:
... -o uat:smb2_seskey_list:<id>,<seskey>
To:
... -o 'uat:smb2_seskey_list:<id>,<seskey>,"",""'
(make sure the quoting is right)
Change-Id: I810d464b6f3e749de39b4428d73e0d6be29f3152
Reviewed-on: https://code.wireshark.org/review/36135
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Add software_update_info() to the software update module, which returns
the name of our update library if we have one. Use it to add automatic
update information to the compiled information in `wireshark --version`.
Add a "release" test suite, which contains a test for automatic updates.
Ping-Bug: 16381
Change-Id: I867a96bdcfde8be541eca2dc0e84b5000276e7dd
Reviewed-on: https://code.wireshark.org/review/36107
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
We want things like aes-256 keys to be displayed completely.
Change-Id: I746f3282440c036cfb60263be40e3b3a6ed859c2
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-on: https://code.wireshark.org/review/35703
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
XXX comment reworded to be more informative and reflect lack of
consensus on removing RFC 3514.
Change-Id: If15b8f5d7c450192b1b6ebbfa463b19f27de177c
Reviewed-on: https://code.wireshark.org/review/35934
Petri-Dish: João Valverde <j@v6e.pt>
Tested-by: Petri Dish Buildbot
Reviewed-by: João Valverde <j@v6e.pt>
A filter for payload makes it easier to exoprt it.
Change-Id: I0732c60c7fac37283fcbe6508d5e27bcd3c603fd
Reviewed-on: https://code.wireshark.org/review/35519
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
dpauxmon relies on libnl that is linux specific. Skip the test
for other platforms.
Fix skip message while here.
Change-Id: I744d83d0a56e41e1c32e45ade8ca0d9dd29876e1
Reviewed-on: https://code.wireshark.org/review/35525
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Tested-by: Dario Lombardo <lomato@gmail.com>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Fixes:
- sdjournal is available on linux only.
- The systemd library has been put in the right group in debian-setup.
Change-Id: Ie022f29da4313d17d55201b6e7ea1ab2ae740e18
Reviewed-on: https://code.wireshark.org/review/35478
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Add a test to verify that PTK can be derived for WPA3 SuiteB-192
captures and that encrypted keydata field is decrypted so that
GTK can be dissected.
NOTE: Capture file contains no encrypted data frames as currently
Wireshark does not support decrypting GCMP-256 encrypted data.
Ping-Bug: 16197
Change-Id: I57fbc14a4b4bca58790c4edcee14f1ef73d73fd5
Reviewed-on: https://code.wireshark.org/review/35068
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Support Extended Key ID for Individually Addressed Frames from
IEEE 802.11 - 2016.
Extended Key ID allows unicast (PTK) keys to also use key ID 1 and has
an additional RSN attribute "KeyID" in EAPOL #3.
Add the additional attribute KeyID to the RSN parser, stop assuming
unicast keys are only using key ID 0 and add a test case to verify
Extended Key ID parsing and decoding.
Change-Id: I43005c74df561be5524fa3738149781f50dafa14
Reviewed-on: https://code.wireshark.org/review/34883
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Jaap Keuter <jaap.keuter@xs4all.nl>
If a ProtoField object was created, but not linked to a Proto, then the
strings field and all elements (depending on type) would leak.
This is a follow-up to g79fef2ae and fixes the real issue in g44870fb1.
Change-Id: I01880a92bb20fae45f68c754b07daeb07630deec
Reviewed-on: https://code.wireshark.org/review/34872
Petri-Dish: Stig Bjørlykke <stig@bjorlykke.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Vasil Velichkov <vvvelichkov@gmail.com>
Reviewed-by: Roland Knall <rknall@gmail.com>
Add the newly create ProtoField to the test_proto.fields in order to be freed.
ERROR: LeakSanitizer: detected memory leaks
Direct leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x55e041f5ef86 in realloc (/home/vasko/sources/wireshark/build_clang/run/tshark+0x222f86)
#1 0x7fd8e0919e7d in g_realloc gmem.c:164:16
#2 0x7fd8e08e457b in g_array_maybe_expand garray.c:820:21
#3 0x7fd8e08e4b49 in g_array_append_vals garray.c:428:3
#4 0x7fd8ef112b6e in value_string_from_table /home/vasko/sources/wireshark/epan/wslua/wslua_proto_field.c:294:9
#5 0x7fd8ef10ee10 in ProtoField_new /home/vasko/sources/wireshark/epan/wslua/wslua_proto_field.c:646:24
Direct leak of 32 byte(s) in 1 object(s) allocated from:
#0 0x55e041f5ef86 in realloc (/home/vasko/sources/wireshark/build_clang/run/tshark+0x222f86)
#1 0x7fd8e0919e7d in g_realloc gmem.c:164:16
#2 0x7fd8e08e457b in g_array_maybe_expand garray.c:820:21
#3 0x7fd8e08e4b49 in g_array_append_vals garray.c:428:3
#4 0x7fd8ef111e2e in range_string_from_table /home/vasko/sources/wireshark/epan/wslua/wslua_proto_field.c:226:17
#5 0x7fd8ef10e43f in ProtoField_new /home/vasko/sources/wireshark/epan/wslua/wslua_proto_field.c:642:24
Fixes: v3.1.1rc0-497-g0371994223 ("wslua: Improve parameter check in ProtoField.new()")
Change-Id: I05b8bf3d78c8a540c6cf33d11dc3f3e01f83b8c5
Reviewed-on: https://code.wireshark.org/review/34834
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
João Valverde