Support decrypting captures with Fast BSS Transition roaming present
by now also scanning (re)association frames for relevant information
elements and feeding it into the dot11decrypt engine.
Both (re)association request and response frames are scanned to allow
for potentially missing one frame and still be able to derive PTKs
needed for successful decryption.
Closes#17145
Change-Id: I08436582e4f83695dc606ddb92ff442d6258ef9b
Fedora and RHEL/CentOS put libsofthsm2.so in a different location
than Debian/Ubuntu, so look there too. This causes test_tls_pkcs11
to pass instead of being skipped (if softhsm2 and the other
prerequisites are installed.)
Add partial support for decrypting captures with connections
established using FT-EAP. To support deriving keys for FT-EAP
the MSK is needed. This change adds MSK as a valid IEEE 802.11
protocol input key type preference as well.
Note that FT-EAP support comes with the following imitations:
- Keys can only be derived from the FT 4-way handshake messages.
- Roaming is not supported.
Add partial support for decrypting captures with connections
established using FT BSS Transition (IEEE 802.11r).
FT BSS Transition decryption comes with the following limitations:
- Only FT-PSK is supported.
- Keys can only be derived from the FT 4-way handshake messages.
- Roaming is not supported.
PEP 394[1] says,
"In cases where the script is expected to be executed outside virtual
environments, developers will need to be aware of the following
discrepancies across platforms and installation methods:
* Older Linux distributions will provide a python command that refers
to Python 2, and will likely not provide a python2 command.
* Some newer Linux distributions will provide a python command that
refers to Python 3.
* Some Linux distributions will not provide a python command at all by
default, but will provide a python3 command by default."
Debian has forced the issue by choosing the third option[2]:
"NOTE: Debian testing (bullseye) has removed the "python" package and
the '/usr/bin/python' symlink due to the deprecation of Python 2."
Switch our shebang from "#!/usr/bin/env python" to "#!/usr/bin/env
python3" in some places. Remove some 2/3 version checks if we know we're
running under Python 3. Remove the "coding: utf-8" in a bunch of places
since that's the default in Python 3.
[1]https://www.python.org/dev/peps/pep-0394/#for-python-script-publishers
[2]https://wiki.debian.org/Python
For WPA security association (SA) entries are created on sucessful
PTK derivation from 4-way handshake frames. WEP though don't use
4-way handshake frames for key derivation and therefore no SA entry
is created. Still WEP decryption implementaton expects to find
an SA otherwise the decryption is skipped.
Fix broken WEP decryption by removing the check for an existing SA
entry and instead form the SA on first successful decryption.
Add also a test for WEP decryption.
Fixes: v3.3.0rc0-1263-g099d241046 ("dot11decrypt: Avoid allocating SA on packet decryption")
Add ui/urls.h to define some URLs on various of our websites. Use the
GitLab URL for the wiki. Add a macro to generate wiki URLs.
Update wiki URLs in comments etc.
Use the #defined URL for the docs page in
WelcomePage::on_helpLabel_clicked; that removes the last user of
topic_online_url(), so get rid of it and swallow it up into
topic_action_url().
Replace bugs.wireshark.org links with their equivalent
gitlab.com/wireshark/wireshark/issues links in the AsciiDoctor buglink
macro and the please_report_bug function. Update the bug URLs in
comments in the tools and test directories.
Add support for TK user input keys. With this Wireshark can
decrypt packet captures where 4WHS frames are missing and
packet captures with non-supported AKMS, for example
802.11r / Fast BSS Transitioning.
Decryption using user TK works as a backup if the normal
decryption flow does not succeed. Having TK decryption keys
added will affect general IEEE 802.11 dissector performance
as each encrypted packet will be tested with every TK.
Worst case scenario is plenty of TKs where none of them
matches encrypted frames.
On successful user TK decryption an SA is formed based on
parameters used to decrypt the frame. This SA is similar to
what is formed when Wireshark detects and derive keys from
4WHS messages. With the SA entry in place the decryption
performance (success case) should be on par with "normal"
decryption flow.
Bug: 16579
Change-Id: I72c2c1e2c6693131d3ba07f8ddb8ff772c1b54a9
Reviewed-on: https://code.wireshark.org/review/37217
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
pytest and `pycodestyle test/suite_*.py --select=W605` warned about it.
Change-Id: I015351d1c00d17aa9f04ab17abed00586ee09e89
Reviewed-on: https://code.wireshark.org/review/36771
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
MFP enabled connections with SHA-256 key management (IEEE 802.11w) use
EAPOL key version == 3. This case was missing making decryption of such
connections fail. Allow key version 3 to handle these too.
Change-Id: If9e3fcc5c3bbfb46e82b39dfed5b2a74787a4f16
Reviewed-on: https://code.wireshark.org/review/36534
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
CCMP-256 decryption support is only available with libgcrypt >= 1.6 so
skip corresponding test if lib is too old.
Fixes: v3.3.0rc0-733-geed31f13be ("ieee80211: Add CCMP-256 decryption support")
Change-Id: I6145f2ec49e19d1356915b9b1d8da54332a4a16a
Reviewed-on: https://code.wireshark.org/review/36414
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Update existing tests to the new smb2_seskey_list syntax and add new
tests for decrypting using different combinations of provided keys.
Change-Id: I86fda351ff736cae6029ec2321c45a02c1917226
Reviewed-on: https://code.wireshark.org/review/36137
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
When dissecting a capture made in the middle of an existing encrypted
session we cannot decrypt the traffic because we don't know:
* what SMB dialect and encryption algorithm was picked during the
session establishment
* which host is the server and which host is the client
Since we know the decrypted payload always starts with a valid header
we use this as an heuristic and try all possible decryption settings.
Change-Id: I1daa297ced98e62cf361b9022871c668e56f8f4b
Reviewed-on: https://code.wireshark.org/review/36136
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Previously users could only give a session key via
uat:smb2_seskey_list:<id>,<seskey>
which was used to generate the decryption keys, as long as the trace
contained the session establishement.
Users have often asked about how to decrypt traffic captured in the
middle of an existing session but this wasn't possible.
This commit extends uat:smb2_seskey_list with 2 extra columns to store
decryption keys so that traffic can be decrypted at any point of the
session.
This has the side effect of changing the current syntax from:
... -o uat:smb2_seskey_list:<id>,<seskey>
To:
... -o 'uat:smb2_seskey_list:<id>,<seskey>,"",""'
(make sure the quoting is right)
Change-Id: I810d464b6f3e749de39b4428d73e0d6be29f3152
Reviewed-on: https://code.wireshark.org/review/36135
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Add a test to verify that PTK can be derived for WPA3 SuiteB-192
captures and that encrypted keydata field is decrypted so that
GTK can be dissected.
NOTE: Capture file contains no encrypted data frames as currently
Wireshark does not support decrypting GCMP-256 encrypted data.
Ping-Bug: 16197
Change-Id: I57fbc14a4b4bca58790c4edcee14f1ef73d73fd5
Reviewed-on: https://code.wireshark.org/review/35068
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Support Extended Key ID for Individually Addressed Frames from
IEEE 802.11 - 2016.
Extended Key ID allows unicast (PTK) keys to also use key ID 1 and has
an additional RSN attribute "KeyID" in EAPOL #3.
Add the additional attribute KeyID to the RSN parser, stop assuming
unicast keys are only using key ID 0 and add a test case to verify
Extended Key ID parsing and decoding.
Change-Id: I43005c74df561be5524fa3738149781f50dafa14
Reviewed-on: https://code.wireshark.org/review/34883
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Jaap Keuter <jaap.keuter@xs4all.nl>
The latest iteration of Microsoft updates to SMB3 added compression to
the protocol. This commit implements decompressing and dissecting
compressed payloads.
The compression algorithms that can be used are "Plain LZ77",
"LZ77+Huffman" and "LZNT1" which you can read more about in the
[MS-XCA] documentation. This set of algorithm is sometimes referred to
as XPRESS.
This commit reuses the existing uncompression API scheme already in
place with zlib and brotli and adds 3 tvb_uncompress_*() function
implemented in:
* epan/tvbuff_lz77.c
* epan/tvbuff_lz77huff.c
* epan/tvbuff_lznt1.c
A new function wmem_array_try_index() was added to the wmem_array API
to make bound checked reads that fail gracefully. New tests for it
have been added as well.
Since both reads (tvb) and writes (wmem_array) are bound checked the
risk for buffer overruns is drastically reduced. LZ77+Huffman has
decoding tables and special care was taken to bound check these.
Simplified versions of the implementations were succesfully tested
against AFL (American Fuzzy Lop) for ~150 millions executions each.
The SMB2/3 dissector was changed to deal with the new transform header
for compressed packets (new protocol_id value) and READ request
flags (COMPRESSED). Badly compressed or encrypted packets are now
reported as such, and the decryption test suite was changed to reflect
that.
This commit also adds a test capture with 1 packet compressed with
each algorithm as returned by Windows Server 2019, along with 3
matching tests in test/suite_dissection.py
Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7
Reviewed-on: https://code.wireshark.org/review/33855
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
epan/dissectors/packet-smb2.c:
- factor out decryption code from transform header dissector
- rewrite CCM decryption to use the proper gcrypt cipher
- add GCM support
- change weird 1 element CCM bitmask to value_string
test/suite_decryption.py: add smb2 GCM decryption test
- add one test
- add a new capture to test smb3.1.1 AES-128-GCM decryption
Change-Id: Id5ab75e1830bc24ace9f9b2f2dbd5a3c20666d52
Reviewed-on: https://code.wireshark.org/review/33600
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
PTK key derivation algorithm for WPA1 uses SHA1 not MD5.
MD5 is used for MIC only.
To avoid regression also add a decrypt test for WPA1 with
GTK rekeying.
Change-Id: Iabcf40c2f74d5dbc1d72cba0718c77020d97f61f
Fixes: v3.1.0rc0-342-g9cf77ec5e1 ("ieee80211: Support decrypting WPA3-Personal / SAE captures")
Reviewed-on: https://code.wireshark.org/review/32691
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Add support for decrypting IEEE 802.11 WPA3-Personal / SAE traffic.
SAE uses AES encryption but a different key derivation function (KDF)
making Wireshark fail to decrypt such captures. Also both KDF and
decryption method is determined based only on EAPOL key description
version. This is not enough to figure out that SAE is being used.
Implement the alternative KDF needed to derive valid PTK. Also
implement a function to parse pairwise + group cipher suites and
auth key management type from RSNE tag. Using this new function
together with a number of new cipher and AKM lookup functions
correct KDF for SAE can be selected.
Bug: 15621
Change-Id: I8f6c917af1c9642c276a244943dd35f850ee3757
Reviewed-on: https://code.wireshark.org/review/32485
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The DTLS and TLS dissectors already share code for parsing the key log
file contents but the actual key material was stored separately. As
implementations (like GnuTLS) write the TLS and DTLS secrets to the same
file (specified by the SSLKEYLOGFILE environment variable), it seems
reasonable to combine them.
This also enables use of the pcapng Decryption Secrets Block for
decryption of DTLS traces. The dtls.keylog_file preference has become
obsolete and can no longer be used (this was not tested anyway).
A new test was added based on dtls12-aes128ccm8.pcap, the master secret
was extracted using the tls.debug_file preference.
Bug: 15252
Change-Id: Idfd52c251da966fe111dea37bc3fb143d968f744
Reviewed-on: https://code.wireshark.org/review/31577
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Stig Bjørlykke <stig@bjorlykke.org>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Add support for loading RSA private key files from PKCS #11 tokens,
identified by PKCS #11 URIs. Add a new 'pkcs11_libs' UAT which can
dynamically load PKCS #11 provider libraries that are not found by
p11-kit.
The configuration GUI will need additional code to discover available
PKCS #11 tokens and will be added later.
This feature requires GnuTLS 3.4 with PKCS #11 support, so Windows,
macOS via Homebrew, Ubuntu 16.04, Debian Stretch. Not supported: RHEL7.
Currently macOS via official packages disables PKCS #11 support, so that
will also not work.
Change-Id: I20646bfd69c6bd13c8c2d27cb65c164a4b0b7a66
Reviewed-on: https://code.wireshark.org/review/30855
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
This should eventually replace the "ssl_keys" UAT which additionally
contains a useless address, port and protocol field. This prepares for
HSM support through PKCS #11.
Change-Id: I59409c98aeedf260d19266d18e14ef7d9b40b582
Reviewed-on: https://code.wireshark.org/review/30977
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
GnuTLS is an optional dependency, allow tests to run without it.
Change-Id: Ib1bd7beaf1d885a157a0e1a630ccc4fbc8786af1
Reviewed-on: https://code.wireshark.org/review/30839
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The case_decrypt_tls.test_tls_rsa_pq test is unexpectedly passing when
GnuTLS is disabled. It checks for '/' in the output, but that also
matches an error message. Use assertRun here and pretty much everywhere
else to catch such issues. Remove a few redundant returncode checks.
Change-Id: I0f9d1dadc0ca73eef9cffb3e2f452aa7c8395c95
Reviewed-on: https://code.wireshark.org/review/30838
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Add a new secrets API to the core, one that can outlive the lifetime of
a single capture file. Expose decryption secrets from wiretap through a
callback and let the secrets API route it to a dissector.
Bug: 15252
Change-Id: Ie2f1867bdfd265bad11fc58f1e8d8e7295c0d1e7
Reviewed-on: https://code.wireshark.org/review/30705
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Continue the conversion from use of globals (the config module) to
fixtures. If a program (like wmem_test or tshark) is unavailable, it
will be skipped now rather than failing the test.
The general conversion pattern is:
- Decorate each class with `@fixtures.uses_fixtures` and (for tests that
run tshark) `@fixtures.mark_usefixtures('test_env')`.
- Convert all `config.cmd_*` to `cmd_*` and add an argument.
- Convert all `config.*_dir` to `dirs.*_dir` and add an argument.
- Convert users of `os.path.join(dirs.capture_file, ...)` to use a new
'capture_file' fixture to reduce boilerplate code. Inline variables if
possible (this conversion was done in an automated way using regexes).
Some other changes: tests that do not require a test environment (like
wmem_test) will use 'base_env' which avoids copying config files,
`env=config.test_env` got removed since this is the default. Some test
classes in suite_clopts were combined. Removed unused imports.
Change-Id: Id5480ffaee7d8d56cf2cb3189a38ae9afa7605a1
Reviewed-on: https://code.wireshark.org/review/30591
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Reduces maintenance costs and makes it possible to simplify code.
pytest supports Python 2.7 and Python 3.4 (or newer), so that is more or
less the minimum target for now.
Change-Id: I0347b6c334bf2fc6c9480ff56e9ccfcd48886dde
Reviewed-on: https://code.wireshark.org/review/30193
Tested-by: Petri Dish Buildbot
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
The new KNXnet/IP dissector replaces the old KNXnet/IP dissector.
The new KNXnet/IP dissector supports the new KNX features
- A_MemoryExtended services
- A_PropertyExt services
- KNX Data Security
- KNXnet/IP Core V2
- KNXnet/IP Device Management V2
- KNXnet/IP Tunneling V2
- KNXnet/IP Routing V2
- KNXnet/IP Security
Change-Id: I3d1d716ef03d16d2720e6a1fcb23c2243d1cd956
Reviewed-on: https://code.wireshark.org/review/29155
Petri-Dish: Roland Knall <rknall@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Roland Knall <rknall@gmail.com>
Rename the "ssl" protocol to "tls" and add an "ssl" alias. Prefer "TLS"
over "SSL" in user interface text and in the documentation.
Fix the test_tls_master_secret test while we're here.
Bug: 14922
Change-Id: Iab6ba2c7c4c0f8f6dd0f6d5d90fac5e9486612f8
Reviewed-on: https://code.wireshark.org/review/29649
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Add TLS 1.3 tests that verify decryption of handshake, application and
early data. Add another test that shows that early data is properly
skipped. This completes TLS 1.3 (RFC 8446) decryption support.
The trace was created using boringssl c4131a4a23a1.
Bug: 12779
Change-Id: Iddd266ecd3f428c95aa3f69616ce55e75d4ccca0
Reviewed-on: https://code.wireshark.org/review/29170
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
This imposes an additional requirement on the key log file, PSKs are
only linked to the most recently seen ephemeral key. This means that the
key log might contain duplicate PSK lines, but at least the dissector
won't have to try all keys and thereby save CPU time.
Bug: 15011
Change-Id: I368fa16269c96c4a1ff3bcb4e376c21f38fa2689
Reviewed-on: https://code.wireshark.org/review/28993
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Invoke IP or data dissector for decrypted transport data.
Bug: 15011
Change-Id: I8fa149c429ae774c16fe7e712d4bfb6b3478ed11
Reviewed-on: https://code.wireshark.org/review/28992
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Transport data decryption will follow later.
Bug: 15011
Change-Id: Ib755e43ff54601405b21aeb0045b15d158bc283b
Reviewed-on: https://code.wireshark.org/review/28991
Reviewed-by: Anders Broman <a.broman58@gmail.com>
As UATs are currently unable to receive keys dynamically without manual
user interaction followed by rescanning of the pcap, add a mechanism
like ssl.keylog_file. Such keys can be extracted using the tools from
contrib/examples/extract-handshakes/ in the WireGuard source tree.
Now decryption of Initiation messages is also possible when keys
(Epriv_i) are captured from the initiator side.
Bug: 15011
Change-Id: If998bf26e818487187cc618d2eb6d4d8f5b2cc0a
Reviewed-on: https://code.wireshark.org/review/28990
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Dario Lombardo