This adds new syntax to read a field from the tree as bytes, instead
of the actual type. This is a useful extension for example to match
matformed strings that contain unicode replacement characters. In
this case it is not possible to match the raw value of the malformed
string field. This extension fills this need and is generic enough
that it should be useful in many other situations.
The syntax used is to prefix the field name with "@". The following
artificial example tests if the HTTP user agent contains a particular
invalid UTF-8 sequence:
@http.user_agent == "Mozill\xAA"
Where simply using "http.user_agent" won't work because the invalid byte
sequence will have been replaced with U+FFFD.
Considering the following programs:
$ dftest '_ws.ftypes.string == "ABC"'
Filter: _ws.ftypes.string == "ABC"
Syntax tree:
0 TEST_ANY_EQ:
1 FIELD(_ws.ftypes.string <FT_STRING>)
1 FVALUE("ABC" <FT_STRING>)
Instructions:
00000 READ_TREE _ws.ftypes.string <FT_STRING> -> reg#0
00001 IF_FALSE_GOTO 3
00002 ANY_EQ reg#0 == "ABC" <FT_STRING>
00003 RETURN
$ dftest '@_ws.ftypes.string == "ABC"'
Filter: @_ws.ftypes.string == "ABC"
Syntax tree:
0 TEST_ANY_EQ:
1 FIELD(_ws.ftypes.string <RAW>)
1 FVALUE(41:42:43 <FT_BYTES>)
Instructions:
00000 READ_TREE @_ws.ftypes.string <FT_BYTES> -> reg#0
00001 IF_FALSE_GOTO 3
00002 ANY_EQ reg#0 == 41:42:43 <FT_BYTES>
00003 RETURN
In the second case the field has a "raw" type, that equates directly to
FT_BYTES, and the field value is read from the protocol raw data.
Modernize the handling of experimental TCP options based on
RFC 6994. In particular use ExID instead of magic (which
in the context of RFC 6994 are the last two bytes of a
32-bit ExID) and add a desciption of ExID based on the
current state of the IANA registry.
Implement out of order buffering and desegmentation for QUIC
CRYPTO frames. Particularly useful for Chrome's "Chaos Protection"
that intentionally introduces them, but handles out of order
CRYPTO frames in different UDP payloads as well. (Buffering
packets at a higher encryption level until the out of order
lower level frames have arrived is a different issue.)
Adds a preference, which defaults to on since if there is
out of order, it's not very useful to turn it off.
Fix#17732. Fix#18215.
Add support to display filters for matching a specific layer within a frame.
Layers are counted sequentially up the protocol stack. Each protocol
(dissector) that appears in the stack is one layer.
LINK-LAYER#1 <-> IP#1 <-> TCP#1 <-> IP#2 <-> TCP#2 <-> etc.
The syntax allows for negative indexes and ranges with the usual semantics
for slices (but note that counting starts at one):
tcp.port#[2-4] == 1024
Matches layers 2 to 4 inclusive.
Fixes#3791.
Add an UAT for configuring fake headers according to the server port, stream
id and direction of the long-lived stream that we start capturing packets
after it is established. That helps to parsing the DATAs captured subsequently.
A testcase also added.
close#17691
- Point all MSP related DATA frames to their MSP instead of
using wmem_tree_lookup32_array_le().
- Add test_grpc_streaming_mode_reassembly testcase for verifying
this feature.
close#17633
"Follow Stream" functionality assumes that all data in a single packet
belongs to the same stream. That is not true for HTTP2 and QUIC, where
we end up having data from unrelated streams.
Filter out the unwanted data directly in the protocol dissector code with
a custom `tap_handler` (as TCP already does).
Close#16093
Add git dissection test cases to existing testing suite for: finding git
packets, finding the Git Protocol version, finding the right amount of
Flush and Delimiter packets, not finding Malformed packets.
Part of #17093
Support decrypting captures with Fast BSS Transition roaming present
by now also scanning (re)association frames for relevant information
elements and feeding it into the dot11decrypt engine.
Both (re)association request and response frames are scanned to allow
for potentially missing one frame and still be able to derive PTKs
needed for successful decryption.
Closes#17145
Change-Id: I08436582e4f83695dc606ddb92ff442d6258ef9b
Changes:
* Replaced large netperfmeter-dccp.pcapng.gz and netperfmeter.pcap.gz captures
by one common small netperfmeter.pcapng.gz for the suites follow_dccp and
netperfmeter.
* Updated test suites "follow_dccp" and "netperfmeter".
This pull request includes:
* The "Follow DCCP stream" feature.
* Updated docbook documentation for the "Follow DCCP stream" feature.
* Test for the feature.
* Corresponding packet trace for the test.
Add partial support for decrypting captures with connections
established using FT-EAP. To support deriving keys for FT-EAP
the MSK is needed. This change adds MSK as a valid IEEE 802.11
protocol input key type preference as well.
Note that FT-EAP support comes with the following imitations:
- Keys can only be derived from the FT 4-way handshake messages.
- Roaming is not supported.
Add partial support for decrypting captures with connections
established using FT BSS Transition (IEEE 802.11r).
FT BSS Transition decryption comes with the following limitations:
- Only FT-PSK is supported.
- Keys can only be derived from the FT 4-way handshake messages.
- Roaming is not supported.
Add case_dissect_protobuf and case_dissect_grpc in test/suite_dissection.py.
Add *.proto into the sub directories of test/protobuf_lang_files/.
Run command like 'pytest --program-path .\run\Debug\ -k "grpc or protobuf"'
in build directory (in windows) to test these cases only.
For WPA security association (SA) entries are created on sucessful
PTK derivation from 4-way handshake frames. WEP though don't use
4-way handshake frames for key derivation and therefore no SA entry
is created. Still WEP decryption implementaton expects to find
an SA otherwise the decryption is skipped.
Fix broken WEP decryption by removing the check for an existing SA
entry and instead form the SA on first successful decryption.
Add also a test for WEP decryption.
Fixes: v3.3.0rc0-1263-g099d241046 ("dot11decrypt: Avoid allocating SA on packet decryption")
This adds a protocol post-dissector for Community ID support to
Wireshark/tshark: https://github.com/corelight/community-id-spec
The protocol is disabled by default. It establishes one new filter
value, "communityid".
Includes test cases and baselines to verify correct Community ID
strings based on similar testsuites in the existing Zeek and Python
implementations.
MFP enabled connections with SHA-256 key management (IEEE 802.11w) use
EAPOL key version == 3. This case was missing making decryption of such
connections fail. Allow key version 3 to handle these too.
Change-Id: If9e3fcc5c3bbfb46e82b39dfed5b2a74787a4f16
Reviewed-on: https://code.wireshark.org/review/36534
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Add a test to verify that PTK can be derived for WPA3 SuiteB-192
captures and that encrypted keydata field is decrypted so that
GTK can be dissected.
NOTE: Capture file contains no encrypted data frames as currently
Wireshark does not support decrypting GCMP-256 encrypted data.
Ping-Bug: 16197
Change-Id: I57fbc14a4b4bca58790c4edcee14f1ef73d73fd5
Reviewed-on: https://code.wireshark.org/review/35068
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Support Extended Key ID for Individually Addressed Frames from
IEEE 802.11 - 2016.
Extended Key ID allows unicast (PTK) keys to also use key ID 1 and has
an additional RSN attribute "KeyID" in EAPOL #3.
Add the additional attribute KeyID to the RSN parser, stop assuming
unicast keys are only using key ID 0 and add a test case to verify
Extended Key ID parsing and decoding.
Change-Id: I43005c74df561be5524fa3738149781f50dafa14
Reviewed-on: https://code.wireshark.org/review/34883
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Jaap Keuter <jaap.keuter@xs4all.nl>
The latest iteration of Microsoft updates to SMB3 added compression to
the protocol. This commit implements decompressing and dissecting
compressed payloads.
The compression algorithms that can be used are "Plain LZ77",
"LZ77+Huffman" and "LZNT1" which you can read more about in the
[MS-XCA] documentation. This set of algorithm is sometimes referred to
as XPRESS.
This commit reuses the existing uncompression API scheme already in
place with zlib and brotli and adds 3 tvb_uncompress_*() function
implemented in:
* epan/tvbuff_lz77.c
* epan/tvbuff_lz77huff.c
* epan/tvbuff_lznt1.c
A new function wmem_array_try_index() was added to the wmem_array API
to make bound checked reads that fail gracefully. New tests for it
have been added as well.
Since both reads (tvb) and writes (wmem_array) are bound checked the
risk for buffer overruns is drastically reduced. LZ77+Huffman has
decoding tables and special care was taken to bound check these.
Simplified versions of the implementations were succesfully tested
against AFL (American Fuzzy Lop) for ~150 millions executions each.
The SMB2/3 dissector was changed to deal with the new transform header
for compressed packets (new protocol_id value) and READ request
flags (COMPRESSED). Badly compressed or encrypted packets are now
reported as such, and the decryption test suite was changed to reflect
that.
This commit also adds a test capture with 1 packet compressed with
each algorithm as returned by Windows Server 2019, along with 3
matching tests in test/suite_dissection.py
Change-Id: I2b84f56541f2f4ee7d886152794b993987dd10e7
Reviewed-on: https://code.wireshark.org/review/33855
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
epan/dissectors/packet-smb2.c:
- factor out decryption code from transform header dissector
- rewrite CCM decryption to use the proper gcrypt cipher
- add GCM support
- change weird 1 element CCM bitmask to value_string
test/suite_decryption.py: add smb2 GCM decryption test
- add one test
- add a new capture to test smb3.1.1 AES-128-GCM decryption
Change-Id: Id5ab75e1830bc24ace9f9b2f2dbd5a3c20666d52
Reviewed-on: https://code.wireshark.org/review/33600
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Lack of handshake reassembly caused Certificate handshake messages to be
reported as "Encrypted Handshake Messages" and broke decryption in some
cases. Fix this by properly tracking handshake fragments and delay
dissection until all fragments are available.
Now when a fragmented Handshake message is found:
* The first fragment will have "(fragmented)" appended to the record
tree item as well as the "Handshake Protocol" item.
* "Reassembled Handshake Message in frame: X" is added for fragments.
* The last reassembled handshake message will be displayed together with
a fragment list.
Note: Previously, handshake records with a message length larger than
the available data was assumed to be encrypted. This restriction had to
be lifted, but can now cause false positives (reporting encrypted data
as unencrypted handshake fragments).
The provided capture is not minimal but should be comprehensive as it is
generated with randomly sized TLS record and TCP segment lengths using
`./tls-handshake-fragments.py hs-frag.pcap --seed=1337 --count=100` and
https://git.lekensteyn.nl/peter/wireshark-notes/tree/crafted-pkt/tls-handshake-fragments.py
(A copy of this script is attached to bug 3303.)
Bug: 3303
Bug: 15537
Bug: 15625
Change-Id: I779925aba30548a76c20e0e37b39d01d2c88a764
Reviewed-on: https://code.wireshark.org/review/32857
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
PTK key derivation algorithm for WPA1 uses SHA1 not MD5.
MD5 is used for MIC only.
To avoid regression also add a decrypt test for WPA1 with
GTK rekeying.
Change-Id: Iabcf40c2f74d5dbc1d72cba0718c77020d97f61f
Fixes: v3.1.0rc0-342-g9cf77ec5e1 ("ieee80211: Support decrypting WPA3-Personal / SAE captures")
Reviewed-on: https://code.wireshark.org/review/32691
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Add support for decrypting IEEE 802.11 WPA3-Personal / SAE traffic.
SAE uses AES encryption but a different key derivation function (KDF)
making Wireshark fail to decrypt such captures. Also both KDF and
decryption method is determined based only on EAPOL key description
version. This is not enough to figure out that SAE is being used.
Implement the alternative KDF needed to derive valid PTK. Also
implement a function to parse pairwise + group cipher suites and
auth key management type from RSNE tag. Using this new function
together with a number of new cipher and AKM lookup functions
correct KDF for SAE can be selected.
Bug: 15621
Change-Id: I8f6c917af1c9642c276a244943dd35f850ee3757
Reviewed-on: https://code.wireshark.org/review/32485
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
If the single byte within a ZeroWindowProbe triggers reassembly within a
subdissector, a new MSP will be created with just a single byte. Be sure
not to mark subsequent segments that contain the full segment data as
retransmission as this prevents the subdissector from seeing the data.
Bug: 15427
Change-Id: I36ae2622689c6606c99cdff70b6beba4b9d25ca7
Reviewed-on: https://code.wireshark.org/review/31732
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Jasper Bongertz <jasper@packet-foo.com>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
When the initial segment is OoO, it was recognized as retransmitted. Fix
this by remembering which frame actually contains the initial segment.
Bug: 15420
Change-Id: If63e2ff581775ff9d396a612839f1bfab30f111f
Reviewed-on: https://code.wireshark.org/review/31720
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
The DTLS and TLS dissectors already share code for parsing the key log
file contents but the actual key material was stored separately. As
implementations (like GnuTLS) write the TLS and DTLS secrets to the same
file (specified by the SSLKEYLOGFILE environment variable), it seems
reasonable to combine them.
This also enables use of the pcapng Decryption Secrets Block for
decryption of DTLS traces. The dtls.keylog_file preference has become
obsolete and can no longer be used (this was not tested anyway).
A new test was added based on dtls12-aes128ccm8.pcap, the master secret
was extracted using the tls.debug_file preference.
Bug: 15252
Change-Id: Idfd52c251da966fe111dea37bc3fb143d968f744
Reviewed-on: https://code.wireshark.org/review/31577
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Stig Bjørlykke <stig@bjorlykke.org>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Support reading and writing pcapng files with DSBs. A DSB may occur
multiple times but should appear before packets that need those
decryption secrets (so it cannot be moved to the end like NRB). The TLS
dissector will be updated in the future to make use of these secrets.
pcapng spec update: https://github.com/pcapng/pcapng/pull/54
As DSBs may be interleaved with packets, do not even try to read it in
pcapng_open (as is done for IDBs). Instead process them during the
sequential read, appending them to the 'wtap::dsbs' array.
Writing is more complicated, secrets may initially not be available when
'wtap_dumper' is created. As they may become available in 'wtap::dsbs'
as more packets are read, allow 'wtap_dumper::dsbs_growing' to reference
this array. This saves every user from checking/dumping DSBs.
If the wtap user needs to insert extra DSBs (while preserving existing
DSBs), they can set the 'wtap_dumper::dsbs_initial' field.
The test file was creating using a patched editcap (future patch) and
combined using mergecap (which required a change to preserve the DSBs).
Change-Id: I74e4ee3171bd852a89ea0f6fbae9e0f65ed6eda9
Ping-Bug: 15252
Reviewed-on: https://code.wireshark.org/review/30692
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
If we have a single capture source and that capture source is pcapng and
we're writing a pcapng file, do the following:
- Pass its SHB and IDBs through unmodified. Don't save or write command
line interface IDBs.
- Save the most recent SHB and IDBs so that we can write them when we're
writing multiple output files.
If we have multiple capture sources, do the following:
- Write Dumpcap's SHB.
- Keep a global list of IDBs, consisting of both command line interfaces
and IDBs read from pcapng sources.
- When reading an EPB or ISB, remap its local interface number to its
corresponding global number.
Add Dumpcap pcapng section tests. Make the application IDs in the
"many_interfaces" captures unique.
Change-Id: I2005934c1f83d839727421960005f106d6c682dd
Reviewed-on: https://code.wireshark.org/review/30085
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
'double' tests have been disabled in aa03833 due to format change
in ntp fields.
Change-Id: Id3ab0a736c164bb7fdfed7b5da8856b512308978
Reviewed-on: https://code.wireshark.org/review/30366
Petri-Dish: Dario Lombardo <lomato@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The new KNXnet/IP dissector replaces the old KNXnet/IP dissector.
The new KNXnet/IP dissector supports the new KNX features
- A_MemoryExtended services
- A_PropertyExt services
- KNX Data Security
- KNXnet/IP Core V2
- KNXnet/IP Device Management V2
- KNXnet/IP Tunneling V2
- KNXnet/IP Routing V2
- KNXnet/IP Security
Change-Id: I3d1d716ef03d16d2720e6a1fcb23c2243d1cd956
Reviewed-on: https://code.wireshark.org/review/29155
Petri-Dish: Roland Knall <rknall@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Roland Knall <rknall@gmail.com>
OoO reassembly assumed that the stream starts with the first data
segment, but this can already be OoO. Use the hint from SYN instead.
The test capture is based on a local capture, post-processed with scapy
to introduce an OoO condition and fixup the frame time.
Bug: 15078
Change-Id: Id0e312bb3d0e7c7f8f1b243a2be9f15c9851c501
Fixes: v2.9.0rc0-1097-gca42331437 ("tcp: add support for reassembling out-of-order segments")
Reviewed-on: https://code.wireshark.org/review/29305
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Add TLS 1.3 tests that verify decryption of handshake, application and
early data. Add another test that shows that early data is properly
skipped. This completes TLS 1.3 (RFC 8446) decryption support.
The trace was created using boringssl c4131a4a23a1.
Bug: 12779
Change-Id: Iddd266ecd3f428c95aa3f69616ce55e75d4ccca0
Reviewed-on: https://code.wireshark.org/review/29170
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
This imposes an additional requirement on the key log file, PSKs are
only linked to the most recently seen ephemeral key. This means that the
key log might contain duplicate PSK lines, but at least the dissector
won't have to try all keys and thereby save CPU time.
Bug: 15011
Change-Id: I368fa16269c96c4a1ff3bcb4e376c21f38fa2689
Reviewed-on: https://code.wireshark.org/review/28993
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Using long-term static public keys, it is possible to identify the
recipient of a handshake message. Add a new UAT where such keys can be
configured. Allow private keys to be configured as well since this
enables decryption of the Initiation handshake message.
Bug: 15011
Change-Id: I0d4df046824eac6c333e0df75f69f73d10ed8e5e
Reviewed-on: https://code.wireshark.org/review/28988
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Verified that the tests failed without the fixes for the linked bugs.
The tests have full statement coverage(*1) for check_follow_fragments
and follow_tcp_tap_listener. For details and Scapy script, see:
https://git.lekensteyn.nl/peter/wireshark-notes/commit/crafted-pkt/badsegments.py?id=4ecf9d858b49e76d8a9c29df01ce1bd523ae6704
(*1) except for `if (data_length <= data_offset) { data_length = 0; }`
Change-Id: I625536df375272cf6c9116231194c39df1217fae
Ping-Bug: 13700
Ping-Bug: 14944
Reviewed-on: https://code.wireshark.org/review/28618
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Kevin Albertson