Currently out-of-order segments will result in cutting a stream into
two pieces while the out-of-order segment itself is ignored. For
example, a stream of segments "ABDCE" is interpreted as "AB", "DE" with
"C" ignored. This behavior breaks TLS decryption or prevent application
layer PDUs (such as HTTP requests/responses) from being reconstructed.
To fix this, buffer segments when a gap is detected.
The proposed approach extends the "multi-segment PDU" (MSP) mechanism
which is normally used for linking multiple, sequential TCP segments
into a single PDU. When a gap is detected between segments, it is
assumed that the segments within this gap are out-of-order and will be
received (or retransmitted) later.
The current implementation has a limitation though, if multiple gaps
exist, then the subdissector will only be called when all gaps are
filled (the subdissector will receive segments later than necessary).
For example with "ACEBD", "ABC" can already be processed after "B" is
received (with "E" still buffered), but due to how MSP are extended, it
must receive "D" too before it reassembles "ABCDE". In practice this
could mean that the request/response times between HTTP requests and
responses are slightly off, but at least the stream is correct now.
(These limitations are documented in the User's Guide.)
As the feature fails at least the 802.11 decryption test where packets
are missing (instead of OoO), hide this feature behind a preference.
Tested with captures containing out-of-order TCP segments from the
linked bug reports, comparing the effect of toggling the preference on
the summary output of tshark, the verbose output (-V) and the two-pass
output (-2 or -2V). Captures marked with "ok" just needed "simple"
out-of-order handling. Captures marked with "ok2" additionally required
the reassembly API change to set the correct reassembled length.
This change does "regress" on bug 10289 though when the preference is
enabled as retransmitted single-segment PDUs are now passed to
subdissectors. I added a TODO comment for this unrelated cosmetic issue.
Bug: 3389 # capture 2907 (HTTP) ok
Bug: 4727 # capture 4590 (HTTP) ok
Bug: 9461 # capture 12130 (TLS/HTTP/RPC-over-HTTP +key 12131) ok
Bug: 12006 # capture 14236 (HTTP) ok2; capture 15261 (HTTP) ok
Bug: 13517 # capture 15370 (HTTP) ok; capture 16059 (MQ) ok
Bug: 13754 # capture 15593 (MySQL) ok2
Bug: 14649 # capture 16305 (WebSocket) ok
Change-Id: If3938c5c1c96db8f7f50e39ea779f623ce657d56
Reviewed-on: https://code.wireshark.org/review/27943
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Split our tests by suite_*.group_* instead of suite_*. There are quite a
few dfilter tests and this should make them more parallelizable.
Change-Id: I52371409618cda70dc99811e8de1fb1ad9d9a3b6
Reviewed-on: https://code.wireshark.org/review/28329
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
"nstime_t{A, B}" is defined as "A + B * 10^9" rather than an integer
part A and fractional part B.
Bug: 14720
Change-Id: I5321db7d5ecea8f976291d2a22667b02162194e2
Reviewed-on: https://code.wireshark.org/review/27775
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Move WIRESHARK_QUIT_AFTER_CAPTURE to the default environment instead of
running the capture tests in a separate environment.
Change-Id: I5198df35f512ffe8c0d7f091a7b50d5fdb9c4ff6
Reviewed-on: https://code.wireshark.org/review/27711
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Run our CTest tests with PYTHONIOENCODING=UTF-8.
If someone runs our tests manually and their output encoding isn't
UTF-8, print replacement characters instead of failing with an error.
Open our log files with "errors='backslashreplace'" in case their
contents aren't UTF-8.
Change-Id: Ifa4d12c2b5e272cf3903f3e0c6102e4d961562f1
Reviewed-on: https://code.wireshark.org/review/27686
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Dot11DecryptTDLSDeriveKey in epan/crypt/dot11decrypt.c requires Libcrypt
1.6 for MIC verification. Skip the test if unavailable.
Change-Id: I0b859c0ecf135182a6249a67b9b5ea9e34ad92b6
Reviewed-on: https://code.wireshark.org/review/27614
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Tested-by: Dario Lombardo <lomato@gmail.com>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Tests that the documented "-z hosts,ipv4" filter actually works and that
the resolved addresses table is correct (it implicitly tests that the
definition of WS_INET_ADDRSTRLEN in wsutil/inet_addr.h is sane).
Change-Id: If7babe665ea5ecb37e38078a9809c88873cb323c
Ping-Bug: 14667
Reviewed-on: https://code.wireshark.org/review/27454
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Change the test suite list in CMakeLists.txt to a static list. Add a
CTest coverage unit test.
Change-Id: I8459f320a2d0707618d6d56abdfce80274fddd2d
Reviewed-on: https://code.wireshark.org/review/27377
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
They've been replaced by the Python scripts.
Change-Id: I8add9c9ea0a6bdd68b2fa3841977863c0ea9a761
Reviewed-on: https://code.wireshark.org/review/27243
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Convert capinfos output to UTF-8 in getCaptureInfo.
Normalize our command paths, otherwise "./run/RelWithDebInfo/..." might
be interpreted as the command "." with flags "/run", "/RelWithDebInfo",
etc. on Windows.
Change-Id: Ib7336a016db3ee0805739fc44913cb9c6895aaad
Reviewed-on: https://code.wireshark.org/review/27239
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Move UAT file creation to config.py.
Run the text2pcap and some of the clopts tests under our default
environment.
Use "in" instead of "has_key".
Change-Id: Ie5c70fb33c29676672bed7bf8205cff0bba77f8a
Reviewed-on: https://code.wireshark.org/review/27234
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Reviewed-by: Gerald Combs <gerald@wireshark.org>
These directories have had trailing slashes for years and users seem to
rely on it, so restore this assumption for backwards compatibility. The
underlying API function (Dir.persconffile_path()) is not changed because
trailing slashes were not documented for that function.
For consistency, ensure that all Lua Dir functions return paths without
trailing slashes.
Bug: 14619
Change-Id: Ia299864999578884b1ad1cd48f1bd883bce6879d
Fixes: v2.5.0rc0-579-gfb052a637f ("Use g_build_filename() instead, fix indentation")
Reviewed-on: https://code.wireshark.org/review/27166
Reviewed-by: João Valverde <j@v6e.pt>
Petri-Dish: João Valverde <j@v6e.pt>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Add the fileformats and I/O suites. Move some more common code to
subprocesstest.py and add a diffOutput method.
Change-Id: I2ec34e46539022bdce78520645fdca6dfc1a8c1a
Reviewed-on: https://code.wireshark.org/review/27183
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
In util_slow_dhcp.py, open stdout as O_BINARY on Windows.
Have ctest pass --verbose to test.py.
Call config.canCapture at test time so that we don't inadvertently skip
some tests.
Stringify our dumpcap config check.
Fix our Gcrypt variable.
Change-Id: I884ec23ddfc7c28b79d4a860c6c43c308598e6db
Reviewed-on: https://code.wireshark.org/review/27182
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Create Python versions of our various test shell scripts. Add CMake
tests for each suite. Tests can now be run directly via test.py, via the
"test" target, or via ctest, e.g.
ctest --verbose --jobs 3
Add a testing chapter to the Developer's Guide.
Add a way to disable ctest in dpkg-buildpackage.
Suites completed:
- capture
- clopts
- decryption
- dissection
Remaining suites:
- fileformats
- io
- mergecap
- nameres
- text2pcap
- unittests
- wslua
Change-Id: I8936e05edefc76a86b6a7a5da302e7461bbdda0f
Reviewed-on: https://code.wireshark.org/review/27134
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Again, no more autotools/libtool, so no more .libs, as that's a
libtoolism.
Change-Id: I909c18b969ca8e04a252ff45f7f3e6bc9d0c8476
Reviewed-on: https://code.wireshark.org/review/27138
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
At one point, I remember a discussion resulting in the official name of
the next-generation replacement for pcap format being changed to
"pcapng", with no hyphen.
Make Wireshark reflect that.
Change-Id: Ie66fb13a0fe3a8682143106dab601952e9154e2a
Reviewed-on: https://code.wireshark.org/review/25214
Reviewed-by: Guy Harris <guy@alum.mit.edu>
This reverts commit b4fc532372.
No longer required.
Change-Id: I0ec135d7b54f2aa1b8369fe8192fd2e725ffd5a8
Reviewed-on: https://code.wireshark.org/review/24409
Reviewed-by: Graham Bloice <graham.bloice@trihedral.com>
Add some debug info to the failing tests by displaying
the LANG var.
Change-Id: I0a7316d06a6aca035b46c3e6211ac8e93425e295
Reviewed-on: https://code.wireshark.org/review/24407
Reviewed-by: Graham Bloice <graham.bloice@trihedral.com>
It was removed from packet_info when circuit API was removed. Remove
tests that call circuit_id for testing.
Change-Id: I88e54b288d9d8f99b63e0828663b3025c57e0ec8
Reviewed-on: https://code.wireshark.org/review/24400
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Michael Mann <mmann78@netscape.net>
Includes adding per-conversation data to store whether we are over
DTLS or UDP and registering as a heuristic sub-dissector for DTLS.
Future changes will add more use of the conversation structure.
Also included is a capture of UDT over DTLS in test/captures/udt-dtls.pcapng.gz,
the associated private key for the session in test/keys/udt-dtls.key and a
new test in the decryption suite to check this works.
Change-Id: I76826d3b35768d0b58f5335063884616968e5784
Reviewed-on: https://code.wireshark.org/review/22533
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
This commit reassembles data frames to build up the full entity body. It does
this for both client/server request and responses. Additionally, it also
decompresses bodies if they have the correct content-encoding header provided
and are not partial bodies.
Bug: 13543
Change-Id: I1661c9ddd09c1f6cf5a08b2b1921f95103aebb52
Reviewed-on: https://code.wireshark.org/review/20737
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Absolute and Relative time fields could not be converted to seconds
without converting to string and parsing to number.
Fixed conversion in generated code that was subject to precision loss
Usage:
f=Field.new("frame.delta_time")
delta=f().value:tonumber()
Change-Id: I6ef91c6238a6c2ed9adf6cae03f8913f0a09332e
Reviewed-on: https://code.wireshark.org/review/22316
Petri-Dish: Stig Bjørlykke <stig@bjorlykke.org>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Implements all seven AEAD_CHACHA20_POLY1305 cipher suites from RFC 7905
(for TLS 1.2) and the final missing one for TLS 1.3 (draft -20).
New test captures (created using OpenSSL_1_1_0-pre6-2528-g042597b0a)
also serve as tests for TLS 1.3 decryption support.
Change-Id: Ice6d639c9c7b7bc23a6ff5fb4832d02694abd8c4
Ping-Bug: 12779
Reviewed-on: https://code.wireshark.org/review/21902
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
This function returns TRUE/FALSE depending whether the specified ProtoField/Dissector
needs to be dissected. By using this function in conjunction with the TreeItem.visible
attribute, Lua dissectors can be significantly sped up by making less C interop calls
which are relatively slow in terms of dissection especially when using sub-protocols
where the dissection of an entire protocol can be skipped.
Added tests for TreeItem:referenced to protofield.lua
Change-Id: I44feacb91a2a5b0e3c28c0ccd8d6b04cccd67261
Reviewed-on: https://code.wireshark.org/review/21387
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Avoid anachronisms, however; there was no "macOS 10.0" or even "OS X
10.0", for example. It was "Mac OS X" until 10.8 (although 10.7 was
sometimes called "OS X" and sometimes called "Mac OS X"), and it was "OS
X" from 10.8 to 10.11.
Change-Id: Ie4a848997dcc6c45c2245c1fb84ec526032375c3
Reviewed-on: https://code.wireshark.org/review/20933
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Peter Wu