wireshark/test
Peter Wu ca42331437 tcp: add support for reassembling out-of-order segments
Currently out-of-order segments will result in cutting a stream into
two pieces while the out-of-order segment itself is ignored. For
example, a stream of segments "ABDCE" is interpreted as "AB", "DE" with
"C" ignored. This behavior breaks TLS decryption or prevent application
layer PDUs (such as HTTP requests/responses) from being reconstructed.
To fix this, buffer segments when a gap is detected.

The proposed approach extends the "multi-segment PDU" (MSP) mechanism
which is normally used for linking multiple, sequential TCP segments
into a single PDU. When a gap is detected between segments, it is
assumed that the segments within this gap are out-of-order and will be
received (or retransmitted) later.

The current implementation has a limitation though, if multiple gaps
exist, then the subdissector will only be called when all gaps are
filled (the subdissector will receive segments later than necessary).
For example with "ACEBD", "ABC" can already be processed after "B" is
received (with "E" still buffered), but due to how MSP are extended, it
must receive "D" too before it reassembles "ABCDE". In practice this
could mean that the request/response times between HTTP requests and
responses are slightly off, but at least the stream is correct now.
(These limitations are documented in the User's Guide.)

As the feature fails at least the 802.11 decryption test where packets
are missing (instead of OoO), hide this feature behind a preference.

Tested with captures containing out-of-order TCP segments from the
linked bug reports, comparing the effect of toggling the preference on
the summary output of tshark, the verbose output (-V) and the two-pass
output (-2 or -2V). Captures marked with "ok" just needed "simple"
out-of-order handling. Captures marked with "ok2" additionally required
the reassembly API change to set the correct reassembled length.

This change does "regress" on bug 10289 though when the preference is
enabled as retransmitted single-segment PDUs are now passed to
subdissectors. I added a TODO comment for this unrelated cosmetic issue.

Bug: 3389   # capture 2907 (HTTP) ok
Bug: 4727   # capture 4590 (HTTP) ok
Bug: 9461   # capture 12130 (TLS/HTTP/RPC-over-HTTP +key 12131) ok
Bug: 12006  # capture 14236 (HTTP) ok2; capture 15261 (HTTP) ok
Bug: 13517  # capture 15370 (HTTP) ok; capture 16059 (MQ) ok
Bug: 13754  # capture 15593 (MySQL) ok2
Bug: 14649  # capture 16305 (WebSocket) ok
Change-Id: If3938c5c1c96db8f7f50e39ea779f623ce657d56
Reviewed-on: https://code.wireshark.org/review/27943
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
2018-06-28 06:10:35 +00:00
..
baseline Test: Add fileformats and I/O. 2018-04-27 19:52:04 +00:00
captures tcp: add support for reassembling out-of-order segments 2018-06-28 06:10:35 +00:00
config ssl,http: rename http-over-ssl to http-over-tls 2016-09-04 13:18:54 +00:00
keys Add support for dissecting UDT over DTLS 2017-07-09 18:25:08 +00:00
lua wslua: fix NSTime:__tostring for negative values 2018-05-25 12:50:18 +00:00
suite_dfilter Test: Add UTF-8 filter tests. 2018-05-30 21:16:38 +00:00
README.test Test: Remove the Bash test scripts. 2018-05-02 04:09:26 +00:00
config.py Test: Add sharkd tests. 2018-06-21 03:30:30 +00:00
hosts.custom Continue to remove $Id$ from top of file 2014-03-31 18:48:06 +00:00
hosts.global Continue to remove $Id$ from top of file 2014-03-31 18:48:06 +00:00
hosts.personal Continue to remove $Id$ from top of file 2014-03-31 18:48:06 +00:00
subprocesstest.py Test: Add sharkd tests. 2018-06-21 03:30:30 +00:00
suite_capture.py Test: Consolidate our environments. 2018-05-22 16:34:03 +00:00
suite_clopts.py Test: Add UTF-8 filter tests. 2018-05-30 21:16:38 +00:00
suite_decryption.py test: require Libgcrypt 1.6 for TDLS test 2018-05-17 15:30:25 +00:00
suite_dissection.py tcp: add support for reassembling out-of-order segments 2018-06-28 06:10:35 +00:00
suite_fileformats.py Test: Add fileformats and I/O. 2018-04-27 19:52:04 +00:00
suite_io.py Test: Add fileformats and I/O. 2018-04-27 19:52:04 +00:00
suite_mergecap.py Test: Add text2pcap. 2018-04-30 21:43:30 +00:00
suite_nameres.py test: add test for resolved addresses 2018-05-12 06:29:55 +00:00
suite_sharkd.py Test: Add sharkd tests. 2018-06-21 03:30:30 +00:00
suite_text2pcap.py Test: Fix capinfos output and command paths. 2018-05-01 18:46:13 +00:00
suite_unittests.py Test+CMake: Make our test grouping more fine-grained. 2018-06-18 19:36:26 +00:00
suite_wslua.py Test: Add wslua. 2018-05-02 00:35:20 +00:00
test.py Test+CMake: Make our test grouping more fine-grained. 2018-06-18 19:36:26 +00:00
util_dump_dhcp_pcap.py Test: Add fileformats and I/O. 2018-04-27 19:52:04 +00:00

README.test

Wireshark Tests

The main testing script is `test.py`. It will attempt to test as much as
possible by default, including packet capture. This means that you will
probably either have to supply a capture interface (`--capture-interface
<interface>`) or disable capture tests (`--disable-capture`). You must
also build the test-programs target in order for the unittests suite to
pass.

To run all tests from CMake do the following:
- Pass `-DTEST_EXTRA_ARGS=--disable-capture` or
  `-DTEST_EXTRA_ARGS=--capture-interface=<interface>`
  as needed for your system.
- Build the “test” target or run ctest, e.g. `ctest --force-new-ctest-process -j 4 --verbose`.

To run all tests directly, run `test.py -p
/path/to/wireshark-build/run-directory <capture args>`.

To see a list of all options, run `test.py -h` or `test.py --help`.

To see a list of all tests, run `test.py -l`.

See the “Wireshark Tests” chapter of the Developer's Guide for details.