When we're walking the list of fragments to free, if we encounter
FD_VISITED_FREE, we can conclude traversal of this fragment list immediately
(and go to the next hash bucket), since everything subsequent to this point in
the list has already been processed by free_all_reassembled_fragments. This
trims an O(n^2) hash table iteration down to O(n).
Before this change, a very ugly 1.1 GByte TFTP capture (with lots of
out-of-order and retransmitted blocks) takes 4 hours to process with
tftp.defragment=TRUE -- output completes after 1.25 hours, and then about
2.75 hours of time is spent doing repeated list traversals within
free_all_reassembled_fragments...(!) With this change, the same test completes
in 1.25 hours, with the cleanup taking just 71 msec.
Tested also with reassemble_test under Valgrind; No issues/leaks were reported.
fragment_add_work() should flag duplicate packets and overlapping packets
that are subsets of the existing reassembly as overlaps (and flag them if
there are conflicts), instead of silently adding them to the reassembly.
Currently the checks are only performed when the new segment is adding
additional bytes to the reassembly.
This is particularly useful for identifying when an earlier reassembly isn't
fully contained in the capture, and the identification / offset number has
wrapped around so that segments during two different reassemblies are being
mixed together. Closes#16872 and #15569.
There's a check for adding a zero length fragment to a reassembly in progress,
but it accidentally checks fd_head->tvb_data (the reassembly in progress)
instead of fd_i->tvb_data (the new fragment) before calling tvb_get_data() on
fd_i->tvb_data. (Note that data / fd_head->tvb_data is created based on the
sum of the lengths of all the fd_i->tvb_data, so the former can only be NULL
if all the latter are, but it's possible for one fragment to be zero length
but not the entire reassembly. Thus this is the necessary and sufficient check.)
Fixes#15569
Mark the "Reassembled in" link item as generated.
Change-Id: Id8f7b26cf77aa5bc82de07799bb1468c8b8a4209
Reviewed-on: https://code.wireshark.org/review/37441
Petri-Dish: Stig Bjørlykke <stig@bjorlykke.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Change all wireshark.org URLs to use https.
Fix some broken links while we're at it.
Change-Id: I161bf8eeca43b8027605acea666032da86f5ea1c
Reviewed-on: https://code.wireshark.org/review/34089
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Make the time stamp precision a 4-bit bitfield, so, when combined with
the other bitfields, we have 32 bits. That means we put the flags at
the same structure level as the time stamp precision, so they can be
combined; that gets rid of an extra "flags." for references to the flags.
Put the two pointers next to each other, and after a multiple of 8 bytes
worth of other fields, so that there's no padding before or between them.
It's still not down to 64 bytes, which is the next lower power of 2, so
there's more work to do.
Change-Id: I6f3e9d9f6f48137bbee8f100c152d2c42adb8fbe
Reviewed-on: https://code.wireshark.org/review/31213
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Currently out-of-order segments will result in cutting a stream into
two pieces while the out-of-order segment itself is ignored. For
example, a stream of segments "ABDCE" is interpreted as "AB", "DE" with
"C" ignored. This behavior breaks TLS decryption or prevent application
layer PDUs (such as HTTP requests/responses) from being reconstructed.
To fix this, buffer segments when a gap is detected.
The proposed approach extends the "multi-segment PDU" (MSP) mechanism
which is normally used for linking multiple, sequential TCP segments
into a single PDU. When a gap is detected between segments, it is
assumed that the segments within this gap are out-of-order and will be
received (or retransmitted) later.
The current implementation has a limitation though, if multiple gaps
exist, then the subdissector will only be called when all gaps are
filled (the subdissector will receive segments later than necessary).
For example with "ACEBD", "ABC" can already be processed after "B" is
received (with "E" still buffered), but due to how MSP are extended, it
must receive "D" too before it reassembles "ABCDE". In practice this
could mean that the request/response times between HTTP requests and
responses are slightly off, but at least the stream is correct now.
(These limitations are documented in the User's Guide.)
As the feature fails at least the 802.11 decryption test where packets
are missing (instead of OoO), hide this feature behind a preference.
Tested with captures containing out-of-order TCP segments from the
linked bug reports, comparing the effect of toggling the preference on
the summary output of tshark, the verbose output (-V) and the two-pass
output (-2 or -2V). Captures marked with "ok" just needed "simple"
out-of-order handling. Captures marked with "ok2" additionally required
the reassembly API change to set the correct reassembled length.
This change does "regress" on bug 10289 though when the preference is
enabled as retransmitted single-segment PDUs are now passed to
subdissectors. I added a TODO comment for this unrelated cosmetic issue.
Bug: 3389 # capture 2907 (HTTP) ok
Bug: 4727 # capture 4590 (HTTP) ok
Bug: 9461 # capture 12130 (TLS/HTTP/RPC-over-HTTP +key 12131) ok
Bug: 12006 # capture 14236 (HTTP) ok2; capture 15261 (HTTP) ok
Bug: 13517 # capture 15370 (HTTP) ok; capture 16059 (MQ) ok
Bug: 13754 # capture 15593 (MySQL) ok2
Bug: 14649 # capture 16305 (WebSocket) ok
Change-Id: If3938c5c1c96db8f7f50e39ea779f623ce657d56
Reviewed-on: https://code.wireshark.org/review/27943
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Change-Id: Ic13f60a3e700f3d8325063079f032eda47eaf22f
Reviewed-on: https://code.wireshark.org/review/19848
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
Register all reassembly tables with a central unit, allowing the
central unit to have the callback that initializes and destroys
the reassembly tables, rather than have dissectors do it individually.
Change-Id: Ic92619c06fb5ba6f1c3012f613cae14982e101d4
Reviewed-on: https://code.wireshark.org/review/19834
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
Also, sort the initializations of structure members by the order in the
structure, to make it easier to check that we've initialized them all.
Bug: 13231
Change-Id: Id2819940d916a5fd5a3f1bf2fc20bd3ee34a75f4
Reviewed-on: https://code.wireshark.org/review/19195
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Fix a memory error found by the buildbot and valgrind in my recent patch
Bug: 13100
Change-Id: Ieb21aa25e048f90ce7192546b0ad3d4718ff07df
Reviewed-on: https://code.wireshark.org/review/18877
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
Add support for defragmentation of fragments that use the defragmentation
scheme of PPP MP (RFC 1990). Instead of getting "sequence_number,
fragment_number, last" as in other protocols, PPP MP provides a single
sequence number that is effectively "seqnum + fragnum", though it provides
flags for both the first and last fragment of a reassembly.
See Appendix A of RFC 4623 (PWE3 Fragmentation and Reassembly) for a list
of protocols that use this style, including PPP MP (RFC 1990), PWE3 MPLS
(RFC 4385), L2TPv2 (RFC 2661), L2TPv3 (RFC 3931), ATM, and Frame Relay.
Also add support for the Multi-class Extension to Multilink PPP (RFC 2686),
which uses some of the previously reserved bits as classes that distinguish
otherwise identical sequence numbers.
Bug: 12548
Change-Id: Ic2ce3c50e61ab2eb50e4d92fd353ca4d2a48fe18
Reviewed-on: https://code.wireshark.org/review/16327
Reviewed-by: Michael Mann <mmann78@netscape.net>
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Do not try to recover from truncated tvbs for fragment_add_seq-like
functions:
- If it is the first block and the dissector requested frag_data_len
number of bytes, we should not lie and pretend that we are fully
reassembled.
- For other blocks, returning NULL as no reassembly was possible makes
sense. But other fragments in the list should not be cleared as there
may be partial fragments which were returned before.
It seems that this special behavior was introduced in
b2c11b5e13 (freeing fragments and
returning NULL as an optimization when fragments are deemed not needed
anymore) and faeb2c2ee1 (for returning
fd_head for the first fragment, "so the first fragment gets dissected as
fragmented packet").
Now in theory unused fragments could stick around, but that also
possible with the normal fragment_add functions.
Bug: 11799
Change-Id: I20829c54e1b2eee25a91fe4de51b19b1458c7789
Reviewed-on: https://code.wireshark.org/review/14082
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Michael Mann <mmann78@netscape.net>
Try to improve address API and also fix some constness warnings
by not overloading the 'data' pointer to store malloc'ed buffers
(use private pointer for that instead).
Second try, now passing test suite.
Change-Id: Idc101cd866b6d4f13500c9d59da5c7a38847fb7f
Reviewed-on: https://code.wireshark.org/review/13946
Petri-Dish: João Valverde <j@v6e.pt>
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: João Valverde <j@v6e.pt>
Documentation changes only (comments and docbook).
Update WSDG with the fragment_add_seq_check API that was introduced in
Wireshark 1.10.
Fix typos and clarify the many functions we have for adding reassembling
fragments.
Change-Id: I38715a8f58e9cf1fe3e34ee4b1a4ae339630282b
Reviewed-on: https://code.wireshark.org/review/14066
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
That leaves less room for getting it wrong.
Change-Id: Iea003fc102ccd14db2924b70fc685033ca34f291
Reviewed-on: https://code.wireshark.org/review/13863
Reviewed-by: Guy Harris <guy@alum.mit.edu>
This reverts commit 13ec77a9fc.
This commit introduces a segmentation fault for Lua code (uncovered by the test suite).
Change-Id: Ibc273d1915cda9632697b9f138f0ae104d3fb65e
Reviewed-on: https://code.wireshark.org/review/13813
Reviewed-by: João Valverde <j@v6e.pt>
Try to improve 'address' API (to be easier/safer) and also avoid
some constness warnings by not overloading the 'data' pointer to
store malloc'ed buffers (use private pointer for that instead).
Change-Id: I7456516b12c67620ceadac447907c12f5905bd49
Reviewed-on: https://code.wireshark.org/review/13463
Petri-Dish: João Valverde <j@v6e.pt>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: João Valverde <j@v6e.pt>
That removes most of the uses of the frame number field in the
frame_data structure.
Change-Id: Ie22e4533e87f8360d7c0a61ca6ffb796cc233f22
Reviewed-on: https://code.wireshark.org/review/13509
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Fixes memleak in reassemble.c
480 bytes in 60 blocks are definitely lost in loss record 3,010 of 3,059
at 0x4C28C10: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0xADA3328: g_malloc (in /usr/lib/libglib-2.0.so.0.4600.1)
by 0xADBA512: g_slice_alloc (in /usr/lib/libglib-2.0.so.0.4600.1)
by 0x6575C7D: fragment_reassembled (reassemble.c:804)
by 0x6577785: fragment_add_seq_check_work (reassemble.c:2027)
by 0x6577880: fragment_add_seq_next (reassemble.c:2068)
by 0x6E614E6: dissect_sccp_message (packet-sccp.c:2875)
by 0x6E63641: dissect_sccp (packet-sccp.c:3401)
by 0x6546CF7: call_dissector_through_handle (packet.c:620)
by 0x6546EA1: call_dissector_work (packet.c:706)
by 0x6547A04: dissector_try_uint_new (packet.c:1163)
by 0x6547A65: dissector_try_uint (packet.c:1189)
Change-Id: I0117b48e1e5d5688c49f264f24387dd6de1d6e08
Reviewed-on: https://code.wireshark.org/review/11541
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
It ends up dragging in libwireshark headers, which programs not linking
with libwireshark shouldn't do. In particular, including
<epan/address.h> causes some functions that refer to libwireshark
functions to be defined if the compiler doesn't handle "static inline"
the way GCC does, and you end up requiring libwireshark even though you
shouldn't require it.
Move plurality() to wsutil/str_util.h, so that non-libwireshark code can
get it without include epan/packet.h. Fix includes as necessary.
Change-Id: Ie4819719da4c2b349f61445112aa419e99b977d3
Reviewed-on: https://code.wireshark.org/review/11545
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Replace CMP_ADDRESS, COPY_ADDRESS, et al with their lower-case
equivalents in the asn1 and epan directories.
Change-Id: I4043b0931d4353d60cffbd829e30269eb8d08cf4
Reviewed-on: https://code.wireshark.org/review/11200
Petri-Dish: Michal Labedzki <michal.labedzki@tieto.com>
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
Trust that the files in epan/ immediately (not dissectors) know what they're
doing so just blindly convert them to captured length.
Change-Id: I872f7d58b2e15ae82c75fd56f4873996fbc97be7
Reviewed-on: https://code.wireshark.org/review/9083
Reviewed-by: Evan Huus <eapache@gmail.com>
Question: "what if we didn't capture the entire fragment due to a too-short
snapshot length?"
Answer: An assertion fails and we leak a bunch of memory.
Don't do that.
Bug: 11129
Change-Id: I0adfb217f0e66ae8f5f6255a4caf7ff940826b59
Reviewed-on: https://code.wireshark.org/review/8128
Petri-Dish: Evan Huus <eapache@gmail.com>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Evan Huus <eapache@gmail.com>
frame
but in different SCTP DATA chunks, whitout the patch the message is
reassembled in both chunks leading to duplicated upper layer PDU:s in the
frame.
Change-Id: Ie31142c38c728018178947544b571622447d8e8f
Reviewed-on: https://code.wireshark.org/review/7716
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Also add an assertion to tvb_generic_clone_offset_len so that it throws an
error *before* allocating memory, as otherwise that memory is leaked.
Bug: 10474
Change-Id: I5036cefac16841914a59670c64979cf599bf7969
Reviewed-on: https://code.wireshark.org/review/4234
Petri-Dish: Evan Huus <eapache@gmail.com>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Evan Huus <eapache@gmail.com>
(Using sed : sed -i '/^ \* \$Id\$/,+1 d')
Fix manually some typo (in export_object_dicom.c and crc16-plain.c)
Change-Id: I4c1ae68d1c4afeace8cb195b53c715cf9e1227a8
Reviewed-on: https://code.wireshark.org/review/497
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Instead of incrementing the offset for each new segment by one we add the length of the segment so that each segment is correctly shown in the segment list.
It proves to be very useful to find which packet (segment) is causing an application dissector to go wrong.
From Matthieu Patou
svn path=/trunk/; revision=53118
much, but I think this way's a little clearer, and it made it much easier to
figure out where the memory leak was.
Fixes the leaks from https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9243
svn path=/trunk/; revision=52448
- support merging chains in tvb_add_to_chain
- when we have an old reassembled TVB, just merge the chains rather than
freeing it (we may still need it as it may already be a data source)
- modelines
Fixes https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9027
#BACKPORT, but it's gonna be messy...
svn path=/trunk/; revision=51825
this code as C++.
Make pointers to raw packet data pointers to guint8, not pointers to
char, as they're octets, not characters.
svn path=/trunk/; revision=50583
João Valverde