It is to tvb_reported_length_remaining() as
tvb_ensure_captured_length_remaining() is to
tvb_captured_length_remaining() - it throws an exception if the offset
is out of range.
(Note that an offset that's just past the end of the {reported,
captured} data is *not* out of range, it just means that there is no
data remaining. Anything *past* that is out of range and thus invalid.)
Process the characters entirely ourselves; that way, we don't have to
worry about tvb_get_string_enc(..., ENC_ASCII) mangling label length
values, can convert non-ASCII characters in labels to the Unicode
REPLACEMENT CHARACTER, and can do bounds checks.
Add support internally to using iconv (always present with glib) to convert
strings from various encodings to UTF-8 (using REPLACEMENT CHARACTER as
recommended), and use that to support GB 18030 and EUC-KR. Replace call
directly to iconv in ANSI 637 for EUC-KR to new API. Update comments
and documentation around character encodings. It is possible to replace
the calls to iconv with an internal decoder later. Tested on Linux and
on Windows (including with illegal characters). Closes#16630.
Implement the Unicode Standard "best practices" for replacing ill-formed
sequences with the Unicode REPLACEMENT CHARACTER. Add wmem_strbuf_append_len
for appending strings with embedded null characters. Clarify why
wmem_strbuf_grow() doesn't always ensure that there's enough room for
a new string, and short-circuit some tests there. Related to #14948
Add an encoding for "unpacked" 3GPP TS 23.038 7-bit strings, in which
each code position is in a byte of its own, rather than with the code
positions packed into 7 bits. Rename the packed encoding to explicitly
indicate that it's packed.
Add an encoding for ETSI TS 102 221 Annex A strings.
Use the new encodings.
Change-Id: I2fad824ca417dcd089fabfdf06f28529c7ee9e87
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Reviewed-on: https://code.wireshark.org/review/37949
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Group them by the data types for which they're used, starting with the
byte-order definitions which (with the inclusion of ENC_NA) are used
with all types.
Put all the ones used for strings together, starting with the character
encodings, with the Zigbee flag and the flags for "this is a string but
we're going to interpret it as a byte array or time stamp".
Make ENC_CHARENCODING_MASK equal to ENC_STR_MASK; no, there's no reason
for ENC_STR_MASK to replace ENC_CHARENCODING_MASK - the opposite should
happen, as ENC_CHARENCODING_MASK at least specifies what the bits set in
it are used for, namely character encodings. If all #defines for
strings should have _STR_ in them, start with the character encoings.
Change-Id: I072420f313086153b4ea4034911fc293453dea00
Reviewed-on: https://code.wireshark.org/review/36962
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Add some ENC_ values for various flavors of packed BCD, and use that
instead of explicitly calling tvb_bcd_dig_to_wmem_packet_str() and
adding the result.
Change-Id: I07511d9d09c9231b610c121cd6ffb3b16fb017a9
Reviewed-on: https://code.wireshark.org/review/36952
Reviewed-by: Guy Harris <gharris@sonic.net>
tvb_find_line_end(), unlike a tvb_find_guint8() looking for an LF,
returns a length that *doesn't* include the line ending, *regardless* of
whether the line ends with CR-LF or just LF, so the query string we
extract is just the query, without any of the line ending.
Update some comments while we're at it to note that the "next_offset"
pointer argument to tvb_find_line_end() and tvb_find_line_end_unquoted()
can be NULL, in which case the offset *past* the line ending isn't
returned. (We pass tvb_find_line_end() NULL in the aforementioned call,
because, in that particular case, we don't care about the next line.)
Change-Id: I1c9746e32c61a79f8cb636d577a2e14a07ecab17
Reviewed-on: https://code.wireshark.org/review/35566
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Add "native" support for the "zig-zag" version of a varint in proto.[ch] and
tvbuff.[ch]. Convert the use of varint in the KAFKA dissector to use the (new)
"native" API.
Ping-Bug: 15988
Change-Id: Ia83569203877df8c780f4f182916ed6327d0ec6c
Reviewed-on: https://code.wireshark.org/review/34386
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Change all wireshark.org URLs to use https.
Fix some broken links while we're at it.
Change-Id: I161bf8eeca43b8027605acea666032da86f5ea1c
Reviewed-on: https://code.wireshark.org/review/34089
Reviewed-by: Guy Harris <guy@alum.mit.edu>
That's what the remaining calls to tvb_get_nstringz() and
tvb_get_nstringz0() are being used to do, even though those routines
were not intended for that purpose - the calls are extracting from a
text protcool, meaning that the strings are *not* null-terminate in the
packet.
Strings - even null-terminated ones - should, in almost all cases, be
extracted by tvb_get_string_enc() or routines that call it, so that an
encoding is specified. In the few cases where we're fetching strings
only to be compared to ASCII constants, or to parse as numbers, we can
get away with this.
Change-Id: I29f0532902c4ade2207de7f06db69c32eafd4132
Reviewed-on: https://code.wireshark.org/review/34072
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
The "Basic code table" in ISO 646 is mostly ASCII, but some code points
either 1) have more than one glyph that can be assigned to them or 2)
have no glyph assigned to them. National versions choose one of the two
glyphs for the code points in group 1) and assign specific glyphs to the
code points in group 2); the International Reference Version assigns the
same glyphs to those code points as does ASCII.
For the "Basic code table" encoding, we map the code points in groups 1)
and 2) to a REPLACEMENT CHARACTER; additional encodings can be added for
the national versions.
Add ENC_ISO_646_IRV (International Reference Version) as an alias for
ENC_ASCII.
Expand some comments, and add some comments, while we're at it.
Change-Id: I4f1b5e426ec193775e919731c5cae1224dc65115
Reviewed-on: https://code.wireshark.org/review/33941
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Clean up some comments while we're at it.
Change-Id: I0cd014bf1d1e7dc740eac1721d5466377938655f
Reviewed-on: https://code.wireshark.org/review/33939
Reviewed-by: Guy Harris <guy@alum.mit.edu>
While we're at it, add the Euro to code page 1251, expand the comments
for 1250 and 1251 and some DOS code pages, and add support for code page
1251 to tvb_get_stringz_enc().
Change-Id: I053d58f87cac26ad7c109e2f1cd8807ffec0622d
Reviewed-on: https://code.wireshark.org/review/33342
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Avoid relying on strptime to parse the day of week (%a) and month name
(%b) since these are locale-dependent. Fixes test suite failures with
tvb.lua and LC_ALL=nl_NL.UTF-8.
Additionally it will now reject four-digit years when using ENC_RFC_822
as that requires two digit years. The only user of this API seems to be
the Lua tests though, so this should not make much of a difference.
Bug: 15437
Change-Id: I75436b93faab23869794d9756b9c3ce6128dd1f4
Reviewed-on: https://code.wireshark.org/review/31698
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
This is intended to be a replacement for get_token_len (from strutil.h) when its used on a tvb. It should be a little safer and remove the need for a dissector to use tvb_get_ptr.
Change-Id: Ib2d4a79718b6fba4eb9acc0129b13be6c8199a43
Reviewed-on: https://code.wireshark.org/review/30892
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
This allows dissectors to check if a portion of the tvb is an ascii string while hiding the use of tvb_get_ptr.
Change-Id: Iaec7559dcfdefb8a5ae23e099ced45e90e611f8f
Reviewed-on: https://code.wireshark.org/review/30291
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
proto_tree_add_item with a zero length argument could end up calling
tvb_get_ptr to retrieve the (empty) backing buffer. This empty tvb was
possibly the result of bad reassembly, but let's gracefully handle it to
avoid a dissector exception.
Call trace for the original exception (only present on the first pass):
proto_report_dissector_bug (format=0x7ffffffecea0 "") at epan/proto.c:1368
ensure_contiguous_no_exception (tvb=0x6060001a5460, offset=0, length=0, pexception=0x7ffffffed060) at epan/tvbuff.c:775
ensure_contiguous (tvb=0x6060001a5460, offset=0, length=0) at epan/tvbuff.c:785
tvb_get_ptr (tvb=0x6060001a5460, offset=0, length=0) at epan/tvbuff.c:906
subset_get_ptr (tvb=0x607000194b90, abs_offset=0, abs_length=0) at epan/tvbuff_subset.c:58
ensure_contiguous_no_exception (tvb=0x607000194b90, offset=0, length=0, pexception=0x7ffffffed3c0) at epan/tvbuff.c:773
ensure_contiguous (tvb=0x607000194b90, offset=0, length=0) at epan/tvbuff.c:785
tvb_get_ptr (tvb=0x607000194b90, offset=0, length=0) at epan/tvbuff.c:906
proto_tree_set_bytes_tvb (fi=0x608000535ca0, tvb=0x607000194b90, offset=0, length=0) at epan/proto.c:3862
proto_tree_new_item (new_fi=0x608000535ca0, tree=0x604000543150, tvb=0x607000194b90, start=0, length=0, encoding=0) at epan/proto.c:2318
proto_tree_add_item_new (tree=0x604000543150, hfinfo=0x7ffff30e91f8, tvb=0x607000194b90, start=0, length=0, encoding=0) at epan/proto.c:3381
proto_tree_add_item (tree=0x604000543150, hfindex=65120, tvb=0x607000194b90, start=0, length=0, encoding=0) at epan/proto.c:3391
dissect_body_data (tree=0x604000543150, pinfo=0x614000000a58, tvb=0x607000194b90, start=0, length=0, encoding=0) at epan/dissectors/packet-http2.c:1974
Change-Id: Icfae83d61ddcc9e26f16eab7f6e0e84e2f0d73ac
Reviewed-on: https://code.wireshark.org/review/29851
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
tvb_generic_clone_offset_len uses tvb_bytes_exist to check that the
requested tvb data is actually available. It did not expect negative
values, that would result in an overly large memory allocation.
Bug: 14678
Change-Id: Ie80095a381e55ca5dbbd5c9d835243549d0b212e
Link: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7179
Reviewed-on: https://code.wireshark.org/review/27526
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
../epan/tvbuff.c: In function 'tvb_new_octet_aligned':
../epan/tvbuff.c:274:26: error: 'abs_offset' may be used uninitialized in this function [-Werror=maybe-uninitialized]
*rem_len = tvb->length - *offset_ptr;
^
../epan/tvbuff.c:486:8: note: 'abs_offset' was declared here
guint abs_offset, rem_length;
^
../epan/tvbuff.c: In function 'tvb_find_line_end':
../epan/tvbuff.c:274:26: error: 'abs_offset' may be used uninitialized in this function [-Werror=maybe-uninitialized]
*rem_len = tvb->length - *offset_ptr;
^
../epan/tvbuff.c:486:8: note: 'abs_offset' was declared here
guint abs_offset, rem_length;
^
../epan/tvbuff.c: In function 'tvb_find_line_end_unquoted':
../epan/tvbuff.c:274:26: error: 'abs_offset' may be used uninitialized in this function [-Werror=maybe-uninitialized]
*rem_len = tvb->length - *offset_ptr;
^
../epan/tvbuff.c:486:8: note: 'abs_offset' was declared here
guint abs_offset, rem_length;
Change-Id: Iba9fe31ac5fcf604d65bbf3bceef0c09004c1b6c
Reviewed-on: https://code.wireshark.org/review/27050
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Add a "contained length" to tvbuffs. For non-subset tvbuffs, that's the
same as the reported length. For a subset tvbuff, that's the amount of
the reported data that was actually present in the "contained data" of
the parent tvbuff.
This is unaffected by the *captured* length of any tvbuff; that differs
from the contained length only if the capture was cut short by a
snapshot length.
If a reference is within the reported data, but not within the contained
data, a ContainedBoundsError exception is thrown. This exception
represents a protocol error, rather than a reference past the captured
data in the packet; we treat it as such.
Change-Id: Ide87f81238eaeb89b3093f54a87bf7f715485af5
Reviewed-on: https://code.wireshark.org/review/27039
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Add 8-bit, 16-bit, 24-bit, and 32-bit "fetch signed value" routines, and
use them rather than casting the result of the 8/16/24/32-bit "fetch
unsigned value" routines to a signed type (which, BTW, isn't sufficient
for 24-bit values, so this appears to fix a bug
in epan/dissectors/packet-zbee-zcl.c).
Use numbers rather than sizeof()s in various tvb_get_ routines.
Change-Id: I0e48a57fac9f70fe42de815c3fa915f1592548bd
Reviewed-on: https://code.wireshark.org/review/26844
Petri-Dish: Guy Harris <guy@alum.mit.edu>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Get rid of CMake's TestBigEndian and Autotools' AC_C_BIGENDIAN checks in
favor of G_BYTE_ORDER. We use G_BYTE_ORDER elsewhere and TestBigEndian
is noticeably slow on Windows.
Change-Id: Idc1326294db9cbee8f6b6b11c2028fc4d19acbf0
Reviewed-on: https://code.wireshark.org/review/26462
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Used to support variable length in QUIC protocol
Bug: 13881
Change-Id: Ia274b1530152376c5fb4e364fc4cf5ab246be1b3
Reviewed-on: https://code.wireshark.org/review/24990
Petri-Dish: Alexis La Goutte <alexis.lagoutte@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Craig Jackson <cejackson51@gmail.com>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
That allows a parallel typedef of ws_in4_addr for guint32.
Change-Id: I03b230247065e0e3840eb87635315a8e523ef562
Reviewed-on: https://code.wireshark.org/review/24073
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Make the description simpler than the code (hopefully).
Change-Id: Iaede1a6eb18e25e379b719f9b66ecbef0c00ea63
Reviewed-on: https://code.wireshark.org/review/20956
Reviewed-by: Martin Kaiser <wireshark@kaiser.cx>
Petri-Dish: Martin Kaiser <wireshark@kaiser.cx>
Reviewed-by: Michael Mann <mmann78@netscape.net>
All cases of the "original" format_text have been handled to add the
proper wmem allocator scope. Remove the "original" format_text
and replace it with one that has a wmem allocator as a parameter.
Change-Id: I278b93bcb4a17ff396413b75cd332f5fc2666719
Reviewed-on: https://code.wireshark.org/review/19884
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
It's only use is in dissectors or other tree creation APIs (where
packet scope is valid), so have it use format_text_wmem with
wmem_packet_scope().
Change-Id: I1f34e284a870c9844c6b27f4ae08a1e7efe54098
Reviewed-on: https://code.wireshark.org/review/19883
Reviewed-by: Michael Mann <mmann78@netscape.net>
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
It's only use is in dissectors, so have it use
format_text_wmem with wmem_packet_scope().
Change-Id: I22121324fd47aee32174b65104458ad2ef329bd7
Reviewed-on: https://code.wireshark.org/review/19856
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
format_text_wsp is fed into by tvb_format_text_wsp and tvb_format_stringzpad_wsp
so those functions need to add a wmem allocated parameter as well.
Most of the changes came from tvb_format_text_wsp and tvb_format_stringzpad_wsp
being changed more so than format_text_wsp.
Change-Id: I52214ca107016f0e96371a9a8430aa89336f91d7
Reviewed-on: https://code.wireshark.org/review/19851
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
Hex digits 0xa through 0xe are '*', '#', 'a', 'b', and 'c',
respectively.
Constify the dgt_set_t argument to tvb_bcd_dig_to_wmem_packet_str(),
while we're at it.
Bug: 13316
Change-Id: I7586f35d23fd262453779d99946e7ccad4b6ffab
Reviewed-on: https://code.wireshark.org/review/19620
Reviewed-by: Guy Harris <guy@alum.mit.edu>
This emphasizes that there is no such thing as *the* routine to
construct a subset tvbuff; you need to choose one of
tvb_new_subset_remaining() (if you want a new tvbuff that contains
everything past a certain point in an existing tvbuff),
tvb_new_subset_length() (if you want a subset that contains everything
past a certain point, for some number of bytes, in an existing tvbuff),
and tvb_new_subset_length_caplen() (for all other cases).
Many of the calls to tvb_new_subset_length_caplen() should really be
calling one of the other routines; that's the next step. (This also
makes it easier to find the calls that need fixing.)
Change-Id: Ieb3d676d8cda535451c119487d7cd3b559221f2b
Reviewed-on: https://code.wireshark.org/review/19597
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Those routines can handle any single-byte character set whose characters
map to characters in the Basic Multilingual Plane; it could be used for
extended ASCII, but we have another routine for that, mapping only
characters with code points > 0x7f, so we just say "nonascii" rather
than "ebcdic".
Change-Id: I3d55b5d58e3e7ab08f3dfbfdb57a0301a30e71d4
Reviewed-on: https://code.wireshark.org/review/19214
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Have a routine that takes a 256-element translation table and uses it to
map various flavors of EBCDIC to Unicode. Have separate translation
tables for "common" EBCDIC (everything that's the same in all EBCDIC
code pages that include the original EBCDIC characters) and EBCDIC code
page 037. Add ENC_EBCDIC_CP037 for code page 037.
Change-Id: Ia882b3c0abef9e30eb54cd47396e6fa0d6342044
Reviewed-on: https://code.wireshark.org/review/19212
Reviewed-by: Guy Harris <guy@alum.mit.edu>
start_offset(offset) and the while loop should be on searched_bytes.
Change-Id: I62343f7cb04f9110f773ba1ada8dbcbfbd262861
Reviewed-on: https://code.wireshark.org/review/18741
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Bug: 13032
Change-Id: I6bf2cc2c43a6262d899a304df6576d9831115966
Reviewed-on: https://code.wireshark.org/review/18350
Petri-Dish: Michael Mann <mmann78@netscape.net>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Michael Mann <mmann78@netscape.net>
João Valverde