This parameter was introduced as a safeguard for bugs
that generate an unbounded string but its utility for
that purpose is doubtful and the way it is being used
creates problems with invalid truncation of UTF-8
strings.
Rename wmem_strbuf_sized_new() with a better name.
Only dissectors are using this function and there is no use case,
as far as I know, that requires its use. Any limitation of length
is imposed transparently by the UI backend.
This function is problematic because it is not Unicode aware and
will truncate a string on an arbitrary byte boundary for multibyte
strings.
Replace its use with a normal strbuf without a length limite and
remove the function because it is not useful and the ITEM_LABEL_LENGTH
parameter does not belong in wmem anyway.
This change adds a protocol dissector for the Management Component
Transport Protocol (MCTP). This is a failry simple datagram-based
protocol for messaging between components within a single platform,
typically over I2C, serial or PCIe.
This dissector just implements the header fields, and sequence-number
based message reassembly. Inner protocols will be added as follow-up
changes.
Linux has support for AF_MCTP data, so decode from the MCTP SLL ltype.
Signed-off-by: Jeremy Kerr <jk@codeconstruct.com.au>
If a character is not a valid Unicode codepoint, i.e. one of
the code points reserved for surrogate pairs or a code point
above 0x10FFFF, don't add it to a wmem_strbuf when converting
from other encodings but add a replacement character instead, by
using a new wmem_strbuf_append_unichar_validated() function.
Now we produce valid UTF-8 in various situations where UCS-2 or UTF-32
can encode unpaired surrogate codepoints. Consolidate some related
checks that are now redundant.
Also add a replacement character to the end of invalid UCS-2 strings
with an odd number of bytes, as done with UTF-16 and UTF-32.
Fix#18508
Add UNICODE_REPLACEMENT_CHARACTER as a #define for the Unicode
REPLACEMENT CHARACTER code point (0x00FFFD), and use that instead of
0xfffd/0xFFFD/0x00FFFD in cases where that value refers to REPLACEMENT
CHARACTER.
Rewrite for speed and correctness.
This implementation is more strict with invalid
first bytes (continuation bytes, invalid codepoints and
some overlong sequences).
Returns 0 instead of -1 for invalid bytes.
Add a convenience function to truncate a UTF-8 string to no more
than certain length, while ensuring that the string ends with
a complete character instead of a partial sequence (by truncating
up to 3 additional bytes as necessary.)
The common use case is when a valid UTF-8 string is copied into
a buffer via snprintf, strlcpy, or strlcat and truncated, to fix
up the end of the string and keep the string valid.
The buffer holding the string must be large enough, and the string
must be valid up to the point of truncation (aside from the possible
partial sequence at the end). For speed, the function does not check
those conditions.
Ping #18412.
This removes the last dependency of the logging subsystem on the
preferences module. The latter is started much later than the former
and this is an issue.
The Windows-only preference "gui.console_open" is stored in the
registry as HKEY_LOCAL_USER\Software\Wireshark\ConsoleOpen. The semantics
are exactly the same. The preference is read by the logging subsystem
for initialization and then again by the preferences (read/write) so
the user can configure it as before.
The code to store the preference also in the preferences file was
kept, for backward compatibility and because it is not incompatible
with using the Registry concurrently.
The elimination of the prefs dependency also allows moving the Windows
console logic to wsutil and add the functionality to wslog directly,
thereby eliminating the superfluous Wireshark/Logray custom log handler.
To be able to read the ws_log_console_open global variable from
libwireshark it becomes necessary to add a new export macro
symbol called WSUTIL_EXPORT.
Add a command line option --log-fatal-domains= and environment variable
WIRESHARK_LOG_FATAL_DOMAINS that aborts the programs if a domain in
the list is logged to.
Negative matches for fatal log domains not implemented for now,
pending a relevant use-case.
For some programs (e.g: unit tests) we want to init the logging
subsystem but having to pass an error output handler is overkill
and annoying.
In that case with a NULL handler printing to stderr by default is
almost certainly the best thing to do.
wmem_strbuf_grow should set the correct size with regard to max_size,
if set. In any case passing the actual free "raw" size to g_strlcpy is
always the correct thing to do.
Use length and size consistently. strbuf->len does not
include the terminating nul. strbuf->alloc_len includes
the terminating nul.
Use consistent language and use "length" to mean size without
nul byte and "size" to mean size with all bytes, including nul.
Some older dissectors that predate Unicode and parse text protocols
are prone to generate invalid UTF-8 strings. This is a bug and can have
safety implications.
For example passing invalid UTF-8 to proto_tree_add_string() is a
common bug. There are safeguards in format_text() but this should
not be relied on as a general solution to the problem.
For one, as the name implies, it is only used with representation of a
field value, which is not the same as the value itself of an FT_STRING field.
Issue #18317 shows another reason why.
For now this compile flag only enables extra checks for string ftypes,
which covers a subset of proto.h APIs including
proto_tree_append_string(). Later is should be extended to other
interfaces.
This is also not expected to be disabled for release builds because
there are still many dissectors that do not correctly handle strings.
More work is needed to 1) identify them and 2) fix them.
Ping #18317
Like wmem_map_remove(), this frees the key/value pair item
in the map but not the key or the value itself (which may
in fact be the same object.) Not generally a problem, as
they'll get freed by the pool. (If someone wants to manage
memory themselves, they should probably be using a GHashTable.)
A given protocol's packet format may depend, for example, on which
lower-level protocol is transporting the protocol in question. For
example, protocols that run atop both byte-stream protocols such as TCP
and TLS, and packet-oriented protocols such as UDP or DTLS, might begin
the packet with a length when running atop a byte-stream protocol, to
indicate where this packet ends and the next packet begins in the byte
stream, but not do so when running atop a packet-oriented protocol.
Dissectors can handle this in various ways:
For example, the dissector could attempt to determine the protocol over
which the packet was transported.
Unfortunately, many of those mechanisms do so by fetching data from the
packet_info structure, and many items in that structure act as global
variables, so that, for example, if there are two two PDUs for protocol
A inside a TCP segment, and the first protocol for PDU A contains a PDU
for protocol B, and protocol B's dissector, or a dissector it calls,
modifies the information in the packet_info structure so that it no
longer indicates that the parent protocol is TCP, the second PDU for
protocol A might not be correctly dissected.
Another such mechanism is to query the previous element in the layers
structure of the packet_info structure, which is a list of protocol IDs.
Unfortunately, that is not a list of earlier protocols in the protocol
stack, it's a list of earlier protocols in the dissection, which means
that, in the above example, when the second PDU for protocol A is
dissected, the list is {...,TCP,A,B,...,A}, which means that the
previous element in the list is not TCP, so, again, the second PDU for
protocol A will not be correctly dissected.
An alternative is to have multiple dissectors for the same protocol,
with the part of the protocol that's independent of the protocol
transporting the PDU being dissected by common code. Protocol B might
have an "over a byte-stream transport" dissector and an "over a packet
transport" dissector, with the first dissector being registered for use
over TCP and TLS and the other dissector being registered for use over
packet protocols. This mechanism, unlike the other mechanisms, is not
dependent on information in the packet_info structure that might be
affected by dissectors other than the one for the protocol that
transports protocol B.
Furthermore, in a LINKTYPE_WIRESHARK_UPPER_PDU pcap or pcapng packet for
protocol B, there might not be any information to indicate the protocol
that transports protocol B, so there would have to be separate
dissectors for protocol B, with separate names, so that a tag giving the
protocol name would differ for B-over-byte-stream and B-over-packets.
So:
We rename EXP_PDU_TAG_PROTO_NAME and EXP_PDU_TAG_HEUR_PROTO_NAME to
EXP_PDU_TAG_DISSECTOR_NAME and EXP_PDU_TAG_HEUR_DISSECTOR_NAME, to
emphasize that they are *not* protocol names, they are dissector names
(which has always been the case - if there's a protocol with that name,
but no dissector with that name, Wireshark will not be able to handle
the packet, as it will try to look up a dissector given that name and
fail).
We fix that exported PDU dissector to refer to those tags as dissector
names, not protocol names.
We update documentation to refer to them as DISSECTOR_NAME tags, not
PROTO_NAME tags. (If there is any documentation for this outside the
Wireshark source, it should be updated as well.)
We add comments for calls to dissector_handle_get_dissector_name() where
the dissector name is shown to the user, to indicate that it might be
that the protocol name should be used.
We update the TLS and DTLS dissectors to show the encapsulated protocol
as the string returned by dissector_handle_get_long_name(); as the
default is "Application Data", it appeaers that a descriptive name,
rather than a short API name, should be used. (We continue to use the
dissector name in debugging messages, to indicate which dissector was
called.)
Strip the timestamps and function names from extcap errors before
displaying them in a dialog. This keeps us from spewing
```
Error by extcap pipe: ** (falcodump:24913) 15:08:20.263535 [falcodump
WARNING] extcap/falcodump.cpp:593 -- main(): cloudtrail plugin error:
failed to list objects: ExpiredToken: The provided token has expired.
status code: 400, request id: M2PYJOT4JGVAA6B9, host id:
+4V8Q+DGV+80Jd3fdc7tODyVdTRvepNgCD9zuvXeL7kzvp2oikaoi9CLMW+UKt/
aR1G2UXIqyQ8=
```
at the user.
Check for warning messages and set our dialog type accordingly.
Windows processes inherit all inheritable handles when a new process is
created using CreateProcess() with bInheritHandles set to TRUE. This can
lead to undesired object lifetime extension. That is, the child process
will keep ineritable handles alive even if it does not use them. Up to
Windows Vista it was not possible explicitly list handles that should be
inherited. Wireshark no longer works on Windows releases earlier than
Vista, so use the new API without checking Windows version.
Require all callers to win32_create_process() to pass in the list of
handles to inherit. Set the listed handles as inheritable shortly before
calling CreateProcess() and set them as not inheritable shortly after
the process is created. This minimizes possibility for other callers
(especially in 3rd party libraries) to inherit handles by accident.
Do not terminate mmdbresolve process on exit. Instead rely on process
exit when EOF is received on standard input. Previously the EOF was
never received because mmdbresolve inherited both ends of standard input
pipe, i.e. the fact that Wireshark closed the write end was not observed
by mmdbresolve because mmdbresolve kept write handle the standard input
pipe open.
The semantics behind ws_pipe_close() were broken since its introduction.
Forcing process termination on Windows, while simply setting variable on
other systems results in more OS specific code sprinkled all over the
place instead of less. Moreover ws_pipe_close() never handled standard
file handles. It is really hard to come up with sensible ws_pipe_close()
replacement, as process exit is actually asynchronous action. It is
recommended to register child watch using g_child_watch_add() instead.
Do not call ws_pipe_close() when deleting capture interface. Things will
break if extcap is still running when interface opts are being freed and
terminating process won't help.
Rework maxmind shutdown to rely on GIOChannel state. For unknown reason
TerminateProcess() is still needed on Windows. The actual root cause
should be identified and fixed instead of giving up hope that it will
ever work correctly on Windows. In other words, TerminateProcess()
should not be used as a pattern, but rather as a last resort.
Remove ws_read_string_from_pipe() as this function encourages bad design
and is no longer necessary. Extcap stderr is read only after the child
process has finished and thus the read will never block.
Close process information thread handle right away as we don't use it.
Remove unused ws_pipe_t member variables.
Close pipe handles when spawning asynchronous processes, so only the
child process holds handle to the other end of the pipe. Closing the
handles makes it possible to use pipes the same way as on other OS, that
is to rely on blocking read() to end when child process finishes.
Do not call CloseHandle() on signal pipe in capture sync after the
handle ownership was transferred to file descriptor. Close the file
descriptor instead.
Instead of using an OS-independent and (somewhat) toolchain-independent,
but x86-only, mechanism to fetch a string that identifies the type(s) of
CPU on the machine, use OS-dependent but instruction-set-independent
mechanisms.
That way, we can get CPU type strings for non-x86 processors - ARM
processors, in particular, but others as well (yes, you can run
Wireshark on an IBM mainframe...).
Fixes#18187.
Add and install default coloring rules and filter buttons for Logray.
Add is_packet_configuration_namespace() and use it to set the default
timestamp type for Logray to "Absolute", which is more appropriate for
log entries.
Switch to the name "Logray" for the log analyzer. Rays are biological
cousins of sharks and more people like the name "Logray" in a completely
unscientific survey here. Apologies for any inconvenience this might
cause.
Windows implements so called CRT handlers, which will catch any
assertions happening inside so called crt routines and either
displays a debug dialog (Cancel, Retry, Ignore) or outright crashes
the application.
See
https://docs.microsoft.com/en-us/cpp/c-runtime-library/parameter-validation?view=msvc-170
for an explanation of the behaviour.
Now, in the current situation here, close will detect (correctly)
that the pipe it is supposed to be closing is already closed. This
happens (again correctly) because it had been closed by the extcap
application.
The change added, checks for a closed pipe first, and if so just
returns -1 (as it should) silently without calling the CRT routine,
therefore not crashing
Add get_configuration_namespace() and use it in code that writes
"generated by" comments at the top of various configuration files.
Update our Logwolf colorfilters.
Libgcrypt 1.8.x is required for a large amount of decryption
support and is the current LTS version of libgcrypt. The 1.6 and
1.7 series have been end-of-life since 2017-06-30 and 2019-06-30,
respectively.
The Linux distributions that have versions of libgcrypt before 1.8.0
are nearing or at end of support (RHEL7, SLES 12, Debian stretch,
Ubuntu 16.04LTS) and can be supported by the Wireshark 3.6 LTS release
series.
Remove an enormous amount of ifdefs based on libgcrypt versions
1.6.0, 1.7.0, and 1.8.0. There will be a second pass for the
commons defines HAVE_LIBGCRYPT_AEAD, HAVE_LIBGCRYPT_CHACHA20, and
HAVE_LIBGCRYPT_CHACHA20_POLY1305, which are now always defined.
The ISAKMP dissector has some comments noting that some workarounds
were used for libgcrypt 1.6 that aren't needed with 1.7; perhaps
that could be updated now.
If we're running in the Logwolf configuration namespace, look for
extcaps in a directory named "extlog". This paves the way for adding
log-specific capture utilities.
Rename init_progfile_dir to configuration_init. Add an argument which
specifies our configuration namespace, which can be "Wireshark"
(default) or "Logwolf".
Add support for display filter binary addition and subtraction.
The grammar is intentionally kept simple for now. The use case
is to add a constant to a protocol field, or (maybe) add two
fields in an expression.
We use signed arithmetic with unsigned numbers, checking for
overflow and casting where necessary to do the conversion.
We could legitimately opt to use traditional modular arithmetic
instead (like C) and if it turns out that that is more useful for
some reason we may want to in the future.
Fixes#15504.
g_utf8_validate_len doesn't exist until glib 2.60, so just
use g_utf8_validate. It does the same thing when the length parameter
is unsigned. Fixes CentOS 7.
Add BASE_SHOW_UTF_8_PRINTABLE and related function tvb_utf_8_isprint
for supporting fields of bytes that are "maybe UTF-8" (default or
SHOULD be UTF-8 but could be something else, with no encoding indicator),
such as SSID fields in IEEE 802.11 (See #16208), certain OctetString
fields in Diameter or PFCP, and other places where
BASE_SHOW_ASCII_PRINTABLE is currently used. Fix#5307
Add a LOG_LEVEL_ECHO that is always active and always non-fatal.
Use that to implement a WS_DEBUG_HERE() macro for quick print outs
during debugging sessions.
The format "23:59:59+2000" is valid but the code assumes that
if the date/time format uses separators, the timezone offset
must have them too. Fix that. Add test cases for timezone
offsets +HHMM and +HH.
Rewrite ws_inet_pton{4,6} and ws_inet_ntop{4,6} without
GLib types.
Check for strerrorname_np() and use that is available,
to simplify error handling.
Add some minimal tests.
Encapsulate the feature requirements for strptime() in a
portability wrapper.
Use _GNU_SOURCE to expose strptime. It should be enough on glibc
without the side-effect of selecting a particular SUS version,
which we don't need and might hide other definitions.
Repeated words were found with:
egrep "(\b[a-zA-Z]+) +\1\b" . -Ir
and then manually reviewed.
Non-displayed strings (e.g., in comments)
were also corrected, to ease future review.
On Windows the POSIX read() and write() don't use the C99/POSIX
types size_t and ssize_t so we must do the same to avoid
gymnastics to squelch narrowing warnings.
This adds two types for that purpose that have the correct
definition for both Windows and POSIX.
Currently used to define ssize_t on platforms that lack it.
Fix some Windows build errors caused by moving the definition into a
separate header.
Fix some narrowing warnings on Windows x64 from changing the definition
of ssize_t from long int to int64_t.
The casts in dumpcap are ugly but necessary. The whole code needs
to be rewritten for portability, or the warnings disabled.
Split ws_regex_matches() into two functions with better semantics
and remove the WS_REGEX_ZERO_TERMINATED symbol.
ws_regex_matches() matches zero terminated strings.
ws_regex_matches_length() matches a string length in code units.
João Valverde