Commit Graph

495 Commits

Author SHA1 Message Date
João Valverde 92c1519dfe dfilter: Add float multiplication/division 2022-04-18 17:09:05 +01:00
João Valverde eb2a9889c3 dfilter: Add abs() function
Add an absolute value function for ftypes.
2022-04-18 17:09:00 +01:00
João Valverde 03e41d4950 ftypes: Fix a macro argument 2022-04-12 09:41:58 +01:00
João Valverde cc5726b63f dfilter: Remove leading colon special meaning
Instead of saying a leading colon will make any token a literal
value, say it is part of the syntax of bytes arrays. This is
useful to write bytes without a separator, and other potentially
ambiguous formats.

The restriction in meaning to bytes and simple numeric values
should make the rules for handling a leading colon (specifically
ommiting it or not) saner without much loss of functionality.
2022-04-07 00:16:07 +01:00
João Valverde 8108e67de7 dfilter: Fix memory leak with leading colon
When retrying fvalue_from_literal() we were leaking the error
message string.

Refactor the code to avoid the retry. This assumes the only
valid use of a leading ':' with a literal is for an IPv6 address.

Bytes with leading ':' are supported but the colon is skipped,
so the parser doesn't see it.

Fixes df0fc8b517.
2022-04-06 18:09:12 +01:00
João Valverde 70582b84f6 dfilter: Allow parsing binary number as a byte array of len 1
Before:

    Filter: eth[0] == 0b00110011

    Instructions:
    00000 READ_TREE		eth -> reg#0
    00001 IF_FALSE_GOTO	4
    00002 MK_RANGE		reg#0[reg#1] -> 0:1
    00003 ANY_EQ		reg#1 == 0b:00:11:00:11 <FT_BYTES>
    00004 RETURN

After:

    Filter: eth[0] == 0b00110011

    Instructions:
    00000 READ_TREE		eth -> reg#0
    00001 IF_FALSE_GOTO	4
    00002 MK_RANGE		reg#0[reg#1] -> 0:1
    00003 ANY_EQ		reg#1 == 33 <FT_BYTES>
    00004 RETURN
2022-04-04 02:02:58 +00:00
João Valverde a81617fda5 Fix build on CentOS 7
CentOS 7 has an old version of GCC that lacks support for C11
macro generics.
2022-03-31 19:48:48 +00:00
João Valverde 8bc214b5bb dfilter: Add remaining arithmetic integer ops 2022-03-31 16:49:42 +01:00
João Valverde 2a9cb588aa dfilter: Add binary arithmetic (add/subtract)
Add support for display filter binary addition and subtraction.

The grammar is intentionally kept simple for now. The use case
is to add a constant to a protocol field, or (maybe) add two
fields in an expression.

We use signed arithmetic with unsigned numbers, checking for
overflow and casting where necessary to do the conversion.
We could legitimately opt to use traditional modular arithmetic
instead (like C) and if it turns out that that is more useful for
some reason we may want to in the future.

Fixes #15504.
2022-03-31 11:27:34 +01:00
João Valverde 9ab2837637 dfilter: Add fvalue duplicate method 2022-03-30 14:05:22 +01:00
João Valverde b9b45a4a8f dfilter: Add ftypes pseudofields
This adds a _ws.ftypes namespace with protocol fields with all
the existing field types.

Currently this is only useful to debug the display filter compiler,
without having to find a real protocol field with the desired type.

Later it may find other uses.
2022-03-28 15:42:32 +01:00
João Valverde ac0a69636b dfilter: Add support for unary arithmetic
This change implements a unary minus operator.

Filter: tcp.window_size_scalefactor == -tcp.dstport

Instructions:
00000 READ_TREE		tcp.window_size_scalefactor -> reg#0
00001 IF_FALSE_GOTO	6
00002 READ_TREE		tcp.dstport -> reg#1
00003 IF_FALSE_GOTO	6
00004 MK_MINUS		-reg#1 -> reg#2
00005 ANY_EQ		reg#0 == reg#2
00006 RETURN

It is supported for integer types, floats and relative time values.
The unsigned integer types are promoted to a 32 bit signed integer.

Unary plus is implemented as a no-op. The plus sign is simply ignored.

Constant arithmetic expressions are computed during compilation.

Overflow with constants is a compile time error. Overflow with
variables is a run time error and silently ignored. Only a debug
message will be printed to the console.

Related to #15504.
2022-03-28 11:20:41 +00:00
João Valverde 3ac17381a6 ftypes: Fix IPv6 bitwise_and buffer overrun
*** CID 1503219:  Memory - illegal accesses  (OVERRUN)
/builds/wireshark/wireshark/epan/ftypes/ftype-ipv6.c: 142 in bitwise_and()
136
137     	prefix = MIN(a->prefix, b->prefix);	/* MIN() like in IPv4 */
138     	prefix = MIN(prefix, 128);		/* sanitize, max prefix is 128 */
139
140     	dst->value.ipv6 = fv_a->value.ipv6;
141     	while (prefix >= 8) {
>>>     CID 1503219:  Memory - illegal accesses  (OVERRUN)
>>>     Overrunning array "bitmasks" of 9 bytes at byte offset 127 using index "prefix" (which evaluates to 127).
142     		dst->value.ipv6.addr.bytes[pos] &= b->addr.bytes[pos] & bitmasks[prefix];
143
144     		prefix -= 8;
145     		pos++;
146     	}
147
2022-03-25 09:13:47 +00:00
João Valverde 0335ebdc3a dfilter: ftype_is_true -> ftype_is_zero 2022-03-23 11:04:41 +00:00
João Valverde 16729be2c1 dfilter: Add bitwise masking of bits
Add support for masking of bits. Before the bitwise operator
could only test bits, it did not support clearing bits.

This allows testing if any combination of bits are set/unset
more naturally with a single test. Previously this was only
possible by combining several bitwise predicates.

Bitwise is implemented as a test node, even though it is not.
Maybe the test node should be renamed to something else.

Fixes #17246.
2022-03-22 12:58:04 +00:00
João Valverde 5f13127a94 ftypes: Remove unnecessary macro 2022-03-16 19:28:41 +00:00
João Valverde 72751919b2 ftypes: Remove shared boolean from fvalue struct
This shared variable hidden behind a macro does not provide any
efficiency gains and just obscures the code. Move the boolean to
the fvalue protocol struct, where it belongs.
2022-03-16 19:25:45 +00:00
João Valverde df0fc8b517 dfilter: Try to be more flexible with leading colons
For an expression starting with a colon (a literal) try to parse
the value with and without colon. This avoids excluding some
valid representations like the IPv6 address "::1".
2022-03-05 11:10:54 +00:00
João Valverde c4f9d8abda dfilter: Rename "unparsed" to "literal"
A literal value is a value that cannot be interpreted as a
registered protocol. An unparsed value can be a literal or
an identifier (protocol/field) according to context and the
current disambiguation rules.

Strictly literal here is to be understood to  mean "numeric
literal, including numeric arrays, but not strings or character
constants".
2022-03-05 11:10:54 +00:00
João Valverde 1278e36152 dfilter: Add more debug code 2022-02-27 23:35:57 +00:00
João Valverde ef31431aeb dfilter: Add a true/false boolean representation
Minor code cleanup.
2022-02-23 23:37:47 +00:00
João Valverde 70d516368b Fix EditorConfig settings 2022-02-23 23:37:47 +00:00
João Valverde 9cc3e7e1bb dfilter: Add support for binary literal constants
Example: 0b1001, 0B111000, etc.
2022-02-23 22:27:59 +00:00
Guy Harris ec0aaf1811 ftype-time: check for NULL from gmtime() and localtime().
On Windows, they return NULL for times prior to the Epoch.
2022-01-04 15:35:18 -08:00
João Valverde 8501dc48dd dfilter: Accept byte arrays without separators
This relaxes the display filter syntax to accept byte arrays without
separators. An expression such as the following becomes valid:

    quic.dcid == b1f0b7cbe0897974

Previously it had to be written as:

    quic.dcid == b1:f0:b7:cb:e0:89:79:74

Partially fixes #17818.
2022-01-03 16:27:16 +00:00
João Valverde dd9ac15ff2 dfilter: Require separators with ISO 8601 time
Require date/time separators when entering a time value, e,g:
    2014-07-04 12:34:56.789+00:00

Separators in the timezone offset are an exception, they are
never mandatory.

This excludes ISO basic format to avoid inputs that could
be entirely numbers indistinguishable from Epoch time, in case
we want to support that in the future.
2022-01-02 10:44:01 +00:00
João Valverde e724a4baf6 dfilter: Use ISO8601 as the default time format
Change from a default custom time representation to ISO8601.
All the existing formats are still supported for backward-
compatibility.

Before:

  Filter: frame.time == "2011-07-04 12:34:56"

  Constants:
  00000 PUT_FVALUE	"Jul  4, 2011 12:34:56.000000000" <FT_ABSOLUTE_TIME> -> reg#1
  (...)

After:

  Filter: frame.time == "2011-07-04 12:34:56"

  Constants:
  00000 PUT_FVALUE	"2011-07-04 12:34:56+0100" <FT_ABSOLUTE_TIME> -> reg#1
  (...)
2021-12-31 15:01:41 +00:00
João Valverde 0047ca961f dfilter: Add support for entering time in UTC
Add the option to enter a filter with an absolute time
value in UTC. Otherwise the value is interpreted in
local time.

The syntax used is an "UTC" suffix, for example:

    frame.time == "Dec 31, 2002 13:55:31.3 UTC"

This also changes the behavior of "Apply Selected as filter".
Fields using a local time display type will use local time
and fields using UTC display type will be applied using UTC.

Fixes #13268.
2021-12-30 17:53:09 +00:00
João Valverde 64572a11f9 dfilter: Use better error messages for absolute times 2021-12-29 02:25:38 +00:00
João Valverde 445dcd3117 epan: Extend abs_time_to_str() with a flags argument 2021-12-28 04:05:20 +00:00
João Valverde a566076839 epan: Move time display types to field_display_e
This makes it easier to understand the code, avoids conflicts
and ugly and unnecessary casts.

The field display enum has evolved over time from integer types
to a type generic parameter.
2021-12-27 22:31:31 +00:00
João Valverde 0d5bfd44a8 Use a wrapper function to call strptime()
Encapsulate the feature requirements for strptime() in a
portability wrapper.

Use _GNU_SOURCE to expose strptime. It should be enough on glibc
without the side-effect of selecting a particular SUS version,
which we don't need and might hide other definitions.
2021-12-27 14:07:32 +00:00
João Valverde 36d5aad962 wsutil: Split ws_regex_matches() into two functions
Split ws_regex_matches() into two functions with better semantics
and remove the WS_REGEX_ZERO_TERMINATED symbol.

ws_regex_matches() matches zero terminated strings.

ws_regex_matches_length() matches a string length in code units.
2021-12-21 00:40:02 +00:00
João Valverde c5a19582e4 epan: Convert to use stdio.h from GLib
Replace:
    g_snprintf() -> snprintf()
    g_vsnprintf() -> vsnprintf()
    g_strdup_printf() -> ws_strdup_printf()
    g_strdup_vprintf() -> ws_strdup_vprintf()

This is more portable, user-friendly and faster on platforms
where GLib does not like the native I/O.

Adjust the format string to use macros from intypes.h.
2021-12-19 19:29:53 +00:00
João Valverde fb0e1a4907 regex: Remove requirement for ssize_t
The type ssize_t is not available on Windows. Because this is
used in the public API we must provide a definition for it.
To avoid having to add a header to fix this use a size_t in
the API instead, and assign SIZE_MAX to represent a null
terminated string.
2021-12-13 23:57:32 +00:00
Moshe Kaplan a523135202 epan: Add header files to Doxygen
Add @file markers for epan
headers so that Doxygen will
generate documentation for them.
2021-11-30 08:46:49 +00:00
Moshe Kaplan 1c3a9af869 Add files with WS_DLL_PUBLIC to Doxygen
Add @file markers for most files that
contain functions exported with
WS_DLL_PUBLIC so that Doxygen will
generate documentation for them.
2021-11-29 21:27:45 +00:00
João Valverde 54bdc20e45 epan: Rewrite ws_escape_string() to use wmem
Return a wmem-allocated string.

Add boolean argument to enable/disable adding double quotes.
2021-11-29 17:47:53 +00:00
João Valverde 44121e2c3b Move escape_string() to wsutil
Move this utility function to wsutil. Rename to
ws_escape_string().

Also add tests.
2021-11-29 17:47:53 +00:00
João Valverde ef8125e3ae Move two functions from epan to wsutil/str_util
Move epan_memmem() and epan_strcasestr() to wsutil/str_util.
Rename to ws_memmem() and ws_strcasestr(). Add compile time
check for a system implementation and use that if available.

We invoke those functions using a wrapper to avoid exposing
_GNU_SOURCE outside of the implementation.
2021-11-28 12:32:51 +00:00
João Valverde 943c282009 dfilter: Parse character constants in lexer
Invalid character constants should be handled in the lexical scanner.

Todo: See if some code could be shared to parse double quoted strings.

It also fixes some unintuitive type coercions to string. Character
constants should be treated as characters, or maybe integers, or
maybe even throw an invalid comparison error, but coverting to a
literal string or byte array is surprising and not particularly
useful:
  '\xFF' -> "'\xFF'" (equals)
  '\xFF' -> "FF"     (contains)

Before:

    Filter: http.request.method contains "\x63"

    Constants:
    00000 PUT_FVALUE	"c" <FT_STRING> -> reg#1
    (...)

    Filter: http.request.method contains '\x63'

    Constants:
    00000 PUT_FVALUE	"63" <FT_STRING> -> reg#1
    (...)

    Filter: http.request.method == "\x63"

    Constants:
    00000 PUT_FVALUE	"c" <FT_STRING> -> reg#1
    (...)

    Filter: http.request.method == '\x63'

    Constants:
    00000 PUT_FVALUE	"'\\x63'" <FT_STRING> -> reg#1
    (...)

After:

    Filter: http.request.method contains '\x63'

    Constants:
    00000 PUT_FVALUE	"c" <FT_STRING> -> reg#1
    (...)

    Filter: http.request.method == '\x63'

    Constants:
    00000 PUT_FVALUE	"c" <FT_STRING> -> reg#1
    (...)
2021-11-24 08:40:20 +00:00
João Valverde 7028646f9e dfilter: Fix invalid character constant error message
This reverts commit d635ff4933.

A charconst cannot be a value string, for that reason it is not
redundant with unparsed.

Maybe character constants should be parsed in the lexical scanner
instead.

Before:
  Filter: ip.proto == '\g'
  dftest: "'\g'" cannot be found among the possible values for ip.proto.

After:
  Filter: ip.proto == '\g'
  dftest: "'\g'" isn't a valid character constant.
2021-11-23 17:35:40 +00:00
João Valverde 72c5efea1b dfilter: Reject invalid character escape sequences
For double quoted strings. This is consistent with single quote
character constants and the C standard. It also avoids common
mistakes where the superfluous backslash is silently suppressed.
2021-11-23 16:48:02 +00:00
João Valverde 274531820a Move regex code to wsutil 2021-11-14 21:00:59 +00:00
João Valverde b9f2e4b7fa Make PCRE2 a required dependency 2021-11-14 21:00:59 +00:00
João Valverde 9df5279af7 dfilter: Remove support for GRegex
PCRE2 is mature, widely used and widely available. Supporting two
different RE implementations, one of which is unmaintained, is
unnecessary and counter-productive.
2021-11-14 21:00:59 +00:00
João Valverde ed8a02af17 dfilter: Add support for PCRE2
PCRE2 is the future of PCRE. The only advantage of GRegex is that
it comes bundled with GLib, which is not an advantage at all.
PCRE2 is widely available, the GRegex abstractions layer are not a
good fit and abstract things that don't need abstracting or that we
could handle better ourselves, there are open bugs (#12997) and
maintenance is spotty at best.

GRegex comes with many of the problems of bundled code, aggravated by
the fact that it completely falls outside of our control.
2021-11-14 21:00:59 +00:00
João Valverde 526ccff3d0 ftypes: Remove unused function declarations
Some registration function declarations were left behind,
remove them. While here move function declarations after
typedefs as is customary.
2021-11-11 09:50:29 +00:00
João Valverde fd78f1ed02 ftypes: Clean up duplicate struct field 2021-11-11 09:50:12 +00:00
João Valverde 5503d5e131 ftypes: Optimize a memory allocation
Avoid pre-allocating a large worst case size character buffer.
2021-11-11 03:56:04 +00:00