This parameter was introduced as a safeguard for bugs
that generate an unbounded string but its utility for
that purpose is doubtful and the way it is being used
creates problems with invalid truncation of UTF-8
strings.
Rename wmem_strbuf_sized_new() with a better name.
Currently the autocompletion engine always suggests a protocol
field completion, even in places where it isn't syntactically
valid.
Fix that by compiling the preamble to the token under the cursor
and checking the returned error. If it is DF_ERROR_UNEXPECTED_END
that indicates a field or literal value was expected. Otherwise
a field replacement is not valid in this position.
Fixes#12811.
Return an struct containing error information. This simplifies
the interface to more easily provide richer diagnostics in the future.
Add an error code besides a human-readable error string to allow
checking programmatically for errors in a robust manner. Currently
there is only a generic error code, it is expected to increase
in the future.
Move error location information to the struct. Change callers and
implementation to use the new interface.
Rename flex macros using parenthesis (mostly a style issue):
DIAG_OFF_FLEX -> DIAG_OFF_FLEX()
DIAG_ON_FLEX -> DIAG_ON_FLEX()
Use the same kind of construct with lemon generated code using
DIAG_OFF_LEMON() and DIAG_ON_LEMON(). Use %include and %code
directives to enforce the desired order with generated code
in the middle in between pragmas.
Fix a clang-specific pragma to use DIAG_OFF_CLANG().
DIAG_OFF(unreachable-code) -> DIAG_OFF_CLANG(unreachable-code).
Apparently GCC is ignoring the -Wunreachable flag, that's why
it did not trigger an unknown pragma warning. From [1}:
The -Wunreachable-code has been removed, because it was unstable: it
relied on the optimizer, and so different versions of gcc would warn
about different code. The compiler still accepts and ignores the
command line option so that existing Makefiles are not broken. In some
future release the option will be removed entirely. - Ian
[1] https://gcc.gnu.org/legacy-ml/gcc-help/2011-05/msg00360.html
Instead of using the abstract type "<RAW>", which might be confusing,
show FT_BYTES, but display the representation with the "@" operator,
so it's not even more confusing in error messages why a field might
flip-flop types.
Refactor the field tostr() function and some other clean ups.
Before:
```
Filter: _ws.ftypes.string ==${@frame.len}
dftest: _ws.ftypes.string and frame.len <RAW> are not of compatible types.
_ws.ftypes.string ==${@frame.len}
^~~~~~~~~
```
After:
```
Filter: _ws.ftypes.string ==${@frame.len}
dftest: _ws.ftypes.string <FT_STRING> and @frame.len <FT_BYTES> are not of compatible types.
_ws.ftypes.string ==${@frame.len}
^~~~~~~~~
```
Extends raw adressing syntax to wok with references. The syntax
is
@field1 == ${@field2}
This requires replicating the logic to load field references, but
using raw values instead. We use separate hash tables for that,
namely "references" vs "raw_references".
This adds new syntax to read a field from the tree as bytes, instead
of the actual type. This is a useful extension for example to match
matformed strings that contain unicode replacement characters. In
this case it is not possible to match the raw value of the malformed
string field. This extension fills this need and is generic enough
that it should be useful in many other situations.
The syntax used is to prefix the field name with "@". The following
artificial example tests if the HTTP user agent contains a particular
invalid UTF-8 sequence:
@http.user_agent == "Mozill\xAA"
Where simply using "http.user_agent" won't work because the invalid byte
sequence will have been replaced with U+FFFD.
Considering the following programs:
$ dftest '_ws.ftypes.string == "ABC"'
Filter: _ws.ftypes.string == "ABC"
Syntax tree:
0 TEST_ANY_EQ:
1 FIELD(_ws.ftypes.string <FT_STRING>)
1 FVALUE("ABC" <FT_STRING>)
Instructions:
00000 READ_TREE _ws.ftypes.string <FT_STRING> -> reg#0
00001 IF_FALSE_GOTO 3
00002 ANY_EQ reg#0 == "ABC" <FT_STRING>
00003 RETURN
$ dftest '@_ws.ftypes.string == "ABC"'
Filter: @_ws.ftypes.string == "ABC"
Syntax tree:
0 TEST_ANY_EQ:
1 FIELD(_ws.ftypes.string <RAW>)
1 FVALUE(41:42:43 <FT_BYTES>)
Instructions:
00000 READ_TREE @_ws.ftypes.string <FT_BYTES> -> reg#0
00001 IF_FALSE_GOTO 3
00002 ANY_EQ reg#0 == 41:42:43 <FT_BYTES>
00003 RETURN
In the second case the field has a "raw" type, that equates directly to
FT_BYTES, and the field value is read from the protocol raw data.
We amend the :<numeric> pattern to not eat the leading
colon. Because the colon can be part of the value (with IPv6 addresses
for example) we want to avoid doing that.
IPv6 addresses are covered by their own rules but this removes the
requirement in the future to handle any special cases and avoids
surprises.
For this reason the colon-prefix syntax is already explicitly defined to
work only for byte arrays and there is currently no universal
syntax for all literal values or even all numbers.
Other numbers can keep using the lexical type "unparsed".
```
run/dftest "_ws.ftypes.uint8 == :fd"
Filter: _ws.ftypes.uint8 == :fd
dftest: ":fd" is not a valid number.
_ws.ftypes.uint8 == :fd
^~~
run/dftest "_ws.ftypes.uint8 == fd"
Filter: _ws.ftypes.uint8 == fd
dftest: "fd" is not a valid number.
_ws.ftypes.uint8 == fd
^~
run/dftest "_ws.ftypes.uint8 == 0xfd"
Filter: _ws.ftypes.uint8 == 0xfd
Syntax tree:
0 TEST_ANY_EQ:
1 FIELD(_ws.ftypes.uint8 <FT_UINT8>)
1 FVALUE(253 <FT_UINT8>)
Instructions:
00000 READ_TREE _ws.ftypes.uint8 <FT_UINT8> -> reg#0
00001 IF_FALSE_GOTO 3
00002 ANY_EQ reg#0 == 253 <FT_UINT8>
00003 RETURN
run/dftest "_ws.ftypes.bytes == fd"
Filter: _ws.ftypes.bytes == fd
Syntax tree:
0 TEST_ANY_EQ:
1 FIELD(_ws.ftypes.bytes <FT_BYTES>)
1 FVALUE(fd <FT_BYTES>)
Instructions:
00000 READ_TREE _ws.ftypes.bytes <FT_BYTES> -> reg#0
00001 IF_FALSE_GOTO 3
00002 ANY_EQ reg#0 == fd <FT_BYTES>
00003 RETURN
run/dftest "_ws.ftypes.bytes == :fd"
Filter: _ws.ftypes.bytes == :fd
Syntax tree:
0 TEST_ANY_EQ:
1 FIELD(_ws.ftypes.bytes <FT_BYTES>)
1 FVALUE(fd <FT_BYTES>)
Instructions:
00000 READ_TREE _ws.ftypes.bytes <FT_BYTES> -> reg#0
00001 IF_FALSE_GOTO 3
00002 ANY_EQ reg#0 == fd <FT_BYTES>
00003 RETURN
```
The <...> syntax for literals, intended to be as generic as
possible, unintentionally introduced an ambiguity with the
relational expression "a < b or a > c".
Literals are values like numbers, bytes, IPv6 addresses or, one
could imagine, UNC paths for example, if an FT_UNC type were to
be added in the future.
We could use a new unique symbol like @...@ but the <...>
syntax is very recent and may not be necessary with ":xxx" so
just remove it.
A byte array can be explicitly declared by prefixing with a colon. It
is not as generic but the main ambiguity that this new syntax attempted
to solve is bytes vs protocol names. We don't want to introduce a new
reserved symbol for now, until other requirements if any are more clear.
Fixes#18418.
Do the first ftype-can check in an arithmetic expressions before
evaluating the second term to be sure we do not allow FT_NONE as a
valid LHS ftype.
$ dftest '_ws.ftypes.none + 1 == 2'
Filter: _ws.ftypes.none + 1 == 2
dftest: FT_NONE cannot +.
_ws.ftypes.none + 1 == 2
^~~~~~~~~~~~~~~
Passing a literal value to abs() on the LHS segfaults, because it
is incorrectly assumed to be a valid field.
We need to check if we actually have a field. While at it improve
the diagnostic of literals.
dfvm.c:206:1: warning: no previous prototype for function 'dfvm_value_tostr'
dfvm.c:550:1: warning: no previous prototype for function 'filter_finfo_fvalues'
dfvm.c:645:1: warning: no previous prototype for function 'filter_refs_fvalues'
All/any equal have their own symbols for operators so cannot
be handled in the same switch case.
Other comparisons don't have different symbols for any/all.
Allow checking if a slice exists. The result is true if the
slice has length greater than zero.
The len() function is implemented as a DFVM instruction instead.
The semantics are the same.
This removes unparsed name resolution during the semantic
check because it feels like a hack to work around limitations
in the language syntax, that should be solved at the lexical
level instead.
We were interpreting unparsed differently on the LHS and RHS.
Now an unparsed value is always a field if it matches a
registered field name (this matches the implementation in 3.6
and before).
This requires tightening a bit the allowed filter names for
protocols to avoid some common and potentially weird conflicting
cases.
Incidentally this extends set grammar to accept all entities.
That is experimental and may be reverted in the future.
This adds support for using the layers filter
with field references.
Before:
$ dftest 'ip.src != ${ip.src#2}'
dftest: invalid character in macro name
After:
$ dftest 'ip.src != ${ip.src#2}'
Filter: ip.src != ${ip.src#2}
Syntax tree:
0 TEST_ALL_NE:
1 FIELD(ip.src <FT_IPv4>)
1 REFERENCE(ip.src#[2:1] <FT_IPv4>)
Instructions:
00000 READ_TREE ip.src <FT_IPv4> -> reg#0
00001 IF_FALSE_GOTO 5
00002 READ_REFERENCE_R ${ip.src <FT_IPv4>} #[2:1] -> reg#1
00003 IF_FALSE_GOTO 5
00004 ALL_NE reg#0 != reg#1
00005 RETURN
This requires adding another level of complexity to references.
When loading references we need to copy the 'proto_layer_num'
and add the logic to filter on that.
The "layer" sttype is removed and replace by a new
field sttype with support for a range. This is a nice
cleanup for the semantic check and general simplification.
The grammar is better too with this design.
Range sttype is renamed to slice for clarity.
[1702/2528] Building C object epan/dfilter/CMakeFiles/dfilter.dir/dfvm.c.o
In function ‘drange_contains_layer’,
inlined from ‘filter_finfo_fvalues’ at /home/jpv/code/wireshark/wireshark/epan/dfilter/dfvm.c:587:21:
/home/jpv/code/wireshark/wireshark/epan/dfilter/dfvm.c:555:41: warning: ‘upper’ may be used uninitialized [-Wmaybe-uninitialized]
555 | if (num >= lower && num <= upper) { /* inclusive */
| ~~~~^~~~~~~~
/home/jpv/code/wireshark/wireshark/epan/dfilter/dfvm.c: In function ‘filter_finfo_fvalues’:
/home/jpv/code/wireshark/wireshark/epan/dfilter/dfvm.c:537:20: note: ‘upper’ was declared here
537 | int lower, upper;
| ^~~~~
