If we don't have an offset, don't print anything with underline.
Also it can underline filters using macros correctly now.
$ tshark -Y 'ip and ${private_ipv4:ip.sr}' -r /dev/null
tshark: Left side of "==" expression must be a field or function, not "ip.sr".
ip and ip.sr == 192.168.0.0/16 or ip.sr == 172.16.0.0/12 or ip.sr == 10.0.0.0/8
^~~~~
Add location tracking as a column offset and length from offset
to the scanner. Our input is a single line only so we don't need
to track line offset.
Record that information in the syntax tree. Return the error location
in dfilter_compile(). Use it in dftest to mark the location of the
error in the filter string. Later it would be nice to use the location
in the GUI as well.
$ dftest "ip.proto == aaaaaa and tcp.port == 123"
Filter: ip.proto == aaaaaa and tcp.port == 123
dftest: "aaaaaa" cannot be found among the possible values for ip.proto.
ip.proto == aaaaaa and tcp.port == 123
^~~~~~
Revert to passing a syntax node from the lexical scanner to the grammar
parser. Using a union is not having a discernible advantage and requires
duplicating a lot of properties of syntax nodes.
This removes the limitation of having only two terms in an
arithmetic expression and allows setting the precedence using
curly braces (like any basic calculator).
Our grammar currently does not allow grouping arithmetic expressions
using parenthesis, because boolean expressions and arithmetic
expressions are different and parenthesis are used with the former.
Instead of saying a leading colon will make any token a literal
value, say it is part of the syntax of bytes arrays. This is
useful to write bytes without a separator, and other potentially
ambiguous formats.
The restriction in meaning to bytes and simple numeric values
should make the rules for handling a leading colon (specifically
ommiting it or not) saner without much loss of functionality.
When retrying fvalue_from_literal() we were leaking the error
message string.
Refactor the code to avoid the retry. This assumes the only
valid use of a leading ':' with a literal is for an IPv6 address.
Bytes with leading ':' are supported but the colon is skipped,
so the parser doesn't see it.
Fixes df0fc8b517.
Fix parsing of some IPv6 addresses and add tests.
Also pass tokens as unparsed unless the user was specfic about
the semantic type. For example the IPv4 address 1.1.1.1 is also a
valid field, but 1.1.1.1/128 is not (because of the slash). However
choose not to enforce the distinction in the lexical scanner and pass
everything as unparsed unless the meaning is explicit in the syntax
with leading dot, colon, or between angle branckets.
Restores support for filters such as "mac-lte", that was broken
in 330d408328.
This means we are not able to support arithmetic expressions with binary
minus without spaces.
$ dftest 'tcp.port == 1-2'
dftest: "1-2" is not a valid number.
After some experimentation I don't think these two existence tests
belong in the grammar, it's an implementation detail and removing it
might avoid some artificial constraints.
Add argument to dfilter_compile_real() to save syntax tree text
representation.
Use it with dftest to print syntax tree.
Misc debug output format improvements.
To allow an arithmetic expressions without spaces, such as "1+2",
we cannot match the expression in other lexical rules using "+". Because
of longest match this becomes the token LITERAL or UNPARSED with semantic value
"1+2". The same goes for all the other arithmetic operators.
So we need to remove [+-*/%] from "word chars" and add very specific
patterns (that won't mistakenly match an arithmetic expression) for
those literal or unparsed tokens we want to support using these characters.
The plus was not a problem but right slash is used for CIDR, minus for
mac address separator, etc.
There are still some corner case. 11-22-33-44-55-66 is a mac
address and not the arithmetic expression with six terms "eleven
minus twenty two minus etc." (if we ever support more than two terms
in the grammar, which we don't currently).
We lift some patterns from the flex manual to match on IPv4 and
IPv6 (ugly) and add MAC address.
Other hypothetical literal lexical values using [+-*/%] are already
supported enclosed in angle brackets but the cases of MAC/IPv4/IPv6 are
are very common and moreover we need to do the utmost to not break backward
compatibily here.
Before:
$ dftest "_ws.ftypes.int32 == 1+2"
dftest: "1+2" is not a valid number.
After:
$ dftest "_ws.ftypes.int32 == 1+2"
Filter: _ws.ftypes.int32 == 1+2
Instructions:
00000 READ_TREE _ws.ftypes.int32 -> reg#0
00001 IF_FALSE_GOTO 4
00002 ADD 1 <FT_INT32> + 2 <FT_INT32> -> reg#1
00003 ANY_EQ reg#0 == reg#1
00004 RETURN
In most, if not all, programming languages logical AND has
higher precedence than logical OR. Apply the principle of
least surprise and do the same for Wireshark display
filters.
Before: ip and tcp or udp => ip and (tcp or udp)
Filter: ip and tcp or udp
Instructions:
00000 CHECK_EXISTS ip
00001 IF_FALSE_GOTO 5
00002 CHECK_EXISTS tcp
00003 IF_TRUE_GOTO 5
00004 CHECK_EXISTS udp
00005 RETURN
After: ip and tcp or udp => (ip and tcp) or udp
Filter: ip and tcp or udp
Instructions:
00000 CHECK_EXISTS ip
00001 IF_FALSE_GOTO 4
00002 CHECK_EXISTS tcp
00003 IF_TRUE_GOTO 5
00004 CHECK_EXISTS udp
00005 RETURN
Fix a failed assertion with constant arithmetic expressions.
Because we do not parse constants on the lexical level it is
more complicated to handle constant expressions with unparsed
values.
We need to handle missing type information gracefully for any
kind of arithmetic expression, not just unary minus.
Add support for display filter binary addition and subtraction.
The grammar is intentionally kept simple for now. The use case
is to add a constant to a protocol field, or (maybe) add two
fields in an expression.
We use signed arithmetic with unsigned numbers, checking for
overflow and casting where necessary to do the conversion.
We could legitimately opt to use traditional modular arithmetic
instead (like C) and if it turns out that that is more useful for
some reason we may want to in the future.
Fixes#15504.
By the time we are using the reference fvalue the tree may have gone
away and with it the fvalue. We need to duplicate the reference
fvalues and take ownership of the memory.
This replaces the current macro reference system with
a completely different implementation. Instead of a macro a reference
is a syntax element. A reference is a constant that can be filled
in the dfilter code after compilation from an existing protocol tree.
It is best understood as a field value that can be read from a fixed
tree that is not the frame being filtered. Usually this fixed tree
is the currently selected frame when the filter is applied. This
allows comparing fields in the filtered frame with fields in the
selected frame.
Because the field reference syntax uses the same sigil notation
as a macro we have to use a heuristic to distinguish them:
if the name has a dot it is a field reference, otherwise
it is a macro name.
The reference is synctatically validated at compile time.
There are two main advantages to this implementation (and a couple of
minor ones):
The protocol tree for each selected frame is only walked if we have a
display filter and if the display filter uses references. Also only the
actual reference values are copied, intead of loading the entire tree
into a hash table (in textual form even).
The other advantage is that the reference is tested like a protocol
field against all the values in the selected frame (if there is more
than one).
Currently the reference fields are not "primed" during dissection, so
the entire tree is walked to find a particular reference (this is
similar to the previous implementation).
If the display filter contains a valid reference and the reference is
not loaded at the time the filter is run the result is the same as a
non existing field for a regular READ_TREE instruction.
Fixes#17599.
This usage devalues a mechanism for warning users that deserves more
attention than this minor suggestion.
The warning is inconvenient for intermediate and advanced users.
This change implements a unary minus operator.
Filter: tcp.window_size_scalefactor == -tcp.dstport
Instructions:
00000 READ_TREE tcp.window_size_scalefactor -> reg#0
00001 IF_FALSE_GOTO 6
00002 READ_TREE tcp.dstport -> reg#1
00003 IF_FALSE_GOTO 6
00004 MK_MINUS -reg#1 -> reg#2
00005 ANY_EQ reg#0 == reg#2
00006 RETURN
It is supported for integer types, floats and relative time values.
The unsigned integer types are promoted to a 32 bit signed integer.
Unary plus is implemented as a no-op. The plus sign is simply ignored.
Constant arithmetic expressions are computed during compilation.
Overflow with constants is a compile time error. Overflow with
variables is a run time error and silently ignored. Only a debug
message will be printed to the console.
Related to #15504.
Add support for masking of bits. Before the bitwise operator
could only test bits, it did not support clearing bits.
This allows testing if any combination of bits are set/unset
more naturally with a single test. Previously this was only
possible by combining several bitwise predicates.
Bitwise is implemented as a test node, even though it is not.
Maybe the test node should be renamed to something else.
Fixes#17246.
The DFVM instructions arguments are generic boxed types but instead
of using FVALUE and PCRE types the code passes aroung REGISTER types
instead. Change that to pass constants in the instruction.
Use a list to allow a variable number of jumps, instead of a fixed
count. The flexibility in the number of jumps a given syntax tree
node might need to handle is useful to add new kinds of
operations.
Syntax tree nodes can mutate and change type so the caching being used
is keepign a stale representation and printing wrong results. Recreate
the string every time the function is called.
We still store the string pointer in the node to be able to pass a const
char * to the caller without leaking memory, as a convenience.
João Valverde