Field blocks (carried in HEADERS, PUSH_PROMISE, and CONTINUATION
frames) are compressed by HPACK. Send them to the follow tap only
after decompression. Update the tests to match the new output.
Ping #18239 (There's still the case of gzip and brotli compressed
DATA frames to handle).
Implement out of order buffering and desegmentation for QUIC
CRYPTO frames. Particularly useful for Chrome's "Chaos Protection"
that intentionally introduces them, but handles out of order
CRYPTO frames in different UDP payloads as well. (Buffering
packets at a higher encryption level until the out of order
lower level frames have arrived is a different issue.)
Adds a preference, which defaults to on since if there is
out of order, it's not very useful to turn it off.
Fix#17732. Fix#18215.
Allow checking if a slice exists. The result is true if the
slice has length greater than zero.
The len() function is implemented as a DFVM instruction instead.
The semantics are the same.
This removes unparsed name resolution during the semantic
check because it feels like a hack to work around limitations
in the language syntax, that should be solved at the lexical
level instead.
We were interpreting unparsed differently on the LHS and RHS.
Now an unparsed value is always a field if it matches a
registered field name (this matches the implementation in 3.6
and before).
This requires tightening a bit the allowed filter names for
protocols to avoid some common and potentially weird conflicting
cases.
Incidentally this extends set grammar to accept all entities.
That is experimental and may be reverted in the future.
This adds support for using the layers filter
with field references.
Before:
$ dftest 'ip.src != ${ip.src#2}'
dftest: invalid character in macro name
After:
$ dftest 'ip.src != ${ip.src#2}'
Filter: ip.src != ${ip.src#2}
Syntax tree:
0 TEST_ALL_NE:
1 FIELD(ip.src <FT_IPv4>)
1 REFERENCE(ip.src#[2:1] <FT_IPv4>)
Instructions:
00000 READ_TREE ip.src <FT_IPv4> -> reg#0
00001 IF_FALSE_GOTO 5
00002 READ_REFERENCE_R ${ip.src <FT_IPv4>} #[2:1] -> reg#1
00003 IF_FALSE_GOTO 5
00004 ALL_NE reg#0 != reg#1
00005 RETURN
This requires adding another level of complexity to references.
When loading references we need to copy the 'proto_layer_num'
and add the logic to filter on that.
The "layer" sttype is removed and replace by a new
field sttype with support for a range. This is a nice
cleanup for the semantic check and general simplification.
The grammar is better too with this design.
Range sttype is renamed to slice for clarity.
Use "True" or "TRUE" instead of "true" and remove case insensivity.
Same for false. This should serve to differentiate booleans a bit
more from protocol names, which should be using lower-case.
Add the de facto standard Lua regex API to Wireshark. Upstream
code is copied verbatim and the module opened in the "rex" table.
This is just a user convenience and developer quality of life improvement
over the GRegex Lua API because it has always been possible to
load lrexlib-pcre2 as a Lua module from Wireshark.
This code has been unmaintained and does not pass the lrexlib test
suite. GRegex itself has been obsolescent for some time, although GNOME
has recently restarted trying to move it to PCRE2.
Remove it in preparation for a move to lrexlib-pcre2.
When processing segments out of order in TCP, it is possible to
get new segments that fill a sequence gap and be able to dissect
at least one PDU but need more data for additional PDUs (that have
data from the contiguous stream bytes.) We can only determine this
after passing the reassembled segments to the subdissector first.
To keep dissection and layer numbers consistent between passes,
split the multisegment PDU, keeping the already dissect PDU(s) in
the current reassembly and creating a new MSP for the parts not yet
dissected.
Update the dissection test to enable the currently skipped test that
require MSP splitting and remove test_tcp_out_of_order_twopass_with_bug
Field infos have a length property that was not stored with the
field value so when using a negative index the end was computed
from the captured length of the frame tvbuff, leading to incorrect
results. The documentation in wireshark-filter(5) describes how
this was supposed to work but as far as I can tell it never worked
properly.
We now store the length and use that (when it is different from -1)
to locate the end of the protocol data in the tvbuff. An extra wrinkle
is that sometimes the length is set after the field value is created.
This is the most common case as the majority of protocols have a
variable length and dissection generally proceeds with a TVB subset from
the current layer (with offset zero) through all remaining layers to the
end of the captured length. For that reason we must use an expedient to allow
changing the protocol length of an existing protocol fvalue, whenever
proto_item_set_len() is called.
Fixes#17772.
Add support to display filters for matching a specific layer within a frame.
Layers are counted sequentially up the protocol stack. Each protocol
(dissector) that appears in the stack is one layer.
LINK-LAYER#1 <-> IP#1 <-> TCP#1 <-> IP#2 <-> TCP#2 <-> etc.
The syntax allows for negative indexes and ranges with the usual semantics
for slices (but note that counting starts at one):
tcp.port#[2-4] == 1024
Matches layers 2 to 4 inclusive.
Fixes#3791.
Revert to passing a syntax node from the lexical scanner to the grammar
parser. Using a union is not having a discernible advantage and requires
duplicating a lot of properties of syntax nodes.
This removes the limitation of having only two terms in an
arithmetic expression and allows setting the precedence using
curly braces (like any basic calculator).
Our grammar currently does not allow grouping arithmetic expressions
using parenthesis, because boolean expressions and arithmetic
expressions are different and parenthesis are used with the former.
Rework the out of order dissection to store the out of order
segments and add them to reassemblies only after the gaps are filled.
This allows reassembly of contiguous segments to be dissected when
they can, instead of having to wait for all segment gaps to be
filled. In cases where a segment has an erroneous later sequence number,
this prevents reassembly from being completely halted.
It is now guaranteed that when the subdissector is called that the
segment from the current frame is either the first segment of the
MSP or has bytes that were requested from the last call of the
subdissector. This makes it easier to split MSPs in a later commit.
MSPs now always have the first segment with the sequence number,
so MSP_FLAGS_MISSING_FIRST_SEGMENT and first_frame_with_seq are
obsolete and can be removed later.
This fixes a long standing TODO in the out of order test in
suite_dissection.py
Dissection is more consistent between the first pass and later
passes, though there is more to be done.
Fix parsing of some IPv6 addresses and add tests.
Also pass tokens as unparsed unless the user was specfic about
the semantic type. For example the IPv4 address 1.1.1.1 is also a
valid field, but 1.1.1.1/128 is not (because of the slash). However
choose not to enforce the distinction in the lexical scanner and pass
everything as unparsed unless the meaning is explicit in the syntax
with leading dot, colon, or between angle branckets.
Restores support for filters such as "mac-lte", that was broken
in 330d408328.
This means we are not able to support arithmetic expressions with binary
minus without spaces.
$ dftest 'tcp.port == 1-2'
dftest: "1-2" is not a valid number.
To allow an arithmetic expressions without spaces, such as "1+2",
we cannot match the expression in other lexical rules using "+". Because
of longest match this becomes the token LITERAL or UNPARSED with semantic value
"1+2". The same goes for all the other arithmetic operators.
So we need to remove [+-*/%] from "word chars" and add very specific
patterns (that won't mistakenly match an arithmetic expression) for
those literal or unparsed tokens we want to support using these characters.
The plus was not a problem but right slash is used for CIDR, minus for
mac address separator, etc.
There are still some corner case. 11-22-33-44-55-66 is a mac
address and not the arithmetic expression with six terms "eleven
minus twenty two minus etc." (if we ever support more than two terms
in the grammar, which we don't currently).
We lift some patterns from the flex manual to match on IPv4 and
IPv6 (ugly) and add MAC address.
Other hypothetical literal lexical values using [+-*/%] are already
supported enclosed in angle brackets but the cases of MAC/IPv4/IPv6 are
are very common and moreover we need to do the utmost to not break backward
compatibily here.
Before:
$ dftest "_ws.ftypes.int32 == 1+2"
dftest: "1+2" is not a valid number.
After:
$ dftest "_ws.ftypes.int32 == 1+2"
Filter: _ws.ftypes.int32 == 1+2
Instructions:
00000 READ_TREE _ws.ftypes.int32 -> reg#0
00001 IF_FALSE_GOTO 4
00002 ADD 1 <FT_INT32> + 2 <FT_INT32> -> reg#1
00003 ANY_EQ reg#0 == reg#1
00004 RETURN
Fix a failed assertion with constant arithmetic expressions.
Because we do not parse constants on the lexical level it is
more complicated to handle constant expressions with unparsed
values.
We need to handle missing type information gracefully for any
kind of arithmetic expression, not just unary minus.
Add support for display filter binary addition and subtraction.
The grammar is intentionally kept simple for now. The use case
is to add a constant to a protocol field, or (maybe) add two
fields in an expression.
We use signed arithmetic with unsigned numbers, checking for
overflow and casting where necessary to do the conversion.
We could legitimately opt to use traditional modular arithmetic
instead (like C) and if it turns out that that is more useful for
some reason we may want to in the future.
Fixes#15504.
This usage devalues a mechanism for warning users that deserves more
attention than this minor suggestion.
The warning is inconvenient for intermediate and advanced users.
This change implements a unary minus operator.
Filter: tcp.window_size_scalefactor == -tcp.dstport
Instructions:
00000 READ_TREE tcp.window_size_scalefactor -> reg#0
00001 IF_FALSE_GOTO 6
00002 READ_TREE tcp.dstport -> reg#1
00003 IF_FALSE_GOTO 6
00004 MK_MINUS -reg#1 -> reg#2
00005 ANY_EQ reg#0 == reg#2
00006 RETURN
It is supported for integer types, floats and relative time values.
The unsigned integer types are promoted to a 32 bit signed integer.
Unary plus is implemented as a no-op. The plus sign is simply ignored.
Constant arithmetic expressions are computed during compilation.
Overflow with constants is a compile time error. Overflow with
variables is a run time error and silently ignored. Only a debug
message will be printed to the console.
Related to #15504.
Add support for masking of bits. Before the bitwise operator
could only test bits, it did not support clearing bits.
This allows testing if any combination of bits are set/unset
more naturally with a single test. Previously this was only
possible by combining several bitwise predicates.
Bitwise is implemented as a test node, even though it is not.
Maybe the test node should be renamed to something else.
Fixes#17246.
Comparisons require a field-like value on one of the sides,
or both. Change this to require on the LHS or both. There is
realy no reason that I can see to allow the relation to commute,
and it allows removing a lot of unnecessary code and extra tests.
John Thacker