The name of the block, in the pcapng specification is the systemd
Journal Export Block; add "export" after "journal" in various
variable/enum/define names.
Update the pcap-ng reader and sysdig event dissector to support the second version of the sysdig event block, which was introduced after Wireshark's original implementation
Only a tiny amount of code outside libwiretap needs to know about
pcap/pcapng LINKTYPE_ values, and all that code needs to know is, for a
given LINKTYPE_ value, what the corresponding WTAP_ENCAP_ value is.
Nothing should need to know, for a given WTAP_ENCAP_ value, what its
LINKTYPE_ value is.
Make it the case that nothing *does* need to know, for a given
WTAP_ENCAP_ value, what its LINKTYPE_ value is. Export
wtap_dump_can_write_encap() and use *that*, in the "import hex dump"
code, what formats can be written to a pcap file.
Do more fixups of the "phy" based on the data rate, so that it reflects
the modulation used for the packet.
Note, in comments, why we're doing this, and that there's no reiable
way, in radiotap, to determine the type of channel on which capturing is
being done, as some packet providers use the channel field to indicate
the channel type and others use it to indicate the modulation.
Only provide the "short preamble" for "11b", as that's now being used to
mean "DSSS modulation" - packets on an 11g channel will be marked as
"11g" if they're OFDM or "11b" if they're DSSS.
Make some other cleanups while we're at it.
It only registers one file type/subtype, so rename it to
wtap_register_file_type_subtype().
That will also force plugins to be recompiled; that will produce compile
errors for some plugins that didn't change to match the new contents of
the file_type_subtype_info structure.
Also check to make sure that the registered file type/subtype supports
at least one type of block; a file type/subtype that doesn't return
*any* blocks and doesn't permit *any* block types to be written is not
very useful. That should also catch most if not all other plugins that
didn't change to match the new contents of the file_type_subtype_info
structure.
Don't make errors registering a file type/subtype fatal; just complain,
don't register the bogus file type/subtype, and drive on.
Register the pcap and pcapng file types/subtypes rather than hardwiring
them into the table.
Call the registration routines for them directly, rather than through a
generated table; they're always supposed to be there, as some code in
Wireshark either writes only one of those formats or defaults to writing
one of those formats. Don't run their source code through the
registration-routine-finder script.
Have the file type/subtype codes for them be directly exported to the
libwiretap core, and provide routines to return each of them, to be used
by the aforementioned code.
When reporting errors with cfile_write_failure_message(), use
wtap_dump_file_type_subtype() to get the file type/subtype value for the
wtap_dumper to which we're writing, rather than hardcoding it.
Have the "export PDU" code capable of supporting arbitrary file
types/subtypes, although we currently only use pcapng.
Get rid of declarations of now-static can_write_encap and
dump_open routines in various headers.
Instead of a "supports name resolution" Boolean and bitflags for types of
comments supported, provide a list of block types that the file
type/subtype supports, with each block type having a list of options
supported. Indicate whether "supported" means "one instance" or
"multiple instances".
"Supports" doesn't just mean "can be written", it also means "could be
read".
Rename WTAP_BLOCK_IF_DESCRIPTION to WTAP_BLOCK_IF_ID_AND_INFO, to
indicate that it provides, in addition to information about the
interface, an ID (implicitly, in pcapng files, by its ordinal number)
that is associated with every packet in the file. Emphasize that in
comments - just because your capture file format can list the interfaces
on which a capture was done, that doesn't mean it supports this; it
doesn't do so if the file doesn't indicate, for every packet, on which
of those interfaces it was captured (I'm looking at *you*, Microsoft
Network Monitor...).
Use APIs to query that information to do what the "does this file
type/subtype support name resolution information", "does this file
type/subtype support all of these comment types", and "does this file
type/subtype support - and require - interface IDs" APIs did.
Provide backwards compatibility for Lua.
This allows us to eliminate the WTAP_FILE_TYPE_SUBTYPE_ values for IBM's
iptrace; do so.
Save a copy of the pathname used to open a file in the wtap structure.
This allows the BER file reader to put a pointer to it in the
pseudo-header; it also would allow file readers to attempt to read
"associated" files that have the same name as the file, but with a
different extension.
Instead of having cf_open() special-case BER files, and calling a
routine in the BER dissector to specify the file name to the dissector,
have separate dissectors for "dissect packet payload as BER" and
"dissect a file as BER", and have the latter get the pathname of the
file from the pseudo-header and determine the ASN.1 syntax from that.
(Side-effect - this means that you can now dissect a BER file, and have
the syntax be determined by the file extension, in TShark as well; the
above cf_open() special-casing was *not* done in TShark, so it didn't
work before. Now the application code doesn't need to do any of that,
so it works in TShark as well as Wireshark.)
Eliminate WTAP_FILE_TYPE_SUBTYPE_ERF and
WTAP_FILE_TYPE_SUBTYPE_SYSTEMD_JOURNAL - instead, fetch the values by
name, using wtap_name_to_file_type_subtype().
This requires that wtap_init() be called before epan_init(); that's
currently the case, but put in comments to indicate why it must continue
to be the case.
Provide a wiretap routine to get an array of all savable file
type/subtypes, sorted with pcap and pcapng at the top, followed by the
other types, sorted either by the name or the description.
Use that routine to list options for the -F flag for various commands
Rename wtap_get_savable_file_types_subtypes() to
wtap_get_savable_file_types_subtypes_for_file(), to indicate that it
provides an array of all file type/subtypes in which a given file can be
saved. Have it sort all types, other than the default type/subtype and,
if there is one, the "other" type (both of which are put at the top), by
the name or the description.
Don't allow wtap_register_file_type_subtypes() to override any existing
registrations; have them always register a new type. In that routine,
if there are any emply slots in the table, due to an entry being
unregistered, use it rather than allocating a new slot.
Don't allow unregistration of built-in types.
Rename the "dump open table" to the "file type/subtype table", as it has
entries for all types/subtypes, even if we can't write them.
Initialize that table in a routine that pre-allocates the GArray before
filling it with built-in types/subtypes, so it doesn't keep getting
reallocated.
Get rid of wtap_num_file_types_subtypes - it's just a copy of the size
of the GArray.
Don't have wtap_file_type_subtype_description() crash if handed an
file type/subtype that isn't a valid array index - just return NULL, as
we do with wtap_file_type_subtype_name().
In wtap_name_to_file_type_subtype(), don't use WTAP_FILE_TYPE_SUBTYPE_
names for the backwards-compatibility names - map those names to the
current names, and then look them up. This reduces the number of
uses of hardwired WTAP_FILE_TYPE_SUBTYPE_ values.
Clean up the type of wtap_module_count - it has no need to be a gulong.
Have built-in wiretap file handlers register names to be used for their
file type/subtypes, rather than building the table in init.lua.
Add a new Lua C function get_wtap_filetypes() to construct the
wtap_filetypes table, based on the registered names, and use it in
init.lua.
Add a #define WSLUA_INTERNAL_FUNCTION to register functions intended
only for internal use in init.lua, so they can be made available from
Lua without being documented.
Get rid of WTAP_NUM_FILE_TYPES_SUBTYPES - most code has no need to use
it, as it can just request arrays of types, and the space of
type/subtype codes can be sparse due to registration in any case, so
code has to be careful using it.
wtap_get_num_file_types_subtypes() is no longer used, so remove it. It
returns the number of elements in the file type/subtype array, which is
not necessarily the name of known file type/subtypes, as there may have
been some deregistered types, and those types do *not* get removed from
the array, they just get cleared so that they're available for future
allocation (we don't want the indices of any registered types to changes
if another type is deregistered, as those indicates are the type/subtype
values, so we can't shrink the array).
Clean up white space and remove some comments that shouldn't have been
added.
Remove most of the built-in file types from the table in
wiretap/file_access.c and, instead, have the file types register
themselves, using wtap_register_file_type_subtypes().
This reduces the source code changes needed to add a new file type from
three (add the handler, add the file type to the table in file_access.c,
add a #define for the file type in wiretap/wtap.h) to one (add the
handler). (It also requires adding the handler's source file to
wiretap/CMakeLists.txt, but that's required in both cases.)
A few remain because the WTAP_FILE_TYPE_SUBTYPE_ #define is used
elsewhere; that needs to be fixed.
Fix the wiretap/CMakefile.txt file to scan k12text.l, as that now
contains a registration routine. In the process, avoid scanning files
that don't implement a file type and won't ever have a registration
routine.
Add a Lua routine to fetch the total number of file types; we use that
in some code to construct the wtap_filetypes table, which we need to do
in order to continue to have all the values that used to come from the
WTAP_FILE_TYPE_SUBTYPE_ types.
While we're at it, add modelines to a file that lacked them.
The "short name" is really just the name, used to look it up. The
"name" is really a description intended solely for human consumption.
Rename the fields, and the functions that access them, to match.
The "description" maintained by Lua for file type handlers is used
*only* for one debugging message; we should probably just eliminate it.
Call it an "internal description" for now.
For most file types, blocks for which we don't have a wtap_block_type_t
aren't "custom", they're just "file-type specific". Add
WTAP_BLOCK_FT_SPECIFIC_REPORT and WTAP_BLOCK_FT_SPECIFIC_EVENT block
types; the "mandatory" part of those blocks includes a
file-type-specific block type value, with specific values assigned to
specific block types (either as part of the file type's definition, or
by us if necessary).
For pcapng files, blocks for which we don't have a wtap_block_type_t are
either "local" (block type has the high-order bit set), are defined in
the current spec but aren't supported yet (which we should fix), or are
*not* defined in the current spec and are *not* "local" (in which case
whoever's using the block number should submit a pull request to the
spec to register the block type *and* give it a specification, so we can
add support). For "local" block types and for not-yet-supported
non-"local" block types, they should be handled as file-type-specific
blocks with the file-type-specific block value being the pcapng block
type code, with plugin support in the pcapng code to read *and* write
those blocks.
Move the structures for the "mandatory" parts of blocks to
wiretap/wtap_opttypes.h, right after the definition of
wtap_block_type_t.
Instead *_register_plugin() is turned into a noop (with a warning).
The test suit is failing with ENABLE_PLUGINS=Off (it was already failing
before and this patch didn't affect that).
Closes#17202.
Change the data structure for that option to have a type field,
indicating that it's either a pcap filter string or a BPF program,
followed by a union with a string-pointer member for pcap filter strings
and an instruction-count-and-pointer-to-instructions structure for BPF
programs.
Have routines to add, set, and fetch that option that handle that
structure; discard the "generic structured option" routines. That means
there's more type checking possible at compile time.
Add more code to handle BPF programs.
When writing pcapng files, check, both for that option and for string
options, whether the option length is too big for the data to fit in a
pcapng option, and don't write it if it is. (XXX - truncate the data?
Report an error?)
S1G adapters should be shipping soon since Silex America has a dev-kit
available, so it is about time to add support for this.
Change-Id: I0225d87f78efbcbe88476921d4fce3d56a3ce0cd
It corresponds to LINKTYPE_ETW in pcap and pcapng files; the structures
in the record format come from the Event Tracing for Windows (ETW) API
rather than directly from Event Trace Log files.
While we're at it, explain what extcap/etl does.
Make some packet size variables unsigned.
Leave some others signed, because they're read with sscanf(), and
sscanf() handles string-to-unsigned conversions in the same crazy way
strtouX() routines do, wherein a leading sign is *not* an error.
Instead, cast them to unsigned after we make sure they're not negative.
New link type DLT_ETW is added for write and read Event Trace on Windows.
This change updates MBIM dissector to decode a MBIM message from
a DLT_ETW packet.
Systemd journal entries aren't file-type-specific; they're found in both
systemd journal entry blocks in pcapng files and in systemd journal
export files. Give it a record type, for use with both file types.
This fixes#16955.
It also means that you can open a systemd journal export file and save
it as a pcapng file.
Instead of grabbing the set of IDBs found at open time, have a loop
using wtap_get_next_interface_description() to read all unread IDBs run
after opening the input file, after reading a packet from the input
file, and after getting an EOF on the input file.
Add a routine wtap_uses_interface_ids() to check whether the file type
and subtype for a dump file uses interface IDs and requires IDBs. If
so, in the aforementioned loop, add the IDBs to the dump stream.
Add a routine wtap_dump_add_idb() to add IDBs to a dump stream. Have it
call a file-format-specific routine to add the IDBs; the only file type
that supports it is pcapng, and it 1) writes out the IDB and 2) adds it
to the set of IDBs for the stream.
Add a wtap_dump_params_init_no_idbs() routine that prevents the IDBs
from the input file from being used to initialize the output file; use
it in cases where we're using the aforementioned loop to copy over IDBs.
Don't require any IDBs to be present when opening a pcapng file for
writing; 1) the simplest pcapng file has just an SHB in it, 2) that
requirement causes dumps that don't provide IDBs at open time to fail,
and 3) the real issue is that we don't want packets with an interface ID
not corresponding to a known IDB, and we already have a check for that.
(There are some hacks here; eventually, when everything processes the
IDBs in such a loop, we may be able to get rid of the "two favors of
dump parameter initialization" hack.)
Fixes#15844.
Addresses the same issue in #15502, but there are other issues there
that also need to be addressed.
In addition, the merge code also needs to be changed to handle this.
In a wtap, keep track of the first interface description not yet fetched
with wtap_get_next_interface_description() and, when
wtap_get_next_interface_description() is called, have it return that
description, as a wtap_block_t for its IDB. If there are no
as-yet-unfetched interface descriptions, return NULL; there may, in the
future, be more interface descriptions for the file, so this should be
called:
* after the file is opened;
* after wtap_read() returns TRUE, indicating that it's returned a
record (and *before* you process the record that wtap_read()
returns, as it might be the interface description for the
interface on which the packet in that record arrived);
* after wtap_read() returns FALSE, indicating an EOF or an error
return (as there might have been interfaces at the end of the
file or before the error point).
At each of those points, the caller should loop until
wtap_get_next_interface_description() returns NULL.
Not used yet (but tested with capinfos, which found a reason why you
have to wait until the end of the file before processing the interface
information - there's now a comment in the code giving that reason).
This will probably be used in the future.
Currently, the only file types that use them are pcapng and IBM's
iptrace; we don't support writing the latter, so this is mainly of
interest for pcapng.
This makes it a bit more obvious what some "is this pcapng?" tests are
really trying to determine, and allows them to automatically support any
new file types that use them.
(With regard to interface descriptions, tere are three types of file:
1) files that contain no interface information;
2) files that contain "just FYI" interface information but that don't
tie packets or other records to particular interfaces;
3) files that contain interface information and tie all packets (and
possibly other records) to an interface.
This tests for files of type 3.)
Microsoft Network Monitor lets you capture on an 802.11 adapter either
in monitor mode or in non-monitor mode; frames captured in non-monitor
mode may have the Protected bit set in the 802.11 header, but are
decrypted and don't incclude encryption information, and may have the
A-MSDU Present flag set in the QoS Control field, but have just a
regular frame payload, not a sequence of A-MSDUs, in the payload field.
Dissect those frames correctly.
Bug: 16758
Change-Id: I42b7e9ce52faa80222692403fa7276c039644343
Reviewed-on: https://code.wireshark.org/review/38082
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Add a tsprec value to the wtap_dump_params structure, giving the
per-file time stamp precision.
In wtap_dump_init_dumper(), when constructing a dummy IDB for files that
don't have one, fill in the tsprecision and time_units_per_second values
based on the tsprec value in the wtap_dump_params structure.
Change-Id: I3708b144d4d0ac0dfbe32bd1c16768a75c942141
Reviewed-on: https://code.wireshark.org/review/37979
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
It generates a fake IDB for files that don't have interface information
and that have a per-file encapsulation type, snapshot length, and time
stamp precision, and adds it to the file's list of IDBs.
Use it for libpcap.
We will use it later for other file formats, so that code such as the
mergecap code to merge into a pcapng file can handle input files that
don't have interface information.
(We should have a way to indicate whether the IDBs are real or fake, so
that capinfos and Statistics > Capture File Properties don't report
meaningless IDB information and make it look as if it's known that the
capture was done on one interface with the properties in question.)
Change-Id: Iec124bf3c7cbd4c69ec2ac7d0dd776e5287f8576
Reviewed-on: https://code.wireshark.org/review/37982
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Also note that the record header might have additional stuff at the end,
although not all record headers do (the header length will indicate
what's there).
Change-Id: I5a9ff1f9cd592448bcc45d18808f4b651cdb2f0d
Reviewed-on: https://code.wireshark.org/review/37921
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Add support to read/write the new EPB options, epb_packetid,
epb_queue and epb_verdict, from/to pcap files.
In addition, it updates the packet-frame dissector to dissect
these new fields.
More details on the options can be found in the PcapNG
specification: https://github.com/pcapng/pcapng
An application using these new fields can be found here:
https://github.com/chaudron/xdp-tools/tree/dev/pcapngII/xdp-dump
Change-Id: I761b8114b437fe573dd2c750e35586ad88494938
Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Reviewed-on: https://code.wireshark.org/review/37412
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Bug: 16255 - support HE MCS to rate conversion
Change-Id: I4a4a6c3d62c167b654d150c397047a55f287e6c8
Reviewed-on: https://code.wireshark.org/review/37255
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Different header, with a different size, an additional field, and with
fields being in a different order.
Distinguish between V1 and V2 by giving the version.
That means we can no longer use the "ethertype" dissector as it stands,
because the packet type field isn't at the end of the header, right
before the payload; pull the "add the type field to the protocol tree"
functionality out of the "ethertype" dissector and leave it up to the
dissector calling it.
Change-Id: I72b8a2483c0a539919fbe5d35fd7e60bff4bf75a
Reviewed-on: https://code.wireshark.org/review/37169
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Make wtap_file_get_shb() take a section number argument, and update code
that called it. In most cases, we convert the code to iterate over
sections; in cases where a big code change would be required, we
temporarily pass it 0 and mark the code as "needs to be updated for
multiple sections".
Eliminate cf_read_section_comment(); in calls outside file.c, other code
directly calls the libwiretap routines it calls and, inside file.c, we
just transplant the code and then fix it not to assume a single SHB.
Change-Id: I85e94d0a4fc878e9d937088759be04cb004e019b
Reviewed-on: https://code.wireshark.org/review/37000
Petri-Dish: Guy Harris <gharris@sonic.net>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <gharris@sonic.net>
Allows opening MP4 (ISO/IEC 14496-12) media files in Wireshark and
viewing their structure.
Change-Id: Ie20b8b89dc69bb52d6faa890e547d90317adecf6
Reviewed-on: https://code.wireshark.org/review/35804
Petri-Dish: Stig Bjørlykke <stig@bjorlykke.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Apparently, at least in some regulatory domains, the DMG PHY now goes
above 66 GHz or may do so in the future; the new/future top appears to
be 71 GHz.
Change-Id: I1ee3f9cff177eed269ccc8318b5c952dbeb526ff
Reviewed-on: https://code.wireshark.org/review/35529
Reviewed-by: Guy Harris <guy@alum.mit.edu>
It turned out that 1 MiB is not enough as atleast the URBs sent by
Android fastbool tools are greater than 1 MiB (1 MiB payload + USBPcap
pseudoheader). Raise the maximum packet size all the way up to 128 MiB.
128 MiB is the upper bound of maximum packet that can be captured by
all official USBPcap releases.
Bug: 15985
Change-Id: Ibbf41f7efae6e0f841e36d39664394e8a8eae77d
Reviewed-on: https://code.wireshark.org/review/34793
Petri-Dish: Tomasz Moń <desowin@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Change all wireshark.org URLs to use https.
Fix some broken links while we're at it.
Change-Id: I161bf8eeca43b8027605acea666032da86f5ea1c
Reviewed-on: https://code.wireshark.org/review/34089
Reviewed-by: Guy Harris <guy@alum.mit.edu>
Dissect raw USB Packets. The actual USB packets to transaction conversion
(which is needed to pass the data to existing USB URB dissector) is not
implemented yet.
Ping-Bug: 15908
Change-Id: Ia75d58882d770fdd8650622d318241743069ad8f
Reviewed-on: https://code.wireshark.org/review/34006
Reviewed-by: Tomasz Moń <desowin@gmail.com>
Petri-Dish: Anders Broman <a.broman58@gmail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Michael Tuexen