Convert doc/*.pod to Asciidoctor. This:
* Means we use the same markup for our man pages, the guides, and
release notes.
* Lets us add versions to our man pages.
* Gives us more formatting options, e.g. AsciiDoc supports `commands`,
nested lists and makes it easy to include version information. The
manpage backend doesn't seem to support tables very well,
unfortunately.
Convert our CMake configuration to produce *roff and html man pages
using Asciidoctor. Add a "manarg" block macro which makes our synopses
wrap correctly.
Similar to the release notes, guides, and FAQ, if Asciidoctor isn't
found the man pages won't be generated or installed.
Move Asciidoctor to the list of package build dependencies in various
places.
This commit includes the conversion script (pod2adoc.py), which will be
removed later.
Line count sanity check:
Man page .pod .adoc
androiddump 260 280
asn2deb 93 105
capinfos 401 471
captype 54 55
ciscodump 241 269
dftest 42 42
dpauxmon 153 169
dumpcap 464 534
editcap 528 583
etwdump 136 156
extcap 157 181
idl2deb 91 103
idl2wrs 120 100
mergecap 206 207
mmdbresolve 75 75
randpkt 107 111
randpktdump 158 184
rawshark 558 610
reordercap 76 78
sdjournal 145 157
sshdump 272 302
text2pcap 274 312
tshark 2135 2360
udpdump 133 151
wireshark-filter 486 479
wireshark 2967 3420
Implement little endian support for tvb_get_bits family of functions.
The big/little endian refers to bit numbering within an octet. In big
endian, the most significant bit is considered bit 0, while in little
endian the least significant bit is considered bit 0.
Add encoding parameters to proto tree bits format family functions.
Specify ENC_BIG_ENDIAN in all dissectors using these functions except in
USB HID that requires ENC_LITTLE_ENDIAN to work correctly.
When formatting bits values, always display most significant bit on the
leftmost position regardless of the encoding. This results in no gaps
between octets and makes the displayed value comprehensible.
Close#4478Fix#17014
Besides the obvious limitation of being unavailable on Windows,
the standard is vague about getopt() and getopt_long() has many
non-portable pitfalls and buggy implementations, that increase
the maintainance cost a lot. Also the GNU libc code currently
in the tree is not suited for embedding and is unmaintainable.
Own maintainership for getopt_long() and use the musl implementation
everywhere. This way we don't need to worry if optreset is available,
or if the $OPERATING_SYSTEM version behaves in subtly different ways.
The API is under the Wireshark namespace to avoid conflicts with
system headers.
Side-note, the Mingw-w64 9.0 getopt_long() implementation is buggy
with opterr and known to crash. In my experience it's a headache to
use the embedded getopt implementation if the system provides one.
Migrate compress-pngs from a Bash script that ran Make to a Python
script, which should be usable on more platforms.
Add Efficient Compression Tool (ect) to the list of compressors.
Add the compressors to the various *-setup.sh scripts, but comment them
out for now.
The Ubuntu build commented on some spelling errors in executable code
files. Fix the errors that don't come from external files containing
the spelling errors (USB product and vendor IDs, PCI IDs, ASN.1
specifications), and fix some errors that don't show up in the
executable code files (e.g., in comments and variable names).
Some checks intended for dissectors don't work well on dissector
*generators*, as they see stuff such as "value_string %s[]" in a format
string used to generate dissector code and get upset because the
purported value_string doesn't end with {0, NULL} (the generator *does*
put a {0, NULL} at the end, but the checker isn't clever enough to
figure that out).
A few of them just needed scratch memory, so allocate and free it
manually after doing any exception-raising checks.
A few others were returning memory, and needed conversion to accept a
wmem scope argument.
When cross-compiling wireshark the lemon tool should be built
using the host machine compiler to be run on the host. Before
cmake this was done via autotools CC_FOR_BUILD but cmake only
supports one compiler toolchain per build and requires some
workarounds like running cmake twice using separately defined
toolchains.
This gets ugly and complicated fast when considering multiple
toolchains, especially for a simple tool like lemon, so just
allow builds to override the C compiler and wipe the cflags.
This way systems like Gentoo/ChromeOS/Yocto with a properly
setup cross-compile environment can just point to the native
BUILD_CC or similar while minimizing complexity.
To quote the comment in plugins/epan/gryphon/packet-gryphon.c, "Note:
using external tfs strings doesn't work in a plugin", so, for plugins,
don't check to make sure that the file doesn't define a
true_false_string that's the same as one defined by tfs.c.
This is a new check added to check_typed_item_calls.py --label
Ignoring cases where item type is FT_NONE, as fpr tjpse
text was appended that otherwise would lack a colon.
For CI, will now return error codes only for those issues
that are definitely bugs that will require fixing. i.e.
- if the type is not compatible with the call
- if a TFS is (case-sensitively) identical to a tfs.c entrywq
We add them when configuring with autotools, so that we build GLib
appropriately for the OS versions we're targeting; do the same when
configuring with Meson.
If this version of macOS comes with a version of libffi, generate a .pc
file for it and install it in /usr/local/lib/pkgconfig, so that
pkg-config finds that version of libffi, and the GLib configuration
process - whether it's done with autotools or Meson - doesn't decide
that there is no libffi and fail or install its own libffi or whatever.
If we're running an external Python 3 package, pip3 will install scripts
in some directory under /Library; set MESON to point to the location
where Meson will be installed, and use that.
Have a meson-done file to indicate that Meson's been installed by us,
and uninstall it only if that's present.
We want to check whether *Apple* provides Python 3, not whether there's
a Python 3 installed; if there is no Apple-provided Python 3, but
there's somebody else's Python 3 installed, leave it alone, don't
uninstall it.
Newer versions of GLib require Meson (they don't support autotools) and
Ninja (they use Ninja rather than Make). Install Meson and, based on
the GLib version, use autotools+make or Meson+Ninja to build GLib.
Move up the installation of Python 3 so that it's available when we
install Meson, as Meson requires Python 3 and is installed with pip3.
Only install an external Python 3 if /usr/bin/python3 doesn't work; on
at least some versions of macOS, /usr/bin/python3 is a wrapper to run
Python 3 from Xcode, and at least some versions of Xcode provide Python
3.
1.10.2 is the latest version, and is the first version to ship as a fat
x86-64/ARM64 binary, so that we have native binaries for both platforms
supported by macOS.
Automated find/replace of wmem_packet_scope() with pinfo->pool in all
files where it didn't cause a build failure.
I also tweaked a few of the docs which got caught up.
Upgrade our vcpkg bundle to one that includes GLib 2.66.4 and libxml2
2.9.10.
Avoid running pkgconfig on Windows so that we don't find Strawberry
Perl's headers.
As of Debian bullseye and Ubuntu 21.04, `qt5-default` is no longer
available. This patch removes it and adds its dependencies instead
as suggested in <https://askubuntu.com/a/1335187/580576>.
Add a -t option to tools/fuzz-test.sh which lets you specify a maximum
fuzz time.
Add an initial "fuzz-test" job which fuzzes test/captures/* for 5
minutes. To do: Fuzz longer using our capture menagerie and report
failures.
Not all shells support [[ ]] compound commands; it's not in the most
recent Single UNIX Specification I could see, and the
ubuntu-clang-other-tests job is reporting
tools/validate-clang-check.sh: 18: [[: not found
Don't use [[ ]].
In addition, if you change extcap/etl.c, it tries to run clang-check on
it, but that file builds, and is only built, on Windows, so clang-check
fails dismally on UN*Xes. Omit it for now.
Update CMake (3.19.7), Qt (5.2.10), and Python (3.9.3) to later bugfix
versions of the current packages. CMake and Python have made tweaks in
the names of the binary packages that support different macOS versions.
Fixes downloading Python 3.9.2+ on macOS 11 after the package suffix
changed from -macos11.0.pkg to -macos11.pkg
Warn about the lack of Qt offline installers for version 5.15 and
greater.
The existing stuff doesn't appear to work (I tried it on 32-bit Ubuntu
18.04, and it did *not* add any flags to the compilation, as it appeared
not to conclude that they were necessary, even though they were).
Pull in the stuff from libpcap, which *does* appear to work. (it does
so in my 32-bit Ubuntu testing).
This should fix#17301.
While we're at it, fix cppcheck.sh so that it doesn't attempt to run
cppcheck on files that have been deleted.
It's not a valid field type, it's only a hack to support regular
expression matching in packet-matching expressions.
Instead, in the packet-matching code, have a separate syntax tree type
for Perl-compatible regular expressions, and a separate instruction to
load one into a register, and have the "matching" operator for field
types take a GRegex * as the second argument.
At least on my just-now-installed Kubuntu 20.04 VM, G++ wasn't installed
by default, and you need that to compile Wireshark (you can avoid it if
you're not building the GUI code, but the GUI code is Qt-based, so it's
in C++). Add both GCC and G++ to the basic list.
Use the versions of lrint and lrintf defined by Visual C++. This should fix
91>C:\buildbot\builders\wireshark-master-64\wireshark-win64-libs\spandsp-0.0.6-win64ws\include\spandsp/fast_convert.h(320,5): error C2169: 'lrint': intrinsic function, cannot be defined (compiling source file C:\buildbot\builders\wireshark-master-64\windows-2019-x64\build\plugins\codecs\G726\G726decode.c) [C:\buildbot\builders\wireshark-master-64\windows-2019-x64\build\cmbuild\plugins\codecs\G726\g726.vcxproj]
91>C:\buildbot\builders\wireshark-master-64\wireshark-win64-libs\spandsp-0.0.6-win64ws\include\spandsp/fast_convert.h(325,5): error C2169: 'lrintf': intrinsic function, cannot be defined (compiling source file C:\buildbot\builders\wireshark-master-64\windows-2019-x64\build\plugins\codecs\G726\G726decode.c) [C:\buildbot\builders\wireshark-master-64\windows-2019-x64\build\cmbuild\plugins\codecs\G726\g726.vcxproj]
for Visual C++ 16.9.1 and later.
I believe this was the original intention, to use these API restricitons
with dissectors only (not that I necessarily agree with that policy either),
and through copy-paste and lack of clear guidelines it spread to other
parts of the build.
Rename the checkAPI groups to make it very clear that this is dissector-only.
This doesn't mean, of course, that good programming practices shouldn't be
followed everywhere. In particular assertions need to be used properly.
Don't use them to catch runtime errors or validate input data.
This commit will be followed by another removing the various ugly hacks
people have been using to get around the checkAPI hammer.
The minimum required version of Qt is now 5.6, and thus the minimum
required version of macOS is 10.8. Reflect that in macos-setup, and
remove version checks and older packages installed to support
Snow Leopard and Lion.
Celcius -> Celsius.
ammendment, framenun and untunelled (with one 'n') are in wireshark_words.txt
but do not seem to be present in our codebase anymore (and are not
correctly-spelled words), so AFAIK they can be removed from the list.
Added a handful of words which don't seem to be in the dictionary on my host
but are real words and are in the codebase.
Removed two contractions which are now handled within tools/check_spelling.py .
Provide a wiretap routine to get an array of all savable file
type/subtypes, sorted with pcap and pcapng at the top, followed by the
other types, sorted either by the name or the description.
Use that routine to list options for the -F flag for various commands
Rename wtap_get_savable_file_types_subtypes() to
wtap_get_savable_file_types_subtypes_for_file(), to indicate that it
provides an array of all file type/subtypes in which a given file can be
saved. Have it sort all types, other than the default type/subtype and,
if there is one, the "other" type (both of which are put at the top), by
the name or the description.
Don't allow wtap_register_file_type_subtypes() to override any existing
registrations; have them always register a new type. In that routine,
if there are any emply slots in the table, due to an entry being
unregistered, use it rather than allocating a new slot.
Don't allow unregistration of built-in types.
Rename the "dump open table" to the "file type/subtype table", as it has
entries for all types/subtypes, even if we can't write them.
Initialize that table in a routine that pre-allocates the GArray before
filling it with built-in types/subtypes, so it doesn't keep getting
reallocated.
Get rid of wtap_num_file_types_subtypes - it's just a copy of the size
of the GArray.
Don't have wtap_file_type_subtype_description() crash if handed an
file type/subtype that isn't a valid array index - just return NULL, as
we do with wtap_file_type_subtype_name().
In wtap_name_to_file_type_subtype(), don't use WTAP_FILE_TYPE_SUBTYPE_
names for the backwards-compatibility names - map those names to the
current names, and then look them up. This reduces the number of
uses of hardwired WTAP_FILE_TYPE_SUBTYPE_ values.
Clean up the type of wtap_module_count - it has no need to be a gulong.
Have built-in wiretap file handlers register names to be used for their
file type/subtypes, rather than building the table in init.lua.
Add a new Lua C function get_wtap_filetypes() to construct the
wtap_filetypes table, based on the registered names, and use it in
init.lua.
Add a #define WSLUA_INTERNAL_FUNCTION to register functions intended
only for internal use in init.lua, so they can be made available from
Lua without being documented.
Get rid of WTAP_NUM_FILE_TYPES_SUBTYPES - most code has no need to use
it, as it can just request arrays of types, and the space of
type/subtype codes can be sparse due to registration in any case, so
code has to be careful using it.
wtap_get_num_file_types_subtypes() is no longer used, so remove it. It
returns the number of elements in the file type/subtype array, which is
not necessarily the name of known file type/subtypes, as there may have
been some deregistered types, and those types do *not* get removed from
the array, they just get cleared so that they're available for future
allocation (we don't want the indices of any registered types to changes
if another type is deregistered, as those indicates are the type/subtype
values, so we can't shrink the array).
Clean up white space and remove some comments that shouldn't have been
added.
NCSI: Extends NCSI dissection based on DSP0222 Version: 1.2.0_2b
Add pci-ids.c and pci-ids.h for mapping PCI IDs(VID,DID,SID,SVID) to string.
Extends NCSI dissection to support DSP0222 Version: 1.2.0_2b.
Extends NCSI dissection to support Mellanox OEM commands.
NCSI: Use TFS for boolean mapped string and added AEN dissectors
1. Use the tfs defined in tfs.c
2. Refine the boolean mapped strings to be TFS style
3. Added dissectors for AEN
NSCI: Fixed erros with gcc 7.5.0
1. Fix compiling errors with gcc 7.5.0 under Ubuntu 18.04
2. Sloved complaints of git pre-commit hook
NCSI: Add "0x" prefix for displaying HEX values
There are codes display HEX values without prefix, added "0x" to fix that.
PCI-IDS: Added PCI ID file and python script to convert it to C codes
1. Added the PCI ID file pci.ids from https://pci-ids.ucw.cz/
2. Added pci-ids-convert.py to convert to epan/dissectors/pci-ids.c
PCI-IDS: Updated the PCI ID list to be Version 2021.01.11
NCSI: Remove trailing spaces and unused href entries
PCI-IDS: Use a fresh copy of pci.ids to generate pci-ids.c
1. Renamed pci-ids-convert.py to make-pci-ids.py
2. make-pci-ids.py uses a fresh copy of pic.ids to generate pci-ids.c
PCI-IDS: Move internal structure to C file
1. Move pci_id_t and pci_vid_index_t from header file to C file.
2. Refined the comments of pci-ids.c
3. Renamed local variable index (shadow variable) to idx
PCI-IDS: Refined binary search codes
PCI-IDS: Moved pci-ids.[ch] to epan/
Moved pci-ids.[ch] to epan/ as they ought to be
This matches what deb-setup does - it has an --install-deb-deps optionto
install tools necessary to build a .deb.
Document it in the WSDG while we're at it.
Currently, only pcapng has one, and it does nothing, but this mechanism
will be used more in the future.
Update comments in epan/dissectors/CMakeLists.txt and ui/taps.h while
we're at it.
Add a standard CONTRIBUTING file which describes how you can contribute
to Wireshark. Make it Markdown with a .md extension since that's the
default in the GitLab UI and elsewhere.
Instead of having the source file containing the top-level registration
routine for the pinfo_stats_tree plugin checked into our repository,
generate it with tools/make-plugin-reg.py, as we do with other plugins.
While we're at it, fix a comment - "DLL" is a Windows term; the
equivalent term in UN*Xes would be "shared object" ("so" or ".so") or
"dynamic library" ("dylib" or ".dylib").
Python only creates the default argument once and reuses it for
further invocations. Instead, of mutating the default list,
set the default argument to be None and then create a
list, if needed. For more info, see
https://docs.python-guide.org/writing/gotchas/
Fix various warnings with the following changes:
Pass a list of include directories to lupdate. Fixes:
ui/qt/proto_tree.cpp:57: Qualifying with unknown namespace/class ::ProtoTree
and similar warnings.
Use QT_TRANSLATE_NOOP instead of QT_TR_NOOP. Fixes:
ui/qt/lte_rlc_statistics_dialog.cpp:993: tr() cannot be called without context
ui/qt/lte_mac_statistics_dialog.cpp:911: tr() cannot be called without context
ui/qt/font_color_preferences_frame.cpp:28: tr() cannot be called without context
ui/qt/font_color_preferences_frame.cpp:29: tr() cannot be called without context
ui/qt/font_color_preferences_frame.cpp:30: Discarding unconsumed meta data
Add Q_OBJECT to the class definition. Fixes:
ui/qt/models/filter_list_model.cpp:120: Class 'FilterListModel' lacks Q_OBJECT macro
The following warnings were not fixed. This might require moving IOGraph
to its own file:
ui/qt/io_graph_dialog.cpp:320: Qualifying with unknown namespace/class ::IOGraphDialog
ui/qt/io_graph_dialog.cpp:555: Qualifying with unknown namespace/class ::IOGraphDialog
ui/qt/io_graph_dialog.cpp:1059: Qualifying with unknown namespace/class ::IOGraphDialog
ui/qt/io_graph_dialog.cpp:1485: Qualifying with unknown namespace/class ::IOGraphDialog
Allow QT version 5.14.x to be installed (if specified as a variable
on the command line.) Remove the ability to install 5.2.x, as QT 5.3
has been the minimum required version since the Wireshark 3.4 branch.
Note that QT no longer providers offline installers for the free releases
of 5.15 and later, so we'll have to come up with a different method.
(See http://download.qt.io/archive/qt/5.15/5.15.0/OFFLINE_README.txt
and https://www.qt.io/blog/qt-offering-changes-2020 )
Python 3.9.1 is the first version of Python to support Big Sur and
Apple Silicon (https://www.python.org/downloads/release/python-391/),
and Python 3.7.6 is the last version with a 64-bit/32-bit binary installer
for macOS X 10.6 (Snow Leopard) to 10.8 (Mountain Lion) provided.
With Ninja, the build is unnecessarily noisy:
[21/81] Generating plugin.c
Generated plugin.c for l16_mono.
[22/81] Generating plugin.c
Generated plugin.c for G711.
Avoid writing `Generated plugin.c for G711` and generate a single line
such as `Generating plugins/codecs/G711/plugin.c` instead.
Do not write the absolute path to plugin.c, this should hopefully help
with reproducible builds that are independent of the build directory.
Apple Silicon requires CMake 3.19.2, but the binaries provided
for 3.19.2 only run on MacOS 10.10 and later, so we have more
bifurcation of the CMake we try to install. Get rid of some of
the old 2.x paths to compensate.
When running on gitlab-ci, the checkout of the project doesn't give
a complete repo. Then git describe fails at giving the number of commits.
In this case just use 0 instead of NULL.
Adds a pre-commit hook for detecting and replacing
occurrences of `g_malloc()` and `wmem_alloc()` with
`g_new()` and `wmem_new()`, to improve the
readability of Wireshark's code, and
occurrences of
`g_malloc(sizeof(struct myobj) * foo)`
with
`g_new(struct myobj, foo)`
to prevent integer overflows
Also fixes all existing occurrences across
the codebase.
It's minizip-$installed_minizip_version-done, not
zlib-$installed_minizip_version-done; the tarball is
zlib-$installed_minizip_version.tar.gz, because it's a contributed file
in the zlib package, but we don't use zlib in the name of the -done
file.
[skip ci]
It has no configure script, so there's no need for "make distclean", and
the Makefile supplied with it has no "make distclean" rule; just do
"make clean".
[skip ci]
In uninstall_autoconf, when running uninstall subfunctions, pass the
arguments to the subfunctions.
When uninstalling Ninja, remove the "we've finished installing this"
indicator file.
Get rid of a debugging "set +x".
Fix/update/expand some comments.
Do uninstalls for dependencies using CMake more similarly.
For LZ4, as it comes with a Makefile rather than any
autotools/CMake/etc. configuration, "make distclean" might not be
necessary, so, as it's not supported, just do "make clean".
For libssh, do all removes in the uninstall in a single command, and use
$DO_RM, so that it uses sudo iff /usr/local isn't writable by us. In
addition, remove the build directory as the equivalent of "make
distclean".
As with libssh, so with brotli.
For a CMake build done in a subdirectory of the source directory, the
equivalent of "make distclean" is "rm -rf {that subdirectory}". Make it
so.
When uninstalling the stuff snappy installs with "rm -rf", use $DO_RM,
so it's done with sudo iff /usr/local isn't writable by us, just as
"make uninstall" is done with $DO_MAKE_UNINSTALL so it's done with sudo
iff /usr/local isn't writable by us.
Fix up the list of what to remove, now that we're building snappy as a
shared library, so that it removes shared libraries rather than the
non-existent static library.
Update a comment while we're at it, as Lua isn't the only dependency
that doesn't support "make uninstall".
The older versions of snappy apparently used autotools and build a
shared library by default; for example, Wireshark 3.2.6 for macOS is
built with snappy, and includes a snappy dynamic library in the app
bundle.
The current version uses CMake and does *not* build a shared library by
default. Instead, it builds a static library, which, when you try to
link it to a C-only shared library...
...does not work.
The linker sees that you're statically linking in a bunch of C++ .o
files and gets upset because it can't find C++ standard library routines
used by that code.
If it's a dynamic library, the library was itself already linked with
the C++ standard library, so the external references to that library
from the snappy library are already marked as having been resolved to
the extent that they're expected to be in the C++ standard library at
run time - and, when the dynamic snappy library is built, it's marked as
depending on the C++ standard library, so the run time linker will, when
it loads the snappy dynamic library, see that the C++ standard library
is required and will load it if it hasn't already been loaded.
Or a distclean target, for that matter.
Do the best we can.
(libpcap and tcpdump support both autotools and CMake, and Wireshark
uses only CMake; all of them support an uninstall target in CMake. Go
forth, read what they did, and sin no more.)
GNU libtool has a libtool program and a libtoolize program.
The development tools for NeXTStEP, apparently, had a libtool program as
well, and the current version of the development tools for the current
version of NeXTStEP, generally known as "Xcode for macOS", still have
that program.
This means that we do some renaming after installing GNU libtool, so
that its "libtool" becomes "glibtool" and its "libtoolize" becomes
"glibtoolize".
That meant we had to compensate for that when running autoreconf when
building and installing minizip.
It turns out we have to do that when running autogen.sh when building
and installing GLib as well.
Enable PKCS #11 support in macOS builds with macos-setup.sh (already
supported on macOS via Homebrew and on all other OSes with GnuTLS 3.4
or greater) by installing p11-kit (and its dependency libtasn1) and
building nettle and GnuTLS against it.
Update versions of xz, lzip, gettext, libgpg, libgpg-error, libgcrypt, gnutls and gmp
to newest releases.
Also update glib but only to last version with autotools support - meson build is left
for another time.
Current versions of glib require a libpcre with unicode enabled which the Catalina system
version does not provide, so install the current version of libpcre as well.
Update some additional tools to commit 3a42bf0de2b9e35efcc3cea38153ab95cb71b352:
brotli, libmaxminddb, lz4, and snappy
Convert wiretap/ascend.y.in from Bison/YACC to Lemon and rename it to
wiretap/ascend_parser.lemon. Tighten up some of our scanning and
parsing. Make the indentation in it and related files consistent. Aside
from the recent IPv4 fragment offset changes, this produces identical
output to the 3.4 branch for the Ascend trace files I have here.
Remove the comment about supporting other commands. Another timeline
might have an Ascend that successfully pivoted to DSL or 15625B+1D
gigabit ISDN, but this one has neither.
This was our last/only Bison/YACC file, so remove Bison/YACC as a
development and packaging dependency and remove references to it from
the documentation.
Big Sur goes to 11, and it appears that next year's (San Juan Capistrano?)
will go to 12, and so on.
Split version numbers into major and minor, and do version-number
comparison (alas, whilst CMake has that built in, the Bourne shell
doesn't, and neither does the Bourne-again shell).
This should fix issue #17043.
Improve script by ignoring common contractions, dealing with
e.g. \n within strings, and finding multiple concatenated words even
when no camelCase is used.
Also includes some actual spelling fixes.
The canonical location for the usb.ids file is
http://www.linux-usb.org/usb.ids. Unfortunately that site isn't
accessible over HTTPS so we were using https://usb-ids.gowdy.us/usb.ids
instead. *That* site is down, so switch to the Linux USB project's
SourceForge repository URL, which appears to house the assets for
www.linux-usb.org, including the usb.ids file.
Re-enable Fedora build and add CentOS 8 and OpenSUSE 15.2 builds.
Fedora 33 does out of build tree cmake builds and needs spec file changes.
CentOS 8 has some changes with cmake and other packages that are similar to
older Fedora, and needs extra repositories enabled to get -devel packages
(still missing -devel for some optional libraries). OpenSUSE Leap 15.2 also
has some changes needed to build. Note that OpenSUSE Leap 15.1 is EOL
at the end of November 2020. Fixes#16971
PEP 394[1] says,
"In cases where the script is expected to be executed outside virtual
environments, developers will need to be aware of the following
discrepancies across platforms and installation methods:
* Older Linux distributions will provide a python command that refers
to Python 2, and will likely not provide a python2 command.
* Some newer Linux distributions will provide a python command that
refers to Python 3.
* Some Linux distributions will not provide a python command at all by
default, but will provide a python3 command by default."
Debian has forced the issue by choosing the third option[2]:
"NOTE: Debian testing (bullseye) has removed the "python" package and
the '/usr/bin/python' symlink due to the deprecation of Python 2."
Switch our shebang from "#!/usr/bin/env python" to "#!/usr/bin/env
python3" in some places. Remove some 2/3 version checks if we know we're
running under Python 3. Remove the "coding: utf-8" in a bunch of places
since that's the default in Python 3.
[1]https://www.python.org/dev/peps/pep-0394/#for-python-script-publishers
[2]https://wiki.debian.org/Python
Speed functions to print hex bytes, escape XML strings and
print out indents by avoiding specifier calls, and building
larger strings before calling fputs().
Someone mentioned this in the sharkfest chat yesterday.
Also, Ostinato relies upon this when importing from pcap.
An example capture I have has gone from 18 to 11 seconds.
It's possible to play opus payload with libopus (https://opus-codec.org/).
Closes#16882.
Helped-by: Pascal Quantin <pascal.quantin@gmail.com>
Signed-off-by: Lin Sun <lin.sun@zoom.us>
Signed-off-by: Yuanzhi Li <ryanlee@mail.ustc.edu.cn>
Add ui/urls.h to define some URLs on various of our websites. Use the
GitLab URL for the wiki. Add a macro to generate wiki URLs.
Update wiki URLs in comments etc.
Use the #defined URL for the docs page in
WelcomePage::on_helpLabel_clicked; that removes the last user of
topic_online_url(), so get rid of it and swallow it up into
topic_action_url().
Add a check to point out where consecutive items have the same filter
but different labels. Quite a few of these look like bugs.
Also, make some REs raw strings, as identified as an issue in
https://gitlab.com/wireshark/wireshark/-/merge_requests/346
Fix some issues discovered by common python linters including:
* switch `None` comparisons to use `is` rather than `==`. Identity !=
equality, and I've spent 40+ hours before tracking down a subtle bug
caused by exactly this issue. Note that this may introduce a problem if
one of the scripts is depending on this behavior, in which case the
comparison should be changed to `True`/`False` rather than `None`.
* Use `except Exception:` as bare `except:` statements have been
discouraged for years. Ideally for some of these we'd examine if there
were specific exceptions that should be caught, but for now I simply
caught all. Again, this could introduce very subtle behavioral changes
under Python 2, but IIUC, that was all fixed in Python 3, so safe to
move to `except Exception:`.
* Use more idiomatic `if not x in y`--> `if x not in y`
* Use more idiomatic 2 blank lines. I only did this at the beginning,
until I realized how overwhelming this was going to be to apply, then I
stopped.
* Add a TODO where an undefined function name is called, so will fail
whenever that code is run.
* Add more idiomatic spacing around `:`. This is also only partially
cleaned up, as I gave up when I saw how `asn2wrs.py` was clearly
infatuated with the construct.
* Various other small cleanups, removed some trailing whitespace and
improper indentation that wasn't a multiple of 4, etc.
There is still _much_ to do, but I haven't been heavily involved with
this project before, so thought this was a sufficient amount to put up
and see what the feedback is.
Linters that I have enabled which highlighted some of these issues
include:
* `pylint`
* `flake8`
* `pycodestyle`
This adds a protocol post-dissector for Community ID support to
Wireshark/tshark: https://github.com/corelight/community-id-spec
The protocol is disabled by default. It establishes one new filter
value, "communityid".
Includes test cases and baselines to verify correct Community ID
strings based on similar testsuites in the existing Zeek and Python
implementations.
Replace bugs.wireshark.org links with their equivalent
gitlab.com/wireshark/wireshark/issues links in the AsciiDoctor buglink
macro and the please_report_bug function. Update the bug URLs in
comments in the tools and test directories.
FT_STRINGZPAD is for null-*padded* strings, where the field is in an
area of specified length, and, if the string is shorter than that
length, all bytes past the end of the string are NULs.
FT_STRINGZTRUNC is for null-*truncated* strings, where the field is in
an area of specified length and, if the string is shorter than that
length, there's a null character (which might be more than one byte, for
UCS-2, UTF-16, or UTF-32), and anything after that is not guaranteed to
have any particular value.
Use IS_FT_STRING() in some places rather than enumerating all the string
types, so that those places get automatically changed if the set of
string types changes.
In File Search Continue requests, the path is a single byte giving the
string length, followed by that many bytes containing the string value.
However, in at least some File Search Continue requests, the string
length value is longer than the string, and there's a NUL, followed by
other non-zero cruft, in the string.
MariaDB and MySQL are not longer drop-in compatible, they differ in very
different directions
for protocol and api. This patch contains support for MariaDB specific
commands and extensions:
- MariaDB specific character sets and collations (also updated MySQL
collations)
- MariaDB extended capabilities in greeting and login packets
- Support for MARIADB_STMT_BULK_EXECUTE command
- Removal of "5.5.5-" prefix in the version string.
check_spelling.py scans Wireshark source or documentation files,
using the general dictionary from pyspellcheck, augmented by the contents
of wireshark_words.txt.
Can scan:
- entire folders (recursively)
- individual files
- open files
- files affected by recent git changes
Remove the --check-addtext and --build flags. They were used for
checkAddTextCalls, which was removed in e2735ecfdd.
Add the sources in ui/qt except for qcustomplot.{cpp,h}. Fix issues in
main.cpp, rtp_audio_stream.cpp, and wireshark_zip_helper.cpp.
Rename "index"es in packet-usb-hid.c.
Add a verify_merge_request routine to validate-commit.py. If the
required CI_MERGE_REQUEST_XXX environment variables are set it uses them
to query the GitLab API to see if "allow_collaboration" is true in the
current merge request.
This is a ham-fisted way of ensuring that committers can rebase and can
be removed if and when https://gitlab.com/gitlab-org/gitlab/-/issues/23308
is fixed.
Install lintian instead of devscripts (which pulls in lintian + many
other packages) in .gitlab-ci.yml. Add lintian to DEBDEPS_LIST in
debian-setup.sh.
Python's lstrip apparently doesn't strip a prefix but instead strips
all supplied characters from beginning of a string. Using lstrip
in generate-nl80211-fields.py script to remove the 'nl80211_' prefix
happened to work for everything but a few NAN related enums.
Introduce a remove_prefix function and regenerate the nl80211
dissector code to fix the abbreviated field names for NAN.
Copy the Buildbot petri dish builder steps to corresponding GitLab CI
jobs. Update validate-commit.py to look for old "Bug:" and "Ping-Bug:"
references and have it call `git stripspace` directly. tools/commit-msg
was specific to Gerrit, so remove it.
Change-Id: Icbc54709052f44c941db9ad6a5dcf596292782a2
'check_tfs.py --common' can look for tfs values that appear multiple times.
Current output prior to these dssector changes was:
('No Extension', 'Extension') appears 3 times in: ['epan/dissectors/packet-bssap.c', 'epan/dissectors/packet-camel.c', 'epan/dissectors/packet-gsm_map.c']
('Optimised for signalling traffic', 'Not optimised for signalling traffic') appears 3 times in: ['epan/dissectors/packet-gsm_a_gm.c', 'epan/dissectors/packet-gsm_map.c', 'epan/dissectors/packet-gtp.c']
('Data PDU', 'Control PDU') appears 3 times in: ['epan/dissectors/packet-pdcp-lte.c', 'epan/dissectors/packet-pdcp-nr.c', 'epan/dissectors/packet-rlc-nr.c']
('Message sent to originating side', 'Message sent from originating side') appears 3 times in: ['epan/dissectors/packet-q2931.c', 'epan/dissectors/packet-q931.c', 'epan/dissectors/packet-q933.c']
('User', 'Provider') appears 3 times in: ['epan/dissectors/packet-q2931.c', 'epan/dissectors/packet-q931.c', 'epan/dissectors/packet-q933.c']
The first and last ones were made common, the others seem a little too specialised.
Checking some of the existing items in tfs.c (using QtCreator's 'Find Usages'),
some of the common items are used a lot, but many of them are not referenced.
Change-Id: Ia4006d2c4fa7cafbc3b004dc7a367a986dbeb0c4
Reviewed-on: https://code.wireshark.org/review/38177
Petri-Dish: Martin Mathieson <martin.r.mathieson@googlemail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Most of the detected non-contiguous mask bitmasks seem to be valid - they often
represent multiple unassigned/reserved bits that have been conflated into
one hf item.
A set of exceptions has been added to the script - a couple of genuine
buts will be addressed presently in a separate commit.
Change-Id: I87fcf6ee532819097c2daf20b4b1338abb4402d8
Reviewed-on: https://code.wireshark.org/review/38103
Reviewed-by: Martin Mathieson <martin.r.mathieson@googlemail.com>
Petri-Dish: Martin Mathieson <martin.r.mathieson@googlemail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Look for calls to certain proto APIs that require hf items of a certain type,
then check that the items passed in have one of the allowed types.
Currently takes around a minute to scan epan/dissectors. There are
a few issues that have not yet been fixed..
Hopefully this can be added to the PetriDish at some point.
Change-Id: Ic9eadcc3f1de03223606b5dca1cb45edcbe95e85
Reviewed-on: https://code.wireshark.org/review/38039
Petri-Dish: Martin Mathieson <martin.r.mathieson@googlemail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Found using tools/check_tfs.py, included in this commit.
Here are the reports that were fixed here:
Examining:
All dissector modules
epan/dissectors/packet-assa_r3.c tfs_mortisepins_flags - could have used tfs_high_low from tfs.c instead: {High,Low}
epan/dissectors/packet-btle.c tfs_present_bit - could have used tfs_present_not_present from tfs.c instead: {Present,Not Present}
epan/dissectors/packet-dhcp.c tfs_fqdn_s - could have used tfs_server_client from tfs.c instead: {Server,Client}
epan/dissectors/packet-docsis-macmgmt.c mdd_tfs_on_off - could have used tfs_on_off from tfs.c instead: {On,Off}
epan/dissectors/packet-docsis-macmgmt.c mdd_tfs_en_dis - could have used tfs_enabled_disabled from tfs.c instead: {Enabled,Disabled}
epan/dissectors/packet-docsis-macmgmt.c req_not_req_tfs - could have used tfs_requested_not_requested from tfs.c instead: {Requested,Not Requested}
epan/dissectors/packet-docsis-tlv.c on_off_tfs - could have used tfs_on_off from tfs.c instead: {On,Off}
epan/dissectors/packet-docsis-tlv.c activation_tfs - could have used tfs_active_inactive from tfs.c instead: {Active,Inactive}
epan/dissectors/packet-docsis.c ena_dis_tfs - could have used tfs_enabled_disabled from tfs.c instead: {Enabled,Disabled}
epan/dissectors/packet-ecmp.c tfs_not_expected_expected - could have used tfs_odd_even from tfs.c instead: {Odd,Even}
epan/dissectors/packet-erf.c erf_link_status_tfs - could have used tfs_up_down from tfs.c instead: {Up,Down}
epan/dissectors/packet-h263.c on_off_flg - could have used tfs_on_off from tfs.c instead: {On,Off}
epan/dissectors/packet-h263.c cpm_flg - could have used tfs_on_off from tfs.c instead: {On,Off}
epan/dissectors/packet-interlink.c flags_set_notset - could have used tfs_set_notset from tfs.c instead: {Set,Not set}
epan/dissectors/packet-ip.c tos_set_low - could have used tfs_low_normal from tfs.c instead: {Low,Normal}
epan/dissectors/packet-ip.c tos_set_high - could have used tfs_high_normal from tfs.c instead: {High,Normal}
epan/dissectors/packet-isakmp.c flag_r - could have used tfs_response_request from tfs.c instead: {Response,Request}
epan/dissectors/packet-isis-lsp.c tfs_metric_supported_not_supported - could have used tfs_no_yes from tfs.c instead: {No,Yes}
epan/dissectors/packet-kerberos.c supported_tfs - could have used tfs_supported_not_supported from tfs.c instead: {Supported,Not supported}
epan/dissectors/packet-kerberos.c set_tfs - could have used tfs_set_notset from tfs.c instead: {Set,Not set}
epan/dissectors/packet-mac-lte.c mac_lte_scell_status_vals - could have used tfs_activated_deactivated from tfs.c instead: {Activated,Deactivated}
epan/dissectors/packet-p_mul.c no_yes - could have used tfs_no_yes from tfs.c instead: {No,Yes}
epan/dissectors/packet-pgm.c opts_present - could have used tfs_present_not_present from tfs.c instead: {Present,Not Present}
epan/dissectors/packet-rsl.c rsl_ms_fpc_epc_mode_vals - could have used tfs_inuse_not_inuse from tfs.c instead: {In use,Not in use}
epan/dissectors/packet-sita.c tfs_sita_on_off - could have used tfs_on_off from tfs.c instead: {On,Off}
epan/dissectors/packet-vines.c tfs_vine_rtp_no_yes - could have used tfs_no_yes from tfs.c instead: {No,Yes}
epan/dissectors/packet-vnc.c button_mask_tfs - could have used tfs_pressed_not_pressed from tfs.c instead: {Pressed,Not pressed}
27 issues found
Change-Id: I7e53b491f20289955c9e9caa8357197d9010a5aa
Reviewed-on: https://code.wireshark.org/review/38087
Petri-Dish: Martin Mathieson <martin.r.mathieson@googlemail.com>
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Switch from WinPcap's WpdPack SDK to a libpcap package built with vcpkg.
We explictly load wpcap.dll on Windows, so make sure we don't link with
pcap.lib.
Move timestamp code from capture-pcap-util-unix.c to
capture-pcap-util.c. Add timestap routines to capture-wpcap.c and make a
couple of other updates.
Change-Id: If0e3dbeb7378c42ed9e3f91b2f15add95d22a2bb
Reviewed-on: https://code.wireshark.org/review/37905
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The script includes the following changes:
- Added pcap masking and anonymization support
- Support to mask/anonymize only portion of field
- Added reading from stdin
- Changed json to ijson library to support large files
- Migrated from text2pcap to scapy for pcap generation
- Added version to script
The development repo is located here
https://github.com/H21lab/json2pcap
Change-Id: I8fc5e282caa604e188f05818f7a2f8875afb8b73
Reviewed-on: https://code.wireshark.org/review/37371
Reviewed-by: Dario Lombardo <lomato@gmail.com>
The intention is to try to run this on the Petri-dish buildbot,
where it could run with '--commits 1' to warn about files touched
in the most recent commit.
Change-Id: Ie924d39e093d1fef8cfbdf02d15bbede386b2862
Reviewed-on: https://code.wireshark.org/review/37826
Petri-Dish: Martin Mathieson <martin.r.mathieson@googlemail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
Avoid string literals while at it to avoid -Wpointer-sign warnings with
GCC 10. This has the additional benefit of avoiding storing the trailing
NUL byte after the data, resulting in a tiny reduction in binary size.
This compound literal syntax is supported since C99 which is permitted
by doc/README.developer.
Change-Id: I35f4d3a46aa78e12915d92136f1de0891131bede
Reviewed-on: https://code.wireshark.org/review/37818
Petri-Dish: Peter Wu <peter@lekensteyn.nl>
Tested-by: Petri Dish Buildbot
Reviewed-by: Anders Broman <a.broman58@gmail.com>
If we've been passed "-l <n>" and the last <n> commits don't match any C
or C++ files or we've been passed "-o" and no files have been changed,
simply exit instead of testing the entire code base. We use "cppcheck.sh
-l <n>" in our CI system and the former behavior is much more useful
(and faster) than the latter.
Change-Id: I1127eabefa854d68f80b0a2dfd05e6895658abc0
Reviewed-on: https://code.wireshark.org/review/37773
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Reviewed-by: Martin Mathieson <martin.r.mathieson@googlemail.com>
cppcheck seems to get confused by the VALS() macro.
Change-Id: Iba59a4886a0461cea9797a09a10e67420b09af19
Reviewed-on: https://code.wireshark.org/review/37639
Petri-Dish: Martin Mathieson <martin.r.mathieson@googlemail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Martin Mathieson <martin.r.mathieson@googlemail.com>
Without this (particularly '.'), for some files it macro definitions cannot
be found the whole file is basically skipped.
Will make overall scan take quite a bit longer.
Change-Id: I7498b23ad9b27edd3a815c7fc51ef8501fa5a56a
Reviewed-on: https://code.wireshark.org/review/37567
Petri-Dish: Martin Mathieson <martin.r.mathieson@googlemail.com>
Tested-by: Petri Dish Buildbot
Reviewed-by: Martin Mathieson <martin.r.mathieson@googlemail.com>
Gerald Combs