Asciidoctor is now required for packaging. Try to make sure it's
installed on CentOS 8 and openSUSE 15.2. Note that CentOS 8 doesn't have
an Asciidoctor package, which complicates our SPEC.
The openSUSE 15.2 RPM Test step occasionally fails when trying to
retrieve repository metadata. Add "--no-remote" to the zypper install
command in the openSUSE 15.2 RPM Test to match the offline behavior of
our other RPM tests.
display on CI build, the text output (and not xml)
store all cppcheck output files on cppcheck folder
(you need to launch cppcheck twice for generate txt and xml)
Use an apostrophe instead of RIGHT SINGLE QUOTATION MARK in our PDF and
EPUB filenames. Some programs (notably Okular) can't open filenames with
extended characters, at least on Windows.
Rename the .build job to .build-linux and require the "docker" tag. Add
a corresponding .test-linux job and have the Linux test stages extend
it. Update various steps accordingly.
In the Windows merge request build job, don't pass
/consoleloggerparameters:PerformanceSummary;NoSummary to msbuild. It
makes the output more verbose and in turn makes errors more difficult to
find.
We have to use dedicated runners for Windows builds, so create a
separate "if-merged" rule for them which includes only the
wireshark/wireshark repository.
Build WiresharkPortable32 or WiresharkPortable64 as appropriate for our
target platform. Add WiresharkPortable64 steps to the Win64 builder.
Update the Developer's Guide. Fixes#17260.
GitLab CI annoyingly shows "⚙ blocked" in the pipeline UI if you have
any jobs that can be run manually instead of a green checkmark even if
all of the other jobs in the pipeline have passed. It's not clear that
"blocked" means "you can run this manually if needed," so adjust the
Documentation rule so that this doesn't happen.
Move common fuzzing configuration items to .fuzz-ubuntu. Build using
Clang, which is what the Buildbot fuzzers did. Add jobs for fuzzing
using Valgrind and randpkt.
This reverts commit b258f90ce5, which was
an attempt to fix a CI build issue by changing our Qt version. That
didn't fix the issue. Switching from CMake 3.20 to 3.19.8 *did*, which
suggests that we've run into
https://gitlab.kitware.com/cmake/cmake/-/issues/21145
Add a -t option to tools/fuzz-test.sh which lets you specify a maximum
fuzz time.
Add an initial "fuzz-test" job which fuzzes test/captures/* for 5
minutes. To do: Fuzz longer using our capture menagerie and report
failures.
Rename the "docbook" job to "documentation". Make sure we can do syntax
highlighting and produce PDFs. Distribute the docs that we build. Allow
the job to be manually run if we don't update any documentation sources.
Build the wsar_html_zip instead of wsar_html and (re-)publish it at
https://www.wireshark.org/download/docs/. Move the doxygen_all job to
the daily schedule section.
The files we need to submit to Coverity might be too large to upload
over HTTP, so use their URL submission method. We won't have a usable
artifact URL until after each cov-build job runs, so we need to submit
our URLs in separate jobs.
There are three ways to reuse configuration blobs in .gitlab-ci.yml: The
"extends" keyword, YAML anchors, and "!reference" tags. As noted in
https://gitlab.com/gitlab-org/gitlab/-/issues/322992, only the last
method works for job rules. Clean up our common rules and apply them
using "!reference".
It looks like "extends" and YAML anchors don't work for scheduling
rules, at least for the way we're using them. Use explicit rules for
scheduled jobs.
Move commonly-used rules to their own hidden jobs. Use ".if-merged" to
ensure that our production build and test jobs are run automatically in
wireshark/wireshark and can be run manually in forks.
Note the new manual behavior in the Developer's Guide.
Trying to upload cov-build output on Windows is currently failing
because the file is too large. Expose the build file as an artifact and
submit its URL instead.
Switch the recent analysis builds from only/when to rules.
Switch the API reference and VS Code Analysis builds to daily.
Remove a no-longer-useful URL.
The test:rpm-fedora job is currently failing with:
$ dnf install -y build/packaging/rpm/RPMS/x86_64/*.rpm
Fedora 34 openh264 (From Cisco) - x86_64 6.4 kB/s | 2.5 kB 00:00
Fedora Modular 34 - x86_64 8.4 MB/s | 3.8 MB 00:00
Fedora Modular 34 - x86_64 - Updates 500 kB/s | 754 kB 00:01
Fedora 34 - x86_64 - Updates 11 MB/s | 5.8 MB 00:00
Fedora 34 - x86_64 21 MB/s | 57 MB 00:02
Error:
Problem: conflicting requests
- nothing provides libminizip.so.2.5()(64bit) needed by wireshark-qt-3.5.0rc0_1661_ge2e4b79d0dd3-1.x86_64
It looks like this is due to Fedora 34 and later shipping with
minizip-3.0: https://src.fedoraproject.org/rpms/minizip.
Disable the test:rpm-fedora job for now until we can find a way to make
it more reliable.
As described at
https://medium.com/@alasher/colored-c-compiler-output-with-ninja-clang-gcc-10bfe7f2b949
both Clang and gcc generate colorized output when they detect a
terminal, but not for piped output, which is the case when using Ninja.
Add an ENABLE_COMPILER_COLOR_DIAGNOSTICS CMake option, and set it to
"ON" when we're using Ninja.
In the merge-req:ubuntu-gcc-ctest and merge-req:ubuntu-clang-other-tests
GitLab CI jobs, generate colorized HTML report artifacts using
ansi2html.
The Windows MR builder was recently migrated to a new machine and
updated to Qt 5.15.2. Since the migration merge request builds have
sometimes failed with
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(240,5): error MSB8066: Custom build for 'C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_de.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_en.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_es.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_fr.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_it.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_ja_JP.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_pl.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_ru.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_sv.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_uk.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\7345cb0fc1b52560d4d2bd48e83ff433\wireshark_zh_CN.qm.rule;C:\builds\wireshark\wireshark\build\CMakeFiles\9829b32238fa3bcc807b02099e4c1642\qtui_autogen.rule' exited with code -1073740940. [C:\builds\wireshark\wireshark\build\ui\qt\qtui_autogen.vcxproj]
Try switching back to Qt 5.15.1. If that doesn't work we might have to
un-migrate the runner.
Fetching the complete Wireshark repository in GitLab's CI currently
takes about 3.5 minutes. GitLab CI lets you do shallow cloning, but only
up to 1000 commits. This isn't sufficient for any of our jobs that might
need a reachable tag since it's currently about 1200 commits away in
master.
Set GIT_DEPTH to 1 (which appears to trigger the required shallow clone
logic in GitLab's CI), then deepen it by setting GIT_FETCH_EXTRA_FLAGS
to "--depth=5000". This is *should* be much faster than a full checkout
and still give us a reachable tag:
$ git rev-list --count v3.3.0rc0..v3.5.0rc0
2330
$ git rev-list --count v3.1.0rc0..v3.3.0rc0
2198
$ git rev-list --count v2.9.0rc0..v3.1.0rc0
3449
Overriding the definition of the rpmbuild macro cmake_build on the
command line, so that it doesn't include the string "--verbose", should
prevent cmake --build from being run with --verbose, and thus prevent it
from running Ninja with the -v flag, and thus prevent a bunch of extra
noisy output from being produced for every build command, and thus
prevent the build log from hitting GitLab's 4MB limit.
Unlike piping the output of "ninja rpm-package" to sed, this means that
the exit status of "ninja rpm-package", rather than the exit status of
sed, is tested.
Filter out the -W and -f options from the output of "ninja rpm-package",
to try to cut down the verbosity.
Do that instead of passing --quiet to rpmbuild, so we get command
progress messages, just without the extra junk.
If the FORCE_CMAKE_NINJA_QUIET environment variable is set, have the
top-level CMakeLists.txt add the -q flag to the arguments to rpmbuild.
That appears to reduce the amount of output.
Set that environment varible in the rpm-fedora build.
See whether unsetting VERBOSE causes the build process not to print the
compile command, complete with a big list of compiler options, for every
single file being built, in the hopes of producing less than the limit
of 4MB of log text.
See whether setting DH_QUIET to 1 and MAKEFLAGS to --silent causes the
build process not to print the compile command, complete with a big list
of compiler options, for every single file being built, in the hopes of
producing less than the limit of 4MB of log text.
Ninja keeps track of its built files in .ninja_log, so if you copy a
pre-built target into a fresh build directory, Ninja will ignore and
overwrite it. This includes the tarball generated by the 'dist' target.
In get-export-release.sh, check for a preexisting tarball and preserve
it by default. This lets us pass the dist tarball from one GitLab CI
stage to other stages without recreating it. It's also arguably the
right thing to do in general, since we record and publish the tarball
hashes for each release and different contents for the same filename can
cause confusion.
Move the dist tarball to the build directory in .gitlab-ci.yml, and add
a note about using the tarball exclusively.
We currently disable shallow clones in our general pipeline settings
because jobs that create packages need a usable `git describe`. None of
our merge request jobs create packages, so clone shallowly in order to
speed things up.
Remove the last package installation instances from .gitlab-ci.yml. Add
a comment at the top noting that dependencies should be added to our
Dockerfiles instead.
The wireshark/wireshark-ubuntu-dev image was recently upgraded to Ubuntu
20.04. Since then the following tests have been failing:
---- FAILED
test/suite_capture.py::case_wireshark_capture::test_wireshark_capture_from_fifo
FAILED
test/suite_capture.py::case_wireshark_capture::test_wireshark_capture_from_stdin
FAILED
test/suite_capture.py::case_tshark_capture::test_tshark_capture_from_fifo
FAILED
test/suite_capture.py::case_tshark_capture::test_tshark_capture_from_stdin
FAILED
test/suite_capture.py::case_dumpcap_capture::test_dumpcap_capture_from_fifo
FAILED
test/suite_capture.py::case_dumpcap_capture::test_dumpcap_capture_from_stdin
FAILED
test/suite_capture.py::case_dumpcap_autostop::test_dumpcap_autostop_filesize
FAILED
test/suite_capture.py::case_dumpcap_autostop::test_dumpcap_autostop_packets
FAILED
test/suite_capture.py::case_dumpcap_ringbuffer::test_dumpcap_ringbuffer_filesize
FAILED
test/suite_capture.py::case_dumpcap_ringbuffer::test_dumpcap_ringbuffer_packets
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_multi_in_multi_out
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_multi_in_single_out
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_single_in_multi_out
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_single_in_single_out
FAILED
test/suite_clopts.py::case_dumpcap_options::test_dumpcap_interface_chars
FAILED
test/suite_clopts.py::case_dumpcap_options::test_dumpcap_invalid_chars
FAILED
test/suite_clopts.py::case_dumpcap_options::test_dumpcap_valid_chars
----
This is because dumpcap can't load our local libraries after setting cap_net_raw+cap_net_admin:
----
-- Begin stderr for command ('/builds/wireshark/wireshark/build/run/dumpcap', '-D') --
/builds/wireshark/wireshark/build/run/dumpcap: error while loading shared libraries: libwsutil.so.0: cannot open shared object file: No such file or directory
-- End stderr for command ('/builds/wireshark/wireshark/build/run/dumpcap', '-D') --
----
Add $(pwd)/run to our rpath.
Re-enable Fedora build and add CentOS 8 and OpenSUSE 15.2 builds.
Fedora 33 does out of build tree cmake builds and needs spec file changes.
CentOS 8 has some changes with cmake and other packages that are similar to
older Fedora, and needs extra repositories enabled to get -devel packages
(still missing -devel for some optional libraries). OpenSUSE Leap 15.2 also
has some changes needed to build. Note that OpenSUSE Leap 15.1 is EOL
at the end of November 2020. Fixes#16971
Adds job of building the HTML versions of the
Wireshark User Guide and Wireshark Developer Guide.
PDF versions are not built because it would take significantly longer
(~10 mins instead of ~5 minutes)
Move the pre-commit and validate-commit checks to their own job. This
lets us quickly re-run validate-commit in case the submitter forgets to
check "Allow commits from members who can merge to the target branch"
Shorten the "merge-request" prefix to "merge-req" in order to make it
easier to distinguish between each job in the GitLab UI.
The Windows runners are constrained by the following:
* We require quite a bit of software not present in the stanadard
runner[1] which takes a long time to install, including Python, Perl,
and Qt.
* You can't specify an arbitrary Docker image like you can with Linux
runners.
As a result we have a project-specific runner for wireshark/wireshark
that runs a custom Windows Docker image. Update the CI rules so that
merge-request:windows only runs for gitlab.com/wireshark/wireshark. The
GitLab documentation recommends rules over only/except, so switch to
them.
Fixup .editorconfig while we're here.
[1]https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md
Git is currently broken in gitlab-runner 13.3.0 + Windows + Docker...
https://gitlab.com/gitlab-org/gitlab/-/issues/239013
...but it appears to work fine in gitlab-runner 13.2.0. I reverted the build machine to that version, so we can enable the merge-request:windows build.
Install lintian instead of devscripts (which pulls in lintian + many
other packages) in .gitlab-ci.yml. Add lintian to DEBDEPS_LIST in
debian-setup.sh.
Copy the Buildbot petri dish builder steps to corresponding GitLab CI
jobs. Update validate-commit.py to look for old "Bug:" and "Ping-Bug:"
references and have it call `git stripspace` directly. tools/commit-msg
was specific to Gerrit, so remove it.
Change-Id: Icbc54709052f44c941db9ad6a5dcf596292782a2
This is going to install the deb artifact created before.
Change-Id: I59a52c8d7fe5dd46bb3ed8684ab3dd477f3baea4
Reviewed-on: https://code.wireshark.org/review/37366
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Lack of capture support is such a rare case, it is not worth spending
too much CI cycles testing it for every build.
Change-Id: Ia9be2b6ba21d0b198e65d746483282ef66cadb84
Reviewed-on: https://code.wireshark.org/review/36626
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Requires an updated wireshark/wireshark-ubuntu-dev image:
https://github.com/wireshark/wireshark-ubuntu-dev-docker/pull/2
Remove -Wframe-larger-than while at it. The default size in our CMake
config is 32k. Clang should not significantly go over it. If so, then it
has to be solved there, and not in the Gitlab config.
Change-Id: I3891fcbd9dec8e5a4597404aa8131f28a1755a02
Reviewed-on: https://code.wireshark.org/review/36369
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
gcc-7 is included with build-essential on Ubuntu 18.04, not installing
an additional gcc-8 saves 110M on disk. The coverage should more or less
be the same, even very old compilers will catch typical build failures.
Change-Id: I03c95dcd1c83b02390489a47de7e2a8d2d757232
Reviewed-on: https://code.wireshark.org/review/36415
Reviewed-by: Dario Lombardo <lomato@gmail.com>
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Add missing output directory for Debian (obj-x86_64-linux-gnu). Restore
a working LD_LIBRARY_PATH setting for OpenSUSE, this has never worked
since v3.1.0rc0-590-gc77ee0008d.
Change-Id: I4f2c2e4636e7dd0233e7f08a88ba3789b021f274
Reviewed-on: https://code.wireshark.org/review/36378
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
As previously in commit 6bb691189c ("gitlab-ci: remove unnecessary jobs,
upgrade versions"), GCC 5 support would only catch two issues. From the
respective commits:
epan/dissectors/packet-synphasor.c:782:7: error: ‘ai’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
ui/qt/wireless_frame.h:63:25: error: in-class initialization of non-static data member is a C++11 extension [-Werror,-Wc++11-extensions]
These are normally just false positive warnings in old compilers, fixed
in newer GCC. It fails only because it was built from git master as
opposed to a release branch where -Werror is disabled by default.
Save some resources, and remove GCC 5. The CentOS-7 job still tests the
old GCC 4.8.5 compiler. Newer GCC versions are handled by other jobs.
Change-Id: I9f49d59e6476ea8bcd9e135dba1c94b37a2f047f
Reviewed-on: https://code.wireshark.org/review/36366
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
The point of testing multiple compiler versions is to catch bugs that
only occur in one particular version. After an analysis of 800+ failed
builds, it appears that a lot of the checks are redundant. Changes:
* Drop GCC 6 and keep GCC 5, Ubuntu 16.04 default GCC is 5.3. Fixes for
warnings in old compilers: commit 829d2c3752 (GCC 5 + 6), commit
e451ea0a4d (Clang 5, GCC 4 + 5). Maybe GCC 5 can also be dropped since
Ubuntu 18.04 is around for a while.
* Drop clang-nopcap, all errors caught by Clang were also caught by GCC.
One change caught by gcc-8-nopcap but not Clang: commit 036c3d6bc7.
* Remove Clang 5: even though it triggers some unique failures, no major
distro ships with it so remove it.
* Drop old Clang versions, newer versions catch all warnings from older.
* Upgrade to Clang 8 which is already installed in the Docker image.
* Remove EOL Fedora 28 and 29, use latest (31).
* Remove EOL openSUSE Leap 42.3, keep 15.1 as the latest.
Reduces from 20 jobs (15 build, 5 test) to 10 jobs (7 build, 3 test).
Change-Id: I4a355dd1a90cbbef9525bd63a8544fa3e22369a2
Reviewed-on: https://code.wireshark.org/review/36281
Reviewed-by: Anders Broman <a.broman58@gmail.com>
The former official image has been deprecated in favor of
opensuse/leap one.
Change-Id: I92de89060f2d8f3a679f0c332eebb65e5c13ba41
Reviewed-on: https://code.wireshark.org/review/33852
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Reviewed-by: Dario Lombardo <lomato@gmail.com>
This enables capture privileges for 'user' that is used in the tests.
travis.yml has been changed as well to reflect the setcap command in
the docs.
Change-Id: I0584e0df417112b1774b86ef8b28121799efe137
Reviewed-on: https://code.wireshark.org/review/33182
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
pytest-3 is the executable name on the docker image (ubuntu derivated).
Remove pip and pytest installation while here, since they're
provided by the docker image.
Change-Id: Iad2e9cafc42cd1e83b2868126abb91d5ee7bbd92
Reviewed-on: https://code.wireshark.org/review/33145
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
openSUSE now ships with 2.0.0 which is incompatible. Use 1.5.8 instead.
Disable ri and rdoc while here.
Change-Id: I40766ae4377f5998dcd58d02ecbdac665f78222a
Reviewed-on: https://code.wireshark.org/review/32547
Reviewed-by: Alexis La Goutte <alexis.lagoutte@gmail.com>
Gerald Combs