Fetching the complete Wireshark repository in GitLab's CI currently
takes about 3.5 minutes. GitLab CI lets you do shallow cloning, but only
up to 1000 commits. This isn't sufficient for any of our jobs that might
need a reachable tag since it's currently about 1200 commits away in
master.
Set GIT_DEPTH to 1 (which appears to trigger the required shallow clone
logic in GitLab's CI), then deepen it by setting GIT_FETCH_EXTRA_FLAGS
to "--depth=5000". This is *should* be much faster than a full checkout
and still give us a reachable tag:
$ git rev-list --count v3.3.0rc0..v3.5.0rc0
2330
$ git rev-list --count v3.1.0rc0..v3.3.0rc0
2198
$ git rev-list --count v2.9.0rc0..v3.1.0rc0
3449
Overriding the definition of the rpmbuild macro cmake_build on the
command line, so that it doesn't include the string "--verbose", should
prevent cmake --build from being run with --verbose, and thus prevent it
from running Ninja with the -v flag, and thus prevent a bunch of extra
noisy output from being produced for every build command, and thus
prevent the build log from hitting GitLab's 4MB limit.
Unlike piping the output of "ninja rpm-package" to sed, this means that
the exit status of "ninja rpm-package", rather than the exit status of
sed, is tested.
Filter out the -W and -f options from the output of "ninja rpm-package",
to try to cut down the verbosity.
Do that instead of passing --quiet to rpmbuild, so we get command
progress messages, just without the extra junk.
If the FORCE_CMAKE_NINJA_QUIET environment variable is set, have the
top-level CMakeLists.txt add the -q flag to the arguments to rpmbuild.
That appears to reduce the amount of output.
Set that environment varible in the rpm-fedora build.
See whether unsetting VERBOSE causes the build process not to print the
compile command, complete with a big list of compiler options, for every
single file being built, in the hopes of producing less than the limit
of 4MB of log text.
See whether setting DH_QUIET to 1 and MAKEFLAGS to --silent causes the
build process not to print the compile command, complete with a big list
of compiler options, for every single file being built, in the hopes of
producing less than the limit of 4MB of log text.
Ninja keeps track of its built files in .ninja_log, so if you copy a
pre-built target into a fresh build directory, Ninja will ignore and
overwrite it. This includes the tarball generated by the 'dist' target.
In get-export-release.sh, check for a preexisting tarball and preserve
it by default. This lets us pass the dist tarball from one GitLab CI
stage to other stages without recreating it. It's also arguably the
right thing to do in general, since we record and publish the tarball
hashes for each release and different contents for the same filename can
cause confusion.
Move the dist tarball to the build directory in .gitlab-ci.yml, and add
a note about using the tarball exclusively.
We currently disable shallow clones in our general pipeline settings
because jobs that create packages need a usable `git describe`. None of
our merge request jobs create packages, so clone shallowly in order to
speed things up.
Remove the last package installation instances from .gitlab-ci.yml. Add
a comment at the top noting that dependencies should be added to our
Dockerfiles instead.
The wireshark/wireshark-ubuntu-dev image was recently upgraded to Ubuntu
20.04. Since then the following tests have been failing:
---- FAILED
test/suite_capture.py::case_wireshark_capture::test_wireshark_capture_from_fifo
FAILED
test/suite_capture.py::case_wireshark_capture::test_wireshark_capture_from_stdin
FAILED
test/suite_capture.py::case_tshark_capture::test_tshark_capture_from_fifo
FAILED
test/suite_capture.py::case_tshark_capture::test_tshark_capture_from_stdin
FAILED
test/suite_capture.py::case_dumpcap_capture::test_dumpcap_capture_from_fifo
FAILED
test/suite_capture.py::case_dumpcap_capture::test_dumpcap_capture_from_stdin
FAILED
test/suite_capture.py::case_dumpcap_autostop::test_dumpcap_autostop_filesize
FAILED
test/suite_capture.py::case_dumpcap_autostop::test_dumpcap_autostop_packets
FAILED
test/suite_capture.py::case_dumpcap_ringbuffer::test_dumpcap_ringbuffer_filesize
FAILED
test/suite_capture.py::case_dumpcap_ringbuffer::test_dumpcap_ringbuffer_packets
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_multi_in_multi_out
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_multi_in_single_out
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_single_in_multi_out
FAILED
test/suite_capture.py::case_dumpcap_pcapng_sections::test_dumpcap_pcapng_single_in_single_out
FAILED
test/suite_clopts.py::case_dumpcap_options::test_dumpcap_interface_chars
FAILED
test/suite_clopts.py::case_dumpcap_options::test_dumpcap_invalid_chars
FAILED
test/suite_clopts.py::case_dumpcap_options::test_dumpcap_valid_chars
----
This is because dumpcap can't load our local libraries after setting cap_net_raw+cap_net_admin:
----
-- Begin stderr for command ('/builds/wireshark/wireshark/build/run/dumpcap', '-D') --
/builds/wireshark/wireshark/build/run/dumpcap: error while loading shared libraries: libwsutil.so.0: cannot open shared object file: No such file or directory
-- End stderr for command ('/builds/wireshark/wireshark/build/run/dumpcap', '-D') --
----
Add $(pwd)/run to our rpath.
Re-enable Fedora build and add CentOS 8 and OpenSUSE 15.2 builds.
Fedora 33 does out of build tree cmake builds and needs spec file changes.
CentOS 8 has some changes with cmake and other packages that are similar to
older Fedora, and needs extra repositories enabled to get -devel packages
(still missing -devel for some optional libraries). OpenSUSE Leap 15.2 also
has some changes needed to build. Note that OpenSUSE Leap 15.1 is EOL
at the end of November 2020. Fixes#16971
Adds job of building the HTML versions of the
Wireshark User Guide and Wireshark Developer Guide.
PDF versions are not built because it would take significantly longer
(~10 mins instead of ~5 minutes)
Move the pre-commit and validate-commit checks to their own job. This
lets us quickly re-run validate-commit in case the submitter forgets to
check "Allow commits from members who can merge to the target branch"
Shorten the "merge-request" prefix to "merge-req" in order to make it
easier to distinguish between each job in the GitLab UI.
The Windows runners are constrained by the following:
* We require quite a bit of software not present in the stanadard
runner[1] which takes a long time to install, including Python, Perl,
and Qt.
* You can't specify an arbitrary Docker image like you can with Linux
runners.
As a result we have a project-specific runner for wireshark/wireshark
that runs a custom Windows Docker image. Update the CI rules so that
merge-request:windows only runs for gitlab.com/wireshark/wireshark. The
GitLab documentation recommends rules over only/except, so switch to
them.
Fixup .editorconfig while we're here.
[1]https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md
Git is currently broken in gitlab-runner 13.3.0 + Windows + Docker...
https://gitlab.com/gitlab-org/gitlab/-/issues/239013
...but it appears to work fine in gitlab-runner 13.2.0. I reverted the build machine to that version, so we can enable the merge-request:windows build.
Install lintian instead of devscripts (which pulls in lintian + many
other packages) in .gitlab-ci.yml. Add lintian to DEBDEPS_LIST in
debian-setup.sh.
Copy the Buildbot petri dish builder steps to corresponding GitLab CI
jobs. Update validate-commit.py to look for old "Bug:" and "Ping-Bug:"
references and have it call `git stripspace` directly. tools/commit-msg
was specific to Gerrit, so remove it.
Change-Id: Icbc54709052f44c941db9ad6a5dcf596292782a2
Gerald Combs