Disable these osmo-gsm-tester jobs, as they have been failing for a long
time and we currently won't spend time on fixing them.
Change-Id: I8db4fbe2c695e28eb9f7e738c5a848fb0071714a
Do not set a node for master-builds and gerrit-verifications to run on,
as this leads to needlessly blocking slots on the jenkins nodes.
Each of these jobs is a matrix job with a slave_axis. The matrix job
itself can run on the "builtin" node, as it practically doesn't cause
load - all it does is start other jobs in the matrix (even if there is
only one other job, which is the default). Jobs started in the matrix
job do run on a configured node already, as the slave_axis parameter is
set.
I've just noticed this problem because currently the raspberry pi nodes
are offline, and so the master-osmo-trx, master-osmo-pcu and other jobs
take up lots of build slots just to wait until the raspberry pi nodes
come online and an actual build can start.
Change-Id: I447dd152fcc942ec48d0bcfbadd496e879c882da
It seems that some users are running nightly builds in production on
EOL Debian 10, which we never imagined and hence didn't keep an archive
of those builds :/
Change-Id: I40c628e032e2c7ce8447657319a7c1206933c163
We typically keep nightly builds for the latest version of each
distribution. That is correct for CentOS and Debian, but we forgot
to update Ubuntu from 21.04 to 22.04 it seems :(
Change-Id: Ic82dfc7e8c4f4a362f7d05b5bc2069d42a0bcdb2
Ubuntu 22.04 is very much a stable/maintained distribution, so we
should keep an archive of "latest" builds for it.
Change-Id: I54988915ec4bcf2ebb1e5cd23ceb049789c56bca
Apparently the output of repoquery isn't always sorted, it started
failing on jenkins after moving the test inside qemu. Add an explicit
sort.
Fix for:
+ comm -23 osmocom_packages_all.txt blacklist.txt
comm: file 1 is not in sorted order
Related: OS#5365
Change-Id: Icb00df102555e06b66b1c2597488b625e3c77f1c
While I'm developing this, Jenkins is currently failing here. Make it
easier to debug this by printing the file contents.
Related: OS#5365
Change-Id: Ifbf4ca7f49c1f4441f84695aea0936515e01ffd4
The DISTRO variable is either debian10 or debian11, fix the broken
check. This condition is there in the first place, because we don't
build the usrp1 backend for centos8.
Change-Id: I987f27db257961faf06824df2dcc8f9db1fedccf
Related: OS#5365
Add services from new projects and enable previously disabled services,
now that this test runs in qemu and services have more permissions like
setting realtime priorities.
Related: OS#5365
Change-Id: Iec7db433cac4c77226e0f1ae2ba502de0d1a8a2b
Change repo-install-test to run inside of qemu instead of docker. This
job needs to run systemd to verify that the systemd services start up
properly. Running systemd inside docker was never officially supported,
it worked with cgroups1 but does not work anymore with cgroups2.
An alternative approach was running inside podman instead of docker
(running systemd inside of podman is officially supported). However we
would have needed various workarounds with podman and wouldn't be able
to test all Osmocom systemd services in the end, due to lack of
permissions (see review of I394918fc61de36acce65ffb33defcb8fc21801c4).
By running with a separate kernel inside qemu we can run all Osmocom
services.
Related: OS#5365
Change-Id: Ie7f1bccb05779cb3614016c0b965b810bbb1471b
Prepare to run repo-install-test inside qemu with kvm instead of
docker, see following patches.
Related: OS#5365
Change-Id: Ia7626fcd397d32e1180e320d4a76a65bcad01b7b
Remove the fallback clean up code, as it also may lead to images getting
removed right before we need to use them. Besides that, it should be
dead code by now since docuum should be running on all our jenkins nodes
to clean up old images based on last use date.
Change-Id: I9ca0c2ba245bdd75d9fb8eaf341055e8c2ab1b55
Don't delete images while they are being used, to fix these errors we
see from time to time in the middle of "docker build" on jenkins:
unknown parent image ID sha256:1b072e35048cd8b680eddabdc641ac678edb1184d222d5e7b3fbe0b3c333129a
This happens because "docker build" creates so-called dangling images
for each step processed of a Dockerfile. The "docker system prune" call
deletes these dangling images (among other things).
Remove the "docker system prune" call. We already have the docuum daemon
to deal with unused images (dangling and not dangling), it removes them
based on last use date so that the used space is always below a
configured limit. As it deletes images that haven't been used the
longest when it reaches the limit, it will not result in the problem
explained above.
Besides images, "docker system prune" also removes unused containers
(instances of images created with 'docker run' without --rm) and
networks. Add "docker container prune" and "docker network prune"
commands to remove them from now on.
Also remove the redundant container removal logic (previous it was
redundant with "docker system prune", now redundant with "docker
container prune").
Related: https://docs.docker.com/config/pruning/
Change-Id: Ia1b466eea43dd135373949e8e3e6b005c169ea0c
After building packages with the OBS scripts, show their contents. This
allows easy checking if config files were correctly packaged etc. By
adding it here, it will also show up at the end of the related jenkins
jobs for gerrit verifications.
Related: OS#5817
Change-Id: Ie30b07f35f7e41990fa352523427d86458291d4d
Podman is in the official debian repositories since debian 10, install
it from there instead of using a third-party repository.
Related: OS#5365
Change-Id: I01549915d2c8dfda5fa605a014a304229c3078f0
Let the gerrit-binpkgs and gerrit-pipeline-result jobs use the
"osmocom-gerrit" label. Having the -debian10/11 label previously was
useful to make sure this doesn't run on an ancient python version that
fails on f-strings. But since the debian9 builders were retired, this is
not needed anymore. These are the only jobs that used these, so now the
labels in the README can be simplified (following patch).
Related: OS#5793
Change-Id: Ieee95d25e036dcd4869d89b10c281179024672e7
The docker container creates the trial dir and some files inside of it
as root, therefore the rm -rf trial-* line fails. Add logic to fix the
permissions before attempting to remove old trial-* and when the script
exits.
Related: OS#5793
Change-Id: I91bd6656f33667f401247721bb34298a6c66fc62
The virtual osmo-gsm-tester has been failing for a long time, but the
test results were still green. This was because the job failed before it
removed old test results, and so those were used. Remove the old results
as early as possible to prevent this in the future.
Change-Id: I23f2f7b24fe41bbded2e05abbe6ea5e717b88e9e
Build the docker image right before using it, instead of building it in
another jenkins job update-osmo-ci-on-slaves via
osmo-ci-docker-rebuild.sh.
The logic in osmo-ci-docker-rebuild.sh was broken. I didn't realize at
the time that this image is only used for the virtual osmo-gsm-tester,
not the physical ones. But only the machines running the physical
osmo-gsm-tester have the /var/tmp/osmo-gsm-tester/state path. The
virtual osmo-gsm-tester isn't running on these machines but on generic
jenkins nodes.
Building the image right before using it makes sense for this job, as it
is the only user of the image. If it was already built from the same
Dockerfile, a cached version is used.
Fix for:
Unable to find image 'osmocom-build/osmo-gsm-tester:latest' locally
Fixes: 9139e76b ("osmo-ci-docker-rebuild: don't always build osmo-gsm-tester")
Change-Id: Icad9459de1d3a3a4e65ecacf7f903433bb504cc9
Use literal style instead of folded style in the shell argument of
osmo-gsm-tester_virtual as otherwise new lines behave unintuitively.
Related: https://yaml-multiline.info/
Change-Id: I862a0ed24bae2c713e857a459797e1a978df0560
Change it to "master" instead of "origin/master" to prepare for building
the docker image right before running it. The Makefile from
docker-playground.git picks up the OSMO_GSM_TESTER_BRANCH environment
variable and expects it to be in that format.
Change-Id: I4ad1ce71fd5b5b57a65267e0be51755f7495b07d
Remove the deb9 lxcs as they are being retired, and add two deb11 lxcs.
Coverity is quite big with 1.2 GiB, so deploying and upgrading this
takes some time. Having it on two hosts should be enough.
Related: OS#5793
Change-Id: I02d43d7cbdbcba5581aa5da0a1effa3db7b445c0
The tarball provided by coverity doesn't have the same version in the
name as in the directory inside the tarball.
Tarball:
cov-analysis-linux64-2022.06.tar.gz
Directory:
cov-analysis-linux64-2022.6.0
Work around this by extracting it into a subdirectory and using glob to
match the directory name.
Related: OS#5793
Change-Id: I2b96d1e47f2697706a042937b2852f0fc5032a7b
Coverity doesn't make the old version available for download and will
probably deprecate the previous version at some point. I'm about to
install it on new hosts, so upgrade to the latest version while at it.
Related: OS#5793
Change-Id: Id5700a9e5a46a9cfafc9959f5b77f9e21dd956b4
Make the binaries_ftp.osmocom.org ssh credentials available to more jobs
that need them.
Related: OS#5798
Change-Id: I4f719c6daba06dcfadd252a4030293bdc0e26337
Prepare to retire build2-deb9build-ansible. With changes in the script,
it doesn't need a persistent workspace anymore, it skips building source
tarballs if they exist on the remote server.
Depends: docker-playground I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce
Related: OS#5793
Change-Id: If56614caaa9c0c44419af596451d48c1970e9014
Remove the node attribute as these can run on any node. I've configured
the three deb11 nodes with "Usage: Use this node as much as possible"
instead of previously "Only build jobs with label expression matching
this node", so simply not specifying a node is now possible. Previously
it wouldn't have any node to run on as all of them were configured that
way.
Related: OS#5793
Change-Id: I2d67daf1cc7930649d097af23c18990cc3ae55fd
Don't attempt to build openbsc 1.4.0 and 1.4.1 in the
osmocom-build-old-tags-against-master job, as it fails to build with a
modern gcc with a potential null pointer dereference warning that gets
treated as error. This was fixed in 1.4.2 with openbsc patch
I93d816a20ba208e9fd32a1fc172a78ccd326e1ba.
A more modern gcc gets used, as this job gets changed to not only run on
jenkins nodes with debian 9 installed, but also more modern versions.
This job does not run in docker, and since it still passes it doesn't
seem worth changing that now.
Related: OS#5793
Change-Id: Ibf4b66195fbd70a68d63929b156a9aac54a699bc
Use new ssh keys for ftp.osmocom.org with consistent naming scheme. It
was not possible to use @ in the ID name for new IDs, so I've used an
underscore instead.
Change-Id: Id0cf70e70944cf1a1963e0046b1470a0485c05f8
Allow overriding the ssh agent credentials per repository, and use
binaries_ftp.osmocom.org where needed. Use docs_ftp.osmocom.org instead
of 97634b9d-a867-4f6d-825d-1da99813e93d so it's clear what the key is
for. I generated a new one with the new ID.
Fixes: OS#5798
Change-Id: I821968e09aaac3f6f6178de33fc909f81cb4abcb
Don't require the debian 9 lxc anymore for publishing docs, the ssh key
problem that made this necessary has been solved.
Closes: OS#5767
Change-Id: Ia4ee524db059a978bedc1d92804fb59d7513d2cb
Oliver Smith
Harald Welte