Commit Graph

1162 Commits

Author SHA1 Message Date
Oliver Smith 266a4363b3 lint: ignore SYMBOLIC_PERMS
As discussed here:
https://gerrit.osmocom.org/c/libosmocore/+/31024/2#message-10cca5d20f7a6455ccb3a4273f1c80bd74d1bb89

Change-Id: I47cfeef55c6cece95fed706b67c117274097977d
2023-01-20 09:25:16 +01:00
Oliver Smith 9e24272311 lint: enable BRACES_NOT_NECESSARY check
As seen in code review here:
https://gerrit.osmocom.org/c/libosmocore/+/30934

... we do care about not using braces for single statement blocks. Let
the linter comment on it.

src/osmo_io.c:143: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io.c:271: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io.c:306: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io_poll.c:63: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io_poll.c:117: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks

Change-Id: I481d1b24a909173520a544ffd567bb8357729f2a
2023-01-19 09:17:02 +01:00
Oliver Smith 9f1889a494 jobs/gerrit-verifications: post comment on start
Post a link to gerrit when starting the build that contains the link to
the pipeline, without sending mail notifications.

This is useful when a gerrit verification build takes rather long, and
it's not clear if a build for gerrit verification was actually started
or not. Also I find this useful when debugging the CI scripts.

Change-Id: I75c5b8874f606739ff557ff0711bb9449a2b4259
2023-01-18 12:35:34 +00:00
Oliver Smith ed604a0bb3 jobs/gerrit-verifications-comment: rename
Rename the jenkins job from gerrit-pipeline-result to
gerrit-verifications-comment, as the next patch will not only use this
job to post a comment at the end of the gerrit verification, but also at
the beginning when the pipeline starts.

Give the pipeline_summary* scripts more generic names as well.

Change-Id: I1b947522aa5f2bb21f5e438db9df3420c998f1bc
2023-01-18 12:35:34 +00:00
Oliver Smith 2e3c8b6603 jobs/ttcn3: fixup for blocker remove in testsuites
Delete the part that was using the "blocking" variable too.

Fixes: 30ccbd3 ("jobs/ttcn3: don't block on same testsuite running")
Change-Id: I23bc6eb99d70447263b79bc4e33524a6932b43eb
2023-01-16 16:26:00 +01:00
Oliver Smith 30ccbd3f70 jobs/ttcn3: don't block on same testsuite running
Don't block jobs if other jobs with the same testsuite are already
running. This used to be necessary as we had subnets hardcoded per
testsuite, and therefore couldn't run it twice on the same host (e.g.
once against latest, once against nightly).

Related: OS#5802
Depends: docker-playground I57152b08ef0f38e17e7019a8df032189b03f56cf
Change-Id: I3159403e2ce2ec184ee48b4ff4f145e718e9b428
2023-01-16 15:23:57 +00:00
Oliver Smith b44f768b7b jobs/gerrit-verifications: osmo-trx: fix matrix
Add the missing comma between "--without-sse, --without-neon", so the
matrix works as expected. Without this patch, the two builds with these
flags didn't run.

Change-Id: I51e58765dd33b15df2e9f2eb09c1260724e4ec9c
2023-01-16 12:55:21 +01:00
Harald Welte 0334fe327e obs: add 2023q1
Change-Id: Id8fa36ac79a6d83d0384e629ef0b8c4db432cc0e
Related: SYS#6283
2023-01-11 21:50:20 +01:00
Vadim Yanitskiy be18bcd4ed coverity: build osmo-trx with MS TRX (--with-mstrx)
Change-Id: I4fbb052eacd8e6b8ddb8130d2e280a248719b250
Related: OS#5599
2022-12-29 18:12:39 +07:00
Harald Welte 4fdee2a0fc coverity: git pull --recursive / update submodules
osmo-trx has just started to use submodules, so let's add the
related git commands here.

Closes: OS#5846
Related: osmo-trx.git Change-Id I36c65a8c725c4da76dc70006cd96b0a2b6878e84
Change-Id: I876fa0152268bd7a43ca4c452d75c26ebadf7a6e
2022-12-27 18:18:29 +01:00
Oliver Smith 48614942c6 jobs/master-builds: use osmo-iuh/master for asn1c
As discussed with Pau, let's build the "osmo-iuh/master" branch of our
asn1c.git repository in jenkins. This branch is what we actually use for
osmo-iuh and "master" should track upstream master.

Related: OS#2435
Change-Id: Ic97a23a175b301e6f42f7ca67deb84dcc3ee3f90
2022-12-27 17:02:38 +00:00
Oliver Smith 57a9e54524 obs: put git submodules into source packages
We started using git submodules with osmo-trx. Adjust the obs scripts
to actually make the git submodules part of the source packages.

Note that this didn't fail in jenkins before with the rpm build
verification, as jenkins updates git submodules on its own.

Fix for:
[  165s] Makefile.am:32: error: required directory ./osmocom-bb/src/host/trxcon does not exist

Change-Id: I51b423f3885d6ead5c21a83bdf8ef6051dc34fe3
2022-12-23 15:10:19 +01:00
Oliver Smith b6d36134a7 jobs/update-osmo-ci-on-slaves-dp: fix node
This trigger job can run on any node, don't have it require a debian9
label that doesn't exist anymore.

Change-Id: I1a23a4c99fe007387f13e330b0b08f6941392829
2022-12-23 11:01:32 +00:00
Harald Welte 14772790d6 ttcn3-testsuites-kernel-git: Use normal e-mail address
There's no need for a hard-coded special list of e-mail addresses for
kernel related tests.

Change-Id: Ida4313997c9ecee13af22414e6a1799a2d9dd2af
2022-12-22 08:54:53 +01:00
Harald Welte d2bcdd73d1 don't report every unstable TTCN3 build
This saves deleting >= 80 e-mails every day.

Closes: OS#5841
Change-Id: I9593a3c3a94ab3f66ed4cd92906e7428a9d04bf5
2022-12-22 08:54:43 +01:00
Oliver Smith 50a54c923b jobs/update-osmo-ci-on-slaves: don't run on ogt
Don't run this job on the osmo-gsm-tester machines. Right now this job
takes quite some time to build various docker images which we don't even
need on osmo-gsm-tester. It also uploads osmo-ci.git and
osmo-gsm-manuals.git, which don't need to be updated on these if we
don't run the osmo-gsm-tester jenkins jobs on them (see previous
commit). According to Pau they would be needed for
osmo-gsm-tester_ttcn3.

Related: OS#5811
Change-Id: I39d72740d896981f069ad339013c1e1e3d4f3bba
2022-12-14 17:29:08 +01:00
Oliver Smith 9086b10d8d jobs/ogt: disable _run-prod, _run-rnd, _ttcn3
Disable these osmo-gsm-tester jobs, as they have been failing for a long
time and we currently won't spend time on fixing them.

Change-Id: I8db4fbe2c695e28eb9f7e738c5a848fb0071714a
2022-12-14 15:11:19 +01:00
Oliver Smith 4f1201ff06 jobs/master-builds,gerrit-verifications: fix node
Do not set a node for master-builds and gerrit-verifications to run on,
as this leads to needlessly blocking slots on the jenkins nodes.

Each of these jobs is a matrix job with a slave_axis. The matrix job
itself can run on the "builtin" node, as it practically doesn't cause
load - all it does is start other jobs in the matrix (even if there is
only one other job, which is the default). Jobs started in the matrix
job do run on a configured node already, as the slave_axis parameter is
set.

I've just noticed this problem because currently the raspberry pi nodes
are offline, and so the master-osmo-trx, master-osmo-pcu and other jobs
take up lots of build slots just to wait until the raspberry pi nodes
come online and an actual build can start.

Change-Id: I447dd152fcc942ec48d0bcfbadd496e879c882da
2022-12-12 12:23:47 +01:00
Harald Welte 47e90f10d2 obs-mirror: Also keep nightly builds for Debian 10
It seems that some users are running nightly builds in production on
EOL Debian 10, which we never imagined and hence didn't keep an archive
of those builds :/

Change-Id: I40c628e032e2c7ce8447657319a7c1206933c163
2022-12-10 12:17:59 +01:00
Harald Welte 94a57624b4 obs-mirror: Keep nightly packages for 22.04, not 21.04
We typically keep nightly builds for the latest version of each
distribution.  That is correct for CentOS and Debian, but we forgot
to update Ubuntu from 21.04 to 22.04 it seems :(

Change-Id: Ic82dfc7e8c4f4a362f7d05b5bc2069d42a0bcdb2
2022-12-10 12:15:12 +01:00
Harald Welte 34643301c3 obs-mirror: Keep latest build also for Ubuntu 22.04
Ubuntu 22.04 is very much a stable/maintained distribution, so we
should keep an archive of "latest" builds for it.

Change-Id: I54988915ec4bcf2ebb1e5cd23ceb049789c56bca
2022-12-10 12:14:37 +01:00
Harald Welte c0c48532be ansible inventory: add gtp0-deb11build
we're moving to debian 11 builders, gtp0 didn't have any

Change-Id: I1d387a460a9d71d537ab4790b4bb3177b1b96b88
2022-12-09 13:26:25 +01:00
Harald Welte 25a92ecb01 ansible inventory: We have recently killed gtp0-deb11build-i586
Change-Id: I15cb177e75f73fb8a06b17e42c99a6b573dc7574
2022-12-09 13:26:25 +01:00
Oliver Smith 9223287228 repo-install-test: sort centos pkgs from repoquery
Apparently the output of repoquery isn't always sorted, it started
failing on jenkins after moving the test inside qemu. Add an explicit
sort.

Fix for:
  + comm -23 osmocom_packages_all.txt blacklist.txt
  comm: file 1 is not in sorted order

Related: OS#5365
Change-Id: Icb00df102555e06b66b1c2597488b625e3c77f1c
2022-12-09 11:56:03 +01:00
Oliver Smith c551a3c297 repo-install-test: print osmocom_packages_all.txt
While I'm developing this, Jenkins is currently failing here. Make it
easier to debug this by printing the file contents.

Related: OS#5365
Change-Id: Ifbf4ca7f49c1f4441f84695aea0936515e01ffd4
2022-12-09 11:56:03 +01:00
Oliver Smith d5106e3d25 jobs/repo-install-test: adjust for qemu
Related: OS#5365
Change-Id: Ie7675811d1a5d6b7cf7e4378c86d747ef6f7a249
2022-12-09 11:56:03 +01:00
Oliver Smith 9f68d7d370 repo-install-test: update test_binaries
Related: OS#5365
Change-Id: Iaad0260b5872d29b41a97aac3ea5ba0af81c6adb
2022-12-09 11:56:03 +01:00
Oliver Smith 36215cfff2 repo-install-test: fix osmo-trx-usrp1 check
The DISTRO variable is either debian10 or debian11, fix the broken
check. This condition is there in the first place, because we don't
build the usrp1 backend for centos8.

Change-Id: I987f27db257961faf06824df2dcc8f9db1fedccf
Related: OS#5365
2022-12-09 11:56:03 +01:00
Oliver Smith f5bd274018 repo-install-test: update services list
Add services from new projects and enable previously disabled services,
now that this test runs in qemu and services have more permissions like
setting realtime priorities.

Related: OS#5365
Change-Id: Iec7db433cac4c77226e0f1ae2ba502de0d1a8a2b
2022-12-09 11:56:03 +01:00
Oliver Smith ed2d877bc6 repo-install-test: fix check for Release.key
$release_key is not defined, replace it with the path.

Change-Id: I6bd13cc6149db1f2ba25ef1ee343115d3c53cf3b
2022-12-09 10:55:15 +00:00
Oliver Smith aae43b731f repo-install-test: add SKIP_PREPARE_VM env var
Change-Id: I96c7aef70d9d2c6318ce4e43c24450461bc00d91
2022-12-09 10:55:15 +00:00
Oliver Smith 048492d1c8 repo-install-test: run inside qemu + kvm
Change repo-install-test to run inside of qemu instead of docker. This
job needs to run systemd to verify that the systemd services start up
properly. Running systemd inside docker was never officially supported,
it worked with cgroups1 but does not work anymore with cgroups2.

An alternative approach was running inside podman instead of docker
(running systemd inside of podman is officially supported). However we
would have needed various workarounds with podman and wouldn't be able
to test all Osmocom systemd services in the end, due to lack of
permissions (see review of I394918fc61de36acce65ffb33defcb8fc21801c4).

By running with a separate kernel inside qemu we can run all Osmocom
services.

Related: OS#5365
Change-Id: Ie7f1bccb05779cb3614016c0b965b810bbb1471b
2022-12-09 10:55:15 +00:00
Oliver Smith d198fe6b38 Cosmetic: repo-install-test: sort env vars comment
Related: OS#5365
Change-Id: Ic21b46e614c7bfd8e0306599b4d33a1b963db376
2022-12-09 10:55:15 +00:00
Oliver Smith 47dde8559f ansible: add rules to install qemu images
Prepare to run repo-install-test inside qemu with kvm instead of
docker, see following patches.

Related: OS#5365
Change-Id: Ia7626fcd397d32e1180e320d4a76a65bcad01b7b
2022-12-09 10:55:15 +00:00
Oliver Smith 21a641d6c2 scripts/docker-cleanup: remove fallback code
Remove the fallback clean up code, as it also may lead to images getting
removed right before we need to use them. Besides that, it should be
dead code by now since docuum should be running on all our jenkins nodes
to clean up old images based on last use date.

Change-Id: I9ca0c2ba245bdd75d9fb8eaf341055e8c2ab1b55
2022-12-09 10:40:58 +01:00
Oliver Smith a7df704d4d scripts/docker-cleanup: fix timing problems
Don't delete images while they are being used, to fix these errors we
see from time to time in the middle of "docker build" on jenkins:
unknown parent image ID sha256:1b072e35048cd8b680eddabdc641ac678edb1184d222d5e7b3fbe0b3c333129a

This happens because "docker build" creates so-called dangling images
for each step processed of a Dockerfile. The "docker system prune" call
deletes these dangling images (among other things).

Remove the "docker system prune" call. We already have the docuum daemon
to deal with unused images (dangling and not dangling), it removes them
based on last use date so that the used space is always below a
configured limit. As it deletes images that haven't been used the
longest when it reaches the limit, it will not result in the problem
explained above.

Besides images, "docker system prune" also removes unused containers
(instances of images created with 'docker run' without --rm) and
networks. Add "docker container prune" and "docker network prune"
commands to remove them from now on.

Also remove the redundant container removal logic (previous it was
redundant with "docker system prune", now redundant with "docker
container prune").

Related: https://docs.docker.com/config/pruning/
Change-Id: Ia1b466eea43dd135373949e8e3e6b005c169ea0c
2022-12-09 10:40:33 +01:00
Oliver Smith 4fe56c0968 obs: build_deb/rpm: show contents after build
After building packages with the OBS scripts, show their contents. This
allows easy checking if config files were correctly packaged etc. By
adding it here, it will also show up at the end of the related jenkins
jobs for gerrit verifications.

Related: OS#5817
Change-Id: Ie30b07f35f7e41990fa352523427d86458291d4d
2022-12-08 15:42:59 +01:00
Oliver Smith 54a36bcb7d ansible/podman: install from debian repos
Podman is in the official debian repositories since debian 10, install
it from there instead of using a third-party repository.

Related: OS#5365
Change-Id: I01549915d2c8dfda5fa605a014a304229c3078f0
2022-12-08 13:04:31 +00:00
Oliver Smith 5d3fb63b3d jobs/README: modernize
Remove outdated debian 9 related information, update section on labels.

Change-Id: I6616f4bdacc90c295db7996855e9aa2b48172b43
2022-12-06 11:17:16 +01:00
Oliver Smith 3b5464d0ed jobs: retire osmocom-gerrit-debian* labels
Let the gerrit-binpkgs and gerrit-pipeline-result jobs use the
"osmocom-gerrit" label. Having the -debian10/11 label previously was
useful to make sure this doesn't run on an ancient python version that
fails on f-strings. But since the debian9 builders were retired, this is
not needed anymore. These are the only jobs that used these, so now the
labels in the README can be simplified (following patch).

Related: OS#5793
Change-Id: Ieee95d25e036dcd4869d89b10c281179024672e7
2022-12-06 11:17:16 +01:00
Oliver Smith fd7c88e992 jobs/update-osmo-ci: remove deb9 nodes
Related: OS#5793
Change-Id: Id2bf3c273848c1de12a3dce3bc007dd205bf4af7
2022-12-06 11:17:16 +01:00
Oliver Smith 9bdd99b2ff ansible/hosts: remove debian 9 lxcs
Related: OS#5793
Change-Id: I465d93e5254ba2f65796fcfb62ea1063a0eb72c6
2022-12-06 11:17:16 +01:00
Oliver Smith 0428da0f6d jobs/osmo-gsm-tester virtual: fix permissions
The docker container creates the trial dir and some files inside of it
as root, therefore the rm -rf trial-* line fails. Add logic to fix the
permissions before attempting to remove old trial-* and when the script
exits.

Related: OS#5793
Change-Id: I91bd6656f33667f401247721bb34298a6c66fc62
2022-12-02 10:50:07 +01:00
Oliver Smith d5e0fe011d jobs/osmo-gsm-tester virtual: send mails on error
Change-Id: If91644a3dfea1af259f7a511a7784cf32d8d23ea
2022-12-01 16:41:10 +01:00
Oliver Smith 23cf9b4b05 jobs/osmo-gsm-tester: remove old test result early
The virtual osmo-gsm-tester has been failing for a long time, but the
test results were still green. This was because the job failed before it
removed old test results, and so those were used. Remove the old results
as early as possible to prevent this in the future.

Change-Id: I23f2f7b24fe41bbded2e05abbe6ea5e717b88e9e
2022-12-01 15:53:30 +01:00
Oliver Smith 4bd7da1c9d jobs/osmo-gsm-tester virtual: fix opts desciption
The default value is what it says in the line above, not empty.

Change-Id: I5ce8a47b43525dcca6bf54184a48f11b0c87c19e
2022-12-01 15:37:18 +01:00
Oliver Smith d3321f9434 jobs/osmo-gsm-tester virtual: don't limit to deb9
Related: OS#5793
Change-Id: I8e7ffa65b4ff6d1816ad167367b0f6ff8c00e64b
2022-12-01 15:04:38 +01:00
Oliver Smith ce5875fd07 jobs/osmo-gsm-tester virtual: fix docker img
Build the docker image right before using it, instead of building it in
another jenkins job update-osmo-ci-on-slaves via
osmo-ci-docker-rebuild.sh.

The logic in osmo-ci-docker-rebuild.sh was broken. I didn't realize at
the time that this image is only used for the virtual osmo-gsm-tester,
not the physical ones. But only the machines running the physical
osmo-gsm-tester have the /var/tmp/osmo-gsm-tester/state path. The
virtual osmo-gsm-tester isn't running on these machines but on generic
jenkins nodes.

Building the image right before using it makes sense for this job, as it
is the only user of the image. If it was already built from the same
Dockerfile, a cached version is used.

Fix for:
  Unable to find image 'osmocom-build/osmo-gsm-tester:latest' locally

Fixes: 9139e76b ("osmo-ci-docker-rebuild: don't always build osmo-gsm-tester")
Change-Id: Icad9459de1d3a3a4e65ecacf7f903433bb504cc9
2022-12-01 15:04:38 +01:00
Oliver Smith cc7af6cd5a jobs/osmo-gsm-tester virtual: use literal style
Use literal style instead of folded style in the shell argument of
osmo-gsm-tester_virtual as otherwise new lines behave unintuitively.

Related: https://yaml-multiline.info/
Change-Id: I862a0ed24bae2c713e857a459797e1a978df0560
2022-12-01 15:04:38 +01:00
Oliver Smith 3307a90691 jobs/osmo-gsm-tester virtual: fix default branch
Change it to "master" instead of "origin/master" to prepare for building
the docker image right before running it. The Makefile from
docker-playground.git picks up the OSMO_GSM_TESTER_BRANCH environment
variable and expects it to be in that format.

Change-Id: I4ad1ce71fd5b5b57a65267e0be51755f7495b07d
2022-12-01 15:04:38 +01:00