Commit Graph

1180 Commits

Author SHA1 Message Date
Oliver Smith 6dde3dcd38 coverity: build osmo-ggsn with --enable-gtp-linux
Change-Id: I5b240dbfe2369a5a54ab6115a8780cf9cc36b537
2023-02-01 12:06:16 +01:00
Oliver Smith 56939cc0a7 coverity/local_test: use all CPU cores
Change-Id: I018fda7cb85bb7e480dbc55d296c2e216098fc5e
2023-02-01 11:30:11 +01:00
Oliver Smith 6a6b19ec6f coverity: check if sdcc is installed
Check at the start if sdcc is installed, as it's required to build
libusrp. It might not be installed when testing locally.

Change-Id: I226f439f7eff657d01c667aae464c080b91d5a82
2023-02-01 10:58:04 +01:00
Oliver Smith 7bd02257c9 coverity/local_test: fix copy error
coverity/common.sh is a symlink to ../scripts/common.sh in this git
repository. The local_test.sh already adjusts the symlink so it points
to the right place after copying related scripts to /tmp/coverity.
However when running the script again, cp would fail with:

cp: '/home/user/code/osmo-dev/src/osmo-ci/coverity/common.sh' and '/tmp/coverity/common.sh' are the same file

Remove the symlink to avoid this error.

Change-Id: I081871e6e06547a90879bc4aa08ac2b5c5ddaeef
2023-02-01 10:58:04 +01:00
Oliver Smith 2f0de239e6 coverity: disable building osmo-isdntap
As it doesn't seem to build properly with default dahdi-linux, disable
it for now.

Related: OS#5886
Change-Id: Icb5cf7517a2ba0f3abb1d44337677f9502069ec6
2023-02-01 10:58:04 +01:00
Oliver Smith 9a637bc596 coverity: osmo-remsim: fix install errors
Set usbdropdir and serialconfdir to fix:
  /usr/bin/mkdir -p '/usr/lib/pcsc/drivers/libifd-osmo-remsim-client.bundle/Contents'
  /usr/bin/mkdir: cannot create directory ‘/usr/lib/pcsc’: Permission denied

Change-Id: I08e72bb06a823752154fe3e2011a42c144c657c4
2023-02-01 10:58:04 +01:00
Oliver Smith 2584debad5 coverity: add libgtpnl
Fix for:
  configure: error: Package requirements (libgtpnl >= 1.2.0) were not met

Change-Id: I534b8a550061e52d4c5ab95ad98e146a77f386bc
2023-02-01 10:58:04 +01:00
Oliver Smith 733626688d ansible/osmocom-jenkins-slave: add libnftables-dev
Install libnftables-dev, so osmo-upf can build successfully in the
coverity jenkins job.

Change-Id: Id72ce8ed5b662085a16f56756f8f93627834d2e2
2023-02-01 10:58:04 +01:00
Harald Welte 8abf46760a coverity: Add many missing projects
Add the following projects to our coverity test job:
  osmo-pcap
  osmo-e1d
  osmo-e1-recorder
  osmo-upf
  osmo-isdntap
  osmo-uecups
  osmo-remsim

Change-Id: Idee49aa4c15fcfb1a469db9e4978523af5608d70
2023-01-31 13:29:48 +01:00
Oliver Smith 4773f26d2d jobs/master,gerrit: fix pysim, sysmo-usim-tool
Don't attempt to create the CCACHE_DIR and chown it to osmocom-build for
the simtester jenkins node, and more generally all that were not
created with the ansible provisioning scripts. In case of simtester,
there is no osmocom-build user and we don't have the space there to use
ccache.

Fix for:
  chown: invalid user: ‘osmocom-build:osmocom-build’

Related: OS#5848
Change-Id: I339d9ba4ad0c959d7325820eb53bfa1f0e04c164
2023-01-25 12:59:30 +00:00
Oliver Smith a788a8b14f jobs/master,gerrit: add docker_run_ccache
Put arguments for ccache into a new docker_run_ccache variable, and use
it with all projects where it makes sense to use ccache.

Other projects use docker_run without the ccache parameters again:
* osmo_ss7, osmo_dia2gsup, pysim:
  Erlang/Python projects
* osmo-e1-hardware, ice40-usbtrace, no2bootloader, osmo-opencm3-projects:
  Don't use the default docker image
* osmo-gsm-tester:
  Only builds manuals (jenkins-build-manuals.sh)

Jenkins job builder's variable substitution doesn't replace {docker_run}
inside docker_run_ccache when used in cmd, so just duplicate the lines.

Related: OS#5848
Change-Id: Ib6d7f309b972979d3a3bc87f8fcf71f9e930c0da
2023-01-25 12:59:18 +00:00
Oliver Smith d909ad2997 jobs/gerrit: openbsc: set pipeline_binpkgs: false
Don't attempt to build deb packages for openbsc in gerrit-verifications.

Fix for:
  ERROR: unknown package: openbsc
  See packages_osmocom and packages_other in obs/lib/config.py

Change-Id: I3cd810366a6d711c1fdbf3d5ff03d507441307bd
2023-01-25 09:17:23 +00:00
Oliver Smith 945687b030 jobs/master,gerrit: use ccache to speed up builds
Related: OS#5848
Change-Id: I81f244ca09e660698df617549d70476fc96255e2
2023-01-24 10:56:03 +00:00
Oliver Smith c914967c53 jobs/master,gerrit: use minimal image for arm
Building debian-buster-jenkins on the raspberry pis may take > 1h,
depending on what changed. It has a lot of things we don't use, so build
and use a minimal image with the rpis instead.

Closes: OS#5863
Depends: docker-playground I4fb4b48b233acaef815c4c27ec6f17f12cfe836b
Change-Id: I73e62fbbf93824c0d37963039de4e00f26a43cbe
2023-01-24 08:04:54 +00:00
Oliver Smith 989c47bc91 scripts/osmo-build.sh / ARTIFACT_STORE: remove
This script was supposed to cache already built Osmocom libraries. But
it was only used with openbsc, didn't work as expected anymore (looking
at openbsc-master, it just builds all deps from source without caching)
and will be replaced with logic that allows using ccache in a future
patch.

Using ccache has the big advantage that we can cache all build objects
where the source files did not change, and not just do it on a
dependency level. Oftentimes only few source files change in our
patches, so we can really cache everything else that is not affected
within any git repository and not just a dependency that didn't change.

ccache also does automatic cache size management to ensure a cache stays
within a given size limit.

Related: OS#5848
Depends: openbsc Ib3272feec76b30412ca60dec204255b64e33831b
Change-Id: Id94d6126b476077d57839e4a884621b8c034f0c6
2023-01-23 13:31:08 +00:00
Alexander Couzens 113b295075 ansible: .gitignore: make poky regex more specific
Otherwise it would also prevent changes to the poky role

Change-Id: I3ac9a6129323f27ba387b44d8293d7b134716af8
2023-01-23 10:57:45 +00:00
Alexander Couzens e64f85a0ee ansible: install-poky-sdk: ensure it's not installed twice
Check if the poky path already exists. Even this wouldn't
detect if the poky has been only installed half way.

Related: OS#5801
Change-Id: I8ff8e72a5974378327d4692341b0a371c667b741
2023-01-23 10:57:30 +00:00
Alexander Couzens 6a290081c5 ansible: replace deprecated include by import_tasks
include_tasks and import_tasks have similar functionality.
Use import_tasks when importing non-variable depending files.
Further use import_tasks when not used within a loop.

Closes: OS#5797
Change-Id: I29766c65b9e1601a20aa8f20a438e8368c65f6a8
2023-01-23 10:57:30 +00:00
Oliver Smith 266a4363b3 lint: ignore SYMBOLIC_PERMS
As discussed here:
https://gerrit.osmocom.org/c/libosmocore/+/31024/2#message-10cca5d20f7a6455ccb3a4273f1c80bd74d1bb89

Change-Id: I47cfeef55c6cece95fed706b67c117274097977d
2023-01-20 09:25:16 +01:00
Oliver Smith 9e24272311 lint: enable BRACES_NOT_NECESSARY check
As seen in code review here:
https://gerrit.osmocom.org/c/libosmocore/+/30934

... we do care about not using braces for single statement blocks. Let
the linter comment on it.

src/osmo_io.c:143: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io.c:271: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io.c:306: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io_poll.c:63: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
src/osmo_io_poll.c:117: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks

Change-Id: I481d1b24a909173520a544ffd567bb8357729f2a
2023-01-19 09:17:02 +01:00
Oliver Smith 9f1889a494 jobs/gerrit-verifications: post comment on start
Post a link to gerrit when starting the build that contains the link to
the pipeline, without sending mail notifications.

This is useful when a gerrit verification build takes rather long, and
it's not clear if a build for gerrit verification was actually started
or not. Also I find this useful when debugging the CI scripts.

Change-Id: I75c5b8874f606739ff557ff0711bb9449a2b4259
2023-01-18 12:35:34 +00:00
Oliver Smith ed604a0bb3 jobs/gerrit-verifications-comment: rename
Rename the jenkins job from gerrit-pipeline-result to
gerrit-verifications-comment, as the next patch will not only use this
job to post a comment at the end of the gerrit verification, but also at
the beginning when the pipeline starts.

Give the pipeline_summary* scripts more generic names as well.

Change-Id: I1b947522aa5f2bb21f5e438db9df3420c998f1bc
2023-01-18 12:35:34 +00:00
Oliver Smith 2e3c8b6603 jobs/ttcn3: fixup for blocker remove in testsuites
Delete the part that was using the "blocking" variable too.

Fixes: 30ccbd3 ("jobs/ttcn3: don't block on same testsuite running")
Change-Id: I23bc6eb99d70447263b79bc4e33524a6932b43eb
2023-01-16 16:26:00 +01:00
Oliver Smith 30ccbd3f70 jobs/ttcn3: don't block on same testsuite running
Don't block jobs if other jobs with the same testsuite are already
running. This used to be necessary as we had subnets hardcoded per
testsuite, and therefore couldn't run it twice on the same host (e.g.
once against latest, once against nightly).

Related: OS#5802
Depends: docker-playground I57152b08ef0f38e17e7019a8df032189b03f56cf
Change-Id: I3159403e2ce2ec184ee48b4ff4f145e718e9b428
2023-01-16 15:23:57 +00:00
Oliver Smith b44f768b7b jobs/gerrit-verifications: osmo-trx: fix matrix
Add the missing comma between "--without-sse, --without-neon", so the
matrix works as expected. Without this patch, the two builds with these
flags didn't run.

Change-Id: I51e58765dd33b15df2e9f2eb09c1260724e4ec9c
2023-01-16 12:55:21 +01:00
Harald Welte 0334fe327e obs: add 2023q1
Change-Id: Id8fa36ac79a6d83d0384e629ef0b8c4db432cc0e
Related: SYS#6283
2023-01-11 21:50:20 +01:00
Vadim Yanitskiy be18bcd4ed coverity: build osmo-trx with MS TRX (--with-mstrx)
Change-Id: I4fbb052eacd8e6b8ddb8130d2e280a248719b250
Related: OS#5599
2022-12-29 18:12:39 +07:00
Harald Welte 4fdee2a0fc coverity: git pull --recursive / update submodules
osmo-trx has just started to use submodules, so let's add the
related git commands here.

Closes: OS#5846
Related: osmo-trx.git Change-Id I36c65a8c725c4da76dc70006cd96b0a2b6878e84
Change-Id: I876fa0152268bd7a43ca4c452d75c26ebadf7a6e
2022-12-27 18:18:29 +01:00
Oliver Smith 48614942c6 jobs/master-builds: use osmo-iuh/master for asn1c
As discussed with Pau, let's build the "osmo-iuh/master" branch of our
asn1c.git repository in jenkins. This branch is what we actually use for
osmo-iuh and "master" should track upstream master.

Related: OS#2435
Change-Id: Ic97a23a175b301e6f42f7ca67deb84dcc3ee3f90
2022-12-27 17:02:38 +00:00
Oliver Smith 57a9e54524 obs: put git submodules into source packages
We started using git submodules with osmo-trx. Adjust the obs scripts
to actually make the git submodules part of the source packages.

Note that this didn't fail in jenkins before with the rpm build
verification, as jenkins updates git submodules on its own.

Fix for:
[  165s] Makefile.am:32: error: required directory ./osmocom-bb/src/host/trxcon does not exist

Change-Id: I51b423f3885d6ead5c21a83bdf8ef6051dc34fe3
2022-12-23 15:10:19 +01:00
Oliver Smith b6d36134a7 jobs/update-osmo-ci-on-slaves-dp: fix node
This trigger job can run on any node, don't have it require a debian9
label that doesn't exist anymore.

Change-Id: I1a23a4c99fe007387f13e330b0b08f6941392829
2022-12-23 11:01:32 +00:00
Harald Welte 14772790d6 ttcn3-testsuites-kernel-git: Use normal e-mail address
There's no need for a hard-coded special list of e-mail addresses for
kernel related tests.

Change-Id: Ida4313997c9ecee13af22414e6a1799a2d9dd2af
2022-12-22 08:54:53 +01:00
Harald Welte d2bcdd73d1 don't report every unstable TTCN3 build
This saves deleting >= 80 e-mails every day.

Closes: OS#5841
Change-Id: I9593a3c3a94ab3f66ed4cd92906e7428a9d04bf5
2022-12-22 08:54:43 +01:00
Oliver Smith 50a54c923b jobs/update-osmo-ci-on-slaves: don't run on ogt
Don't run this job on the osmo-gsm-tester machines. Right now this job
takes quite some time to build various docker images which we don't even
need on osmo-gsm-tester. It also uploads osmo-ci.git and
osmo-gsm-manuals.git, which don't need to be updated on these if we
don't run the osmo-gsm-tester jenkins jobs on them (see previous
commit). According to Pau they would be needed for
osmo-gsm-tester_ttcn3.

Related: OS#5811
Change-Id: I39d72740d896981f069ad339013c1e1e3d4f3bba
2022-12-14 17:29:08 +01:00
Oliver Smith 9086b10d8d jobs/ogt: disable _run-prod, _run-rnd, _ttcn3
Disable these osmo-gsm-tester jobs, as they have been failing for a long
time and we currently won't spend time on fixing them.

Change-Id: I8db4fbe2c695e28eb9f7e738c5a848fb0071714a
2022-12-14 15:11:19 +01:00
Oliver Smith 4f1201ff06 jobs/master-builds,gerrit-verifications: fix node
Do not set a node for master-builds and gerrit-verifications to run on,
as this leads to needlessly blocking slots on the jenkins nodes.

Each of these jobs is a matrix job with a slave_axis. The matrix job
itself can run on the "builtin" node, as it practically doesn't cause
load - all it does is start other jobs in the matrix (even if there is
only one other job, which is the default). Jobs started in the matrix
job do run on a configured node already, as the slave_axis parameter is
set.

I've just noticed this problem because currently the raspberry pi nodes
are offline, and so the master-osmo-trx, master-osmo-pcu and other jobs
take up lots of build slots just to wait until the raspberry pi nodes
come online and an actual build can start.

Change-Id: I447dd152fcc942ec48d0bcfbadd496e879c882da
2022-12-12 12:23:47 +01:00
Harald Welte 47e90f10d2 obs-mirror: Also keep nightly builds for Debian 10
It seems that some users are running nightly builds in production on
EOL Debian 10, which we never imagined and hence didn't keep an archive
of those builds :/

Change-Id: I40c628e032e2c7ce8447657319a7c1206933c163
2022-12-10 12:17:59 +01:00
Harald Welte 94a57624b4 obs-mirror: Keep nightly packages for 22.04, not 21.04
We typically keep nightly builds for the latest version of each
distribution.  That is correct for CentOS and Debian, but we forgot
to update Ubuntu from 21.04 to 22.04 it seems :(

Change-Id: Ic82dfc7e8c4f4a362f7d05b5bc2069d42a0bcdb2
2022-12-10 12:15:12 +01:00
Harald Welte 34643301c3 obs-mirror: Keep latest build also for Ubuntu 22.04
Ubuntu 22.04 is very much a stable/maintained distribution, so we
should keep an archive of "latest" builds for it.

Change-Id: I54988915ec4bcf2ebb1e5cd23ceb049789c56bca
2022-12-10 12:14:37 +01:00
Harald Welte c0c48532be ansible inventory: add gtp0-deb11build
we're moving to debian 11 builders, gtp0 didn't have any

Change-Id: I1d387a460a9d71d537ab4790b4bb3177b1b96b88
2022-12-09 13:26:25 +01:00
Harald Welte 25a92ecb01 ansible inventory: We have recently killed gtp0-deb11build-i586
Change-Id: I15cb177e75f73fb8a06b17e42c99a6b573dc7574
2022-12-09 13:26:25 +01:00
Oliver Smith 9223287228 repo-install-test: sort centos pkgs from repoquery
Apparently the output of repoquery isn't always sorted, it started
failing on jenkins after moving the test inside qemu. Add an explicit
sort.

Fix for:
  + comm -23 osmocom_packages_all.txt blacklist.txt
  comm: file 1 is not in sorted order

Related: OS#5365
Change-Id: Icb00df102555e06b66b1c2597488b625e3c77f1c
2022-12-09 11:56:03 +01:00
Oliver Smith c551a3c297 repo-install-test: print osmocom_packages_all.txt
While I'm developing this, Jenkins is currently failing here. Make it
easier to debug this by printing the file contents.

Related: OS#5365
Change-Id: Ifbf4ca7f49c1f4441f84695aea0936515e01ffd4
2022-12-09 11:56:03 +01:00
Oliver Smith d5106e3d25 jobs/repo-install-test: adjust for qemu
Related: OS#5365
Change-Id: Ie7675811d1a5d6b7cf7e4378c86d747ef6f7a249
2022-12-09 11:56:03 +01:00
Oliver Smith 9f68d7d370 repo-install-test: update test_binaries
Related: OS#5365
Change-Id: Iaad0260b5872d29b41a97aac3ea5ba0af81c6adb
2022-12-09 11:56:03 +01:00
Oliver Smith 36215cfff2 repo-install-test: fix osmo-trx-usrp1 check
The DISTRO variable is either debian10 or debian11, fix the broken
check. This condition is there in the first place, because we don't
build the usrp1 backend for centos8.

Change-Id: I987f27db257961faf06824df2dcc8f9db1fedccf
Related: OS#5365
2022-12-09 11:56:03 +01:00
Oliver Smith f5bd274018 repo-install-test: update services list
Add services from new projects and enable previously disabled services,
now that this test runs in qemu and services have more permissions like
setting realtime priorities.

Related: OS#5365
Change-Id: Iec7db433cac4c77226e0f1ae2ba502de0d1a8a2b
2022-12-09 11:56:03 +01:00
Oliver Smith ed2d877bc6 repo-install-test: fix check for Release.key
$release_key is not defined, replace it with the path.

Change-Id: I6bd13cc6149db1f2ba25ef1ee343115d3c53cf3b
2022-12-09 10:55:15 +00:00
Oliver Smith aae43b731f repo-install-test: add SKIP_PREPARE_VM env var
Change-Id: I96c7aef70d9d2c6318ce4e43c24450461bc00d91
2022-12-09 10:55:15 +00:00
Oliver Smith 048492d1c8 repo-install-test: run inside qemu + kvm
Change repo-install-test to run inside of qemu instead of docker. This
job needs to run systemd to verify that the systemd services start up
properly. Running systemd inside docker was never officially supported,
it worked with cgroups1 but does not work anymore with cgroups2.

An alternative approach was running inside podman instead of docker
(running systemd inside of podman is officially supported). However we
would have needed various workarounds with podman and wouldn't be able
to test all Osmocom systemd services in the end, due to lack of
permissions (see review of I394918fc61de36acce65ffb33defcb8fc21801c4).

By running with a separate kernel inside qemu we can run all Osmocom
services.

Related: OS#5365
Change-Id: Ie7f1bccb05779cb3614016c0b965b810bbb1471b
2022-12-09 10:55:15 +00:00