Create debian bookworm (12) based containers for building all Osmocom
projects in master/gerrit/ttcn3 jenkins jobs.
This is a combination of debian-bullseye-build (ttcn3) and
debian-bullseye-jenkins(-arm) (master/gerrit), which were used for this
purpose but had several differences. Combining them solves the
following problems:
* The naming was confusing, as both the -jenkins and -build containers
were used in jenkins and were used to build the Osmocom stack.
* This lead to adding the dependencies to the wrong containers / not
adding them to both containers.
* Now we are sure that if a program builds from source in the master
and gerrit verifications, it will also build in the ttcn3 jobs.
Other notable changes from the debian 11 containers:
* Python2 is finally gone (OS#5950, for the few projects that still
need it like openbsc, I'll use debian 11 for the jenkins jobs)
* Removed osc and git-buildpackage, now that OBS package building is
done in different docker containers (see osmo-ci/scripts/obs)
* Combined multiple RUN commands into one as they the image build
down, used set -x to display what commands run exactly
* Use UID variable instead of hardcoding 1000
* Optimized order of commands, so e.g. the LLVM for Arm that we will
rarely change gets installed/cached before the big list of apt pkgs
* Replaced old git.osmocom.org urls
* Removed apt-get upgrade; the base image will get upgraded from time
to time, no need to upgrade within the image
* Add --depth=1 option to git clones and remove temporary git clones
* Removed generating locales code, it was only done by one of the two
images and I couldn't find in the git log why we would need this (if
we do need it we can add it back later)
The following patches add more debian bookworm containers and have
similar improvements.
Related: OS#6057
Change-Id: I49aaf62b5b97775f923453611df3b91354a640a0
Fix current jenkins test breakage:
Adjust HNBGW_Tests.cfg after changes to osmo-ttcn3-hacks in
"hnbgw: prepare cn pool: add multiple MSCs and SGSNs"
Ia29565cabc072de9aa46565b57232e1eda65874f
Change-Id: I912e8ad1f528d0d726b778a02a0f54c77b298f54
Prepare for osmo-bsc I8b3d5efb769437a5d3036e1e627b8d477275d93e where
jenkins.sh will explicitly build meas_vis and configure.ac will stop if
libcdk5-dev is not installed.
Currently meas_vis only gets built if libcdk5-dev is installed, meaning
right now it doesn't get built by the jenkins verification and
master-build jobs (only by the debian package build verification).
Related: OS#5173
Change-Id: I015b6d7cb834e99ea5d04206ba5f8c519c4e6af1
an MME testsuite exists in osmo-ttcn3-hacks but it is not executed in
docker yet.
Related: OS#5760
Change-Id: Ic1ce1dace47353b0dcf321c923a78567e3ddefb9
OsmoBTS currently uses PCUIF v.10 but will move to v.11 soon. (see
Depends). Unfortuantely this means that we have to execute the TTCN3
testsuite in master with PCUIF v.11 and in latest with PCUIF v.10. This
will be the case until the current master becomes the new latest on the
next release.
Depends: osmo-bts.git I25816ac12e63cc6b641eb414e6bc7eaa9c85fc25
Depends: osmo-ttcn3-hacks.git I08de02e951e10bc8b4381cc2ad32e63f2747e3c4
Change-Id: Ia28bc0d6d3cbfe63be19443db86631fb67bb80fb
Related: OS#5927
It was recently decided it's a good practice to always specify the role
and sctp-role for all ASPs configured in the VTY, since it's an
important configuration providing feedback on the network setup
expectancies.
Change-Id: Ia495bc3c5dd4421e5730c74b2f5dc4e4cdc1a673
While inspecting the running docker container I found that the script
already runs as redmine user. I had assumed that it runs as root
earlier. Running 'su' as redmine user fails, therefore it currently does
not run the fetch_changesets command.
Related: OS#6083
Change-Id: I195311b93868eb5fd73ad6e3c165d48f3b7b53ce
Do "docker run" directly inside the Makefile, without wrapping it
through the "run" target in make/Makefile. Use $(USER)/ttcn3-bts-test as
image name, instead of $(REGISTRY)/$(USER)/ttcn3-bts-test. The
$(REGISTRY)/ part is hardcoded in make/Makefile.
This fixes TTCN3-centos-bts-test, because it uses the ttcn3-bts-test
image that gets built in jenkins.sh before running 'make cfg'. Without
the patch, an image downloaded some time ago from the registry gets
used (a new image is not pulled), leading to the stale config.
Fixes: OS#6065
Change-Id: Iebd81e814d84e8cf8360568419ceda7ffc38e742
Until recently, the asp-clnt-* ASPs, which have specific handling in osmo_sccp_simple_client_on_ss7_id(),
were being always forcedly set to sctp-role CLIENT by code in that
function.
This prevented user of that API from explicitly configuring the ASP as
"sctp-role server" through the VTY as the option would be overwritten silently.
Now, the sctp-role from config is followed if the ASP is
defined/configured through the VTY (not dynamically created at the time
osmo_sccp_simple_client_on_ss7_id() is called).
Since the default for a VTY-specified ASP is to be in "sctp-role
server", the config files need to be updated to properly configure the
ASP to be in "sctp-role client", which is the desired mode here.
Same applies for "role", where the default is SG but it is actually used
as "ASP" here.
Change-Id: Idf84502ffa199926a5f0ee616313b515743811ab
Depends: osmo-ttcn3-hacks.git Change-Id I4eb5b5f6b4b24df079b4c74e2a2e2ebb8769b0bd
In osmo-ttcn3-hacks.git I81052ece7d1cc8b43da6155356ed1c4d9620acdc
we're adding support for so-called "norctrx"tests, covering
operation without routing-context IEs. For this, we need additional
AS/ASP definitions in the osmo-stp.cfg
Change-Id: Ie00d96d04dd36c71c79f3b73713c3fe1c26a5145
Related: OS#6003
Every test needs to wait for T3*N3 seconds before exiting in order to
make sure the queues waiting for duplicates are cleared, otherwise state
from previous state may affect next tests.
Since 1.10.0, osmo-ggsn supports having those timers changed through
VTY. Let's set them to a lower value than default specified by spec, in
order to speed up each test 15 seconds -> 2 seconds.
Only ttcn3-ggsn (against osmo-ggsn) and not ttcn3-ggsn-ogs (against
open5gs-smfd) is updated, since the later doesn't support changing the
default timer values using the config files yet.
Related: OS#5485
Change-Id: Ica346bd791a67b72fad597702d750383e8ac5895
Add iproute2 to make "ss" available. It is used in the openbsc testsuite
to dump socket statistics. If it isn't available, it does not cause
tests to fail, but prints an error message that looks like it might be
the real cause for the test failures:
> ss -tn
/bin/sh: 1: ss: not found
I've also asked Harald if we should just retire the master-openbsc job,
but we want to keep it around to ensure new libosmocore builds with this
old codebase.
Change-Id: I306caeda9bcd4020734eb3b343420c8115958ded
Add a bullseye based container that can be used instead of
debian-buster-jenkins-arm. The master-builds jenkins jobs will attempt
to use this jenkins job for the arm builds after setting the main
default image from debian-buster-jenkins to debian-bullseye-jenkins.
Related: OS#5949
Change-Id: I8e1741f86ffb8abd658d1e4e0415dfd11fb1a8a1
Prepare to let jenkins master-builds use debian-bullseye-jenkins
instead of debian-buster-jenkins. Add almost everything from the buster
Dockerfile to the bullseye Dockerfile that is not there, except for:
dh-systemd (not available in debian11)
Also add python2, as some projects still require it to run tests
(OS#5950).
Related: OS#5949
Change-Id: I51925d0ab9e5a779379efab59c381ef12fb60929
Add the ghostscript package, it contains ps2pdf which is now used in
osmo-gsm-manuals to shrink pdfs.
Related: SYS#6380
Change-Id: If7b30d0c948604211f4f5242aca8564c310a6ce3
As "docker kill" / "docker container kill" (alias) doesn't block until
the given container stops, make sure to always run "docker wait"
afterwards.
Closes: OS#5928
Change-Id: I0242ece96541d8036ebbf8b0f498ebf231db26b5
This Dockerfile allows building a given repo and branch of
titan.core.git which can then be used by any ttcn-*-test/ environment by
simply changing its Dockerfile "FROM" line from
$REGISTRY/$USER/debian-bullseye-titan to
$REGISTRY/$USER/debian-bullseye-titan-mater.
This is useful to debug or develop titan.core or test whether a given
testsuite works fine with a newer version of titan.core.
Change-Id: I19ee98a319ccad167d06c4f183fe80ecac909483
When running jenkins.sh as root or any other user with UID different
than UID=1000 of the 'osmocom' user we have in open5gs-{master,latest}
images, programs spawned by osmo-uecups-daemon (e.g. ping) may fail to
start. In such cases, osmo-uecups-daemon reports unexpected program
termination with exit code=512. This is happening because the
stdout and stderr of a spawned program are being redirected to
"$VOL_BASE_DIR/pgw-tester/TESTCASE.prog.std{out,err}", for which
the 'osmocom' user needs to have write permissions.
Change-Id: Icb6b30e618e290d974a919fef34c6b5cb7eeb648
Related: OS#5913
As stated in the ticket, both open5gs-{nrf,smf} daemons refuse to
start since build #326 because of the following error:
[sbi] ERROR: TLS enabled but no server key (../lib/sbi/context.c:186)
The problem is that the recent open5gs (v2.6.0-39-g22be888 at the
moment of writing) is attempting to employ TLS verification for SBI
even if it's not explicitly configured in the config files.
commit 05fbaf69587488e53b5e741a9ada9f9fa5749322
Author: Sukchan Lee <acetcom@gmail.com>
Date: Sat Feb 18 10:58:29 2023 +0900
[SBI] HTTP2-TLS verification - ConfFile Changed
Our config files are slightly outdated, so let's take a chance
to update them and specify the missing TLS params. Check out
open5gs-{nrf,smf,upf}.yaml from the latest git version.
The updated files are confirmed to work with both:
* latest release for Debian v2.5.8, and
* latest git version v2.6.0-39-g22be888.
Change-Id: I27adbab6a6b95ddf4c6d39803e4f7dd079f11a4c
Related: OS#5913
In ttcn3-pgw-test we're running open5gs-upfd as a non-root user.
We do have this user in open5gs-master, but not in open5gs-latest.
Change-Id: I9139fc52fa6973cf495559d3c4202d9f804a2079
Related: OS#5913
ping is needed for ttcn3-pgw-test. In the case of osmo-uecups-master
it's available from the parent $DISTRO-build image, but for the -latest
it needs to be installed by the inherited image itself.
Change-Id: Ia1b9124f09203c7dcf6d26dee78b09d5a139a04f
Related: OS#5913
The following warning message appears when running apt-key:
apt-key is deprecated. Manage keyring files in trusted.gpg.d instead
The apt-key man page mentions that the "use of apt-key is deprecated,
except for the use of apt-key del in maintainer scripts to remove
existing keys from the main keyring". What's more, "apt-key will last
be available in Debian 11 and Ubuntu 22.04".
Additionally, while the deprecation message says to "manage keyring
files in trusted.gpg.d instead", the Debian wiki states otherwise.
It's recommended to store the keys in /usr/share/keyrings instead,
and then use the 'signed-by' option to reference them.
ee284d5917
Change-Id: I1cae4f49e72f5e4f9d703cdb8f6d117e18e3567c
Give OsmoBSC time to connect to OsmoSTP, so BSSMAP RESET from the
testsuite doesn't fail in OsmoSTP with:
MTP-TRANSFER.req for DPC 187: no route!
Without this patch, it would fail every other time on my system when
only having one test enabled in BSC_Tests.cfg.
Change-Id: Ic7f80bdfe2fe58e93c2f2e3baa6d7645fcdc3062
Add a parameter that can be used to reproduce the failure locally with
the last list from jenkins, without having SSH set up to download the
list of existing tarballs.
Change-Id: I9af310c24aa65c21cc4bd365d3df7804cba02df7
Probably copy pasted from another script, doesn't make much sense here
since only the release tarballs are built. Nothing gets compiled here,
the variable was not used.
Change-Id: Ie74ee6976759f9634503b21e964e93efcc925679
Run configure with the --with-mstrx argument, so LIBTRXCON_DIR is not
pointing at an empty directory.
Fix for:
make[2]: Entering directory '/build/_temp/repos/osmo-trx/osmocom-bb'
make[2]: *** No rule to make target 'distdir'. Stop.
Change-Id: I2f067409f98119d4e35cf86cc2c816fc3c19e3f5
We're not *building* anything for the open5gs-latest, but simply
downloading and installing DEBs from Sukchan's opensuse.org repo.
Change-Id: Iec7ccab28c37d274a731e08a2f893ec0514c1068
Fixes: 62ac21133a
Add a new network_clean_remove_all_ttcn3 function and use it in the fr
related testsuites to ensure no network is running before the test
starts. We just had the situation where the network link between
gtp0-deb10fr (where these testsuites run exclusively, and only one at a
time) and the jenkins host went down. And so the clean up trap
apparently did not run and starting a new test fails as the old network
still exists and has the network devices attached.
Related: OS#5802
Change-Id: I6a9464a11edcba978be08764bec9de19760a5c77
Add a minimal image to use on the raspberry pis, so it doesn't need to
spend 1-2 hours building debian-buster-jenkins on the raspberry pis,
full of things we don't actually use there.
I've verified that osmo-bts, osmo-pcu, osmo-trx build with this image
(the only projects which we build on the rpis).
Related: OS#5863
Change-Id: I4fb4b48b233acaef815c4c27ec6f17f12cfe836b
Make ttcn3-bts-test work as expected again by replacing the subnets
hardcoded in the confmerge files and Makefile.
Related: OS#5802
Change-Id: I733619f80b4e2456da070649634d3c942bb6d95f
Make ccache available inside debian-buster-jenkins. An upcoming osmo-ci
patch will make use of it in gerrit-verification and master jobs. With
this docker-playground patch alone it's not used yet since
/usr/lib/ccache is not in the default PATH.
Related: OS#5848
Change-Id: I164c7835f99436e8d768a864f36118eaf799d109
As we run more testsuites in parallel, the load was higher and so this
timeout was reached.
Related: OS#5802
Change-Id: I823010106a99a4906e32198ed42b36f9fe662c4e
Put the current SUBNET into NET_NAME, not the previous one. Without this
patch it works too, but this makes more sense.
Related: OS#5802
Change-Id: I6c90505fd6c0ba62244ff67709b0bf31bf44b449
Fix the bug that running a test twice on the same host results in the
second test stopping the containers and network of the first test:
* Have the subnet name in the network name, so there's no collision of
network names when cleaning up the network.
* Don't clean up the network when creating a new network. Rely on the
clean up trap to clean up the network instead.
* Don't set NET_NAME when sourcing jenkins-common.sh anymore, as
network_create will set it. There's also network_bridge_create, which
only gets used by osmo-ran (which we don't run in jenkins). Add a
check to this function to ensure NET_NAME is set before calling it,
which osmo-ran is already doing.
Related: OS#5802
Change-Id: Ifcd384272c56d585e220e2588f2186dc110902ed
Oliver Smith
Neels Hofmeyr
Daniel Willmann
Philipp Maier
Pau Espin
Harald Welte
Vadim Yanitskiy