Refactor code to build the initramfs and kernel as user, instead of
running related code as root. Put the smoke_test.log into /tmp instead
of the cache dir, so we don't write as root into the cache dir. We cat
it on error anyway, so we don't need to have it in the workspace.
Change-Id: I1670dd86f2670f42eb99e4527d483965d8222402
Ensure the user that runs docker owns files in cache and data dirs
during fix_perms, if an old job created them as root.
Change-Id: I43af48bd87e27abad5ad52e734256d78651f67c3
Make it possible to e.g. run only one of the generic/oml/hopping
configurations.
I made a version of this patch a few weeks ago for testing OS#6375,
and I'm submitting it now so I can add multiple configurations for
ttcn3-ggsn-test in the next patch with the same logic.
Change-Id: I1dce5e6aa4b5d67f9f8c96ced611ab2875c353c8
Fix the TTCN3-centos-pcu-test-2023q1 jenkins job where currently most
tests are failing. This is because 2023q1 is using an older PCUIF
version that is no longer supported by osmo-ttcn3-hacks.git. As
discussed, use a 2023q1 branch of osmo-ttcn3-hacks where the support was
not removed yet.
Fixes: SYS#6638
Change-Id: Id82425aca4a8ff87feb8a130d31712bc62fa27ed
- Add a docker container to run osmo_dia2gsup from git master.
- Add infrastructure to run the osmo-dia2gsup container against
osmo-ttcn3-hacks.git/dia2gsup testsuite.
Related: OS#5757
Change-Id: I646096d5a010df02af75aefa33b023b55997a6d7
Use $DOCKER_ARGS in the docker run call to kernel-test/prepare.sh, which
builds the initramfs. I use this to make a local build of osmo-ggsn +
depends available in the docker container in /usr/local with the
ttcn3.sh script in osmo-dev.git.
Change-Id: Ie2500fa1bd9d57e2531463ce93fdd9930ad829d3
As "docker kill" / "docker container kill" (alias) doesn't block until
the given container stops, make sure to always run "docker wait"
afterwards.
Closes: OS#5928
Change-Id: I0242ece96541d8036ebbf8b0f498ebf231db26b5
We're not *building* anything for the open5gs-latest, but simply
downloading and installing DEBs from Sukchan's opensuse.org repo.
Change-Id: Iec7ccab28c37d274a731e08a2f893ec0514c1068
Fixes: 62ac21133a
Add a new network_clean_remove_all_ttcn3 function and use it in the fr
related testsuites to ensure no network is running before the test
starts. We just had the situation where the network link between
gtp0-deb10fr (where these testsuites run exclusively, and only one at a
time) and the jenkins host went down. And so the clean up trap
apparently did not run and starting a new test fails as the old network
still exists and has the network devices attached.
Related: OS#5802
Change-Id: I6a9464a11edcba978be08764bec9de19760a5c77
Make ttcn3-bts-test work as expected again by replacing the subnets
hardcoded in the confmerge files and Makefile.
Related: OS#5802
Change-Id: I733619f80b4e2456da070649634d3c942bb6d95f
As we run more testsuites in parallel, the load was higher and so this
timeout was reached.
Related: OS#5802
Change-Id: I823010106a99a4906e32198ed42b36f9fe662c4e
Put the current SUBNET into NET_NAME, not the previous one. Without this
patch it works too, but this makes more sense.
Related: OS#5802
Change-Id: I6c90505fd6c0ba62244ff67709b0bf31bf44b449
Fix the bug that running a test twice on the same host results in the
second test stopping the containers and network of the first test:
* Have the subnet name in the network name, so there's no collision of
network names when cleaning up the network.
* Don't clean up the network when creating a new network. Rely on the
clean up trap to clean up the network instead.
* Don't set NET_NAME when sourcing jenkins-common.sh anymore, as
network_create will set it. There's also network_bridge_create, which
only gets used by osmo-ran (which we don't run in jenkins). Add a
check to this function to ensure NET_NAME is set before calling it,
which osmo-ran is already doing.
Related: OS#5802
Change-Id: Ifcd384272c56d585e220e2588f2186dc110902ed
Try multiple subnet numbers until successfully creating a network. This
way we can run the same ttcn3 testsuite multiple times in parallel
without conflicts (e.g. once against latest, once against nightly). Also
we don't need to make sure each new testsuite has a unique subnet
number anymore.
I've considered also adjusting network_bridge_create, but that gets used
exclusively by osmo-ran/jenkins.sh, a script which we don't actually run
in jenkins. It seems that in this script it makes more sense to not get
a random subnet number.
Related: OS#5802
Change-Id: I57152b08ef0f38e17e7019a8df032189b03f56cf
Use 'set +x' before echo, so messages are not printed twice and the
output becomes more readable.
Example:
+ echo Creating network ttcn3-bts-test
Creating network ttcn3-bts-test
Related: OS#5802
Change-Id: I213cb80bd1d9401281ae1fb7394b2319453550dd
Use debian:bullseye instead of debian-bullseye-build. Building the
latter requires a lot of time, it's definitively not needed for just
running chmod. Use debian:bullseye, since we use it as base for most
images and it's therefore very likely that it's already downloaded
(and if not, it would get downloaded here).
Change-Id: I88f513a0a8ee04628842a7aa1b663c390d2032cd
Add "--rm" to each "docker run" command, so they don't continuously fill
up disk space.
Fix this even in the pipework script. We don't use the code path there,
but by always having --rm after "docker run" (same line or next line),
a new lint script in osmo-ci I8ab9c291504475d670bdefc50c4524c5bdd4c880
can help us avoid this in the future.
In ttcn3-ggsn-test/jenkins.sh, move one existing --rm in a later line
upwards so the linter can find it.
Related: SYS#5827, OS#5099
Related: https://github.com/jpetazzo/pipework/pull/248
Change-Id: I48b01c43fedf379b8a565eaab0369806d7831bd8
CentOS Linux 8 is EOL, attempting to install packages in it results in
an error. CentOS Linux is a rebuild of RHEL (stable versions). The
CentOS projects recommends to use CentOS Stream instead, which is a
build of the "public development branch for RHEL".
After the early EOL was announced on 2020-12-08, alternative projects
AlmaLinux and Rocky Linux have been established as binary compatible
forks of RHEL 8 (stable versions).
Both Alma and Rocky seem to be solid projects, see related Wikipedia
articles and their sources. Pick Alma and adjust the whole tree to use
the almalinux:8 docker image instead of centos:centos8.
Fix for:
Error: Failed to download metadata for repo 'appstream': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
Related: SYS#5818
Related: https://www.centos.org/cl-vs-cs/
Related: https://www.centos.org/centos-linux-eol/
Related: https://bugs.centos.org/view.php?id=18394
Related: https://en.wikipedia.org/wiki/AlmaLinux
Related: https://en.wikipedia.org/wiki/Rocky_Linux
Change-Id: I30e1a773b901b1d2187214445116c7f2aecc4e36
After the default UPSTREAM_DISTRO was changed from debian:stretch to
debian:bullseye, the "debian9-repo-install-test" container has gotten
built with bullseye instead of stretch. This is the reason for failures
of the jenkins job Osmocom-repo-install-debian9 we have seen over the
past days.
With this patch applied, it runs through again:
https://jenkins.osmocom.org/jenkins/job/Osmocom-repo-install-debian9/339/
Change-Id: I98a19184ba936114c03cd5cc4f54a3173cbd9cfe
Add missing container for -latest tests. Currently this causes the
-latest tests to fail.
Related: OS#4969
Change-Id: I1230e87784bc21b5a6424db0bd8734181ead9bfd
So far we were using ancient Debian 9 (stretch) for our IUT
containers. Let's upgrade that to Debian 11 (bullseye).
Related: OS#4969
Change-Id: Ic6bece9cb695e6eccfcb1e83fdbf1048724a3cf9
So far we were executing all our TTCN-3 tests from a container
image with Debian stretch (9) plus a custom more recent eclipse-titan
package from the osmocom feed.
Let's update the container base OS from stretch (9) to bullseye (11)
while using the same packaged eclipse-titan version (8.0.0) for running
the tests. So this should be a low-risk change, as titan runtime
remains identical.
I've executed all test suites locally and couldn't see any regressions.
Related: OS#4969
Change-Id: Ib3bdfa3bec8f8ef42c55ca61cdee8fbca923874f
Support running multiple TTCN-3 testsuites in parallel when the
jenkins.sh scripts get called from outside of jenkins too.
Closes: OS#5358
Change-Id: Iaaf5719f758553687f2ab57975be275e7ac07e36
Pull debian-stretch-titan from registry.osmocom.org if the repository is
enabled. Otherwise the image gets only pulled the first time, and does
not get updated once it becomes outdated.
Fixes: OS#5336
Change-Id: I1cf998c21e4ee1f723c3b783703e339328377f3e
Do not build debian-stretch-titan, when registry.osmocom.org is enabled.
The image that would be built at this point is not useful, since other
images like ttcn3-msc-test will use the debian-stretch-titan image from
the registry instead of the one that was just built locally.
Related: OS#5336
Change-Id: I7127e3ebac3a6a985c3ba50ba8c7cb8c5de978d9
Write a line like 'Misc_Helpers.mp_osmo_repo := "nightly"' into the
TTCN-3 config file (e.g. BSC_Tests.cfg), before starting the testsuite.
This allows executing different code paths in the tests based on the
repository.
Related: OS#5327
Change-Id: Ic06532f7a67e59458652c5cf4c8f6fee8113e703
Allow to optionally skip listing the installed packages. This is useful
to speed up tests when running locally.
Related: osmo-dev I3d9976d07f088e2d106d23259889a4bffe1abbcb
Change-Id: Iaafb81e9ee94850ae3b6318574ad5370d74a2838
Since recently, we also have a new IMAGE_SUFFIX '2021q1-centos8'.
We cannot rely on image_suffix_is_latest() anymore, because this
prefix does not qualify as 'latest'. If one needs to enable a
feature that is only available in master, then here is a way:
if ! image_suffix_is_master; then
# Patch the configuration files
fi
Change-Id: I65df66cd2e14934bed8114618d6ffad7831d1025
Adjust docker_depends and docker_dir_from_image_name for 2021q1. Extend
README.md with examples.
Related: SYS#5370
Change-Id: Icf1e3f1928658ff3d82190c4176252cf3590a1f5
Display an overview of installed Osmocom packages at the start of each
ttcn3-*/jenkins.sh script (and others making use of
docker_images_require).
Closes: OS#5091
Change-Id: I62670017d790fd67c6a0d8d464386c0f8e94c1a8
Add a new centos8-obs-latest container with shared code (just like
debian-stretch-obs-latest) and adjust the package install blocks in
osmo-*-latest.
Related: OS#5049
Change-Id: I274b31d4903df07e99fd434af25bdb72991f6923
Move common code from osmo-*-latest to debian-stretch-obs-latest:
* add the OBS repository key
* add the OBS repository with "latest" feed
* install telnet
* invalidate the cache once the repository is updated
This is in preparation to make osmo-*-latest work with centos8,
preventing the following issues we would have without this refactoring:
* avoid need to read the repository index of both debian-stretch and
centos8, and to invalidate the cache if either of them changes
(instead of just the relevant one). This is currently a problem in
osmo-*-master/Dockerfile, Id69f3eaf37f823bac0947366f27a25edd277a4cb
will fix it there.
* no need to add boilerplate code to install the centos8 repository to
each osmo-*-latest/Dockerfile
While at it, also remove deprecated MAINTAINER lines.
Related: OS#5049
Related: https://docs.docker.com/engine/reference/builder/#maintainer-deprecated
Change-Id: I443d52eba416ea298cc5421e280ad2f4ca59283b
Disable KVM if /dev/kvm is missing, as it is apparently the case on our
jenkins nodes. This makes the tests run a bit slower, but not much. Add
kernel_test_wait_for_vm to sleep until the VM is booted up (~5s instead
of ~1s without KVM), so the first test in the testsuite does not fail
when using KVM.
The variable is useful to test the non-KVM code path even if KVM is
available on the host.
Related: OS#3208
Change-Id: I1f337af1e2de6db05b22636bc31a535404235559
Make it possible to run ttcn3-ggsn-test against osmo-ggsn with the GTP
kernel module. This feature is enabled by environment variables
explained in ttcn3-ggsn-test/jenkins.sh, and can either use the
pre-built kernel from Debian, or build a kernel from source.
The kernel runs inside QEMU inside the usual osmo-ggsn-master docker
image.
Thanks to Pau for providing the kernel config and kernel-test scripts
that this patch is based on! I ended up modifying them quite a bit to
fit the docker use case, and to build the initrd (so we can use the
stock debian kernel).
Related: OS#3208
Change-Id: I54e752048eed2a170a1cf4e0fd410de255b9de5d
Don't call fix_perms from collect_logs anymore, it looks like it was
only added there to avoid adding it at the end of each jenkins.sh. Now
that we have clean_up_common, put it there.
Related: OS#3208
Change-Id: I6e7e5b35198f420e130b1282eb948b96ccd6c8e0
Add set_clean_up_trap() in jenkins-common.sh and run it at the beginning
of the jenkins.sh files. Move the common clean up code from the end of
every jenkins.sh file into clean_up_common(), which gets called by the
trap. Add a custom clean_up() function to those jenkins.sh files that
need additional clean up.
Replace explicit container stop commands (for containers attached to the
docker network) with one call to network_clean() in clean_up_common(). It
kills all containers attached to the docker network.
The motivation for this change is the upcoming optional build of initrd
and kernel during ttcn3-ggsn-test/jenkins.sh. After building these, a
short smoke test will be performed to make sure we can boot the kernel
and initrd, before continuing to run the entire testsuite against it. If
building or the smoke test fails, we must do a proper clean up of the
network and fix permissions.
Related: OS#3208
Change-Id: I807c6221d22fec31db23e2516e477a3eb43347fb
Oliver Smith
Pau Espin
Vadim Yanitskiy
Harald Welte