Display an overview of installed Osmocom packages at the start of each
ttcn3-*/jenkins.sh script (and others making use of
docker_images_require).
Closes: OS#5091
Change-Id: I62670017d790fd67c6a0d8d464386c0f8e94c1a8
Add a new centos8-obs-latest container with shared code (just like
debian-stretch-obs-latest) and adjust the package install blocks in
osmo-*-latest.
Related: OS#5049
Change-Id: I274b31d4903df07e99fd434af25bdb72991f6923
Move common code from osmo-*-latest to debian-stretch-obs-latest:
* add the OBS repository key
* add the OBS repository with "latest" feed
* install telnet
* invalidate the cache once the repository is updated
This is in preparation to make osmo-*-latest work with centos8,
preventing the following issues we would have without this refactoring:
* avoid need to read the repository index of both debian-stretch and
centos8, and to invalidate the cache if either of them changes
(instead of just the relevant one). This is currently a problem in
osmo-*-master/Dockerfile, Id69f3eaf37f823bac0947366f27a25edd277a4cb
will fix it there.
* no need to add boilerplate code to install the centos8 repository to
each osmo-*-latest/Dockerfile
While at it, also remove deprecated MAINTAINER lines.
Related: OS#5049
Related: https://docs.docker.com/engine/reference/builder/#maintainer-deprecated
Change-Id: I443d52eba416ea298cc5421e280ad2f4ca59283b
Disable KVM if /dev/kvm is missing, as it is apparently the case on our
jenkins nodes. This makes the tests run a bit slower, but not much. Add
kernel_test_wait_for_vm to sleep until the VM is booted up (~5s instead
of ~1s without KVM), so the first test in the testsuite does not fail
when using KVM.
The variable is useful to test the non-KVM code path even if KVM is
available on the host.
Related: OS#3208
Change-Id: I1f337af1e2de6db05b22636bc31a535404235559
Make it possible to run ttcn3-ggsn-test against osmo-ggsn with the GTP
kernel module. This feature is enabled by environment variables
explained in ttcn3-ggsn-test/jenkins.sh, and can either use the
pre-built kernel from Debian, or build a kernel from source.
The kernel runs inside QEMU inside the usual osmo-ggsn-master docker
image.
Thanks to Pau for providing the kernel config and kernel-test scripts
that this patch is based on! I ended up modifying them quite a bit to
fit the docker use case, and to build the initrd (so we can use the
stock debian kernel).
Related: OS#3208
Change-Id: I54e752048eed2a170a1cf4e0fd410de255b9de5d
Don't call fix_perms from collect_logs anymore, it looks like it was
only added there to avoid adding it at the end of each jenkins.sh. Now
that we have clean_up_common, put it there.
Related: OS#3208
Change-Id: I6e7e5b35198f420e130b1282eb948b96ccd6c8e0
Add set_clean_up_trap() in jenkins-common.sh and run it at the beginning
of the jenkins.sh files. Move the common clean up code from the end of
every jenkins.sh file into clean_up_common(), which gets called by the
trap. Add a custom clean_up() function to those jenkins.sh files that
need additional clean up.
Replace explicit container stop commands (for containers attached to the
docker network) with one call to network_clean() in clean_up_common(). It
kills all containers attached to the docker network.
The motivation for this change is the upcoming optional build of initrd
and kernel during ttcn3-ggsn-test/jenkins.sh. After building these, a
short smoke test will be performed to make sure we can boot the kernel
and initrd, before continuing to run the entire testsuite against it. If
building or the smoke test fails, we must do a proper clean up of the
network and fix permissions.
Related: OS#3208
Change-Id: I807c6221d22fec31db23e2516e477a3eb43347fb
Similar to what is already done with DISTRO, which points to given image
of ours based on name. This time we do the same with upstream images,
such as debian:stretch or centos:centos8.
This way, for instance calling docker_images_require
"osmo-bsc-latest-centos8" would try to build the
osmo-bsc-latest/Dockerfile file starting from a centos8 image.
Change-Id: I33cb21aa024396974559fd98f9f3c64e2c351eda
We previously only allocated IPv4 sub-nets to the network segments
created with 'docker network'. Let's fix that by assigning both
IPv4 and IPv6 address ranges to each docker network.
Related: OS#4700
Change-Id: I8802208fddcce1ffa57e5626575d23d02b320d99
When my disk runs full, jenkins-common.sh fails to create a VOL_BASEDIR right
at the start, and hence no testsuite gets its external volumes mounted
properly. However, the test suite still starts up with missing cfg files etc,
and fails in non-obvious ways, until I find out the disk is full some minutes
later. Instead, verify that the base dir really exists or bail out right there.
Change-Id: I6540797c8e0ee3e7b09d4a80592d5e270e7d9adc
Make it possible to call the ttcn3-*/jenkins.sh scripts with:
IMAGE_SUFFIX="master-centos8"
The existing docker_images_require lines of these jenkins.sh scripts
will then call docker_images_require with arguments like
"osmo-mgw-master-centos8". For example, from ttcn3-mgw-test/jenkins.sh:
docker_images_require \
"osmo-mgw-$IMAGE_SUFFIX" \
"ttcn3-mgw-test"
Let docker_images_require build the image from
osmo-mgw-master/Dockerfile (-centos8 is cut off from the dirname) and
with DISTRO=centos8 as argument.
Collisions with the debian-stretch images are avoided by setting IMAGE
to the full image name (e.g. osmo-mgw-master-centos8).
Related: OS#4564
Change-Id: I598a262fe1a7ed4dd89e13c53e4ded103c6e3b91
Add a function to automatically resolve the most common dependencies,
osmo-* need debian-stretch-build and ttcn3-* need debian-stretch-titan.
Do this in a shell function in jenkins-common.sh, so we can extend it in
the next patch with distribution specific dependencies (osmo-*-centos8
needs centos8-build).
Related: OS#4564
Change-Id: I98760c6dc3206529c0e267338c9f6f8e1d5be33a
We can't create the net if it exists, and it can't be removed until all
attached containers are dead, so ensure this is the case upon net
creation. This fixes test failures due to stale nets and half-killed
test runs.
Change-Id: Id6d13b233ebfd808d8dfe83b6d1d1ba20c3392c8
When fixing the permissions at the end of a test run, it really doesn't
matter if we're running a current debian-stretch-build image, or if we
are using an ancient one. All we need is a 'chmod' binary.
Change-Id: Ib008ee72f40aeefb307c0ba74a57b70daae9d07b
Related: OS#4060
Prevent building docker images on top of outdated Debian images, where
the package download feeds have been disabled. Use 'docker build --pull'
instead of 'docker build', whenever the "FROM" line in the Dockerfile
does not contain '$USER' (meaning this is an upstream image).
Checking the FROM line is necessary, as downloading '$USER' images will
fail (we never upload those).
Related: OS#3869
Change-Id: I1076bbb7d77bdc99f5d60d641c09ce965fb9dfc6
Use the debian-jessie-build container to ensure all files under
${VOL_BASE_DIR} are readable.
Call this directly from collect_logs so we handle all ttcn3 jobs.
Fixes: OS#3800
Change-Id: I124c554eee69ec5586547d3c96b8ec571e0653a7
docker_images_require() must fail when a new image can not be built,
even if an old image already exists.
We have this situation in a lot of the -latest tests right now, which
makes it harder to find the actual error.
Note that we can't simply use #!/bin/sh -e in all the jenkins.sh
scripts that source jenkins_common.sh, because they must be able to
clean up the docker containers on failure. So they can't abort
somewhere in the middle of the scripts. We could rewrite them to do the
clean up in a trap, but this should be done in a separate commit.
Related: OS#3767
Change-Id: I7039089457b62b8798a79c5efd62bd91622986d3
This new function can be used on top of each *-test/jenkins.sh to
automatically build the required Docker images before starting a
testsuite.
The top-level Makefile should also be able to build all Docker images
on which a job depends. But in that Makefile the list of dependencies
can not be specified dynamically, as it will be necessary for OS#3268:
ttcn3-mgw-test may depend on osmo-mgw-latest or on osmo-mgw-master.
This will be used by Idbb708ab16cb71bab5069127945b63388222369e.
Related: OS#3268
Change-Id: I0afdf3400282634fca9b31d57eaeedd6b5c28aa1
On jenkins, place all logs and manage docker volumes in the workspace instead
of a /tmp/* dir. Use $WORKSPACE/logs as docker volume base to begin with, thus
there needs to be no copy from /tmp to $WORKSPACE/logs.
On non-jenkins runs, place all in a /tmp/* dir still, but also skip copying of
the logs: just have a /tmp/logs symlink to the last tmpdir.
Change-Id: I8cf6014725ae8ba602be5f3ec31dfb8e49ff993e
To remove code dup and prepare a change to where logs are written, add
collect_logs() to jenkins_common.sh and call that from each jenkins.sh after
the tests are done.
The 'rm -rf' is already done before a test starts. No need to do that again
after each test.
Change-Id: I5d8472ec36b07c828685b1bd7718e31392d168a3
Oliver Smith
Pau Espin
Harald Welte
Neels Hofmeyr
Eric Wild
Daniel Willmann