docker-playground/README.md

227 lines
7.7 KiB
Markdown

Osmocom docker playground
=========================
This repository contains some humble attempts at creating some Docker
containers + related stacks around Osmocom. So far, the main focus is
test automation.
## Running a testsuite
All testsuite folders start with `ttcn3` or `nplab`. Run the following
to build/update all required containers from the "master" branch and
start a specific testsuite:
```
$ cd ttcn3-mgw-test
$ ./jenkins.sh
```
Environment variables:
* `IMAGE_SUFFIX`: the version of the Osmocom stack to run the testsuite
against. Default is `master`, set this to `latest` to test the last
stable releases.
* `OSMO_TTCN3_BRANCH`: [osmo-ttcn3-hacks.git](https://git.osmocom.org/osmo-ttcn3-hacks/)
branch, which will be used when building a `ttcn3-*` docker image.
Defaults to `master`.
* `OSMO_BSC_BRANCH`, `OSMO_MSC_BRANCH`, ...: branch of the appropriate
Osmocom project. Defaults to `master`.
* `NO_DOCKER_IMAGE_BUILD`: when set to `1`, it won't try to update the
containers (see "caching" below)
* `DOCKER_ARGS`: pass extra arguments to docker, e.g. to mount local sources
for building as done in osmo-dev.git/ttcn3/ttcn3.sh
* `TEST_CONFIGS`: for tests that can run with multiple config sets (e.g.
`ttcn3-bts-test`), run only some of them. See `TEST_CONFIGS_ALL` in the
`jenkins.sh` for possible values.
### Run only one test
Run only `TC_gsup_sai` in `ttcn3-hlr-test`:
```
$ cd ttcn3-hlr-test
$ export DOCKER_ARGS="-e TEST_NAME=TC_gsup_sai"
$ ./jenkins.sh
```
Run only `TC_est_dchan` in `ttcn3-bts-test`, with the `generic` configuration:
```
$ cd ttcn3-bts-test
$ export DOCKER_ARGS="-e TEST_NAME=TC_est_dchan"
$ export TEST_CONFIGS="generic"
$ ./jenkins.sh
```
### Using nightly packages from a different date
Pick a date from [here](https://downloads.osmocom.org/obs-mirror/) and use it:
```
$ export OSMOCOM_REPO_PATH="obs-mirror/20230316-061901"
$ cd ttcn3-bsc-test
$ ./jenkins.sh
```
### More examples
latest (debian):
```
$ export IMAGE_SUFFIX="latest"
$ cd ttcn3-mgw-test
$ ./jenkins.sh
```
latest-centos8:
```
$ export IMAGE_SUFFIX="latest-centos8"
$ cd ttcn3-mgw-test
$ ./jenkins.sh
```
2021q1-centos8:
```
export OSMOCOM_REPO_TESTSUITE_MIRROR="https://downloads.osmocom.org"
export OSMOCOM_REPO_MIRROR="https://downloads.osmocom.org"
export OSMOCOM_REPO_PATH="osmo-maintained"
export OSMOCOM_REPO_VERSION="2021q1"
export IMAGE_SUFFIX="2021q1-centos8"
$ cd ttcn3-mgw-test
$ ./jenkins.sh
```
## Kernel test
OsmoGGSN can be configured to either run completely in userspace, or to
use the GTP-U kernel module. To test the kernel module, OsmoGGSN and
the kernel module will run with a Linux kernel (either the pre-built
one from Debian, or a custom built one) in QEMU inside docker. As of
writing, `ttcn3-ggsn-test` is the only testsuite where it makes
sense to test kernel modules. But the same environment variables could
be used for other testsuites in the future.
Environment variables:
* `KERNEL_TEST`: set to 1 to run the SUT in QEMU
* `KERNEL_TEST_KVM`: set to 0 to disable KVM acceleration
* `KERNEL_BUILD`: set to 1 to build the kernel instead of using the
pre-built one
* `KERNEL_REMOTE_NAME`: git remote name (to add multiple git
repositories in the same local linux clone, default: net-next)
* `KERNEL_URL`: git remote url (default: net-next.git on kernel.org)
* `KERNEL_BRANCH` branch to checkout (default: master)
* `KERNEL_SKIP_REBUILD`: set to 1 to not build the kernel again if already
built with `KERNEL_BUILD=1`
* `KERNEL_SKIP_SMOKE_TEST`: don't boot up the kernel in QEMU once before
running the testsuite
The OBS repository mirror consists of
`${OSMOCOM_REPO_MIRROR}/${OSMOCOM_REPO_PATH}/${OSMOCOM_REPO_VERSION}`,
e.g. `https://downloads.osmocom.org/packages/osmocom:/latest/`.
### Creating kernel config fragments
For the kernel tests, we are storing kernel config fragments in the git
repository instead of full kernel configs. Generate them as follows:
```
$ cd _cache/linux
$ cp custom.config .config
$ make olddefconfig
$ cp .config custom-updated.config
$ make defconfig # config to which to diff
$ scripts/diffconfig -m .config custom-updated.config > fragment.config
```
Verify that it was done right:
```
$ make defconfig
$ scripts/kconfig/merge_config.sh -m .config fragment.config
$ make olddefconfig
$ diff .config custom-updated.config # should be the same
```
## Building containers manually
Most folders in this repository contain a `Dockerfile`. Build a docker
container with the same name as the folder like this:
```
$ cd debian-stretch-build
$ make
```
## Caching
All folders named `osmo-*-latest` and `osmo-*-master` build the latest
stable or most recent commit from `master` of the corresponding Osmocom
program's git repository. When you have built it already, running `make`
will only do a small HTTP request to check if the sources are outdated
and skip the build in case it is still up-to-date.
## Dependencies
Folders that don't have a `jenkins.sh` usually only depend on the
container that is specified in the `FROM` line of their `Dockerfile`.
Testsuites depend on multiple containers, they are defined on top of
each `jenkins.sh`:
```shell
. ../jenkins-common.sh
IMAGE_SUFFIX="${IMAGE_SUFFIX:-master}"
docker_images_require \
"osmo-stp-$IMAGE_SUFFIX" \
"osmo-bsc-$IMAGE_SUFFIX" \
"osmo-bts-$IMAGE_SUFFIX" \
"ttcn3-bsc-test"
```
#### Reasoning for this implementation
Before having the `docker_images_require` lines, there used to be a
top-level `Makefile` for resolving dependencies between the containers.
But it was prone to mistakes: when new folders in the repository
were added without related targets in the `Makefile`, `make` would
always assume that the targets where the always existing folders and
therefore never build the containers.
In order to implement testing `latest` in addition to `master`
([OS#3268](https://osmocom.org/issues/3268)), it would have been
necessary to add further complexity to the `Makefile`. Instead it was
decided to scrap the file, and just keep the short list of dependencies
right above where they would be needed in the `jenkins.sh`.
## Obtaining gdb backtrace from crash
If for instance TTCN3 test is producing a crash on a program running in docker,
eg. osmo-msc, it is desirable to get a full crash report. This section describes
how to do so.
First, open `osmo-$program/Dockerfile` and add lines to install `gdb` plus
`$program` dependency debug packages. For instance:
```
+RUN apt-get install -y --no-install-recommends \
+ gdb \
+ libosmocore-dbg libosmo-abis-dbg libosmo-netif-dbg libosmo-sigtran-dbg osmo-msc-dbg && \
+ apt-get clean
```
In same `Dockerfile` file, modify configure to build with debug symbols enabled
and other interesting options, such as `--enable-sanitize`:
```
- ./configure --enable-smpp --enable-iu && \
+ export CPPFLAGS="-g -O0 -fno-omit-frame-pointer" && \
+ export CFLAGS="-g -O0 -fno-omit-frame-pointer" && \
+ export CXXFLAGS="-g -O0 -fno-omit-frame-pointer" && \
+ ./configure --enable-smpp --enable-iu --enable-sanitize && \
```
Finally open the script you use to run the program (for instance
`ttcn3-$program-master/jenkins.sh`), and modify it to launch the process using
gdb, and to print a full backtrace when control returns to gdb (when the process
crashes):
```
-/bin/sh -c "osmo-msc -c /data/osmo-msc.cfg >>/data/osmo-msc.log 2>&1"
+/bin/sh -c "gdb -ex 'run' -ex 'bt full' --arg osmo-msc -c /data/osmo-msc.cfg >>/data/osmo-msc.log 2>&1"
```
## See also
* [Overhyped Docker](http://laforge.gnumonks.org/blog/20170503-docker-overhyped/)
for related rambling on why this doesn't work as well as one would
want.
* [Osmocom wiki: Titan TTCN3 Testsuites](https://osmocom.org/projects/cellular-infrastructure/wiki/Titan_TTCN3_Testsuites)