Commit Graph

42 Commits

Author SHA1 Message Date
Harald Welte 8e3b3ff43c ttcn3-bts-test: use osmo-config-merge to generate config files
This uses the config files from osmo-ttcn3-hacks.git instead of keeping
local copies around here, which has shown many times will get out of
sync.  Extract the original files (*.src.cfg) from the testsuite image
(ttcn3-bts-test) and merge them with the local *.confmerge files using
osmo-config-merge.  Use the resulting files (*.gen.cfg).

Unfortunately, osmo-config-merge is not capable of removing text blocks
or substituting specific parts, so we fall-back to using sed where it's
needed.  A good example is 'log gsmtap', as there is no
command for removing a logging target of such type.

Co-Authored-By: Vadim Yanitskiy <>
Co-Authored-By: Oliver Smith <>
Change-Id: If15461240f3037c142c176fc7da745a1701ae3f8
2022-02-10 22:49:24 +06:00
Vadim Yanitskiy edaa9d3033 ttcn3-bts-test: move generic config files into a directory
Change-Id: I1bc5c51306525c6829e9d8e2991555a99c9b49a2
2022-02-10 19:08:37 +06:00
Vadim Yanitskiy bcf017e455 ttcn3-{bts,bsc,msc,hlr}-test: set OSMO_SUT_{HOST,PORT}
This enables the test suite to obtain talloc reports between the
test case executions, which get stored together with the PCAP files.
Let's see how it works with the mentioned projects first;  others
can be configured later.  See [1] for more details.

Change-Id: Icd4c2d80db934535d499598282ed9416d8088163
Related: [1] I1b03b17426d8760c55976e3b78ca2f3af248c055
Related: OS#5328
2021-12-01 13:48:50 +00:00
Oliver Smith 305ce55b74 ttcn3: write Osmocom repository to configs
Write a line like 'Misc_Helpers.mp_osmo_repo := "nightly"' into the
TTCN-3 config file (e.g. BSC_Tests.cfg), before starting the testsuite.
This allows executing different code paths in the tests based on the

Related: OS#5327
Change-Id: Ic06532f7a67e59458652c5cf4c8f6fee8113e703
2021-11-26 18:12:50 +01:00
Vadim Yanitskiy 76e2529e42 ttcn3-bts-test: use 'gsmtap-remote-host' parameter
Command line argument '-i' has been deprecated.

Change-Id: Ib01069a55a27e79c67b76abe39763c1b645075e5
Related: I17676a21c4e0c9cbc88f2c5c53a39c6c6c473ca1
2021-03-18 02:34:11 +01:00
Oliver Smith 033c6ab1be */ set -e after setting clean up trap
Abort the script and trigger the clean up script, whenever any of the
commands below to prepare the testsuite are failing. This saves time
with figuring out why suddenly all or most tests are failing, and avoids
running the entire testsuite on jenkins if it's obviously not going to

Related: OS#3208
Change-Id: Ie68da2affda8c96b3a515a857a921a05f1bf8ef7
2021-02-26 09:32:34 +01:00
Oliver Smith 12b961942a add common clean up trap
Add set_clean_up_trap() in and run it at the beginning
of the files. Move the common clean up code from the end of
every file into clean_up_common(), which gets called by the
trap. Add a custom clean_up() function to those files that
need additional clean up.

Replace explicit container stop commands (for containers attached to the
docker network) with one call to network_clean() in clean_up_common(). It
kills all containers attached to the docker network.

The motivation for this change is the upcoming optional build of initrd
and kernel during ttcn3-ggsn-test/ After building these, a
short smoke test will be performed to make sure we can boot the kernel
and initrd, before continuing to run the entire testsuite against it. If
building or the smoke test fails, we must do a proper clean up of the
network and fix permissions.

Related: OS#3208
Change-Id: I807c6221d22fec31db23e2516e477a3eb43347fb
2021-02-26 09:32:06 +01:00
Pau Espin c63a131a60 Remove test backward compatibility with older osmo-* versions
These changes are no longer needed since master versions of whole CNI
have been recently released.

Related: OS#5042
Change-Id: I6f491b58d4576b22c8ff3ab56a7b9e17163a8739
2021-02-25 16:41:47 +01:00
Vadim Yanitskiy c52dbbe7f0 ttcn3-bts-test: fix: sleep 1 second before respawning the IUT
There has been a lot of changes merged to both osmo-{bsc,bts}, so
now the later seems to be respawning faster then before.  Because
of that, osmo-bts-trx is now reaching the limit of 500 runs on
Jenkins right in the middle of ttcn3-bts-test execution: exiting after 500 runs

causing the remaining test cases to fail due to:

  "BTS_Tests.ttcn:253 : Timeout waiting for RSL bring up"

Most likely, the IUT respawns several times in the windows between
the test cases, i.e. after TITAN finishes one test case and before
it starts another.  Let's instruct to sleep 1 second
before starting the process again.

Change-Id: I09e5a8e7c4d28424cd0d004a8bae3cf0ade2d838
2020-12-16 14:07:12 +00:00
Vadim Yanitskiy 0e2a3d6bb1 ttcn3-{pcu,bts}-test: use PCUIFv9 for 'latest' images
Change-Id: I6f072e4db07fd6907b91925549edcf62d9c93fb7
Related: I9350c4a54434c3d46ce9424f382ca0057e58d053
Related: SYS#4868, SYS#4915
2020-09-22 02:00:41 +07:00
Pau Espin 1f45fb08ff Add function to print docker network related parameters
Change-Id: Ie00561de7a4494065156a124565e2190151e6019
2020-08-27 12:49:43 +02:00
Vadim Yanitskiy 149de2822d ttcn3-bts-test/fh: fix: properly rename classnames
Not only append ':hopping' to classnames equal 'BTS_Tests', but
also 'BTS_Tests_SMSCB' and 'BTS_Tests_LAPDm', so they would not
shadow non-hopping test cases in the test results analyzer.

Change-Id: Ie6ed7844c2d0dfd839181bce971506284da03d28
Related: SYS#4868, OS#4546
2020-08-13 19:27:39 +00:00
Vadim Yanitskiy c91dae9d1d ttcn3-bts-test/fh: fix: properly include BTS_Tests.cfg
Include the correct configuration file, not the one from the image.

Change-Id: Iecae5cddfbbce4c3bc4077c6c09f527914c06e4b
Related: SYS#4868, OS#4546
2020-08-13 19:27:39 +00:00
Vadim Yanitskiy 617835b50b ttcn3-bts-test/ use bts-tester-{generic,oml,virtphy,hopping}
Let's use separate directories to store the artifacts of different
TTCN-3 test case groups.  This would prevent the test suite from
overwriting *.log and *.pcap files when running both generic and
hopping configurations together, and simplify the overall hierarchy.

Change-Id: I7c41cfb395047433e22e9c65c8c00ae284717b32
Related: SYS#4868, OS#4546
2020-08-13 19:27:39 +00:00
Vadim Yanitskiy 075a487ad9 ttcn3-bts-test/ do not create unused directory
Change-Id: I01efad7f86723301807237d9fae515e82cae125b
2020-08-13 19:27:39 +00:00
Harald Welte 306a51dd30 Enable IPv6 in docker networking
We previously only allocated IPv4 sub-nets to the network segments
created with 'docker network'.  Let's fix that by assigning both
IPv4 and IPv6 address ranges to each docker network.

Related: OS#4700
Change-Id: I8802208fddcce1ffa57e5626575d23d02b320d99
2020-08-13 15:01:58 +00:00
Vadim Yanitskiy 27a063fc2f ttcn3-bts-test/ enable frequency hopping test cases
Change-Id: I2b8ee5037146203801ab92a1b4fe6703522d8093
Related: SYS#4868, OS#4546
2020-08-08 01:37:35 +07:00
Pau Espin fc5cab9e3a ttcn3-bts: Fix most OML test failing after osmo-bts shutdown optimization
Since osmo-bts.git 0a6bdb024f5429a35c56178496158c9b1c3ee032, the process
is quickly terminated during shutdown if no TRX is running. Since in the
OML tests the TTCN3 components implements the OML IPA server side, most
of the time when osmo-bts starts it finds no one listening on the
address and exits immediately. This makes script run osmo-bts
lots of times per seconds (since osmo-bts terminates quicker now). As a
result, it quickly consumes the 500 launch threshold and exits without
starting osmo-bts being launched anymore.
As a result, only first tests pass, and all the later ones fail because
osmo-bts never connects.

Let's add a sleep 1 between respawns in only when running OML
tests (not needed in the others since there's an osmo-bsc always
running). This also clears a lot of output in the logs of osmo-bts
processes restarting over and over again.

Change-Id: I10d3e1ef8325dd78f9ae2b95fbfd1c6d893bf8a4
2020-07-08 12:37:29 +00:00
Neels Hofmeyr 04c20edefd collect core dumps: '--ulimit core=-1' and WORKDIR=/data everywhere
Change-Id: Ief73b53ed9da6f5af82975bc36d851277d5d3185
2020-06-21 20:58:22 +00:00
Vadim Yanitskiy 0bc32c4eab ttcn3-bts-test/ fix latest: enable IPA stream ID patching
Change-Id: I9d76acc717f5aafbfbecafb30160b02486d341fd
Depends: I66cecc9ea24ba79e1a03492e3fda2874951d37a0
2020-06-09 21:05:56 +07:00
Vadim Yanitskiy 511e02f3c5 ttcn3-bts-test/ fix: properly start virtphy
Change-Id: I2c92ca7d8839a5e141a756f0e1ca26f5f27ca143
2020-06-07 20:46:38 +00:00
Pau Espin ed0b9c92bb ttcn3-bts: Log fake_trx stdout to file
This allows easily checking if fake_trx fails to start early during
startup, for instance due to a python exception.

Change-Id: I3e13a3b1f78d071618e4848aa19bcba1eb300d18
2020-06-07 19:27:28 +00:00
Oliver Smith 5505b95a15 ttcn3-*-test: remove implicit dependencies
Don't list debian-stretch-build and debian-stretch-titan as dependencies
anymore, now that docker_images_require finds them automatically.

Related: OS#4564
Change-Id: I3cb6b693f3dd9df84fb29c0217bb3dae454cc18f
2020-05-30 09:14:06 +02:00
Vadim Yanitskiy c5f1a673bd ttcn3-bts-test: enable 3 additional transceivers for BTS#0
This is needed for the forthcoming frequency hopping test cases.

Change-Id: Ib5ad31388ae25399ad09739aac3fdcb0b3a1f78b
Depends: I86afb55ecc6703ce7a229aaa626223f9331a4778
Related: OS#4546
2020-05-28 10:31:28 +00:00
Oliver Smith f48c09d3bc ttcn3-*-test/jenkins*.sh: add $DOCKER_ARGS
Allow to pass options to each "docker run" line in the various scripts. I'm using this to run the testsuites and Osmocom
components in docker with the same configurations as jenkins, but with
Osmocom and testsuite binaries built outside and mounted into the

DOCKER_ARGS="-v /usr/local:/usr/local:ro -v $hacks:/osmo-ttcn3-hacks:ro"

This allows very short test cycles without a second set of configs. The
full script that makes use of this feature is in osmo-dev.git,

Change-Id: I8d1f78aee926f8f6ec0b704e16639bc83f525816
2020-05-06 06:22:12 +00:00
Vadim Yanitskiy bcfd877466 ttcn3-bts-test: enable time / date logging for
Depends: Ie5d14a261e17af554f7132b03d58549a4831dcdb
Change-Id: I7f82e4f573db8f72256718c04bb1ace23788ba38
2020-04-14 18:00:16 +00:00
Pau Espin 1385d5e2be ttcn3-bts: Fix trxcon docker volume dir created as root
If directory is not created beforehand by the user, docker (root) will
create the directory before binding it to the container.

Related: OS#4313
Fixes: a66f6ebb23
Change-Id: I3f02faef1aa5846c48a3776191c3862bac913d10
2019-12-09 11:50:31 +01:00
Pau Espin a66f6ebb23 ttcn3-bts: Mount trxcon dir so logs can be retrieved outside of docker
Change-Id: Icd243b62b8c80cdb2c2f09e1308f00565589c0e7
2019-12-04 14:11:38 +01:00
Pau Espin c3f157b939 Update ttcn3-*/ to log stdout+stderr of process to log file
This way we get VTY parsing failures as well as backtraces upon process

Related: OS#4212
Change-Id: I3d9f6de638dbc07391a32c6a0cbd469dc5adf2b5
2019-11-20 17:17:45 +01:00
Harald Welte 1d60e6be3e ttcn3-bts-test: Execute OML tests without BSC
Run the testsuite one more time; this time for OML tests which
require us to run without a BSC.  We can run those with trxcon
and omso-bts-trx and don't need virtphy for them.

Related: OS#4031
Depends: osmo-ttcn3-hacks I82bb85b684d2db3a6b1e96a68ffde03e22affa6b
Change-Id: I97db9c74c56ca93c75e5e5226955799cd945eec0
2019-05-29 21:59:43 +02:00
Harald Welte da2ddb8757 ttcn3-bts-test: Add virtphy based PCUIF tests
Some tests require GPRS features not yet present in trxcon, so we have
to fall back using virtphy.

Change-Id: I2f17f03194a4dfa0abca51b0d3e22b3934d08a9a
2019-05-29 21:59:43 +02:00
Harald Welte 70c92ef1c2 ttcn3-bts-test/ Move start of containers into shell functions
Change-Id: If9ac580aa5f0745468f69d413aa732891acb4a1c
2019-05-29 21:59:43 +02:00
Harald Welte 7c53c9e60d Switch all osmo-*-master and ttcn3-*-test from jessie to stretch
Jessie will be EOL in June 2019, let's make the switch to stretch
before that.

Change-Id: I37a534d07081f3d321869c86d49d03171c8da0c6
2019-03-30 09:56:36 +00:00
Oliver Smith 6687896c64 ttcn3-*-test: don't hardcode /usr/local/bin/
Remove hardcoded /usr/local/bin/ paths from This caused the
*-latest tests to fail, as they have the Osmocom binaries in /usr/bin
instead of /usr/local/bin.

Related: OS#3767
Change-Id: I9d1427e66028ffd15fed211b2dab5adc37718666
2019-01-28 13:58:02 +01:00
Oliver Smith 761b7acd22 ttcn3-bts-test-latest: use osmocom-bb-host-master
Do not try to run a osmocom-bb-host-latest docker image, which does not
exist as we are not tagging release versions for osmocom-bb (yet).

This was done right in the docker_images_require call at the top
already (which is why it did not try to build such a "-latest" image),
but then the script tried to use the "-latest" images anyway (instead
of hardcoding "-master").

Related: OS#3767
Change-Id: Ib743f09c71d93bf4c4ad947ef62b37de6b2d7ecc
2019-01-25 12:00:00 +01:00
Vadim Yanitskiy bc933e91f5 ttcn3-bts-test/ use logging options of
Change-Id: I32bf66b1320b752450a8cb064348b3c86b1c03cd
2019-01-14 15:40:33 +07:00
Oliver Smith ccf62eb231 IMAGE_SUFFIX, docker_images_require()
Default value "master" of the IMAGE_SUFFIX environment variable can be
changed to "latest" to test the latest stable builds instead of the
nightly ones. Use docker_images_require() to make sure that the required
images are existing and up-to-date before running the tests.

Related: OS#3268
Change-Id: Idbb708ab16cb71bab5069127945b63388222369e
2018-10-26 10:19:05 +02:00
Pau Espin 21831c4ef6 ttcn3-bts-test: Log fake_trx output to file
Change-Id: I82a80b045e6f298e5254f42f51dbf8c84013f45c
2018-10-25 18:05:35 +00:00
Vadim Yanitskiy fbb5366dee Rename 'osmocom-bb-trxcon' to 'osmocom-bb-host-master'
At the moment, this Docker image does contain not only trxcon,
but also FakeTRX, virt_phy, and other host applications, so
let's name it properly...

Change-Id: Ieba2411230efdedc13afcd62b8b3a2153fbaf0cb
2018-08-08 05:13:49 +07:00
Neels Hofmeyr 2f61bf81e9 have one common function to collect logs
To remove code dup and prepare a change to where logs are written, add
collect_logs() to and call that from each after
the tests are done.

The 'rm -rf' is already done before a test starts. No need to do that again
after each test.

Change-Id: I5d8472ec36b07c828685b1bd7718e31392d168a3
2018-04-24 13:46:05 +00:00
Vadim Yanitskiy 9d3b695d46 ttcn3-bts-test/ fix path
Change-Id: Ie9931651d477dc95c561a5ed554d5360b465d4da
2018-04-10 18:08:10 +07:00
Harald Welte c0da6a0e7a Add osmo-bts-test + dependencies 2018-03-01 15:19:49 +00:00