Commit Graph

386 Commits

Author SHA1 Message Date
Oliver Smith f9a889e29d debian-stretch-build-dist: new Dockerfile
Provide all build dependencies needed to pass ./configure of most
Osmocom repositories. This will be used to run osmo-release-tarballs.sh
in jenkins.

Related: OS#3870
Change-Id: I991fe41a1f55ccd6a66815ec3d7f239850397d9a
2019-07-02 13:19:39 +02:00
Pau Espin 2bd38c7846 ggsn: Set correct name for osmo-ggsn log file
Change-Id: I65803bdd4d2bcd79c690841bfc3cc62c8260289d
2019-06-30 11:55:11 +00:00
Oliver Smith 7721e142dc ttcn3-mgw-test-latest: disable conn_timeout_test
Depends: I8bbf15baa45679d5812a5a9184520ef9b9e73bba (osmo-ttcn3-hacks)
Change-Id: I05ff77246462ba90e8704969e0982662027ad630
2019-06-28 11:22:02 +02:00
Oliver Smith 0436d95f2e regen_doc: check OSMO_INTERACT_VTY, fix default
Change default from osmo-interact-vty.py to osmo_interact_vty.py, which
is the name of the script in osmo-python-tests.git.

Check if that script is in PATH before building and running the docker
container.

Change-Id: I763c7379b710160e0155f1fde091137754141335
2019-06-27 09:57:01 +02:00
Daniel Willmann b39b785fdf Have only one bts/bsc configured in osmo-bsc and bscnat
The way our counter reporting works is that each counter group is
reported even if the same one exists multiple times. To avoid that
change the default config (used by regen_doc.sh) to only configure one
bts / bsc.

Change-Id: I57468d983580b5ed4f15961ac1eac9bed9096556
Related: OS#1700
2019-06-24 14:12:01 +00:00
Daniel Willmann 3c9fd3bef8 osmo-sgsn-master: Bind to localhost instead of a specific IP by default
This does not break ttcn3-sgsn-test since it uses a custom config file
with the correct IPs.

Change-Id: I2a922139c50d06217e5e6105ea65bf673daa10a8
Related: OS#1700
2019-06-24 14:12:01 +00:00
Daniel Willmann a0eb1ed06d osmo-mgw: Bind to 0.0.0.0 by default
Don't bind to a specific IP otherwise starting osmo-mgw will fail (if we
don't configure that exact IP address)

Related: OS#1700
Change-Id: Ic5828957a29d4f317e1ebf4f03b5f5359f6250e8
2019-06-24 14:12:01 +00:00
Oliver Smith 88a74a5de1 ttcn3-{hlr,mgw,pcu}-test: log to file
Add missing file output which all other tests already have.

Related: OS#4067
Change-Id: Ifb4a39e6e2987f4dc04369508fe0dff0913a02ff
2019-06-24 13:52:06 +00:00
Harald Welte 26ad849722 redmine: migrate from 3.4 to 3.4-passenger
This provides much more reasonable performance on our high-end server

Change-Id: Idaa19566d1d5ac4ea0ff0296d57d1f9ca2bd599b
2019-06-24 14:50:29 +02:00
Daniel Willmann 63023eb7e6 regen_doc.sh: Add optional parameter to customize docker execution
The osmo-nitb-master image can run not only osmo-nitb, but also
osmo-bsc_nat. In that case we can't use the default command from
Dockerfile, but should provide our own. This argument makes it possible
to do so.

Change-Id: I52c3cb2cdcfb143e71ac2c1aa40bb466e41a0f5c
Related: OS#1700
2019-06-19 14:04:46 +02:00
Daniel Willmann ec06cc5981 osmo-nitb-master: Add default config file for osmo-bsc_nat
Change-Id: I44bb395009582f5e1399f277e15203553c2dcf78
Related: OS#1700
2019-06-19 14:04:46 +02:00
Daniel Willmann 3b96cf3bb4 Add script to regenerate VTY and counter documentation
This script needs access to the VTY interface, so I changed some
remaining config files to bind the VTY interface to 0.0.0.0.

There are a couple of services that do that already.

Related: OS#1700
Change-Id: Ic5cf7f59de0c18d4c5568b89433c2b11db2a72a5
2019-06-18 18:18:08 +00:00
Harald Welte b40355a588 jenkins-common.sh: Don't rebuild debian-stretch-build during fix_perms
When fixing the permissions at the end of a test run, it really doesn't
matter if we're running a current debian-stretch-build image, or if we
are using an ancient one.  All we need is a 'chmod' binary.

Change-Id: Ib008ee72f40aeefb307c0ba74a57b70daae9d07b
Related: OS#4060
2019-06-13 11:46:34 +02:00
Pau Espin b14af6729b bscnat: Bind bsc-nat CTRL on 0.0.0.0
Other tests already do it, and allows using the CTRL iface in docker
setups where 127.0.0.1 is not used.

Change-Id: I6f16dc322d38f9f0870241f3416c81df8ec5481b
2019-06-11 12:11:07 +02:00
Harald Welte 2771b03658 ttcn3-bts-tests: Add BTS_Tests_LAPDm.control to config file
There's a number of new tests that aren't executed yet.

Change-Id: I3bfb4483f39d4dbf3f506f86470e4f2d3b6b7af2
2019-06-04 15:38:34 +00:00
Pau Espin ee49c7ba88 ttcn3-*: Enable extended-timestamp in VTY cfg
Having extended-timestamp allows for easier matching between osmo-*.log
files and pcap traces.

Change-Id: I98cdc0adc24edc966d5648d2c215c6ad7c749209
2019-06-03 17:59:42 +02:00
Pau Espin bf0d5075cb msc: Disable osmux for ttcn3-msc-test-latest
Latest tagged version of osmo-msc doesn't support "osmux on/off" VTY
commands, so we have to enable osmux use on that version to avoid most
TTCN3 tests failing during set up phase.

Depends: osmo-ttcn3-hacks.git I53d58b2d905905ebf1df322d0389b3715a48212f
Change-Id: I6b3be3981978661de2fa90be130f6f1811a3d1f9
2019-05-31 18:05:40 +02:00
Pau Espin 0e59f1e8bf bsc: Fix: Disable osmux for ttcn3-bsc-test-sccplite-latest
Recent commit added bits to disable "osmux on/off" vty commands on
"-latest" tested, but forgot to add required line in
sccplite/BSC_Tests.cfg for sed command in jenkins.sh script to work.

Fixes: 2cdab85e5f
Change-Id: I9b4ce2d7151d65b4c83417a59fd01672971b3937
2019-05-31 14:44:58 +02:00
Harald Welte 1d60e6be3e ttcn3-bts-test: Execute OML tests without BSC
Run the testsuite one more time; this time for OML tests which
require us to run without a BSC.  We can run those with trxcon
and omso-bts-trx and don't need virtphy for them.

Related: OS#4031
Depends: osmo-ttcn3-hacks I82bb85b684d2db3a6b1e96a68ffde03e22affa6b
Change-Id: I97db9c74c56ca93c75e5e5226955799cd945eec0
2019-05-29 21:59:43 +02:00
Harald Welte da2ddb8757 ttcn3-bts-test: Add virtphy based PCUIF tests
Some tests require GPRS features not yet present in trxcon, so we have
to fall back using virtphy.

Change-Id: I2f17f03194a4dfa0abca51b0d3e22b3934d08a9a
2019-05-29 21:59:43 +02:00
Harald Welte 70c92ef1c2 ttcn3-bts-test/jenkins.sh: Move start of containers into shell functions
Change-Id: If9ac580aa5f0745468f69d413aa732891acb4a1c
2019-05-29 21:59:43 +02:00
Harald Welte cbcdf97f92 osmoocm-bb-host-master: Build virtphy, not only trxcon
There are some BTS_Tests which (so far) don't yet work with trxcon
hence require virtphy.

Change-Id: I0aabd73e2778b9144aa7945f25cae5cefea4111b
2019-05-29 21:59:43 +02:00
Pau Espin cb3d19f277 ttcn3-sgsn: Run log_merge.sh after tests
Change-Id: If03422bb8fb126fe6f204a4f8bc6edf0cbb7f194
2019-05-29 14:34:46 +00:00
Pau Espin 2cdab85e5f bsc: Disable osmux for ttcn3-bsc-test-latest
Latest tagged version of osmo-bsc doesn't support "osmux on/off" VTY
commands, so we have to enable osmux use on that version to avoid most
TTCN3 tests failing during set up phase.

Depends: osmo-ttcn3-hacks I61e4c59b2926f3f70cb6d0190a8683861e54179a
Change-Id: I745b210f43cb147d323a15f78c294f70ee077729
2019-05-28 18:42:01 +02:00
Pau Espin 808801a9fa ttcn3-bscnat: Prepare cfgs for VTY & Osmux support
* Adding "osmux on" for each BSC will be required to enable Osmux later
on, but still it doesn't change behavior now because Osmux must be
enabled globally too (will be done through VTY).

* Set up correct bind and remote IP addr to have VTY working later on.

* Use extended timestamp to have subsecond accuracy to match with pcap
easily.

Change-Id: I794a17bdeaff9ab9316d7a6acb1e58dfd815b645
2019-05-27 18:06:06 +00:00
Harald Welte 77bf937cac ttcn3-bts-test: Fix BSC VTY access from test suite
Some tests, particularly those for SMSCB, want to access the BSC
VTY.  This was enabled in the ttcn3 code, but the configs we're
using in the docker environment didn't properly refelect that yet.

Change-Id: I6b6e8b47bfe85c15fb57a25b649029f872477146
2019-05-23 17:17:31 +00:00
Harald Welte 317aba0226 ttcn3-bscnat-test: Use unique sub-network 172.18.15.0/24
For some reason, the bscnat tests were using the same subnet as the
sip-connector tests, causing address clashes when trying to execute
both test suites simultaneously:

+ echo Creating network ttcn3-sip-test
Creating network ttcn3-sip-test
+ docker network create --internal --subnet 172.18.11.0/24 ttcn3-sip-test
Error response from daemon: Pool overlaps with other one on this address space

Change-Id: I62b01b4b291ed9d729139fcf9b0102459e7f5a6a
2019-05-23 16:00:26 +02:00
Oliver Smith 61f2be7bd4 Fix git checkout for branches and commits
Replace 'git checkout -f -B $BRANCH origin/$BRANCH && \' in all
Dockerfiles that accept branch variables ($OSMO_TTCN3_BRANCH,
$OSMO_MGW_BRANCH, ...) with the following:
	git checkout $BRANCH && \
	(git symbolic-ref -q HEAD && git reset --hard origin/$BRANCH || exit 1); \

This allows using branch names and commit hashes in the $BRANCH
variables. Using commits is needed for the bisect script added in [1].

The second line ("(git symbolic...") checks if we are in detached HEAD
state after the checkout, and if we are not, pulls in all new commits
from origin/$BRANCH. Note that it ends in ';' instead of '&&', because
the command in the next line should be executed even if
"git symbolic-ref" does not exit with 0 (detached HEAD state).

Here is an example, to illustrate that the new command does the right
thing. Clone a repository and be 50 commits behind origin/master:
$ git clone "https://git.osmocom.org/osmo-mgw"
$ cd osmo-mgw
$ git reset --hard origin/master~50

With BRANCH="master":
$ git checkout master && \
  (git symbolic-ref -q HEAD && git reset --hard origin/master || exit 1); \
  echo "done"
Already on 'master'
...
done
$ git status
Your branch is up-to-date with 'origin/master'.

With BRANCH="85978d":
$ git checkout 85978d && \
  (git symbolic-ref -q HEAD && git reset --hard origin/85978d || exit 1); \
  echo "done"
Note: checking out '85978d'.
...
done
$ git status
HEAD detached at 85978dad

Related previous changes:
* [2] made it work for commit hashes, but broke using branch names other
      than master, and pulling in new commits from master
* [3] made branches other than master work again, but did not fix
      pulling in new commits from master
* [4] reverted [3] and the git checkout related part from [2]

[1] Change-Id: I11f7e61a9b30d58a0fdfcaf77dde447806bf661f
[2] Change-Id: If3bc5fae07cf63c4fef68306be4c4328660bc869
[3] Change-Id: I2ff745c8d19b777d876170d5717c082ceb68a1f3
[4] Change-Id: Ie6da0f9ea96f11407e38545a6b3cf22ef9cadc25

Related: OS#4015
Change-Id: I4004980baf0b7d6096702b6f3067ccbdb369a28c
2019-05-22 11:18:56 +02:00
Harald Welte 54553bef85 osmo-bts-test: Enable BTS_Tests_SMSCB.control for CBCH testing
Support for this was introdcued in osmo-ttcn3-hacks
Change-Id If4b5a906a0841c0a8c3d7c4e9e5a3d1208ecf16a, but as we have
a separate .cfg file here, we must update it, too.

Change-Id: Iab0e9a46a8ee429619e9ef9159a631dc44ab5aeb
2019-05-21 18:33:28 +02:00
Harald Welte 06ac6e455e Revert "Simplify git checkout, allow branches and commits"
Unfortuately this commit will check out *local* master (i.e. the
previous checkout) rather than the origin/master as it's supposed to
be. This means that ever since merging this patch, our ttcn3 tests
were running some "undefined" stale versions and not current master.

This reverts commit 26565bb729.

Change-Id: Ie6da0f9ea96f11407e38545a6b3cf22ef9cadc25
2019-05-21 17:53:11 +02:00
Alexander Couzens 73663b01f2 RFC: allow to set configuration and/or specific testcases
Allows to pass the configuration file and/or tests to
start-testsuite.sh.

This way a single test can be specified by the command line.

If this RFC gets accepted, I would convert all other Docker.files

Change-Id: I17e480321f1e7bfa287349f438ff6990e30f1114
2019-05-16 18:55:41 +02:00
Oliver Smith 26565bb729 Simplify git checkout, allow branches and commits
Replace the following statements:

a) "git checkout -f -B $BRANCH origin/$BRANCH"
b) "git checkout -f -B $BRANCH $BRANCH"

with:

c) "git checkout -f $BRANCH"

Regarding a), we don't need to specify 'origin/' for each branch, since
we are cloning the repositories in the same Dockerfile, and therefore we
know for sure that there is only one remote and branch names won't be
ambiguous. Removing the 'origin/' allows to put commit hashes into the
branch variables (like done in the new bisect script [1]).

Version b) does not work with branch names:
$ git checkout -f -B osmith/check-imei-before-lu osmith/check-imei-before-lu
fatal: Cannot update paths and switch to branch 'osmith/check-imei-before-lu' at the same time.
Did you intend to checkout 'osmith/check-imei-before-lu' which can not be resolved as commit?

New version c) works with both commits and branches, and it is shorter.

[1] Change-Id: I11f7e61a9b30d58a0fdfcaf77dde447806bf661f

Change-Id: I2ff745c8d19b777d876170d5717c082ceb68a1f3
2019-05-07 10:24:52 +02:00
Oliver Smith 269331461a debian-*-build: don't download "Release" file
Stop jenkins jobs from failing, due to the download errors we are
getting from time to time, by simply not downloading these files. They
are redundant, because we are already running 'docker build --pull'
before trying to build the debian-*-build images [1], and the docker
images are released at least as often, as the "Release" file gets
updated [2].

This is in line with how it is done in osmo-ci.git.

[1]: Change-Id: I1076bbb7d77bdc99f5d60d641c09ce965fb9dfc6
[2]: https://hub.docker.com/_/debian

Change-Id: Ieedb310814412f235e56bdc65517404842eac00f
2019-05-07 06:31:16 +00:00
Vadim Yanitskiy 857d60059c gr-gsm-master/Dockerfile: use the official archlinux/base image
According to [1], the image 'base/archlinux' has been deprecated.
It's recommended to use the official ArchLinux docker image [2].

[1] https://hub.docker.com/r/base/archlinux
[2] https://hub.docker.com/r/archlinux/base

Change-Id: I7e5d57662e9d8ca2b5037c24427ce1c44de15813
2019-05-05 18:59:16 +03:00
Alexander Couzens c924d417c5 debian-stretch-titan: install procps
The ttcn3-tcpdump-stop.sh uses `ps` to find out if tcpdump is still
running.

Change-Id: I8db4ad2262a3a734201e39bc065c1c383738992c
2019-05-02 18:04:18 +02:00
Harald Welte df7d1bc260 ttcn3-msc-test: Execute MSC_Tests_Iu.control
The new IuCS related tests have been added as 'MSC_Tests_Iu'
module to take advantage of the hierarchical display format in the
Jenkins tests results analyzer.  Let's execute them.

Change-Id: I80ccfce4874d14039bbcb273c3b450db2a7780dd
2019-05-01 08:25:18 +02:00
Harald Welte e0816ed331 ttcn3-msc-tests: Add IuCS for one simulated RNC
We need to update the MSC_Tests.cfg as well as the osmo-stp.cfg
to provsion a virtual RNC link.  As the new RNC uses routing key 2,
we shift the MSC routing key to 3.

Change-Id: I10a249b1a851436fd3c20face6ccc94b304bd3e4
2019-04-30 15:24:16 +00:00
Daniel Willmann 6e601e60ad Add script to bisect test failures with ttcn3 and docker
With this script you can now use docker ttcn3 test results to bisect a
regression and find the offending commit.

Use like this from the osmo-* git repository:
$ git bisect start <bad-rev> <good-rev>
$ git bisect run ~/scm/osmo/docker-playground/osmo-bisect.sh <component-to-test> <testcase>
e.g.:
$ git bisect run ~/scm/osmo/docker-playground/osmo-bisect.sh bsc BSC_Tests.TC_ho_in_fail_no_detect

Change-Id: I11f7e61a9b30d58a0fdfcaf77dde447806bf661f
2019-04-26 12:07:15 +02:00
Daniel Willmann b59a9ed8d4 Add OSMO_*_BRANCH environment variables for build args
Previously we could only set OSMO_TTCN3_BRANCH as environment variable
to build a test other than master.

This patch adds environment variables for all osmo-*-master images which
allow docker tests to be executed for an arbitrary commit.

The origin/ prefix from the git checkout command is removed so the
*_BRANCH variable doesn't have to contain branch names, but van also
contain arbitrary commits. This shouldn't have any adverse effect as we
only have one remote in the checkout.

Change-Id: If3bc5fae07cf63c4fef68306be4c4328660bc869
2019-04-25 13:56:14 +02:00
Harald Welte a7deb5d7a8 debian-stretch-titan: include libfftranscode for our Iu related tests
Change-Id: Ifd7d682df90f1b17823d8358a8a498f149e4591b
2019-04-21 11:00:45 +00:00
Harald Welte 6657ba7889 Use ftp.debian.org as a "blind workaround" for trouble with ftp.de
We get a lot of build failures recently like this:

Step 3/4 : ADD  http://ftp.de.debian.org/debian/dists/stretch/Release /tmp/Release
ADD failed: Get http://ftp.de.debian.org/debian/dists/stretch/Release: dial tcp 141.76.2.4:80: i/o timeout
../make/Makefile:44: recipe for target 'docker-build' failed
make: *** [docker-build] Error 1
make: Leaving directory '<https://jenkins.osmocom.org/jenkins/job/ttcn3-sip-test/ws/debian-stretch-build'>
+ exit 1

Let's hope that this works around the connectivity problems with
ftp.de.debian.org

Change-Id: I5a0eb058d95372c44fce042de90c4127a3021a93
2019-04-17 09:10:48 +02:00
Oliver Smith bdfb823f05 nplab-m3ua-test: support running without jenkins
Don't use $WORKSPACE anymore, use $VOL_BASE_DIR like in every other
jenkins.sh in this repository. Make it possible to run jenkins.sh
outside of jenkins (where $WORKSPACE is not set), because
jenkins_common.sh will set up $VOL_BASE_DIR to point to /tmp/logs.

Change-Id: I200b4ee1760a879cbc0a80a30a062a3f2a8e703d
2019-04-05 08:30:28 +02:00
Oliver Smith e021ae6ab3 nplab-m3ua-test: add sigtran-tests dependency
Make sure to build sigtran-tests before building nplab-m3ua-tests. This
has been working by coincidence without the dependency so far, because
nplab-sua-test had the dependency listed and that test ran on the same
host.

Change-Id: I9df1d2b188e86b3c99c6ec793b6eb644ab3c27c9
2019-04-05 08:20:20 +02:00
Oliver Smith e8cd52f074 gitignore: add .release
Ignore .release files generated by make/Makefile.

Change-Id: I9ea8e4326f6da2ff95cf45999637c89f884a9e19
2019-04-04 10:46:14 +00:00
Oliver Smith 052fe1bb8a make/Makefile: fix recursive 'PULL' error
Don't abort with the following error, when running 'make' without
setting the PULL environment variable:

 ../make/Makefile:20: *** Recursive variable 'PULL' references itself (eventually). Stop.

This fixes a regression from Change-Id
I1076bbb7d77bdc99f5d60d641c09ce965fb9dfc6.

Related: OS#3869
Change-Id: I4b4772e9ed6e07ab00943154265c9cbdea22a2f5
2019-04-04 08:16:44 +02:00
Oliver Smith a8be3ff52f jenkins-common.sh: pull upstream base images
Prevent building docker images on top of outdated Debian images, where
the package download feeds have been disabled. Use 'docker build --pull'
instead of 'docker build', whenever the "FROM" line in the Dockerfile
does not contain '$USER' (meaning this is an upstream image).

Checking the FROM line is necessary, as downloading '$USER' images will
fail (we never upload those).

Related: OS#3869
Change-Id: I1076bbb7d77bdc99f5d60d641c09ce965fb9dfc6
2019-04-02 11:38:26 +02:00
Harald Welte 7c53c9e60d Switch all osmo-*-master and ttcn3-*-test from jessie to stretch
Jessie will be EOL in June 2019, let's make the switch to stretch
before that.

Change-Id: I37a534d07081f3d321869c86d49d03171c8da0c6
2019-03-30 09:56:36 +00:00
Harald Welte cffe1022b5 print branch name and git commit hash during docker container build
This is quite useful when looking at jenkins logs to know which exact
version was built at the time.

Change-Id: Id52c382b454e2beecf46820752aeff15b2c1a0ae
2019-03-29 16:40:40 +00:00
Daniel Willmann 2e3a5c1c53 ttcn3-bscnat-test: osmo-bsc_nat config file fixes
* Log to file in osmo-bsc_nat
* Fix number of endpoints so the test passes

Change-Id: Ie90ef6a6e09a3c47955297dedf78121e2c5cbe31
2019-03-29 11:14:36 +00:00
Daniel Willmann 1b5ed832c8 osmo-cn-latest: Add default osmo-msc.cfg
Change-Id: I821f76ace7dfd666a6ba6aa382eafa6527af7ba6
2019-03-28 13:22:22 +01:00