Commit Graph

44 Commits

Author SHA1 Message Date
Oliver Smith f997b4e771 network_create: find free subnet automatically
Try multiple subnet numbers until successfully creating a network. This
way we can run the same ttcn3 testsuite multiple times in parallel
without conflicts (e.g. once against latest, once against nightly). Also
we don't need to make sure each new testsuite has a unique subnet
number anymore.

I've considered also adjusting network_bridge_create, but that gets used
exclusively by osmo-ran/jenkins.sh, a script which we don't actually run
in jenkins. It seems that in this script it makes more sense to not get
a random subnet number.

Related: OS#5802
Change-Id: I57152b08ef0f38e17e7019a8df032189b03f56cf
2023-01-16 15:19:29 +00:00
Oliver Smith 36c6ff8f11 osmo-gsm-tester: fix getting latest commit
Get the latest commit from gerrit.osmocom.org, because after we moved
repositories from git.osmocom.org it currently answers with a 404.

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I0e2215d067dec1e2327f765b5a97ba7512812647
2022-12-01 14:50:39 +01:00
Oliver Smith 73a272ecd4 osmo-gsm-tester: use debian's soapysdr-module-lms7
After switching from build.opensuse.org to obs.osmocom.org's published
packages, the soapysdr0.7-module-lms7 binary package we had does not
exist anymore.

I'm not sure where it came from, from the name it looks like it was
built from version 0.7 sources, but we only have a soapysdr_0.5.4
package in OBS that was added for debian 8/ubuntu 16.04 compatibility 4
years ago, and builds have been disabled for all architectures:
https://build.opensuse.org/package/show/network:osmocom:nightly/soapysdr

Instead of relying on this leftover binary package, use the one provided
by debian instead. This osmo-gsm-tester docker container is based on
debian-buster, which has soapysdr 0.6 packaged. If we should really need
0.7, we could switch it to bullseye or build it from source while
building the container, or add it back to obs.osmocom.org.

I verified that the container builds with this change.

Fix for:
  E: Unable to locate package soapysdr0.7-module-lms7

Change-Id: Ie2408258eda407d9db6c2f0f5fbe475861756778
2022-06-29 10:20:01 +02:00
Oliver Smith c6094726ae */Dockerfile: drop MAINTAINER line
MAINTAINER is deprecated, drop the line in all Dockerfiles as suggested
by Vadim.

Related: https://docs.docker.com/engine/reference/builder/#maintainer-deprecated
Change-Id: If1ef3032af2075d792c526ae744ec4c0c091da3a
2022-05-13 18:17:06 +02:00
Oliver Smith a0a991f412 make/Makefile: always use docker run --rm
Add --rm to the "docker run" line in make/Makefile, and remove it from
RUN_ARGS.

Related: https://osmocom.org/projects/osmocom-servers/wiki/Docker_cache_clean_up
Related: SYS#5827
Change-Id: Iea063aa39a08397ea64d2018b433991a130e7f56
2022-02-14 07:46:23 +00:00
Vadim Yanitskiy 44393058f6 make/Makefile: allow overriding '-it' for target 'run'
Allocating a pseudo-TTY (-t, --tty) fails when 'run' is executed
inside of a Jenkins or cron script.  This change fixes
ttcn3-bts-test, which invokes 'run' to fetch the config files.

Change-Id: If22f682be4f004c5bb43e65098079a4f4fe6158d
Fixes: If15461240f3037c142c176fc7da745a1701ae3f8
2022-02-11 15:21:16 +06:00
Pau Espin 5882c8b6db osmo-gsm-tester: Update srsLTE references to srsRAN
Change-Id: I58d4c26337382ad9e66a569a482e77442137383a
2021-05-12 18:23:10 +02:00
Oliver Smith 948e450794 osmo-gsm-tester: open5gs deps only for x86_64
mongo-db is only available for x86_64 from their third party
repository. Don't attempt to install it for another architecture. As
this is part of the open5gs dependencies, don't install any of them
either.

This should fix the currently failing "update-osmo-ci-on-slaves"
jenkins job. I've considered disabling the build of the osmo-gsm-tester
container for ARM altogether, but the osmo-gsm-tester manual explicitly
mentions ARM trails.

Change-Id: Ieed36b1adf4b9ae147c7ee7ade22d4855c3dbac8
2021-04-14 09:39:00 +02:00
Pau Espin 0e6108a007 ogt: Add support to build open5gs
meson from open5gs requires debian10's version. physicial sysmocom OGT
setup already migrated to debian10, do the same here.

Related: SYS#4805
Change-Id: Id329726f93251b617219beeb674046065c30a436
2021-04-12 14:00:19 +02:00
Pau Espin 51493aa388 ogt: Update base image to debian buster
The OGT build setup and the physicial setup use now debian buster. Let's
update this container too to buster to avoid different versions of libs
during build and runtime (such as libasan).

Change-Id: If5adfb3c52ec5ded2adaca984e7a8c05f5866d34
2021-03-22 12:24:12 +01:00
Pau Espin d909f50176 ogt: Drop old arfcn leftovers from resources.conf
Those arfcn objects were never really used smartly and new versions of
OGT don't support it.

Change-Id: I0cbcd0f3df92393f03f29c81013e1f4b6b9aa830
2021-03-16 15:52:59 +00:00
Vadim Yanitskiy ef0f49aa4b Share a single instance of Release.key between all images
This change employs the same approach as [1] for Release.key.

Change-Id: Ic51d6e6557ee0b56d21d8ff4feaf04bc02e41a2e
Related: [1] I3ec86c8610b3b43d39ea8e3da444861d317ced4e
2021-03-10 18:47:03 +01:00
Oliver Smith 033c6ab1be */jenkins.sh: set -e after setting clean up trap
Abort the script and trigger the clean up script, whenever any of the
commands below to prepare the testsuite are failing. This saves time
with figuring out why suddenly all or most tests are failing, and avoids
running the entire testsuite on jenkins if it's obviously not going to
work.

Related: OS#3208
Change-Id: Ie68da2affda8c96b3a515a857a921a05f1bf8ef7
2021-02-26 09:32:34 +01:00
Oliver Smith 12b961942a jenkins-common.sh: add common clean up trap
Add set_clean_up_trap() in jenkins-common.sh and run it at the beginning
of the jenkins.sh files. Move the common clean up code from the end of
every jenkins.sh file into clean_up_common(), which gets called by the
trap. Add a custom clean_up() function to those jenkins.sh files that
need additional clean up.

Replace explicit container stop commands (for containers attached to the
docker network) with one call to network_clean() in clean_up_common(). It
kills all containers attached to the docker network.

The motivation for this change is the upcoming optional build of initrd
and kernel during ttcn3-ggsn-test/jenkins.sh. After building these, a
short smoke test will be performed to make sure we can boot the kernel
and initrd, before continuing to run the entire testsuite against it. If
building or the smoke test fails, we must do a proper clean up of the
network and fix permissions.

Related: OS#3208
Change-Id: I807c6221d22fec31db23e2516e477a3eb43347fb
2021-02-26 09:32:06 +01:00
Oliver Smith 8f60d951da Cosmetic: debian-stretch-jenkins, osmo-gsm-tester: fix indent
Change-Id: Ia6b12b2c35f69eb0434797b29ee0ef676b032250
2021-01-13 15:56:06 +01:00
Oliver Smith 8f2210f528 debian-stretch-jenkins, osmo-gsm-tester: add osmo-ci
Clone osmo-ci.git, and install all files from scripts/* to
/usr/local/bin. Together with osmo-ci
I45bd7ea56849dab839a98cfd52806f7cd288094e, this will make overwriting
PATH obsolete.

Related: OS#4911
Change-Id: I0820f13cd740f7d4a8999d279cef6ae293a67d81
2021-01-12 08:26:25 +00:00
Pau Espin 7d7a06b749 common: Introduce OSMOCOM_REPO_MIRROR param
This allows easily changing to another osmocom repo base URL if the
default one is down.

Related: OS#4862
Change-Id: I8010b08f3dabacfb3c13a44eece6c7a490e0742e
2020-12-02 14:27:02 +01:00
Pau Espin 1210086f95 ogt: Install dependency python3-watchdog
Change-Id: I9136cef6b5a7be085c840dd798bc21493f0a47a6
2020-11-10 17:26:44 +01:00
Pau Espin d5bd67d602 ogt: Create grbroker target directory
This is needed on the target remote hosts, where osmo-gsm-tester will
install a copy of the script to run remotely at runtime.

Change-Id: I59151dfe7091628707cd016db42171d22729a60c
2020-10-21 12:34:05 +00:00
Pau Espin 2ac3deec0b osmo-gsm-tester: Install gnuradio dependency
The python bindings of gnuradio are used to control a zmq connection
broker between srsENB and srsUE to emulate attenuations.

Change-Id: Ic4e8c4d6a5d1e20dd336ff15868c7c3f51dea323
2020-10-16 16:13:15 +02:00
Pau Espin 3cee04dddb ogt: Update installed patchelf to required version 0.11
osmo-gsm-tester requires patchelf 0.11 since a while ago to solve some
bugs on older versions. See osmo-ci
d276db78afbb3f28fc6a4a949bca131d0cfac1d4 for more information.

Related: OS#4389

Change-Id: I3a0c2168420d2a4ee2302a191824f571ec17786b
2020-10-05 12:44:09 +02:00
Pau Espin 1f45fb08ff jenkins-common.sh: Add function to print docker network related parameters
Change-Id: Ie00561de7a4494065156a124565e2190151e6019
2020-08-27 12:49:43 +02:00
Harald Welte 306a51dd30 Enable IPv6 in docker networking
We previously only allocated IPv4 sub-nets to the network segments
created with 'docker network'.  Let's fix that by assigning both
IPv4 and IPv6 address ranges to each docker network.

Related: OS#4700
Change-Id: I8802208fddcce1ffa57e5626575d23d02b320d99
2020-08-13 15:01:58 +00:00
Neels Hofmeyr 04c20edefd collect core dumps: '--ulimit core=-1' and WORKDIR=/data everywhere
Change-Id: Ief73b53ed9da6f5af82975bc36d851277d5d3185
2020-06-21 20:58:22 +00:00
Pau Espin db9514e2fe osmo-gsm-tester: Update resources.conf syntax changes
Change-Id: Iaa3e43d6878933b1f01026d58b01b1009fe4cd78
2020-06-11 17:17:21 +02:00
Pau Espin eeaa003c5a osmo-gsm-tester: Fix typo in path in jenkins.sh
Change-Id: Icdbd94a9e3de81ae60ce18da86ad57ebf70fb04f
2020-05-12 16:42:51 +02:00
Pau Espin 16901f8053 osmo-gsm-tester: Point config parameter to file instead of its dirname
Since osmo-gsm-tester.git Change-Id Ieca65b71b543c44cfcec8e83efd0fe053c432e55,
the -c parameter holds the config file and not the directory where it is contained.

Change-Id: I5aa0507d0e82616ee3cca74573fea6bdb7459b53
2020-05-12 12:53:36 +00:00
Pau Espin cfad27be56 osmo-gsm-tester: Pass conf dir through -c cmd line arg instead of env var
As of osmo-gsm-tester.git Change-Id
I7eb69f870d0dcb5906d45ae067d6bed1aabf5862, the enviornment variable
OSMO_GSM_TESTER_CONF is deprecated in favour of already existing -c cmd
line argument. Let's use it while still providing the docker setup with
a way (using same env var name) to modify where the conf dir is to be
found inside the docker container (for instance, to provide an external
conf dir).

Change-Id: I54ea9dc4d0bb205debf312f5a37053bc60cfcdb7
2020-05-07 18:49:11 +02:00
Pau Espin 42bb1385ee osmo-gsm-tester: Update path to sysmocom's specific setup
Since osmo-gsm-tester.git I364e0859c0851b9167907a4d948b659a0f6a19fb the
example/ directory has been renamed to sysmocom/

Change-Id: I8ca085e02330be51de12066da0555c1bc0c8b767
2020-04-10 21:02:43 +02:00
Pau Espin f2373cc0ad osmo-gsm-tester/resources.conf: Add 2g,4g modem features attr
Since osmo-gsm-tester.git I38bcf6abf789f52c3ed0bee7911567fa872e2491
these features are included and used in the "4g" suite to allow
selecting 4G modems different than srsue.

Change-Id: Ie1942fd661b91218780bc52bb000d192259e2b69
2020-04-03 19:04:02 +02:00
Pau Espin 428858e642 osmo-gsm-tester: Require debian-stretch-jenkins as Dockerfile suggests
Some days ago the Dockerfile was changed to pull from
debian-stretch-jenkins instead of debian-stretch-build, since the first
one has extra runtime dependencies required by osmo-gsm-tester.
However, the jenkins script is still requiring the old dependency. Let's
update it accordingly.

Change-Id: If33aab60e04da7beb4cd0715d5cb3145c001823a
2020-03-31 19:31:59 +02:00
Harald Welte 1bccbb8a66 update OBS Release.key; the old one expired today.
Change-Id: I0dd7d7f54a3eb93ca677f3bdc5e00f62694aaa9e
2020-03-30 18:06:58 +02:00
Pau Espin 52be9c2e40 osmo-gsm-tester: Print colored run output succes/failure
This way one can quickly see if osmo-gsm-tester run fine or an issue
ocurred and logs need to be checked.

Change-Id: I0b267889f403fc1505f9dd859568bc15a120dd81
2020-03-20 19:17:54 +01:00
Pau Espin 8e8f671711 osmo-gsm-tester: Allow changing host's trial directory
It's set to /tmp/trial by default for convenience, since it's a
temporary directory and easy to remember and to access. Still, let's
give the opportunity to change it (if user wants it to be stored in
persistent data or if has low amounts of non-persistent memory).

Change-Id: I2a7ea361c6156c97bd8d2d94cf1b6856344d4d9f
2020-03-20 19:17:35 +01:00
Pau Espin cafa320eba osmo-gsm-tester: Use non 'inst' name as target install dir, and remove it once archived
osmo-gsm-tester uses also the "inst/" directory, so using that name to
install from cmake is not a good idea here, since they will end up in
the same directory under /tmp/trial/inst.
Also, make sure the temporary install dst directory is removed after
creating the archive, since at that point is no longer needed.

Change-Id: Ib6610487140f0703d5817241bc4a63027696c1d4
2020-03-20 19:17:35 +01:00
Pau Espin 41230705dc osmo-gsm-tester: Build srsLTE inside the docker instance if inst not provided by host
osmo-gsm-tester.git/contrib/jenkins-build-srslte.sh cannot be used
because there's a need to split the fetch step from the build step.
That's because we want to fetch in the host (in order to use host's user
ssh key for potential private repos) while still building inside the
docker container, where we have all the dependencies (and where the
binaries are going to be run afterwards by osmo-gsm-tester).

During the changes above it was also noticed that /tmp/trial on the host
got its UID+GID changed (because we switch user inside the container to
downscale privileges). Fix it by chown'ing the directory back to the
host user's UID+GID.

Change-Id: Ia0db8fba840392a0b7883cfc15e45c2e80407a16
2020-03-19 20:13:58 +01:00
Pau Espin 853ddcd792 osmo-gsm-tester: Base docker image on debian-stretch-jenkins
That image contains extra runtime dependencies (such as libua5.3) which
may be required by software run by osmo-gsm-tester.

Change-Id: I5c6239e1beb0cdc6d2b1477ddb60d66debe5d1aa
2020-03-12 15:51:08 +00:00
Pau Espin 9145a91594 osmo-gsm-tester: Allow passing osmo-gsm-tester params from jenkins.sh caller
After this patch, one can run osmo-gsm-tester with specific parameters
(debug level, suite and tests to run, etc.) by using
OSMO_GSM_TESTER_OPTS variable:
OSMO_GSM_TESTER_OPTS="-s 4g -t ping -l error" ./jenkins.sh

Change-Id: I2e8ff51a3152125148c6a117a987e8c11a576e1e
2020-03-12 15:49:48 +00:00
Pau Espin d61b741a61 osmo-gsm-tester: Allow setting rtprio inside the container
This fixes for instance srsLTE being unable to create a thread with
realtime priority.

Change-Id: If0d033c6e912795804e6d7322c22fdfc6602028b
2020-03-12 15:49:48 +00:00
Pau Espin 4b686a004e osmo-gsm-tester: Copy limits.d files to docker image
These are required to allow setting rt prio for jenkins user inside the
docker container, as well as for being allowed to create coredump files.

Change-Id: I8ed6193196d739c1416976d431d07b8a0c55ee2a
2020-03-02 18:04:24 +01:00
Pau Espin 2a9cf7a53a osmo-gsm-tester: Improvements anv fixes to run 4G network with latest ogt version
* Fix wrong KI.
* Add an extra IP address to slave so that ENB and EPC can be run on
  different IP addresses and their GTP ports don't collide.
* Add required rf parameters to resources.conf

Change-Id: I0198bc99f75e9567ee717aa79e434ec28705fac9
2020-03-02 17:28:39 +01:00
Pau Espin 79032097dc osmo-gsm-tester: Add missing srsLTE dependencies (soapy)
Currently srsLTE trials link against soapySDR library to have LimeSDR
support. We need to move installation of those new packages further
below in the Dockerfiler because that lib version is only available with
osmocom repositories (which were added at a later point in the file).

Change-Id: Ia0e506954ffebd94adb75fec6bf481e41f9ec436
2020-03-02 17:28:39 +01:00
Pau Espin 3e59758fd7 osmo-gsm-tester: new versions require python3-numpy
Change-Id: Ic1a9edbaf4151aabd157f859ec6fbec30d4f27b0
2020-03-02 16:39:53 +01:00
Pau Espin c9fa2a8808 Introduce osmo-gsm-tester docker setup
This set up allows running a typical osmo-gsm-tester set up with a main
unit (ogt-master) attached to one (or more) slaves where resources are
located and processes are run.

As an example, current config allows running an iperf3 test over an srsLTE
stack using the ZeroMQ backend.

Change-Id: Iac7c0b613048ce332642d3957c55cc7c53b56d42
2020-03-01 19:23:00 +00:00