Commit Graph

677 Commits

Author SHA1 Message Date
Oliver Smith 31daedcd2b osmo-*-master: move common code to *-build
Move common code from osmo-*-master to debian-stretch-build and
centos8-build:
* add the OBS repository key (debian only)
* add the OBS repository with "nightly" feed (debian only, already done
  for centos8)
* install telnet
* invalidate the cache once the repository is updated

While at it, remove deprecated MAINTAINER lines.

This fixes the need to read both the repository index of debian-stretch
and centos8 and having to invalidate the cache if either of them changes
(instead of reading just the relevant index and invalidating the cache
only if the relevant one changes).

A similar change was done with osmo-*-latest in
I443d52eba416ea298cc5421e280ad2f4ca59283b.

Related: OS#5049
Related: https://docs.docker.com/engine/reference/builder/#maintainer-deprecated
Change-Id: Id69f3eaf37f823bac0947366f27a25edd277a4cb
2021-03-04 14:45:51 +01:00
Oliver Smith 69b3ecb912 osmo-*-latest: support centos8
Add a new centos8-obs-latest container with shared code (just like
debian-stretch-obs-latest) and adjust the package install blocks in
osmo-*-latest.

Related: OS#5049
Change-Id: I274b31d4903df07e99fd434af25bdb72991f6923
2021-03-04 14:42:32 +01:00
Oliver Smith effdb1a452 debian-stretch-obs-latest: new container
Move common code from osmo-*-latest to debian-stretch-obs-latest:
* add the OBS repository key
* add the OBS repository with "latest" feed
* install telnet
* invalidate the cache once the repository is updated

This is in preparation to make osmo-*-latest work with centos8,
preventing the following issues we would have without this refactoring:
* avoid need to read the repository index of both debian-stretch and
  centos8, and to invalidate the cache if either of them changes
  (instead of just the relevant one). This is currently a problem in
  osmo-*-master/Dockerfile, Id69f3eaf37f823bac0947366f27a25edd277a4cb
  will fix it there.
* no need to add boilerplate code to install the centos8 repository to
  each osmo-*-latest/Dockerfile

While at it, also remove deprecated MAINTAINER lines.

Related: OS#5049
Related: https://docs.docker.com/engine/reference/builder/#maintainer-deprecated
Change-Id: I443d52eba416ea298cc5421e280ad2f4ca59283b
2021-03-04 14:39:46 +01:00
Oliver Smith 46797c80ad osmo-msc-latest: implicit libdbd-sqlite3 install
Remove the explicit mention of libdbd-sqlite3, as it is in the depends
of osmo-msc now.

Related: OS#3771
Change-Id: I0b6d8ea4295021ce5470ee342ed7d89b42886348
2021-03-04 11:19:02 +01:00
Oliver Smith b2a0f11a49 osmo-ggsn-latest: add more kernel build depends
Add everything needed to run ttcn3-ggsn-test-kernel-* jobs.
osmo-ggsn-master needs less dependencies, as more get installed in the
base image debian-stretch-build.

Related: OS#3208
Change-Id: Iaef87c3418b8e6f1e427b2abd9d40e9e28dc63e9
2021-03-04 10:55:40 +01:00
Harald Welte 2d831f1b50 Add osmo-gbproxy-latest container
Change-Id: I285cb51da03b793eab994ff3c00de16097e3f992
2021-03-02 13:17:01 +01:00
Harald Welte 71156da68f add osmo-smlc-latest container
Change-Id: I714758ddd832e77b592dec5d17ef3568420d11ff
2021-03-02 11:17:04 +00:00
Oliver Smith 58c1c19946 README.md: describe OSMO_*_BRANCH
Copy description from wiki, so the README.md describes all environment
variables, and the wiki page can link to the README.md.

Change-Id: I23077edf153b76f7936f0b874e87bb7424766b19
2021-03-01 09:39:34 +01:00
Oliver Smith 292e35c1d9 kernel-test: verify REMOTE_NAME and URL
If the checked out git repository already has a remote with the name
KERNEL_REMOTE_NAME, make sure that the KERNEL_URL matches the remote
URL.

Motivation for this change is osmo-ci
I5d4202a67a24d9c15a5b211fa29ce9d5b5a9d9c1, which will expose the
parameters in the jenkins job.

Related: OS#3208
Change-Id: I18b29011e3e8e2577bdf1a9c64c4370309cc8399
2021-03-01 08:50:57 +01:00
Harald Welte c877821758 ttcn3-cbc-test: fix copy+paste error regarding osmo-stp.cfg
Change-Id: I1517238e997605cbcf265b97526694d17137f00e
2021-02-28 11:15:18 +01:00
Oliver Smith cb2a8326c1 kernel-test: don't try to use KVM if not available
Disable KVM if /dev/kvm is missing, as it is apparently the case on our
jenkins nodes. This makes the tests run a bit slower, but not much. Add
kernel_test_wait_for_vm to sleep until the VM is booted up (~5s instead
of ~1s without KVM), so the first test in the testsuite does not fail
when using KVM.

The variable is useful to test the non-KVM code path even if KVM is
available on the host.

Related: OS#3208
Change-Id: I1f337af1e2de6db05b22636bc31a535404235559
2021-02-26 16:18:26 +00:00
Pau Espin 5a31e0adc8 ttcn3-sip: Drop uneeded module param MNCC_Emulation.mp_mncc_version
Since -latest was updated to newer releases, this param is never set to
changed anymore, so there's no need to set it here explicitly.

Change-Id: Ia9d6b7f383d6532f8f2da381e80f98af82a98e74
Related: OS#5042
2021-02-26 13:36:06 +01:00
Pau Espin 06b256e337 ttcn3-msc: Drop uneeded module params
Since -latest was updated to newer releases, there's no need to change
the values of these params. Furthermore, they will be completely dropped soon.

Change-Id: I6121b435c5b984eab64d953e9d3f26eecc0e7bc5
Related: OS#5042
2021-02-26 13:35:03 +01:00
Pau Espin 6c696b2294 ttcn3-hlr: Drop uneeded module param HLR_Tests.mp_hlr_supports_dgsm
Since -latest was updated to newer releases, this param is never set to
false anymore, and it already defaults to true, so there's no need to
set it here explicitly. Furthermore, it will be completely dropped soon.

Change-Id: Icab36eba29ad4fadd07ec804ef713910b80ce706
2021-02-26 13:04:07 +01:00
Pau Espin 6ef96e4980 bts-test: Drop uneeded module param RSL_Emulation.mp_rslem_patch_ipa_cid
Since we updated -latest to new releases, this param is never set to
true anymore. Furtermore, it will be dropped soon completely.

Related: OS#5042
Depends: osmo-ttcn3-hacks.git Change-Id I39e9c1986ea682d54dcb22b31798ca91f1677949
Change-Id: I4bda577f1ed6964a2114df1aba366177998f0854
2021-02-26 13:02:02 +01:00
Pau Espin a0026cd724 ttcn3-bsc: Get rid of uneeded param BSC_Tests.mp_media_mgw_offer_ipv6
This param is true by default and there's no one setting it to false
anymore (and soon will be removed).

Related: OS#5042
Change-Id: I1780320c69ec4a9f086677fd4884dec674f8332e
2021-02-26 12:53:21 +01:00
Pau Espin bd79c36e29 Get rid of uneeded param StatsD_Checker.mp_enable_stats
This param is currently set to true by default and will be removed soon.

Related: OS#5042
Depends: osmo-ttcn3-hacks.git Change-Id I4e2049c109986906d3c985ca2282174b1abff581
Change-Id: I3c7e7dc985a339d884dc388b33bf8a6ec986c5b7
2021-02-26 12:45:20 +01:00
Oliver Smith 79e48fd3ae ttcn3-ggsn-test: add kernel test
Make it possible to run ttcn3-ggsn-test against osmo-ggsn with the GTP
kernel module. This feature is enabled by environment variables
explained in ttcn3-ggsn-test/jenkins.sh, and can either use the
pre-built kernel from Debian, or build a kernel from source.

The kernel runs inside QEMU inside the usual osmo-ggsn-master docker
image.

Thanks to Pau for providing the kernel config and kernel-test scripts
that this patch is based on! I ended up modifying them quite a bit to
fit the docker use case, and to build the initrd (so we can use the
stock debian kernel).

Related: OS#3208
Change-Id: I54e752048eed2a170a1cf4e0fd410de255b9de5d
2021-02-26 09:32:39 +01:00
Oliver Smith fd6f09e0c2 osmo-ggsn-master: add configure --enable-gtp-linux
Related: OS#3208
Change-Id: I8cf3de4775aa82bfdad1bf2c40aab3b50e910db3
2021-02-26 09:32:39 +01:00
Oliver Smith cf43f686ac jenkins-common: put fix_perms() in clean_up_common
Don't call fix_perms from collect_logs anymore, it looks like it was
only added there to avoid adding it at the end of each jenkins.sh. Now
that we have clean_up_common, put it there.

Related: OS#3208
Change-Id: I6e7e5b35198f420e130b1282eb948b96ccd6c8e0
2021-02-26 09:32:39 +01:00
Oliver Smith 033c6ab1be */jenkins.sh: set -e after setting clean up trap
Abort the script and trigger the clean up script, whenever any of the
commands below to prepare the testsuite are failing. This saves time
with figuring out why suddenly all or most tests are failing, and avoids
running the entire testsuite on jenkins if it's obviously not going to
work.

Related: OS#3208
Change-Id: Ie68da2affda8c96b3a515a857a921a05f1bf8ef7
2021-02-26 09:32:34 +01:00
Oliver Smith 12b961942a jenkins-common.sh: add common clean up trap
Add set_clean_up_trap() in jenkins-common.sh and run it at the beginning
of the jenkins.sh files. Move the common clean up code from the end of
every jenkins.sh file into clean_up_common(), which gets called by the
trap. Add a custom clean_up() function to those jenkins.sh files that
need additional clean up.

Replace explicit container stop commands (for containers attached to the
docker network) with one call to network_clean() in clean_up_common(). It
kills all containers attached to the docker network.

The motivation for this change is the upcoming optional build of initrd
and kernel during ttcn3-ggsn-test/jenkins.sh. After building these, a
short smoke test will be performed to make sure we can boot the kernel
and initrd, before continuing to run the entire testsuite against it. If
building or the smoke test fails, we must do a proper clean up of the
network and fix permissions.

Related: OS#3208
Change-Id: I807c6221d22fec31db23e2516e477a3eb43347fb
2021-02-26 09:32:06 +01:00
Pau Espin c63a131a60 Remove test backward compatibility with older osmo-* versions
These changes are no longer needed since master versions of whole CNI
have been recently released.

Related: OS#5042
Change-Id: I6f491b58d4576b22c8ff3ab56a7b9e17163a8739
2021-02-25 16:41:47 +01:00
Harald Welte d725a9a2df add osmo-cbc-latest container
Change-Id: Ia0071a52a7cdb4f8e53570e05d0d4c2848d0577a
2021-02-23 20:50:05 +01:00
Harald Welte bb39336e33 cbc: New ttcn3-cbc-test container
Change-Id: I4386aeee8ba6d7aec7f879bad28208b7f27a45d9
2021-02-22 10:40:33 +01:00
Harald Welte 5efdd15292 cbc: Add osmo-cbc-master container
Change-Id: I7f144899725cd0f049f860beced45e37b3242a93
2021-02-22 10:40:33 +01:00
Alexander Couzens 6e9f0dcc88 gbproxy: update sns configuration to use ip-sns-bind
With libosmocore commit the SNS configuration requires a ip-sns-bind vty configuration
to define the binds which should be use.

Related: SYS#5354
Depends-on: I9ab8092bf286e7d90e92f5702a5404425e959c84 (libosmocore)
Change-Id: I11541540468584fc17a18fa294da5c9813c28d64
2021-02-16 21:24:29 +01:00
Oliver Smith a2104231f2 ttcn3-gbproxy-test: remove broken merge logs cmd
Remove commands that result in a docker error message, like in
Ie6f8270230e1f0372b63d610b7868ea8719b18ad.

Change-Id: I9dec8d27c0f09cd8ee75255c8dfa314c70387ee9
2021-02-16 17:36:35 +00:00
Harald Welte cb688f8e49 gerrit: update to 3.2.7 for security reasons
Change-Id: Ib51cb5466ec2eddeb76e1853a4c1206f9a8e5fd0
2021-02-15 12:29:22 +01:00
Oliver Smith 19594be9e8 Revert "RFC: allow to set configuration and/or specific testcases"
Fix error message reported by Pau:
  docker: invalid reference format: repository name must be lowercase.

This happens in the "docker run" command after the "Starting container
to merge logs" message, because the arguments to --entrypoint are not in
quotation marks. The command fails.

Logs were merged anyway, because
If03422bb8fb126fe6f204a4f8bc6edf0cbb7f194 had already reverted half the
patch and added the log merging back to the Dockerfile.

This reverts commit 73663b01f2.

Change-Id: Ie6f8270230e1f0372b63d610b7868ea8719b18ad
2021-02-12 19:07:26 +01:00
Oliver Smith 6f6724178b ttcn3: use REGISTRY arg with debian-stretch-titan
Allow jenkins to fetch the image from our private docker registry.
Outside of jenkins, the image is built locally just like before.

Related: OS#5017
Change-Id: I46cc176ea09d8badc359b627d7ce2f459211258c
2021-02-11 16:27:23 +01:00
Oliver Smith ce824cc3e1 ttcn3-{fr,gbproxy}-test: use ttcn3-docker-run too
Move the shared pipework script to the base image, and call it in
ttcn3-docker-run.sh if WAIT_FOR_NETDEV is set. Use ttcn3-docker-run.sh
as entrypoint in both Dockerfiles and remove the custom entrypoint
scripts (which are the same as ttcn3-docker-run.sh now).

Related: OS#5017
Change-Id: I2889abe94ed522a205890e5cb39e09cbebfbd820
2021-02-11 15:45:40 +01:00
Oliver Smith 2b38b8e779 ttcn3: move shared run code to script
While I'm at it with tidying up the Dockerfiles, create a
ttcn3-docker-run.sh with shared run code.

Related: OS#5017
Change-Id: Id90769707158f0488eca2313c57b99ea7a4a27c8
2021-02-11 15:45:40 +01:00
Oliver Smith 914a486ef1 debian-stretch-titan: prepare: start/stop links
Create symlinks to ttcn3-*-start.sh and ttcn3-*-stop.sh in the
prepare.sh script instead of in each of the ttcn3 Dockerfiles.

Related: OS#5017
Change-Id: I634cf71cff7936efe6e0461d21dc1119de922ada
2021-02-11 15:45:40 +01:00
Oliver Smith 9206cf6b28 debian-stretch-titan: prepare: run 'make deps'
Run 'make deps' in ttcn3-docker-prepare.sh, if needed.

Related: OS#5017
Change-Id: I9085203b27d310b63a035fd71f7c2a78d02710d4
2021-02-11 15:45:40 +01:00
Oliver Smith 892d910444 ttcn3: move prepare code to shared script
Move the git fetch/checkout code and make call to build the testsuite,
to debian-stretch-titan/ttcn3-docker-prepare.sh. In the next patch, I
will extend the script to update deps right before building too (e.g.
because OSMO_TTCN3_BRANCH changed).

Related: OS#5017
Change-Id: I4b5bedf058dc527e821f9b7204c632820e671af9
2021-02-11 15:45:40 +01:00
Oliver Smith 73d23109b1 debian-stretch-titan: update deps on change
After the initial clone of osmo-ttcn3-hacks and deps, let docker
download deps/Makefile to invalidate the cache if the file changed.

Run "git pull" and "make deps" afterwards, but only if the Makefile is
different. The "if" saves time in the initial build of the image, it
avoids the "git fetch" on every dependency repository during "make deps".

Related: OS#5017
Change-Id: I56673312cfb23375d67900016aaac1931f677275
2021-02-11 15:45:40 +01:00
Oliver Smith 61fed0ad39 ttcn3: move initial clone to debian-stretch-titan
Clone the osmo-ttcn3-hacks and all dependency repositories less often by
moving related commands to the shared debian-stretch-titan image.

Remove the 'git checkout -f -B master origin/master' line, because the
master branch is checked out by default.

While at it, move the shared "git config" commands too, and move them
before cloning the repositories, so they don't run again whenever the
deps change (logic to invalidate the cache if deps change will be added
in the next patch).

Related: OS#5017
Change-Id: I2bb142dce061eba4b6a828c4e435510e309989fd
2021-02-11 15:45:40 +01:00
Oliver Smith db3f98113c ttcn3: remove /root/projects/git symlink
Remove leftover from old TTCN-3 build scripts, before refactoring ttcn3
Dockerfiles. This line has already been removed in 357ec806 from 2017 for
ggsn-test.

In osmo-ttcn3-hacks.git, this is only referenced in the obsolete
bin/install.script (looks like we could remove that, together with the
rest of the bin dir?).

Related: OS#5017
Change-Id: Id23e7fae58ba246916a38aa0a10035d4f67f7588
2021-02-11 15:45:40 +01:00
Harald Welte 1927b88800 osmo-sip-master: Don't install packages not required
We don't need to install dozens of packages that osmo-sip-connector
never had as an upstream dependency.

Change-Id: I604c9843866c100f12574f1ca37ff853a1b64f42
2021-02-11 12:52:21 +01:00
Harald Welte 57b6011e28 redmine: Make changeset keywords work with OS# annotation
In Osmocom we annotate osmocom issues as OS#1234 and not just as #1234,
in order to distinguish them from redmine or coverity issues.

Change-Id: I04a97434433a022f47a759a8219458e8772ae71e
Related: OS#5005, OS#3291
2021-02-10 16:57:26 +01:00
Daniel Willmann 60f4d39500 gbproxy: Fix ip-sns VTY config
Change-Id: I8faa5d29bfffe50c76c1b907717144ff893126a3
Related: SYS#5354
Depends: Ida979f3b9daa5f7280a629441e4006a7635653b0 (libosmocore.git)
2021-02-10 13:57:31 +01:00
Daniel Willmann ba06957cf4 Add osmo-gbproxy-* container for osmo-gbproxy
Uses the new repo split off of osmo-sgsn

Related: OS#4992
Change-Id: I1e7ede4d1da54e3bcb286213b34401bd9dcf7cc1
2021-02-04 16:48:23 +01:00
Harald Welte bd94a1443e ttcn3-fr-tests: Configure MTU to 1700
On our deb10fr VMs we have the modified kernel HDLC module supporting
the frame relay MTU up to 1700 bytes, so let's extend our tests to cover
that.  Mainline kernels only support 1500 bytes.

Depends: osmo-ttcn3-hacks.git I8e38ecf6b270c81bd73ee43b1fa0b259a999c14b
Change-Id: Id25532edbf5e1a918ad038723da464723d6f03f0
2021-02-04 15:01:57 +01:00
Harald Welte ed6163df50 ttcn3-fr-tests: Disable junit-xml generation for FRNET_Tests
FRNET_Tests is not a test suite, but just a stub against which we
execute FR_Tests.  Hence, we don't want it to generate a junit-xml,
as that doesn't contain any errors and only upsets our jenkins
test results analyzer, assuming the tests have failed.

Change-Id: Idec2bf8da30ccd211b21901d56979143ce3a172f
Requires: osmo-ttcn3-hacks.git Id296e62fb86731492d42370173a48f217b2fbdc0
2021-02-04 10:38:15 +01:00
Harald Welte 18564ba9c2 New ttcn3-fr-test container
Change-Id: I32433871e25de46ef58b70af46400cbb4dc0dbc5
2021-02-03 22:20:20 +01:00
Vadim Yanitskiy 0b9f7c338b osmo-pcu-master/Dockerfile: add missing libosmoctrl dependency
Since Id35f40d05f3e081f32fddbf1fa34cb338db452ca, osmo-pcu also
depends on libosmoctrl.  Debian based images work just fine,
because they depend on libosmocore-dev, while for CentOS we
need to specify all sub-libraries explicitly.

Change-Id: I6cccb77432b2b8fcb9f90c5205f7f1fb9dc41910
2021-02-01 14:19:44 +01:00
Pau Espin 0d6b421aa7 ttcn3-pcu: Update configs to support NACC tests
Change-Id: If027812940579a7f344a36543e85518cc096905b
2021-02-01 11:21:06 +00:00
Alexander Couzens c83c5d627f sgsn: change configuration to the new NS2
Change the whole vty configuration for NS to be more flexible
and support more setups. Old configurations are invalid.

For further information see:
https://osmocom.org/projects/libosmocore/wiki/Network_service_(NS)

Depends-on: Ie9306ab4d4738c2c57a69987086e22771b30657e (osmo-sgsn)
Change-Id: I65fb4e52d22617cf8488d8ab77c970f84995560d
2021-01-28 21:55:24 +01:00
Harald Welte 366ace7461 ttcn3-gbproxy-test: s/sgsn/gbproxy/
the jenkins.sh script was copied from the SGSN test suite and still
mentioned sgsn at several places when actually intending to mention
gbproxy.

Change-Id: I8709121d8501da4afc1c8dda70356096d331a9b0
2021-01-25 12:48:02 +00:00