Commit Graph

1001 Commits

Author SHA1 Message Date
Oliver Smith 4bbe87db96 release-tarball: add libosmo-pfcp, libgtpnl
Change-Id: I4096c21352366c16167efce5c2a97f4e04c1addb
2023-02-08 15:44:39 +01:00
Oliver Smith b769d12021 release-tarball: add var to use list from jenkins
Add a parameter that can be used to reproduce the failure locally with
the last list from jenkins, without having SSH set up to download the
list of existing tarballs.

Change-Id: I9af310c24aa65c21cc4bd365d3df7804cba02df7
2023-02-08 14:16:02 +01:00
Oliver Smith c23f21f31c release-tarball: remove PARALLEL_MAKE variable
Probably copy pasted from another script, doesn't make much sense here
since only the release tarballs are built. Nothing gets compiled here,
the variable was not used.

Change-Id: Ie74ee6976759f9634503b21e964e93efcc925679
2023-02-08 14:16:02 +01:00
Oliver Smith 2ef19d24d2 release-tarball: fix osmo-trx 1.5.0
Run configure with the --with-mstrx argument, so LIBTRXCON_DIR is not
pointing at an empty directory.

Fix for:
  make[2]: Entering directory '/build/_temp/repos/osmo-trx/osmocom-bb'
  make[2]: *** No rule to make target 'distdir'.  Stop.

Change-Id: I2f067409f98119d4e35cf86cc2c816fc3c19e3f5
2023-02-08 14:16:02 +01:00
Oliver Smith c801b72336 release-tarball: support git submodules
osmo-trx 1.5.0 is now using a git submodule for osmocom-bb, so add
support for it.

Change-Id: I35e9ef6d976ae8cf2bfb4f3c410e6b2cb3ca7841
2023-02-08 14:15:59 +01:00
Oliver Smith 1790b9a7e6 release-tarball: add missing cd /build
Fix for:
  upload: no tarballs generated, nothing to do.

Change-Id: I047546b26f379c5302d8558b92b77e4fa311a0a0
2023-02-08 14:13:39 +01:00
Pau Espin 666488c8e4 ttcn3-*: Update test setup after Osmocom CNI release 202302
Change-Id: If34b043a55177e6c0d1be3949fc2a2682258cc84
2023-02-08 12:36:27 +01:00
Vadim Yanitskiy 4a2d9d1c0c jenkins-common.sh: open5gs-latest does not require debian-bullseye-build
We're not *building* anything for the open5gs-latest, but simply
downloading and installing DEBs from Sukchan's opensuse.org repo.

Change-Id: Iec7ccab28c37d274a731e08a2f893ec0514c1068
Fixes: 62ac21133a
2023-02-08 17:44:38 +07:00
Oliver Smith 46e4362ae2 FR testsuites: clean up all networks before start
Add a new network_clean_remove_all_ttcn3 function and use it in the fr
related testsuites to ensure no network is running before the test
starts. We just had the situation where the network link between
gtp0-deb10fr (where these testsuites run exclusively, and only one at a
time) and the jenkins host went down. And so the clean up trap
apparently did not run and starting a new test fails as the old network
still exists and has the network devices attached.

Related: OS#5802
Change-Id: I6a9464a11edcba978be08764bec9de19760a5c77
2023-01-30 13:45:30 +01:00
Oliver Smith 7a49258ca7 debian-buster-jenkins-arm: new image
Add a minimal image to use on the raspberry pis, so it doesn't need to
spend 1-2 hours building debian-buster-jenkins on the raspberry pis,
full of things we don't actually use there.

I've verified that osmo-bts, osmo-pcu, osmo-trx build with this image
(the only projects which we build on the rpis).

Related: OS#5863
Change-Id: I4fb4b48b233acaef815c4c27ec6f17f12cfe836b
2023-01-23 13:08:54 +00:00
Oliver Smith f71e0f2856 ttcn3-bts-test: replace subnets during 'make cfg'
Make ttcn3-bts-test work as expected again by replacing the subnets
hardcoded in the confmerge files and Makefile.

Related: OS#5802
Change-Id: I733619f80b4e2456da070649634d3c942bb6d95f
2023-01-18 12:35:42 +01:00
Oliver Smith 2437f4fe8e debian-buster-jenkins: install ccache
Make ccache available inside debian-buster-jenkins. An upcoming osmo-ci
patch will make use of it in gerrit-verification and master jobs. With
this docker-playground patch alone it's not used yet since
/usr/lib/ccache is not in the default PATH.

Related: OS#5848
Change-Id: I164c7835f99436e8d768a864f36118eaf799d109
2023-01-18 09:52:24 +01:00
Oliver Smith 76533bea88 kernel_test_wait_for_vm: increase timeout to 15s
As we run more testsuites in parallel, the load was higher and so this
timeout was reached.

Related: OS#5802
Change-Id: I823010106a99a4906e32198ed42b36f9fe662c4e
2023-01-17 13:42:04 +01:00
Oliver Smith 381e2f3a0e network_create: set NET_NAME after next SUBNET
Put the current SUBNET into NET_NAME, not the previous one. Without this
patch it works too, but this makes more sense.

Related: OS#5802
Change-Id: I6c90505fd6c0ba62244ff67709b0bf31bf44b449
2023-01-17 13:41:29 +01:00
Oliver Smith 998bc92f07 jenkins-common: replace subnet in scm files too
Replace the subnet in *.scm configs of nplab-sua-test and
nplab-m3ua-test too.

Related: OS#5802
Change-Id: Ia734df3721a4ae3408a17d33c835282b79174132
2023-01-17 12:47:52 +01:00
Oliver Smith d2827d91ce network_create: put SUBNET in NET_NAME
Fix the bug that running a test twice on the same host results in the
second test stopping the containers and network of the first test:

* Have the subnet name in the network name, so there's no collision of
  network names when cleaning up the network.

* Don't clean up the network when creating a new network. Rely on the
  clean up trap to clean up the network instead.

* Don't set NET_NAME when sourcing jenkins-common.sh anymore, as
  network_create will set it. There's also network_bridge_create, which
  only gets used by osmo-ran (which we don't run in jenkins). Add a
  check to this function to ensure NET_NAME is set before calling it,
  which osmo-ran is already doing.

Related: OS#5802
Change-Id: Ifcd384272c56d585e220e2588f2186dc110902ed
2023-01-17 09:39:52 +01:00
Oliver Smith f997b4e771 network_create: find free subnet automatically
Try multiple subnet numbers until successfully creating a network. This
way we can run the same ttcn3 testsuite multiple times in parallel
without conflicts (e.g. once against latest, once against nightly). Also
we don't need to make sure each new testsuite has a unique subnet
number anymore.

I've considered also adjusting network_bridge_create, but that gets used
exclusively by osmo-ran/jenkins.sh, a script which we don't actually run
in jenkins. It seems that in this script it makes more sense to not get
a random subnet number.

Related: OS#5802
Change-Id: I57152b08ef0f38e17e7019a8df032189b03f56cf
2023-01-16 15:19:29 +00:00
Oliver Smith 4cb0fac1b0 ttcn3-*/jenkins*.sh: use $SUBNET everywhere
Use the SUBNET variable instead of hardcoding it in some places. Split
commands across multiple lines while at it to improve readability.

Related: OS#5802
Change-Id: I08f83089ef97f5f92d4bbfa5743301e7375e9f0f
2023-01-16 11:41:11 +00:00
Vadim Yanitskiy ea90e44ab0 */Dockerfile: clone from gerrit.osmocom.org using https://
* Do not clone using deprecated git://, use https:// instead.
* Clone from the main repo gerrit.osmocom.org (if possible).
* Use the gitiles plugin to fetch the last commit from Gerrit.

Change-Id: I204fa6e3e3ebbdb8732393b897fca35659d61ae8
2023-01-06 23:10:13 +07:00
Vadim Yanitskiy aa27022cf8 ttcn3-upf-test: fetch /tmp/commit from gerrit, not from cgit
Change-Id: Id327720258a9ece1efb79db0d7c4fcbe19864c92
2023-01-06 23:09:04 +07:00
Oliver Smith 9a2556824a scripts/regen_doc: use SUBNET variable
Add a $SUBNET parameter and use it throughout the script, similar to
other ttcn3-*/jenkins.sh scripts. This is in preparation for figuring
out a free network in network_create() automatically in a future patch.

Related: OS#5802
Change-Id: I3bda4bd7280469ad5ee02104295134a2e0b768e3
2022-12-22 13:59:10 +01:00
Oliver Smith 68cddfad72 ttcn3-ggsn-test: cp initrd-ggsn.sh to VOL_BASE_DIR
Prepare for a future patch where a free subnet will be figured out
automatically instead of having a hardcoded one, by moving
initrd-ggsn.sh to $VOL_BASE_DIR as the subnet will be replaced in files
in this directory.

Related: OS#5802
Change-Id: Ie4f35b77a2bff20f2a0479890a429754768cafd2
2022-12-22 13:26:41 +01:00
Oliver Smith 0974da58cd jenkins-common: tweak echo messages
Use 'set +x' before echo, so messages are not printed twice and the
output becomes more readable.

Example:
  + echo Creating network ttcn3-bts-test
  Creating network ttcn3-bts-test

Related: OS#5802
Change-Id: I213cb80bd1d9401281ae1fb7394b2319453550dd
2022-12-22 13:26:41 +01:00
Oliver Smith a47ab326c5 *-repo-install-test: remove
Remove the docker containers for repo-install-test, as it is now running
in qemu + kvm.

Related: OS#5365
Depends: osmo-ci Ie7f1bccb05779cb3614016c0b965b810bbb1471b
Change-Id: Ic40fddcc6b30318b3a6e63fe99b010939b95be25
2022-12-08 14:14:08 +01:00
Harald Welte c4d1cc59b0 ttcn3-docker-run.sh: Use interface "up" flag, not operstate
Don't use the "operstate" sysfs attribute to determine if an interface
is "up", use the actual UP-flag (0x01) in the "flags" sysfs attribute.

The "operstate" attribute may at least occasionally be "unknown" and
remain in that state (causing jenkins jobs to wait indefinitely), while
the flags (which we don't look at before this patch) indicates it is "up".

This is a fixup to the below commit:

commit d2014603a7
Author: Harald Welte <laforge@osmocom.org>
Date:   Wed Feb 3 22:05:43 2021 +0100

    debian-stretch-titan: Wait for interface to be _up_ not just its existance

Change-Id: Ib5c3bbe470ce874217437c2518df5ae07f0d8301
Closes: OS#5803
2022-12-02 11:15:34 +00:00
Oliver Smith 36c6ff8f11 osmo-gsm-tester: fix getting latest commit
Get the latest commit from gerrit.osmocom.org, because after we moved
repositories from git.osmocom.org it currently answers with a 404.

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I0e2215d067dec1e2327f765b5a97ba7512812647
2022-12-01 14:50:39 +01:00
Oliver Smith 193feff80f release-tarball-build-dist: fix permission errors
Run all commands inside the docker containers as a user with the same
UID as on the host system, not as root. This is a good idea in general
and also fixes permission problems when jenkins tries to wipe the
workspace.

Related: OS#5793
Change-Id: I0d949e5ab4b1a709d18a316def0a98c3b244457e
2022-11-30 17:28:11 +01:00
Oliver Smith 858178e4e3 release-tarball-build-dist: fix missing dir crash
Don't fail if no tarball was generated and therefore the
_release_tarballs directory does not exist.

Related: OS#5793
Change-Id: Ib7c551e5376776eb267140bc4b897d5bb1634cbc
2022-11-30 17:28:08 +01:00
Oliver Smith 757396a61b release-tarball-build-dist: check exists on server
Previously this job relied on always running on the same jenkins node,
and having the previously built release tarballs in a workspace
directory that does not get removed. It would only skip building a
release tarball if the file already exists locally.

As part of OS#5793 this job needs to be moved away from that jenkins
node. When trying to run the job on a different node, it will build all
release tarballs again which takes unnecessarsily long and actually
fails on libosmocore-0.9.4 due to a missing python2.

This probably happens because the job was refactored at some point and I
didn't realize that the python2 dep is now missing as the previous
tarballs existed.

In general it doesn't make much sense to build previous release tarballs
again and to always keep the legacy environments around for that. Change
the code to check if a tarball exists on the server, and skip building
the tarball in that case.

Related: OS#5793
Related: https://jenkins.osmocom.org/jenkins/view/All%20no%20Gerrit/job/Osmocom-release-tarballs/1281/console
Change-Id: I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce
2022-11-30 16:52:13 +01:00
Oliver Smith c4f160e2c2 release-tarball-build-dist: move rsync into docker
Prepare to use ssh inside the docker container to check if tarballs
already exist, see the next patch.

Remove the '' around UserKnownHostsFile as using SSH_COMMAND in the next
patch without rsync fails otherwise with:
  command-line: line 0: Bad configuration option: 'userknownhostsfile

Change-Id: I6dfe5d587d05063c6cf2c9ec683e7b7e499b79c1
2022-11-30 16:51:27 +01:00
Oliver Smith 59f3aa83d5 release-tarball-build-dist: order install pkgs
Change-Id: I059f400db20e858b379704363e32d39a7897984a
2022-11-30 16:04:20 +01:00
Harald Welte 9f5e67b95d upgrade gerrit to 3.4.8
Change-Id: I0be6e5f0c616b6c671635da62ba02afff2985592
2022-11-27 10:47:23 +01:00
Pau Espin 0ecbb795c5 ttcn3-{pgw,ggsn}-test: Update smf.yml with new metrics section location
The "metrics" section was moved inside each program node to avoid
clashes when several network programs are run in the same process and
use the same config file.

Depends: open5gs.git 2d4fbf92eb1be9ffe84df6dc103ee7401d1d3b0f
Change-Id: Iae16bfccfbaaed37701e671ba619f8082f55506d
2022-11-21 11:28:52 +01:00
Vadim Yanitskiy 9b6cc6ac70 open5gs-latest: migrate from debian:buster to debian:bullseye
Change-Id: Ia09aa712a916686f7220da9a5d2f4a02a8b7b8db
2022-11-15 22:41:23 +07:00
Vadim Yanitskiy ba5f99aac0 ttcn3-ggsn-test-ogs: update freeDiameter-smf.conf
Recent open5gs changed location of the TLS key and certificate files:

  commit 0859dd44530a944076c207c750188572effc2709
  Author: Sukchan Lee <acetcom@gmail.com>
  Date:   Sat Nov 12 09:37:43 2022 +0900

      Follow-up on #1865

Updating paths makes the open5gs-master work, however the -latest
still installs these files to '/etc/freeDiameter/'.  Let's remove
both 'TLS_Cred' and 'TLS_CA' params, we are not using TLS anyway.

Change-Id: Iacd7287f217db7a23ceece1d4508ef674b696d35
2022-11-15 22:41:23 +07:00
Vadim Yanitskiy ff221f9af5 ttcn3-pgw-test: update freeDiameter-smf.conf
Recent open5gs changed location of the TLS key and certificate files:

  commit 0859dd44530a944076c207c750188572effc2709
  Author: Sukchan Lee <acetcom@gmail.com>
  Date:   Sat Nov 12 09:37:43 2022 +0900

      Follow-up on #1865

Updating paths makes the open5gs-master work, however the -latest
still installs these files to '/etc/freeDiameter/'.  Let's remove
both 'TLS_Cred' and 'TLS_CA' params, we are not using TLS anyway.

Change-Id: Ic90f713fb127f8d5407c18553b29b3c2be141420
2022-11-15 22:41:08 +07:00
Daniel Willmann df4522ade4 hnbgw: Only build with_pfcp on master
The functionality is not in -latest yet so running osmo-hnbgw with this
configuration fails which in turn fails the test in jenkins instead of
just marking it unstable.

Change-Id: I4309c323c1d61e8f22dae499c407d57999f6f13a
2022-11-07 10:17:36 +00:00
Oliver Smith a0cd1869b2 debian-buster-jenkins: add more pysim related deps
Install all depends to properly generate the manuals. Otherwise it
prints errors like the following and the resulting manual has empty
sections:
  /build/docs/shell.rst:329: ERROR: Failed to import "LinFixedEF" from "pySim.filesystem".
  No module named 'smartcard'

Get the requirements.txt directly from the git repository, so we don't
need to update the list of dependencies here. Remove the dependencies
from the list that I added recently for pysim, and which are are also
in requirements.txt.

Add swig, as otherwise pyscard fails to install.

Related: OS#5497
Change-Id: I13a0b0dc8be81bf4034b08b1ef1100f1420fa3a1
2022-11-04 10:19:05 +00:00
Daniel Willmann 98c48a5cfa ttcn3-bsc-test: Fix ttcn3-bsc_sccplite-latest
Re-add the msc 0->mgw config line since the "latest" version of osmo-bsc still uses
it. This was probably caused by fixing fallout from osmo-bsc.git commit
da4af65a51ee8b8d7b380330c0301de293347563 in master.

The config line is ignored in master so should be safe to keep until a
new osmo-bsc version is released.

Change-Id: I85231bf66a78232567eb91b92b8542ca7f728beb
2022-11-02 19:24:17 +01:00
Oliver Smith 729efb0202 debian-buster-jenkins: add pysim doc/lint depends
The lint and docs parts of pysim's contrib/jenkins.sh will run on a
different jenkins node than the tests (which has to run on simtester).
Add the depends for lint and docs to debian-buster-jenkins, so we can
run them in docker.

The python libraries construct, pytlv, pyyaml are needed so pylint
doesn't print false errors like "Undefined variable 'TLV'".

Related: OS#5497
Change-Id: Id5c75725d2fab46b29773fa4f637fa2d73fa7291
2022-10-24 14:13:00 +00:00
Pau Espin 9a330b6fc3 ttcn3-bsc: Update osmo-bsc cfg files to use mgw pooling VTY commands
OsmoBSC has supported this VTY interface since more than a year ago.
Let's update the config files to use the new "mgw" node.

The recently submitted VTY commands without the redundant "mgw" prefix
are still not used here in order to have the config file work with
latest release which still doesn't support those.

Change-Id: Iabf117f9e6de02cac16e44d9a0ca32a30d71847c
Related: SYS#5987
2022-10-19 13:50:13 +02:00
Oliver Smith 4077d8aad5 debian-buster-jenkins: skip llvm for arch !x86_64
The pre-built LLVM toolchain we are downloading is built for x86_64, so
this fails on other architectures.

Related: OS#5260
Change-Id: Ie5a851d8c4e808ced09b20d7478679bf1ecc1cbb
2022-10-17 11:06:34 +02:00
Oliver Smith a4d67a24f1 debian-buster-jenkins: add LLVMEmbeddedToolchainForArm
This is needed for simtrace2. As the bootloader goes beyond partition
size in modern gcc, we'll use clang instead.

Related: OS#5260
Related: simtrace2 I2aa2e20e75e334560dbe1f6db9fd1491873ff91f
Change-Id: I4f3fd7f7a40e5fa00bbfc713c1365f573febfe41
2022-10-13 14:47:41 +02:00
Oliver Smith 0fec30c698 debian-buster-jenkins: order pkgs alphabetically
Change-Id: I32dffc65239f1ad650b9e8d6ed6e4fe48878b916
2022-10-13 14:47:41 +02:00
Oliver Smith bc443c4e97 Revert "debian-buster-build: add LLVMEmbeddedToolchainForArm"
Wrong docker container, debian-buster-jenkins is used for the build, not
debian-buster-build. The next patches will adjust the right one.
This reverts commit 1ce91ecfb0.

Change-Id: Idff1d5a263181070133743c781615b1102311efb
2022-10-13 14:47:35 +02:00
Oliver Smith 1ce91ecfb0 debian-buster-build: add LLVMEmbeddedToolchainForArm
This is needed for simtrace2. As the bootloader goes beyond partition
size in modern gcc, we'll use clang instead.

Related: OS#5260
Related: simtrace2 I2aa2e20e75e334560dbe1f6db9fd1491873ff91f
Change-Id: Ib82a53fa7edc62d21e772efbb9b2c049d1b50c4d
2022-10-13 13:11:04 +02:00
Oliver Smith f721f00ec1 debian-buster-build: order pkgs alphabetically
Change-Id: I16da41b9bcb3d1137218b0ed5c2f6adf2a723fce
2022-10-13 11:34:22 +02:00
Pau Espin 5761378906 ttcn3-mgw: Increase number of enpoints in osmo-mgw.cfg
This is needed by test TC_crcx_osmux_257

Related: osmo-ttcn3-hacks.git Change-Id I74e6c760e076fe2547b6297a54524255fb8b
Related: SYS#5987
Change-Id: I14e3abdf63c585e4518b0c0a61dbc57e6710a998
2022-10-07 13:25:24 +02:00
Pau Espin 8bf8057668 ttcn3-mgw: jenkins.sh Fix centos tests
The sed lines dropping the new configs must be applied to any non-master
run.

Fixes: e077fec684
Change-Id: I369bb61f3979bd9118fad17c3d202253411ba14b
2022-10-07 13:25:24 +02:00
Pau Espin e077fec684 ttcn3-mgw: Set osmux bind-ip(-v6) VTY configs
Until now we forgot to properly configure the osmux socket local ip, but
that was fine because 0.0.0.0 was taken as a default.
With addition of IPv6, this changed, and the socket is not bound unless
an IP address is set (to allow conditional use of IPv4, IPv6 or both).

Depends: osmo-mgw.git Change-Id I446cd7da217e9f4a74995d7784ae55dcc60a29b7
Change-Id: Ib7e9428a10238fae395a1707d6034dc5c97b9024
2022-10-05 08:45:45 +00:00