Commit Graph

1056 Commits

Author SHA1 Message Date
Oliver Smith 9ecd2280d8 ansible/jenkins/osmocom_build_deps: drop py2 pkgs
Drop python2 packages. It's deprecated, packages don't exist in
debian 11 and our jenkins nodes don't need it anymore.

Related: OS#5453
Change-Id: Id2da7adab83f240e7e0b45c018f91c9e639fc80d
2022-05-18 15:15:03 +02:00
Oliver Smith f511264bdd ansible/jenkins/osmocom_build_deps: drop dh-systemd
This package is no longer available in debian 11. Even in debian 9 the
description says:
  "This package is for transitional purposes and can be removed safely."

Related: OS#5453
Change-Id: I6f72de4d7fb0aab4b153dfc0affbc6f11d20ee4c
2022-05-18 15:05:31 +02:00
Oliver Smith e2ef4fed0e ansible/hosts: add gtp0-deb11build-i586
Related: OS#5453
Change-Id: I107dfb543090989c8b7c746be381aa19f0afa366
2022-05-18 15:00:47 +02:00
Harald Welte e50ad03b46 docker: Fix vfs storage cleanup task
The 'grep' logic of Change-Id I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
seems to be inverted.  if "overlay2" is found (grep successful) the
exit code of grep is '0'.

Change-Id: I334b0c71ef626ffd2c84097e902a5164faa3fa8b
Related: OS#5827
2022-05-17 22:15:16 +02:00
Oliver Smith c825551ca8 jobs: TTCN3-centos: set OSMOCOM_REPO_TESTSUITE_MIRROR
In previous patch Id114d55edc2ba92f7851243ab7df51db332079c4, only the
OSMOCOM_REPO_MIRROR should have been overridden. However
OSMOCOM_REPO_TESTSUITE_MIRROR defaults to OSMOCOM_REPO_MIRROR, so
explicitly set it in this patch to the opensuse mirror.

This fixes problems while building debian-bullseye-titan caused by the
different directory structure of both mirrors, and OSMOCOM_REPO_PATH
intentionally not taken into account while building the testsuite
docker image.

This is part 2/2 of fixing TTCN3-centos-bts-test, which has been failing
since OSMOCOM_REPO_MIRROR was changed in osmo-ci.git. Part 1 is in
docker-playground I78e0445a1b82e5cd72ecaa86e8fe25516b19f485.

Change-Id: I8201f34a0f9747eea78c358a626cc3981d7407c4
2022-05-13 11:15:00 +02:00
Harald Welte c614362382 New osmocom-obs-worker role for our OBS workers
Related: OS#5556
Change-Id: I7a6d82e89e3a840739f71b90cdebe89bf8383c4a
2022-05-10 19:43:13 +02:00
Oliver Smith 526f1e12d3 jobs/ttcn3-testsuites: centos: use osmocom mirror
Lately we've been seeing a lot of build failures from the TTCN3-centos
jobs running against the nightly repository. It seems that the mirror we
get redirected to isn't syncing properly. Use the Osmocom mirror
instead.

Fix for:
  Nightly packages of the Osmocom project (CentOS 190 kB/s | 1.7 kB     00:00
  Nightly packages of the Osmocom project (CentOS 1.8 kB/s | 634  B     00:00
  [91mErrors during downloading metadata for repository 'network_osmocom_nightly':
    - Status code: 404 for 7062058d3e234003a6c3595331cf3fc100-filelists.xml.gz (IP: 134.76.12.6)
    - Status code: 404 for ad91db8cb719947d9abc971661c588e8c6-primary.xml.gz (IP: 134.76.12.6)
  [0m[91mError: Failed to download metadata for repo 'network_osmocom_nightly': Yum repo downloading error: Downloading error(s): repodata/ad91db8cb7273beaefca0e473d9c4dd12d12786819947d9abc971661c588e8c6-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/7062058d3e5395b4b02570496e41e19e8fd306f5234003a6c3595331cf3fc100-filelists.xml.gz - Cannot download, all mirrors were already tried without success

Change-Id: Id114d55edc2ba92f7851243ab7df51db332079c4
2022-05-05 15:05:50 +02:00
Oliver Smith 382aebd040 treewide: rename rpi4-raspbian10 to rpi4-raspbian11
The raspberries have been upgraded to the debian 11 based raspberry pi
OS. Adjust the name to avoid confusion.

Related: OS#5453
Change-Id: I0b2e08b11d83243e81b2bb4412ffcc9bc59821ff
2022-05-02 13:29:50 +00:00
Harald Welte 032c0de4b8 dahdi-linux: Add gerrit build verification against master
Now that master-dahdi-linux-master is passing, we should verify any
future patches also against master.

Change-Id: I86cfd7e868dcaeeb6c6dbc6f08046a70b4582cc7
2022-04-24 19:48:47 +02:00
Harald Welte 5b933d8aea dahdi: use buster, not stretch
stretch has too old gcc which doesn't support -Wno-format-truncation
which is used in dahdi-linux since 2018.

Related: OS#5407
Change-Id: I4fca722e67b2c649dea4f9799d277cd568bcc493
2022-04-24 18:20:58 +02:00
Oliver Smith 7abb4ca810 OBS, ttcn3-testsuites: add 2022q1
Related: SYS#5887
Change-Id: Idadb541b49e57b4403a6a6e4c966bcba9c6651db
2022-04-13 19:38:18 +02:00
Pau Espin 99578e8653 jobs: Add ttcn3-pgw-test jenkins job
Change-Id: I363a7a8eb524abfeb45c81603c29cea96f86a787
2022-04-07 17:55:39 +02:00
Oliver Smith 0c250b7c52 OBS nightly-packages.sh: add osmo-bsc-nat
Related: SYS#5560
Change-Id: I47cd67d5dcf3eb49b486e99840449e1c6a41b4ab
2022-03-29 16:45:24 +02:00
Oliver Smith 1abb6f35bf lint: checkpatch: don't require space in T=-1234
Inside struct osmo_tdef we write non-spec timers as T=-1234. Do not
complain about having no space before the minus character.

Example:
    { .T=-25, .default_val=5, .desc="Timeout for ..." },

Note that the Linux kernel coding style also requires spaces around the
equals signs. We follow that everywhere except for struct osmo_tdef,
and the linter has already been adjusted for that in
I1f0b9ed5bd49ef9b5ab0e347b9260e71df34ff9c.

Change-Id: I0885e84ad99c7d333a5930c411fd1273badb0fcb
2022-03-03 10:41:48 +01:00
Oliver Smith fe6512a5ba lint: checkpatch_osmo.sh: exclude .patch files
Change-Id: I734452bc78d7e026292883f81bc07c3baf58e309
2022-03-02 11:43:39 +01:00
Oliver Smith f5ab1346db jobs/ttcn3-testsuites: adjust nplab-m3ua timers
Fix sporadic errors related to the clean up script running while images
being built:

  failed to get digest sha256:8ea2ba9a1...

Closes: OS#5447
Change-Id: If722fddce8de65a1ae012a46d6a202f430e0c69d
2022-02-28 10:42:24 +01:00
Daniel Willmann 5ecc602efe Reenable ttcn3-hnbgw-test
Now that mgw support is in osmo-hnbgw master and docker/ttcn3 testsuite
is working we can enable the test again.

Change-Id: I4c7d1ee876e90d52827b2d8cc57342298dd44061
Related: OS#5152
2022-02-24 11:43:39 +01:00
Oliver Smith 8922843ec2 jobs/ttcn3-testsuites.yml: add ttcn3-fr-test
Add a configuration for the job to the yml file. It looks like this job
was created manually, or the configuration was just not pushed to
gerrit.

This makes e-mail notifications consistent and fixes missing colors in
build output.

Change-Id: I14995dea0a0d223b78e20b49953d5c814c1ad4a1
2022-02-22 11:55:13 +00:00
Oliver Smith f271bea63d jobs/gerrit-lint: don't verify dahdi-linux
Closes: OS#5465
Change-Id: I41ede04849da033a497daf39b34828bed00ab724
2022-02-22 11:33:25 +01:00
Harald Welte 8349494c9e jenkins-jobs: Add master + gerrit for dahdi-tools
Change-Id: I9c007e070e168bfe6509f50dbc3b4e0846c31356
2022-02-21 21:23:58 +01:00
Pau Espin ef0e042ae8 jobs/ttcn3-testsuites.yml: Add ttcn3-ggsn-test-ogs
Change-Id: I61950f4fc613faae3a731ace394b76c8754bd3c4
2022-02-21 13:08:48 +00:00
Oliver Smith 432aff8deb scripts/osmo-ci-docker-rebuild: x86: skip deb11
Don't attempt to build debian-bullseye (11) in docker on debian 10 x86,
as it results in:

  GPG error: http://security.debian.org/debian-security bullseye-security InRelease: At least one invalid signature was encountered.
  E: The repository 'http://security.debian.org/debian-security bullseye-security InRelease' is not signed.

This is a temporary measure until we've upgraded the only jenkins node
that runs debian 10 on x86, gtp0-deb10build32.

Related: OS#5453
Change-Id: Iba7679c027ca89a29393c220d01b195cfce4fec3
2022-02-21 10:30:04 +01:00
Oliver Smith f225c1da76 ansible/ttcn3: don't install titan/libfftranscode
With the last patch, the jenkins infrastructure will always run
eclipse-titan in docker. Therefore do not install eclipse-titan and
libfftranscode on jenkins nodes anymore.

This avoids the problem that we couldn't upgrade libfftranscode with the
existing ansible rules.

libosmocore-utils is still being installed, as osmo-config-merge must be
installed on the jenkins nodes.

Closes: OS#5452
Change-Id: Ie46fe3dcb286a6258021fad9c35be10442544761
2022-02-21 08:08:00 +00:00
Oliver Smith b712491a56 jobs/gerrit-verifications: build ttcn3 in docker
Build the testsuites in the same docker container during gerrit
verification, as it will be the case when building and running the
testsuites in the various ttcn3-* jenkins jobs.

This avoids the need to install eclipse-titan and libfftranscode on the
jenkins nodes.

Related: OS#5452
Change-Id: I6a667255bb55a945b4c6279d711142aa252c6996
2022-02-21 08:08:00 +00:00
Neels Hofmeyr 950b8ea380 allow to terminate value_string[] with '{0}'
For a long time I was using '{}' to indicate a nulled out struct. But
apparently '{0}' is the favored way to write that. Let's allow using
this notation to terminate a value_string[].

Change-Id: Id2f5ba897ec83f34f8d3c4425353c0baf309bb6d
2022-02-18 14:04:13 +00:00
Oliver Smith c100a6d098 lint: checkpatch: ignore MISSING_SPACE
Don't require breaking strings at spaces. This is not useful for strings
of hex characters, e.g.:

	{ 123, "ffffffffffffffffffffffffffffffffffffffff"
	       "ffffffffffffffffffffffffffffffffffffffff"
	       "ffffffffffffffffffffffffffffffffffffffff"

Change-Id: Ib6be7744418530ae3ddbf373e67d1d7f25a60508
2022-02-18 14:18:46 +01:00
Pau Espin ef9b2c4062 Use docker image debian-bullseye-erlang instead of debian-buster-erlang
Let's use newer debian to avoid problems with older erlang. Furthermore,
the new image builds rebar3 to avoid issues with unmatching erlang
versions.

Change-Id: I7b1956c515daccf6ab1ff87031c6fec649cadb4b
2022-02-15 20:36:10 +01:00
Oliver Smith bc9bd3df75 lint: checkpatch: show hint about typedefs_osmo.txt
Change-Id: Icaee700a9cc7896cdbda6cce93db41b6b50a7fa3
2022-02-15 13:51:09 +01:00
Oliver Smith cf1fdbf030 lint: checkpatch: add typedefs_osmo.txt
Fix false error:
  ERROR:SPACING: need consistent spacing around '*' (ctx:WxV)

in a line with a typedef:
  int handle_rab_ass_req(..., ranap_message *message);

Change-Id: Icbc46d0c788ca75684a48a73f739265f4a09b09c
2022-02-15 13:47:06 +01:00
Oliver Smith 7a40ff9aa1 ansible/docuum: set default to 40 GB
After looking at total available disk space on most of the hosts,
storing up to 40 GB of images seems a more reasonable default. We can
increase it again if it should really not be enough.

Related: OS#5099
Change-Id: Iad679252b027003391f1e2368149cb10ea0c729e
2022-02-14 15:33:37 +01:00
Oliver Smith ffd73ca5e5 ansible/docuum: fix docker_max_image_space replace
When changing the value, replace the previous line instead of adding a
new line at the bottom.

Related: OS#5099
Change-Id: I35a3671a026cc72cf1488c8668e8a0e675de2530
2022-02-14 15:33:32 +01:00
Oliver Smith 98a4f57706 ansible/docuum: fix delgroup error
Check if a group with the same GID as the docker group on the host
exists, before attempting to remove it.

Related: OS#5099
Change-Id: I24af5c1d2b1c488b37f29f1609ee56692eff569e
2022-02-14 15:33:27 +01:00
Oliver Smith 5bf4022a5b jobs/gerrit-lint: run on docker-playground too
Change-Id: I7213155ccd61a006231609b1d7ae1eec187b3e7b
2022-02-11 15:44:54 +01:00
Oliver Smith 3ab7e59de8 ansible: docker: clean vfs storage leftovers
Docker is configured via ansible to use the overlay2 storage driver
since August 2020 (I20728d6017204c3978e23376baa89de6e91fed1e). Clean up
an unused vfs dir if it is present on the system, as it was the case on
build2-deb9build-ansible with a 190 GiB vfs dir.

Related: OS#5827
Change-Id: I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
2022-02-11 15:44:54 +01:00
Oliver Smith 88521fbc14 scripts/docker-cleanup.sh: conditional img clean
Only run the simple image clean code if docuum is not running. It works
well enough in most cases, but has the drawbacks that it never deletes
"latest" images or images not matching "^osmocom-build", and may delete
images that are still being used (OS#5447). With the other tool, all
images are considered for removal, and the ones that have not been used
the longest time are removed first.

Related: OS#5477, OS#5066, SYS#5827
Change-Id: I1cef0833c096de0fa5acf77156bb5dd362e2ef9c
2022-02-11 15:44:44 +01:00
Oliver Smith b5ebf6ea6b scripts/docker-cleanup.sh: use "docker system prune"
Do not only clean up dangling images, but also containers, volumes and
networks.

Related: SYS#5827
Change-Id: If441b251de50063f0229d36fb1bc260a4cb1dd87
2022-02-11 15:44:16 +01:00
Oliver Smith 2b77e64c48 scripts/docker-cleanup.sh: delete containers too
Related: SYS#5827
Change-Id: I73b2f13875286c1aaa5424809edab2202f41768b
2022-02-11 15:44:16 +01:00
Oliver Smith e94b29e837 scripts/docker-cleanup.sh: use set -x
Change-Id: Iba170128e55a9778467c3d3bcf33a91321a8c29f
2022-02-11 15:44:16 +01:00
Oliver Smith 17f82a260d ansible/roles/docker: install docuum
Allow setting a max amount of space that docker images should take up,
and delete the least recently used images when the amount is exceeded.

Related: https://osmocom.org/projects/osmocom-servers/wiki/Docker_cache_clean_up
Change-Id: I640b1e607feca87e7a578946ae4b8332ce854ab1
2022-02-11 15:44:16 +01:00
Oliver Smith f07b5c8a64 ansible/setup-jenkins-slave: accept rel-change
Fix for:
  W:This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
  E:Repository 'http://raspbian.raspberrypi.org/raspbian buster InRelease' changed its 'Suite' value from 'stable' to 'oldstable'

Related: https://github.com/ansible/ansible/issues/48352
Change-Id: I90ee8dd49001d6810f2118e9feff3375e87052ea
2022-02-11 15:44:16 +01:00
Oliver Smith 6695a91c0b ansible/ansible.cfg: make command output readable
Have a readable output, instead of the default that wraps the output
with stripped new lines inside json.

Related: https://github.com/ansible/ansible/issues/27078#issuecomment-364560173
Change-Id: I88d584a1808d82d75906b350e3bffe9bc73a8c67
2022-02-11 15:44:16 +01:00
Oliver Smith 41e218a355 lint: catch missing --rm flag for 'docker run'
Not having the --rm flag causes docker containers to be stored forever,
so make sure we always add it to the "docker run" commands that we run
on jenkins via scripts in various repositories.

Depends: docker-playground I48b01c43fedf379b8a565eaab0369806d7831bd8
Related: https://osmocom.org/projects/osmocom-servers/wiki/Docker_cache_clean_up
Related: SYS#5827, OS#5099
Change-Id: I8ab9c291504475d670bdefc50c4524c5bdd4c880
2022-02-11 15:44:16 +01:00
Oliver Smith 2e88fa2663 scripts/osmocom-packages-docker: docker run --rm
Change-Id: I357ae8071d27088a2c0a74f962ac1589bff43a70
2022-02-11 15:44:16 +01:00
Oliver Smith 0b22171bc3 editorconfig: new file
Auto-configure editors with editorconfig plugin installed to indent yaml
files with two spaces in this repository.

Change-Id: I008ee040e2c15db11a35a061faa4270bacdbd10d
2022-02-11 15:44:16 +01:00
Vadim Yanitskiy f6f49d23a1 tasks/ttcn3-slave.yml: also install libosmocore-utils
This package provides osmo-config-merge, which is going to be used
by docker-playground for generating configuration files.

Change-Id: I87e54440a6b399aa0f0461ebf668991b5cdd0b07
Related: https://gerrit.osmocom.org/c/docker-playground/+/20622
2022-02-11 13:59:06 +00:00
Harald Welte 5cff7924ec jobs/master-builds.yml: Add master-pysim to publish manual
We only had gerrit build verification for pysim so far, but no 'master'
job that periodically builds/tests master (and publishes the manual).

As a reult, the pySim manual was still the original version from April
2021, while the code has of course progressed quite a bit in git since.

Change-Id: I4dce05cdb5f2c574256378d5fb66cff1c329c141
Closes: OS#5271
2022-02-09 17:48:04 +01:00
Pau Espin 1982f52f8e Deploy osmo-ci to ogt nodes and update it periodically
We need osmo-ci in those nodes in order for the docker old images to be
cleaned through cron entry installed by ansible docker role, which uses
/home/jenkins/osmo-ci/scripts/docker-cleanup.sh.

Related: OS#5441
Change-Id: Ieb6db0b24dce6099cfa24871a957471ca2c8bcee
2022-02-07 11:29:26 +01:00
Oliver Smith 7ad596e4d9 jobs/master-builds-dahdi.yml: new file
Related: OS#5407
Depends: dahdi-linux I904ab66a1ecd72492642ac2cc4cb102c7283c590
Change-Id: I8a479d0a214ff56c5b66f468a1c2b1b2f674e091
2022-02-04 15:05:04 +01:00
Oliver Smith b3cef938c9 jobs/gerrit-verifications-dahdi.yml: new file
Add gerrit verifications for the out-of-tree dahdi-linux kernel module.
Put this in a separate file (not gerrit-verifications.yml), because:
* not only gerrit git repo gets cloned, but also linux.git, and caching
  needs to be done properly for it
* so we can have one separate job per linux version that gets tested

Test v4.19 and v5.10 for now, as suggested in the issue.

Related: OS#5407
Depends: docker-playground Id72d19ad08681cd7cb3194de2226292f19e96df5
Change-Id: I65c7663a6b8045796536a727e4adf5554f16b2a9
2022-02-04 14:40:17 +01:00
Oliver Smith b7bf1b735d jobs/gerrit-lint: add dahdi-linux
Related: OS#5407
Change-Id: I7fdb40e40fb8cf8c9072a4e898aeffa43d1d7ae0
2022-02-04 12:36:28 +01:00