Commit Graph

178 Commits

Author SHA1 Message Date
Oliver Smith 1fc7514296 ansible: fix updating /opt/coverity/current
Remove the previous /opt/coverity/current symlink, before creating the
new one. Otherwise the new symlink would be created inside "current"
pointing at the old version, instead of replacing "current" with the
new symlink.

  # ln -svf /opt/coverity/2023.6.2/* /opt/coverity/current
  '/opt/coverity/current/cov-analysis-linux64-2023.6.2' -> '/opt/coverity/2023.6.2/cov-analysis-linux64-2023.6.2'
  # readlink /opt/covervity/current
  /opt/coverity/2022.06/cov-analysis-linux64-2022.6.0

Add the verbosity flag to the command while at it.

(The glob is unrelated, why it is needed is described in
 I2b96d1e47f2697706a042937b2852f0fc5032a7b)

Related: SYS#6685
Change-Id: I4ecb4997829b3cc61c839d089bda44f821ca8b85
2023-12-08 13:40:08 +00:00
Oliver Smith 39dd96b390 coverity: upgrade to 2023.6.2
Change-Id: I91f087fa3b39e1e0bc48739f92c4c23b3c0aa34f
2023-12-08 09:34:32 +01:00
Harald Welte a05542cf58 ansible/jenkins-slave/osmo-ci: Install 'make' required for osmo-ci
TASK [osmocom-jenkins-slave : rebuild osmocom docker images for jenkins build tests] ******************
fatal: [lx2-raspbian11build-ansible]: FAILED! => {
    "changed": true,
    "cmd": "./scripts/osmo-ci-docker-rebuild.sh",
    "delta": "0:00:04.129667",
    "end": "2023-11-18 14:09:52.142684",
    "rc": 1,
    "start": "2023-11-18 14:09:48.013017"
}

STDERR:

+ make -C ../debian-bookworm-build-arm BUILD_ARGS=--pull UPSTREAM_DISTRO=debian:bookworm DISTRO=debian-bookworm IMAGE=osmocom-build/debian-bookworm-build-arm OSMO_TTCN3_BRANCH=master
./scripts/osmo-ci-docker-rebuild.sh: 203: make: not found

Change-Id: I785e4224e729accec15beaa3aef8e01f12624d9d
2023-11-18 15:22:43 +01:00
Harald Welte 506e58eb5b ttcn3-slave: Install sudo
The sudo tool is required by the ansible 'become_user' used later in ttcn3-slave.yml

TASK [osmocom-jenkins-slave : ensure jenkins folder exists] *******************************************
fatal: [lx2-raspbian11build-ansible]: FAILED! => {
    "changed": false,
    "rc": 127
}

MODULE_STDOUT:
/bin/sh: 1: sudo: not found

Related: OS#5873
Change-Id: I2d50ef2dee8e060ff4f84810f2bba62d4801d638
2023-11-18 15:13:18 +01:00
Harald Welte ced38b31bc obs-worker: support more recent opensuse versions
We don't want to install new builders with 15.3, and the playbook
should just detect the installed version rather than making assumptions

Change-Id: I96f9896c0eeab120633d5800198d8b93371f7a5a
2023-11-18 11:38:08 +01:00
Oliver Smith 4e3b7babda ansible: qemu: netcat -> netcat-openbsd
Make the ansible rule compatible with debian 12. The same package is
already in debian 11 and debian 10.

Fix for:
  E: Package 'netcat' has no installation candidate

Change-Id: Idcc90ad7a2a104a9b4f5a76c6c89d74604651530
2023-10-19 07:30:20 +00:00
Oliver Smith 367f550da9 ansible: jenkins: debian: add more packages
Add python3-setuptools, as it is needed to run the
update-osmo-python-on-slaves jenkins job.

Add bc, as it is needed in docker-playground.git's network_create() in
jenkins-common.sh.

Related: OS#6186
Change-Id: I1ff0711b7b29ba4dc78f51f257b986d7a0994a96
2023-10-19 07:30:20 +00:00
Oliver Smith 7edd780403 ansible: jenkins: debian: sort packages
Related: OS#6186
Change-Id: If4ce784821cc40a2e795afafa452c99bf0588253
2023-10-19 07:30:20 +00:00
Oliver Smith 119bc1f44a ansible: jenkins: use default-jre-headless
Use default-jre-headless instead of giving the openjdk version, so we
don't need to adjust it to whatever the current version in debian is.

The default-jre-headless package is at least available since debian 10:
https://packages.debian.org/search?suite=buster&arch=any&searchon=names&keywords=default-jre-headless

Related: OS#6186
Change-Id: I6c5c74ff3d75dca2f89765d95c4b4c6d67ace703
2023-10-19 07:30:20 +00:00
Oliver Smith 7d90d5b17f ansible: jenkins: debian: remove qemu package
This is a dummy package that isn't available anymore in debian 12.

https://packages.debian.org/buster/qemu says:
> This package can safely be removed.

We install qemu-user-static and qemu-system-arm, which contain qemu
binaries.

Related: OS#6186
Change-Id: I4d5bd376b1e370d9636b9a2e9e710f5f20b9ae98
2023-10-19 07:30:20 +00:00
Oliver Smith 975edb5099 ansible: ttcn3: install netcat-openbsd
Make nc available on jenkins nodes running ttcn3 tests, as it is now
used by ttcn3-hss-test-ogs.

Related: docker-playground Iebeefd9689ae2efa54678b99ff929ca255ec64d1
Change-Id: I397d4fe33813b083b61cdabb1d00ab4ea63a456b
2023-10-16 11:17:34 +02:00
Oliver Smith 228efc3f13 ansible: osmocom_build_deps: remove
The builds for almost all jenkins jobs are done in docker now, we
definitely don't need to install all of these dependencies on all
jenkins nodes anymore.

Remove the file for now, so it doesn't look like they still get used and
so we don't need to keep it up-to-date (e.g. inkscape was just replaced
with rsvg-convert).

When we deploy a new jenkins node and notice that we still do need a few
of these dependencies, we can bring them back later.

Change-Id: I7c87bd516d1bcdbcc21f65909dabbd3ccc38eede
2023-08-17 14:01:39 +02:00
Oliver Smith 82ff45e7f7 repo-install-test: add debian12
Related: OS#6057
Change-Id: I51fa3c5212b5e29f4c2804384781693e8de2c097
2023-08-03 11:03:58 +02:00
Alexander Couzens 607cedc308 ansible: install-coverity: ensure it's not installed twice
Check if the coverity path already exists. Even this wouldn't
detect if the coverity has been only installed half way.

Related: OS#5801
Change-Id: I95549983bb6bd47e04eb37c73afe5409637f87d3
2023-03-23 08:52:06 +00:00
Alexander Couzens f2e49f051c ansible: install-coverity: add missing tag coverity
Change-Id: I86be4c72ad3bce070ddf7f8270492b6cf8ea826e
2023-03-23 08:52:06 +00:00
Alexander Couzens 6d7432e774 ansible: install-poky-sdk: add tag poky to the apt install task
Change-Id: Id8e433a789aff8df241687f1f109a161e725a29a
2023-03-23 08:52:06 +00:00
Oliver Smith 733626688d ansible/osmocom-jenkins-slave: add libnftables-dev
Install libnftables-dev, so osmo-upf can build successfully in the
coverity jenkins job.

Change-Id: Id72ce8ed5b662085a16f56756f8f93627834d2e2
2023-02-01 10:58:04 +01:00
Alexander Couzens e64f85a0ee ansible: install-poky-sdk: ensure it's not installed twice
Check if the poky path already exists. Even this wouldn't
detect if the poky has been only installed half way.

Related: OS#5801
Change-Id: I8ff8e72a5974378327d4692341b0a371c667b741
2023-01-23 10:57:30 +00:00
Alexander Couzens 6a290081c5 ansible: replace deprecated include by import_tasks
include_tasks and import_tasks have similar functionality.
Use import_tasks when importing non-variable depending files.
Further use import_tasks when not used within a loop.

Closes: OS#5797
Change-Id: I29766c65b9e1601a20aa8f20a438e8368c65f6a8
2023-01-23 10:57:30 +00:00
Oliver Smith 47dde8559f ansible: add rules to install qemu images
Prepare to run repo-install-test inside qemu with kvm instead of
docker, see following patches.

Related: OS#5365
Change-Id: Ia7626fcd397d32e1180e320d4a76a65bcad01b7b
2022-12-09 10:55:15 +00:00
Oliver Smith 54a36bcb7d ansible/podman: install from debian repos
Podman is in the official debian repositories since debian 10, install
it from there instead of using a third-party repository.

Related: OS#5365
Change-Id: I01549915d2c8dfda5fa605a014a304229c3078f0
2022-12-08 13:04:31 +00:00
Oliver Smith b0f36fe227 ansible/coverity: extract into subdirectory
The tarball provided by coverity doesn't have the same version in the
name as in the directory inside the tarball.

Tarball:
	cov-analysis-linux64-2022.06.tar.gz
Directory:
	cov-analysis-linux64-2022.6.0

Work around this by extracting it into a subdirectory and using glob to
match the directory name.

Related: OS#5793
Change-Id: I2b96d1e47f2697706a042937b2852f0fc5032a7b
2022-12-01 13:08:28 +01:00
Oliver Smith 8125b33b9a ansible: coverity: update version to 22.06
Coverity doesn't make the old version available for download and will
probably deprecate the previous version at some point. I'm about to
install it on new hosts, so upgrade to the latest version while at it.

Related: OS#5793
Change-Id: Id5700a9e5a46a9cfafc9959f5b77f9e21dd956b4
2022-12-01 13:08:07 +01:00
Oliver Smith 1a3ee20141 ansible/jenkins: add gerrit ssh key to known_hosts
Add the key to known hosts to prevent host key verification failed
errors when attempting to use the gerrit API from jenkins (to post
review comments) or when attempting to clone over ssh from gerrit (as
we do for some reason apparently).

Change-Id: I50981f201c5ca1425e30677b2578491f5111fc0c
2022-11-28 14:13:34 +01:00
Oliver Smith 9fbcd1c35b ansible/install-poky-sdk: install more depends
Install missing depends file and gcc, and sort the list alphabetically
while at it.

Fixes: OS#5794
Change-Id: Ia0530859d65818e55498bc97af139b8773445dcf
2022-11-28 14:10:50 +01:00
Oliver Smith f9c9069151 ansible/ttcn3-slave: fix adding apt key
Downloading the apt key from obs.osmocom.org returns a 404 error unless
an Accept HTTP header is sent. (Yes, really.) Also using apt-key is
deprecated, so change it to download the key with get_url as shown in
the ansible docs example.

Fix for:
  Failed to download key at https://obs.osmocom.org/projects/osmocom/public_key: HTTP Error 404: Not Found

Related: https://github.com/ansible/ansible/issues/50436#issuecomment-451205256
Related: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_key_module.html#examples
Change-Id: Idb23231d629ba8a75510deace0d1f130e889d535
2022-10-13 10:51:55 +02:00
Oliver Smith 7fc58133d6 Replace most remaining git.osmocom.org urls
Replace the legacy git.osmocom.org clone URLs with either gerrit or
gitea in most places. Besides not using legacy urls anymore, the
advantage of using gerrit directly is that we don't have a delay of 10
min to several hours on the repository.

Not replaced:
* ansible/roles/osmocom-workstation: not sure if still used, would need
  a bit of a rewrite to use logic similar to osmo_git_clone_url().

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I90409c9e7f4db4e1765647021625f9acfccaa78b
2022-09-26 15:07:59 +02:00
Harald Welte 5455b42405 update git URLs (git -> https; gitea/gerrit)
Unencrypted git:// protocol offers no integrity or authentication,
making it subject to tampering.  Use https:// instead.

https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
https://blog.readthedocs.com/github-git-protocol-deprecation/

Change-Id: Ia44e4127ff87a3d4d747225d3258188a0a891f1b
2022-08-13 04:02:51 +07:00
Harald Welte d5b6cba4e9 ansible: Use downloads.osmocom.org instead of opensuse.org
Change-Id: Ib60eb87c7b0c17d20bbbfa8b94d8262f0c165b1c
Related: OS#5557
2022-07-13 07:41:55 +00:00
Oliver Smith 4f8cf4049f ansible/gsm-tester: use debian's soapysdr-module-lms7
Same reasoning as in docker-playground
Ie2408258eda407d9db6c2f0f5fbe475861756778.

Change-Id: I957dc42df640e513554f54e597a12df05e0fd740
2022-06-29 09:33:30 +00:00
Harald Welte bab323da52 install-coverity: update to 2021.12
Starting in February 2022, coverity stopped to support the old
2019.03 version that we were using.  Unfortunately that didn't
generate any e-mail notification to the project admins so it
remained undetected until now :(

What sucks is that the filename states 2021.12 but then the
actual directory created upon extraction is 2021.12.1, meaning
our ansible rule breaks.  We have to rename the tarball to 2021.12.1
before running the playbook...

Closes: OS#5594
Change-Id: I950834cc2ad70979a53a563610bbc9feef1f5f20
2022-06-28 15:14:24 +00:00
Oliver Smith cd9c983b61 ansible: get OBS pub key from proper URL
Same reasoning as in previous patch
Idd0fb6e07cba959a36269244b0c7b5c62aaffeee.

Change-Id: I630654ade0edddd23f9fe7c578d1f329f5f49603
2022-06-10 15:35:46 +02:00
Oliver Smith cfcd2f8bec ansible/gsm-tester-bts/Release.key: update
The old one exipired, this one is valid until 2024-08-02.

Related: docker-playground I56a8708ae55cc4cc5928cd80bb57d63f32e22423
Change-Id: I9526481aaee8b84348de399bf2f58f4bb0d137e8
2022-06-10 11:17:32 +02:00
Oliver Smith 185317396b ansible: rpis: enable docuum
Now that all rpis have been upgraded to debian 11, enable docuum like
it's already done for x86_64.

Change-Id: I7ab54f03819704cf60a69a417de6d14cb5cd7a56
2022-05-30 16:54:51 +02:00
Oliver Smith 9ecd2280d8 ansible/jenkins/osmocom_build_deps: drop py2 pkgs
Drop python2 packages. It's deprecated, packages don't exist in
debian 11 and our jenkins nodes don't need it anymore.

Related: OS#5453
Change-Id: Id2da7adab83f240e7e0b45c018f91c9e639fc80d
2022-05-18 15:15:03 +02:00
Oliver Smith f511264bdd ansible/jenkins/osmocom_build_deps: drop dh-systemd
This package is no longer available in debian 11. Even in debian 9 the
description says:
  "This package is for transitional purposes and can be removed safely."

Related: OS#5453
Change-Id: I6f72de4d7fb0aab4b153dfc0affbc6f11d20ee4c
2022-05-18 15:05:31 +02:00
Harald Welte e50ad03b46 docker: Fix vfs storage cleanup task
The 'grep' logic of Change-Id I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
seems to be inverted.  if "overlay2" is found (grep successful) the
exit code of grep is '0'.

Change-Id: I334b0c71ef626ffd2c84097e902a5164faa3fa8b
Related: OS#5827
2022-05-17 22:15:16 +02:00
Harald Welte c614362382 New osmocom-obs-worker role for our OBS workers
Related: OS#5556
Change-Id: I7a6d82e89e3a840739f71b90cdebe89bf8383c4a
2022-05-10 19:43:13 +02:00
Oliver Smith f225c1da76 ansible/ttcn3: don't install titan/libfftranscode
With the last patch, the jenkins infrastructure will always run
eclipse-titan in docker. Therefore do not install eclipse-titan and
libfftranscode on jenkins nodes anymore.

This avoids the problem that we couldn't upgrade libfftranscode with the
existing ansible rules.

libosmocore-utils is still being installed, as osmo-config-merge must be
installed on the jenkins nodes.

Closes: OS#5452
Change-Id: Ie46fe3dcb286a6258021fad9c35be10442544761
2022-02-21 08:08:00 +00:00
Oliver Smith 7a40ff9aa1 ansible/docuum: set default to 40 GB
After looking at total available disk space on most of the hosts,
storing up to 40 GB of images seems a more reasonable default. We can
increase it again if it should really not be enough.

Related: OS#5099
Change-Id: Iad679252b027003391f1e2368149cb10ea0c729e
2022-02-14 15:33:37 +01:00
Oliver Smith ffd73ca5e5 ansible/docuum: fix docker_max_image_space replace
When changing the value, replace the previous line instead of adding a
new line at the bottom.

Related: OS#5099
Change-Id: I35a3671a026cc72cf1488c8668e8a0e675de2530
2022-02-14 15:33:32 +01:00
Oliver Smith 98a4f57706 ansible/docuum: fix delgroup error
Check if a group with the same GID as the docker group on the host
exists, before attempting to remove it.

Related: OS#5099
Change-Id: I24af5c1d2b1c488b37f29f1609ee56692eff569e
2022-02-14 15:33:27 +01:00
Oliver Smith 3ab7e59de8 ansible: docker: clean vfs storage leftovers
Docker is configured via ansible to use the overlay2 storage driver
since August 2020 (I20728d6017204c3978e23376baa89de6e91fed1e). Clean up
an unused vfs dir if it is present on the system, as it was the case on
build2-deb9build-ansible with a 190 GiB vfs dir.

Related: OS#5827
Change-Id: I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
2022-02-11 15:44:54 +01:00
Oliver Smith 17f82a260d ansible/roles/docker: install docuum
Allow setting a max amount of space that docker images should take up,
and delete the least recently used images when the amount is exceeded.

Related: https://osmocom.org/projects/osmocom-servers/wiki/Docker_cache_clean_up
Change-Id: I640b1e607feca87e7a578946ae4b8332ce854ab1
2022-02-11 15:44:16 +01:00
Oliver Smith f07b5c8a64 ansible/setup-jenkins-slave: accept rel-change
Fix for:
  W:This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
  E:Repository 'http://raspbian.raspberrypi.org/raspbian buster InRelease' changed its 'Suite' value from 'stable' to 'oldstable'

Related: https://github.com/ansible/ansible/issues/48352
Change-Id: I90ee8dd49001d6810f2118e9feff3375e87052ea
2022-02-11 15:44:16 +01:00
Vadim Yanitskiy f6f49d23a1 tasks/ttcn3-slave.yml: also install libosmocore-utils
This package provides osmo-config-merge, which is going to be used
by docker-playground for generating configuration files.

Change-Id: I87e54440a6b399aa0f0461ebf668991b5cdd0b07
Related: https://gerrit.osmocom.org/c/docker-playground/+/20622
2022-02-11 13:59:06 +00:00
Pau Espin 702fd8ff94 Add new repo osmo-hnbgw (nightly) to build jobs and scripts
Change-Id: I2fdbd651e4b50191e1a67e183590de99d6366ca5
2022-01-05 11:39:41 +01:00
Pau Espin 2146640769 Add osmo-hnodeb to ansible/jenkins/coverity setup
Change-Id: Ie4b03aefeda13d216706bd648145689cf2ef93bb
2021-10-25 11:06:48 +02:00
Harald Welte d793650519 some more git.sysmocom.de https (+path) changes due to gitea migration
Change-Id: Ie1599b356932ffa83d70e08b96179b7ca4909b14
2021-05-17 08:16:34 +02:00
Pau Espin 72e63562fd ansible: osmocom_build_deps.yml: Install missing open5gs dep libnghttp2-dev
The official Build Guide lists it as a dependency, and meson fails
without it.

Change-Id: I6b40acd7dd5bdc596b26227bba83e3f87a841d00
2021-03-15 19:23:53 +01:00