Commit Graph

207 Commits

Author SHA1 Message Date
Oliver Smith 733626688d ansible/osmocom-jenkins-slave: add libnftables-dev
Install libnftables-dev, so osmo-upf can build successfully in the
coverity jenkins job.

Change-Id: Id72ce8ed5b662085a16f56756f8f93627834d2e2
2023-02-01 10:58:04 +01:00
Alexander Couzens 113b295075 ansible: .gitignore: make poky regex more specific
Otherwise it would also prevent changes to the poky role

Change-Id: I3ac9a6129323f27ba387b44d8293d7b134716af8
2023-01-23 10:57:45 +00:00
Alexander Couzens e64f85a0ee ansible: install-poky-sdk: ensure it's not installed twice
Check if the poky path already exists. Even this wouldn't
detect if the poky has been only installed half way.

Related: OS#5801
Change-Id: I8ff8e72a5974378327d4692341b0a371c667b741
2023-01-23 10:57:30 +00:00
Alexander Couzens 6a290081c5 ansible: replace deprecated include by import_tasks
include_tasks and import_tasks have similar functionality.
Use import_tasks when importing non-variable depending files.
Further use import_tasks when not used within a loop.

Closes: OS#5797
Change-Id: I29766c65b9e1601a20aa8f20a438e8368c65f6a8
2023-01-23 10:57:30 +00:00
Harald Welte c0c48532be ansible inventory: add gtp0-deb11build
we're moving to debian 11 builders, gtp0 didn't have any

Change-Id: I1d387a460a9d71d537ab4790b4bb3177b1b96b88
2022-12-09 13:26:25 +01:00
Harald Welte 25a92ecb01 ansible inventory: We have recently killed gtp0-deb11build-i586
Change-Id: I15cb177e75f73fb8a06b17e42c99a6b573dc7574
2022-12-09 13:26:25 +01:00
Oliver Smith 47dde8559f ansible: add rules to install qemu images
Prepare to run repo-install-test inside qemu with kvm instead of
docker, see following patches.

Related: OS#5365
Change-Id: Ia7626fcd397d32e1180e320d4a76a65bcad01b7b
2022-12-09 10:55:15 +00:00
Oliver Smith 54a36bcb7d ansible/podman: install from debian repos
Podman is in the official debian repositories since debian 10, install
it from there instead of using a third-party repository.

Related: OS#5365
Change-Id: I01549915d2c8dfda5fa605a014a304229c3078f0
2022-12-08 13:04:31 +00:00
Oliver Smith 9bdd99b2ff ansible/hosts: remove debian 9 lxcs
Related: OS#5793
Change-Id: I465d93e5254ba2f65796fcfb62ea1063a0eb72c6
2022-12-06 11:17:16 +01:00
Oliver Smith 6bb5f4ce84 hosts: update coverity-slaves
Remove the deb9 lxcs as they are being retired, and add two deb11 lxcs.
Coverity is quite big with 1.2 GiB, so deploying and upgrading this
takes some time. Having it on two hosts should be enough.

Related: OS#5793
Change-Id: I02d43d7cbdbcba5581aa5da0a1effa3db7b445c0
2022-12-01 13:08:38 +01:00
Oliver Smith b0f36fe227 ansible/coverity: extract into subdirectory
The tarball provided by coverity doesn't have the same version in the
name as in the directory inside the tarball.

Tarball:
	cov-analysis-linux64-2022.06.tar.gz
Directory:
	cov-analysis-linux64-2022.6.0

Work around this by extracting it into a subdirectory and using glob to
match the directory name.

Related: OS#5793
Change-Id: I2b96d1e47f2697706a042937b2852f0fc5032a7b
2022-12-01 13:08:28 +01:00
Oliver Smith 8125b33b9a ansible: coverity: update version to 22.06
Coverity doesn't make the old version available for download and will
probably deprecate the previous version at some point. I'm about to
install it on new hosts, so upgrade to the latest version while at it.

Related: OS#5793
Change-Id: Id5700a9e5a46a9cfafc9959f5b77f9e21dd956b4
2022-12-01 13:08:07 +01:00
Oliver Smith 1a3ee20141 ansible/jenkins: add gerrit ssh key to known_hosts
Add the key to known hosts to prevent host key verification failed
errors when attempting to use the gerrit API from jenkins (to post
review comments) or when attempting to clone over ssh from gerrit (as
we do for some reason apparently).

Change-Id: I50981f201c5ca1425e30677b2578491f5111fc0c
2022-11-28 14:13:34 +01:00
Oliver Smith 9fbcd1c35b ansible/install-poky-sdk: install more depends
Install missing depends file and gcc, and sort the list alphabetically
while at it.

Fixes: OS#5794
Change-Id: Ia0530859d65818e55498bc97af139b8773445dcf
2022-11-28 14:10:50 +01:00
Harald Welte f04bfa8cc5 ansible: add {host2,build3}-deb11build-ansible
The playbook currently doesn't deploy, see https://osmocom.org/issues/5794

Change-Id: I15439bb85c1c03a96231ea195d6ad1679a0a1af1
2022-11-28 12:30:21 +01:00
Oliver Smith 1c7e7c21c3 ansible/setup-simtest: pre-install pysim depends
Avoid spending time to install the depends of pysim in each CI job.

Closes: OS#5497
Change-Id: I1e9ddb93f05108146a219bd28cf1e9332764bf8e
2022-10-20 09:18:25 +00:00
Oliver Smith f9c9069151 ansible/ttcn3-slave: fix adding apt key
Downloading the apt key from obs.osmocom.org returns a 404 error unless
an Accept HTTP header is sent. (Yes, really.) Also using apt-key is
deprecated, so change it to download the key with get_url as shown in
the ansible docs example.

Fix for:
  Failed to download key at https://obs.osmocom.org/projects/osmocom/public_key: HTTP Error 404: Not Found

Related: https://github.com/ansible/ansible/issues/50436#issuecomment-451205256
Related: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_key_module.html#examples
Change-Id: Idb23231d629ba8a75510deace0d1f130e889d535
2022-10-13 10:51:55 +02:00
Oliver Smith ed03bc78ed ansible/hosts: add build2-deb11build-ansible
Add it to the config. I've used this to enable ipv6 in docker, so
libosmocore builds don't fail in the socket test when building rpm/deb
packages on that host in docker.

Related: OS#2385
Change-Id: Ibe8e2a638fcbc7c0a69c211385a870b1a82b6fef
2022-10-13 10:51:55 +02:00
Oliver Smith 7fc58133d6 Replace most remaining git.osmocom.org urls
Replace the legacy git.osmocom.org clone URLs with either gerrit or
gitea in most places. Besides not using legacy urls anymore, the
advantage of using gerrit directly is that we don't have a delay of 10
min to several hours on the repository.

Not replaced:
* ansible/roles/osmocom-workstation: not sure if still used, would need
  a bit of a rewrite to use logic similar to osmo_git_clone_url().

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I90409c9e7f4db4e1765647021625f9acfccaa78b
2022-09-26 15:07:59 +02:00
Harald Welte 5455b42405 update git URLs (git -> https; gitea/gerrit)
Unencrypted git:// protocol offers no integrity or authentication,
making it subject to tampering.  Use https:// instead.

https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
https://blog.readthedocs.com/github-git-protocol-deprecation/

Change-Id: Ia44e4127ff87a3d4d747225d3258188a0a891f1b
2022-08-13 04:02:51 +07:00
Harald Welte d5b6cba4e9 ansible: Use downloads.osmocom.org instead of opensuse.org
Change-Id: Ib60eb87c7b0c17d20bbbfa8b94d8262f0c165b1c
Related: OS#5557
2022-07-13 07:41:55 +00:00
Harald Welte ac694eacb6 ansible/hosts: Replace failed OBS/qemu with real arm machine
We gave up on trying to fix OBS cross-building via qemu, and instead
bought a real ARM64 machine.

Change-Id: I5f3ffe451d029ba459575881bf88f2527fe9fd15
2022-07-13 07:41:55 +00:00
Oliver Smith 4f8cf4049f ansible/gsm-tester: use debian's soapysdr-module-lms7
Same reasoning as in docker-playground
Ie2408258eda407d9db6c2f0f5fbe475861756778.

Change-Id: I957dc42df640e513554f54e597a12df05e0fd740
2022-06-29 09:33:30 +00:00
Harald Welte bab323da52 install-coverity: update to 2021.12
Starting in February 2022, coverity stopped to support the old
2019.03 version that we were using.  Unfortunately that didn't
generate any e-mail notification to the project admins so it
remained undetected until now :(

What sucks is that the filename states 2021.12 but then the
actual directory created upon extraction is 2021.12.1, meaning
our ansible rule breaks.  We have to rename the tarball to 2021.12.1
before running the playbook...

Closes: OS#5594
Change-Id: I950834cc2ad70979a53a563610bbc9feef1f5f20
2022-06-28 15:14:24 +00:00
Oliver Smith cd9c983b61 ansible: get OBS pub key from proper URL
Same reasoning as in previous patch
Idd0fb6e07cba959a36269244b0c7b5c62aaffeee.

Change-Id: I630654ade0edddd23f9fe7c578d1f329f5f49603
2022-06-10 15:35:46 +02:00
Oliver Smith cfcd2f8bec ansible/gsm-tester-bts/Release.key: update
The old one exipired, this one is valid until 2024-08-02.

Related: docker-playground I56a8708ae55cc4cc5928cd80bb57d63f32e22423
Change-Id: I9526481aaee8b84348de399bf2f58f4bb0d137e8
2022-06-10 11:17:32 +02:00
Oliver Smith 185317396b ansible: rpis: enable docuum
Now that all rpis have been upgraded to debian 11, enable docuum like
it's already done for x86_64.

Change-Id: I7ab54f03819704cf60a69a417de6d14cb5cd7a56
2022-05-30 16:54:51 +02:00
Oliver Smith 9ecd2280d8 ansible/jenkins/osmocom_build_deps: drop py2 pkgs
Drop python2 packages. It's deprecated, packages don't exist in
debian 11 and our jenkins nodes don't need it anymore.

Related: OS#5453
Change-Id: Id2da7adab83f240e7e0b45c018f91c9e639fc80d
2022-05-18 15:15:03 +02:00
Oliver Smith f511264bdd ansible/jenkins/osmocom_build_deps: drop dh-systemd
This package is no longer available in debian 11. Even in debian 9 the
description says:
  "This package is for transitional purposes and can be removed safely."

Related: OS#5453
Change-Id: I6f72de4d7fb0aab4b153dfc0affbc6f11d20ee4c
2022-05-18 15:05:31 +02:00
Oliver Smith e2ef4fed0e ansible/hosts: add gtp0-deb11build-i586
Related: OS#5453
Change-Id: I107dfb543090989c8b7c746be381aa19f0afa366
2022-05-18 15:00:47 +02:00
Harald Welte e50ad03b46 docker: Fix vfs storage cleanup task
The 'grep' logic of Change-Id I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
seems to be inverted.  if "overlay2" is found (grep successful) the
exit code of grep is '0'.

Change-Id: I334b0c71ef626ffd2c84097e902a5164faa3fa8b
Related: OS#5827
2022-05-17 22:15:16 +02:00
Harald Welte c614362382 New osmocom-obs-worker role for our OBS workers
Related: OS#5556
Change-Id: I7a6d82e89e3a840739f71b90cdebe89bf8383c4a
2022-05-10 19:43:13 +02:00
Oliver Smith 382aebd040 treewide: rename rpi4-raspbian10 to rpi4-raspbian11
The raspberries have been upgraded to the debian 11 based raspberry pi
OS. Adjust the name to avoid confusion.

Related: OS#5453
Change-Id: I0b2e08b11d83243e81b2bb4412ffcc9bc59821ff
2022-05-02 13:29:50 +00:00
Oliver Smith f225c1da76 ansible/ttcn3: don't install titan/libfftranscode
With the last patch, the jenkins infrastructure will always run
eclipse-titan in docker. Therefore do not install eclipse-titan and
libfftranscode on jenkins nodes anymore.

This avoids the problem that we couldn't upgrade libfftranscode with the
existing ansible rules.

libosmocore-utils is still being installed, as osmo-config-merge must be
installed on the jenkins nodes.

Closes: OS#5452
Change-Id: Ie46fe3dcb286a6258021fad9c35be10442544761
2022-02-21 08:08:00 +00:00
Oliver Smith 7a40ff9aa1 ansible/docuum: set default to 40 GB
After looking at total available disk space on most of the hosts,
storing up to 40 GB of images seems a more reasonable default. We can
increase it again if it should really not be enough.

Related: OS#5099
Change-Id: Iad679252b027003391f1e2368149cb10ea0c729e
2022-02-14 15:33:37 +01:00
Oliver Smith ffd73ca5e5 ansible/docuum: fix docker_max_image_space replace
When changing the value, replace the previous line instead of adding a
new line at the bottom.

Related: OS#5099
Change-Id: I35a3671a026cc72cf1488c8668e8a0e675de2530
2022-02-14 15:33:32 +01:00
Oliver Smith 98a4f57706 ansible/docuum: fix delgroup error
Check if a group with the same GID as the docker group on the host
exists, before attempting to remove it.

Related: OS#5099
Change-Id: I24af5c1d2b1c488b37f29f1609ee56692eff569e
2022-02-14 15:33:27 +01:00
Oliver Smith 3ab7e59de8 ansible: docker: clean vfs storage leftovers
Docker is configured via ansible to use the overlay2 storage driver
since August 2020 (I20728d6017204c3978e23376baa89de6e91fed1e). Clean up
an unused vfs dir if it is present on the system, as it was the case on
build2-deb9build-ansible with a 190 GiB vfs dir.

Related: OS#5827
Change-Id: I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
2022-02-11 15:44:54 +01:00
Oliver Smith 17f82a260d ansible/roles/docker: install docuum
Allow setting a max amount of space that docker images should take up,
and delete the least recently used images when the amount is exceeded.

Related: https://osmocom.org/projects/osmocom-servers/wiki/Docker_cache_clean_up
Change-Id: I640b1e607feca87e7a578946ae4b8332ce854ab1
2022-02-11 15:44:16 +01:00
Oliver Smith f07b5c8a64 ansible/setup-jenkins-slave: accept rel-change
Fix for:
  W:This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
  E:Repository 'http://raspbian.raspberrypi.org/raspbian buster InRelease' changed its 'Suite' value from 'stable' to 'oldstable'

Related: https://github.com/ansible/ansible/issues/48352
Change-Id: I90ee8dd49001d6810f2118e9feff3375e87052ea
2022-02-11 15:44:16 +01:00
Oliver Smith 6695a91c0b ansible/ansible.cfg: make command output readable
Have a readable output, instead of the default that wraps the output
with stripped new lines inside json.

Related: https://github.com/ansible/ansible/issues/27078#issuecomment-364560173
Change-Id: I88d584a1808d82d75906b350e3bffe9bc73a8c67
2022-02-11 15:44:16 +01:00
Vadim Yanitskiy f6f49d23a1 tasks/ttcn3-slave.yml: also install libosmocore-utils
This package provides osmo-config-merge, which is going to be used
by docker-playground for generating configuration files.

Change-Id: I87e54440a6b399aa0f0461ebf668991b5cdd0b07
Related: https://gerrit.osmocom.org/c/docker-playground/+/20622
2022-02-11 13:59:06 +00:00
Pau Espin 1982f52f8e Deploy osmo-ci to ogt nodes and update it periodically
We need osmo-ci in those nodes in order for the docker old images to be
cleaned through cron entry installed by ansible docker role, which uses
/home/jenkins/osmo-ci/scripts/docker-cleanup.sh.

Related: OS#5441
Change-Id: Ieb6db0b24dce6099cfa24871a957471ca2c8bcee
2022-02-07 11:29:26 +01:00
Pau Espin 702fd8ff94 Add new repo osmo-hnbgw (nightly) to build jobs and scripts
Change-Id: I2fdbd651e4b50191e1a67e183590de99d6366ca5
2022-01-05 11:39:41 +01:00
Pau Espin 2146640769 Add osmo-hnodeb to ansible/jenkins/coverity setup
Change-Id: Ie4b03aefeda13d216706bd648145689cf2ef93bb
2021-10-25 11:06:48 +02:00
Harald Welte d793650519 some more git.sysmocom.de https (+path) changes due to gitea migration
Change-Id: Ie1599b356932ffa83d70e08b96179b7ca4909b14
2021-05-17 08:16:34 +02:00
Harald Welte e82f47ccfc ansible/hosts: new IPs for GTP0 slaves
the old ones were DHCP-allocated and after a power outage, they changed.

I've now moved to static allocated addresses.

Change-Id: I0d77cc8ae72d854312b742c6c64b10a731a9f5ba
2021-05-17 08:10:12 +02:00
Pau Espin 72e63562fd ansible: osmocom_build_deps.yml: Install missing open5gs dep libnghttp2-dev
The official Build Guide lists it as a dependency, and meson fails
without it.

Change-Id: I6b40acd7dd5bdc596b26227bba83e3f87a841d00
2021-03-15 19:23:53 +01:00
Pau Espin c427bd6cf4 ansible: ogt: uhubctl has new cmdline syntax to specify ports
Change-Id: Ie57d85dc0a66e053247cd5bb92a4b2dea4eb4309
2021-03-15 14:51:11 +01:00
Pau Espin 8b9f1a8d55 ansible: ogt: prod setup has 12 modems nowadays
Change-Id: I6e401b292b2587733396c4160c008b894fc066a5
2021-03-15 14:50:28 +01:00