Commit Graph

185 Commits

Author SHA1 Message Date
Oliver Smith 4f8cf4049f ansible/gsm-tester: use debian's soapysdr-module-lms7
Same reasoning as in docker-playground
Ie2408258eda407d9db6c2f0f5fbe475861756778.

Change-Id: I957dc42df640e513554f54e597a12df05e0fd740
2022-06-29 09:33:30 +00:00
Harald Welte bab323da52 install-coverity: update to 2021.12
Starting in February 2022, coverity stopped to support the old
2019.03 version that we were using.  Unfortunately that didn't
generate any e-mail notification to the project admins so it
remained undetected until now :(

What sucks is that the filename states 2021.12 but then the
actual directory created upon extraction is 2021.12.1, meaning
our ansible rule breaks.  We have to rename the tarball to 2021.12.1
before running the playbook...

Closes: OS#5594
Change-Id: I950834cc2ad70979a53a563610bbc9feef1f5f20
2022-06-28 15:14:24 +00:00
Oliver Smith cd9c983b61 ansible: get OBS pub key from proper URL
Same reasoning as in previous patch
Idd0fb6e07cba959a36269244b0c7b5c62aaffeee.

Change-Id: I630654ade0edddd23f9fe7c578d1f329f5f49603
2022-06-10 15:35:46 +02:00
Oliver Smith cfcd2f8bec ansible/gsm-tester-bts/Release.key: update
The old one exipired, this one is valid until 2024-08-02.

Related: docker-playground I56a8708ae55cc4cc5928cd80bb57d63f32e22423
Change-Id: I9526481aaee8b84348de399bf2f58f4bb0d137e8
2022-06-10 11:17:32 +02:00
Oliver Smith 185317396b ansible: rpis: enable docuum
Now that all rpis have been upgraded to debian 11, enable docuum like
it's already done for x86_64.

Change-Id: I7ab54f03819704cf60a69a417de6d14cb5cd7a56
2022-05-30 16:54:51 +02:00
Oliver Smith 9ecd2280d8 ansible/jenkins/osmocom_build_deps: drop py2 pkgs
Drop python2 packages. It's deprecated, packages don't exist in
debian 11 and our jenkins nodes don't need it anymore.

Related: OS#5453
Change-Id: Id2da7adab83f240e7e0b45c018f91c9e639fc80d
2022-05-18 15:15:03 +02:00
Oliver Smith f511264bdd ansible/jenkins/osmocom_build_deps: drop dh-systemd
This package is no longer available in debian 11. Even in debian 9 the
description says:
  "This package is for transitional purposes and can be removed safely."

Related: OS#5453
Change-Id: I6f72de4d7fb0aab4b153dfc0affbc6f11d20ee4c
2022-05-18 15:05:31 +02:00
Oliver Smith e2ef4fed0e ansible/hosts: add gtp0-deb11build-i586
Related: OS#5453
Change-Id: I107dfb543090989c8b7c746be381aa19f0afa366
2022-05-18 15:00:47 +02:00
Harald Welte e50ad03b46 docker: Fix vfs storage cleanup task
The 'grep' logic of Change-Id I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
seems to be inverted.  if "overlay2" is found (grep successful) the
exit code of grep is '0'.

Change-Id: I334b0c71ef626ffd2c84097e902a5164faa3fa8b
Related: OS#5827
2022-05-17 22:15:16 +02:00
Harald Welte c614362382 New osmocom-obs-worker role for our OBS workers
Related: OS#5556
Change-Id: I7a6d82e89e3a840739f71b90cdebe89bf8383c4a
2022-05-10 19:43:13 +02:00
Oliver Smith 382aebd040 treewide: rename rpi4-raspbian10 to rpi4-raspbian11
The raspberries have been upgraded to the debian 11 based raspberry pi
OS. Adjust the name to avoid confusion.

Related: OS#5453
Change-Id: I0b2e08b11d83243e81b2bb4412ffcc9bc59821ff
2022-05-02 13:29:50 +00:00
Oliver Smith f225c1da76 ansible/ttcn3: don't install titan/libfftranscode
With the last patch, the jenkins infrastructure will always run
eclipse-titan in docker. Therefore do not install eclipse-titan and
libfftranscode on jenkins nodes anymore.

This avoids the problem that we couldn't upgrade libfftranscode with the
existing ansible rules.

libosmocore-utils is still being installed, as osmo-config-merge must be
installed on the jenkins nodes.

Closes: OS#5452
Change-Id: Ie46fe3dcb286a6258021fad9c35be10442544761
2022-02-21 08:08:00 +00:00
Oliver Smith 7a40ff9aa1 ansible/docuum: set default to 40 GB
After looking at total available disk space on most of the hosts,
storing up to 40 GB of images seems a more reasonable default. We can
increase it again if it should really not be enough.

Related: OS#5099
Change-Id: Iad679252b027003391f1e2368149cb10ea0c729e
2022-02-14 15:33:37 +01:00
Oliver Smith ffd73ca5e5 ansible/docuum: fix docker_max_image_space replace
When changing the value, replace the previous line instead of adding a
new line at the bottom.

Related: OS#5099
Change-Id: I35a3671a026cc72cf1488c8668e8a0e675de2530
2022-02-14 15:33:32 +01:00
Oliver Smith 98a4f57706 ansible/docuum: fix delgroup error
Check if a group with the same GID as the docker group on the host
exists, before attempting to remove it.

Related: OS#5099
Change-Id: I24af5c1d2b1c488b37f29f1609ee56692eff569e
2022-02-14 15:33:27 +01:00
Oliver Smith 3ab7e59de8 ansible: docker: clean vfs storage leftovers
Docker is configured via ansible to use the overlay2 storage driver
since August 2020 (I20728d6017204c3978e23376baa89de6e91fed1e). Clean up
an unused vfs dir if it is present on the system, as it was the case on
build2-deb9build-ansible with a 190 GiB vfs dir.

Related: OS#5827
Change-Id: I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
2022-02-11 15:44:54 +01:00
Oliver Smith 17f82a260d ansible/roles/docker: install docuum
Allow setting a max amount of space that docker images should take up,
and delete the least recently used images when the amount is exceeded.

Related: https://osmocom.org/projects/osmocom-servers/wiki/Docker_cache_clean_up
Change-Id: I640b1e607feca87e7a578946ae4b8332ce854ab1
2022-02-11 15:44:16 +01:00
Oliver Smith f07b5c8a64 ansible/setup-jenkins-slave: accept rel-change
Fix for:
  W:This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
  E:Repository 'http://raspbian.raspberrypi.org/raspbian buster InRelease' changed its 'Suite' value from 'stable' to 'oldstable'

Related: https://github.com/ansible/ansible/issues/48352
Change-Id: I90ee8dd49001d6810f2118e9feff3375e87052ea
2022-02-11 15:44:16 +01:00
Oliver Smith 6695a91c0b ansible/ansible.cfg: make command output readable
Have a readable output, instead of the default that wraps the output
with stripped new lines inside json.

Related: https://github.com/ansible/ansible/issues/27078#issuecomment-364560173
Change-Id: I88d584a1808d82d75906b350e3bffe9bc73a8c67
2022-02-11 15:44:16 +01:00
Vadim Yanitskiy f6f49d23a1 tasks/ttcn3-slave.yml: also install libosmocore-utils
This package provides osmo-config-merge, which is going to be used
by docker-playground for generating configuration files.

Change-Id: I87e54440a6b399aa0f0461ebf668991b5cdd0b07
Related: https://gerrit.osmocom.org/c/docker-playground/+/20622
2022-02-11 13:59:06 +00:00
Pau Espin 1982f52f8e Deploy osmo-ci to ogt nodes and update it periodically
We need osmo-ci in those nodes in order for the docker old images to be
cleaned through cron entry installed by ansible docker role, which uses
/home/jenkins/osmo-ci/scripts/docker-cleanup.sh.

Related: OS#5441
Change-Id: Ieb6db0b24dce6099cfa24871a957471ca2c8bcee
2022-02-07 11:29:26 +01:00
Pau Espin 702fd8ff94 Add new repo osmo-hnbgw (nightly) to build jobs and scripts
Change-Id: I2fdbd651e4b50191e1a67e183590de99d6366ca5
2022-01-05 11:39:41 +01:00
Pau Espin 2146640769 Add osmo-hnodeb to ansible/jenkins/coverity setup
Change-Id: Ie4b03aefeda13d216706bd648145689cf2ef93bb
2021-10-25 11:06:48 +02:00
Harald Welte d793650519 some more git.sysmocom.de https (+path) changes due to gitea migration
Change-Id: Ie1599b356932ffa83d70e08b96179b7ca4909b14
2021-05-17 08:16:34 +02:00
Harald Welte e82f47ccfc ansible/hosts: new IPs for GTP0 slaves
the old ones were DHCP-allocated and after a power outage, they changed.

I've now moved to static allocated addresses.

Change-Id: I0d77cc8ae72d854312b742c6c64b10a731a9f5ba
2021-05-17 08:10:12 +02:00
Pau Espin 72e63562fd ansible: osmocom_build_deps.yml: Install missing open5gs dep libnghttp2-dev
The official Build Guide lists it as a dependency, and meson fails
without it.

Change-Id: I6b40acd7dd5bdc596b26227bba83e3f87a841d00
2021-03-15 19:23:53 +01:00
Pau Espin c427bd6cf4 ansible: ogt: uhubctl has new cmdline syntax to specify ports
Change-Id: Ie57d85dc0a66e053247cd5bb92a4b2dea4eb4309
2021-03-15 14:51:11 +01:00
Pau Espin 8b9f1a8d55 ansible: ogt: prod setup has 12 modems nowadays
Change-Id: I6e401b292b2587733396c4160c008b894fc066a5
2021-03-15 14:50:28 +01:00
Oliver Smith 9b493f64ce ansible: jenkins-slave: enable ntpd
Enable ntpd to prevent clock difference issues reported by jenkins. I
just saw them appearing with one of the new raspberry pis.

Related: https://jenkins.osmocom.org/jenkins/computer/ (shows clock diff)
Change-Id: I2685e60221b0f3baa3f9e6076c01acf37cd38444
2021-03-10 17:06:10 +01:00
Oliver Smith 796fd16b30 ansible: remove raspbian default user "pi"
Add a new subtask that deletes Raspbian's default user "pi" with
UID=1000, so we can use the UID for the jenkins user. When the ansible
playbook is running, root access via ssh is already configured, so we
don't need the user "pi" anymore.

Related: OS#5055
Change-Id: If0cee6d35016c487eca02752625732dd03fad350
2021-03-08 17:29:20 +01:00
Oliver Smith 13e705f1ff ansible: explicitly give jenkins user UID=1000
docker-playground.git assumes that its containers run as UID=1000,
otherwise some will fail with errors like:

  + osmo-clean-workspace.sh
  + chmod -R +w .
  chmod: changing permissions of '.': Operation not permitted

Make this assumption explicit with "uid: 1000" in "add user jenkins".
This causes ansible to verify that the UID is really 1000, and if it is
not, it changes the UID. A follow-up patch will remove Raspbian's
default user of the same UID.

Related: OS#5055
Change-Id: I6efc9e74049e179d2f86655217195e76789ba77d
2021-03-08 17:28:57 +01:00
Oliver Smith aee8b3f14b ansible: fix apt deprecation warning about loops
Fix deprecation warnings like the following:
  [DEPRECATION WARNING]: Invoking "apt" only once while using a loop
  via squash_actions is deprecated. Instead of using a loop to supply
  multiple items  and specifying `name: "{{ item }}"`, please use
  `name: ['libjansson-dev', 'libulfius-dev']` and remove the loop.
  This feature will be removed in version 2.11.

Related: OS#5055
Change-Id: I57b9ce690e8167249468954f3e62d14c7a92fcfc
2021-03-08 15:47:17 +01:00
Oliver Smith 025c38d521 ansible/hosts: update rpi4-raspbian10build-*
Replace old rpi4-deb9build-ansible with the new entries:
* rpi4-raspbian10build-1
* rpi4-raspbian10build-2
* rpi4-raspbian10build-3

The new jenkins nodes do not use lxc anymore (see related issue).

Related: OS#5055
Change-Id: I5d6588320613711251abcb664a5614ad49215725
2021-03-08 15:40:05 +01:00
Oliver Smith 829f8b948d ansible: tasks/osmo-ci: clone osmo-gsm-manuals
Perform the initial clone of osmo-gsm-manuals.git, just like osmo-ci.git.

Related: OS#4912
Change-Id: I39a09a4297c7c78357b3d4db4ab89547fa08db59
2021-01-13 11:33:16 +01:00
Harald Welte 11f89dfc6a ansible: add new deb9build slave on gtplab0
Change-Id: Ide72397cff15c2caa785a117ff095d6f08cb411b
2020-11-28 22:52:11 +01:00
Harald Welte bf07865cb3 ansible: Fix OBS directory naming
It's really annoying that OBS uses Debian_8.0 and Debian_9.0 (for 8.x
and 9.x) and then Debian_10 for 10.x.  Let's work around this by some
conditional vars_file trickery, hopefully once and for all.

Change-Id: I45c4752774f44332b3f9aaf875f49043c03ec5ca
2020-11-28 22:33:02 +01:00
Harald Welte 8df41af010 osmocom-jenkins-slave: Re-order
Install TTCN3 before installing build-deps, as build-deps
requires libulfius, which is only provided by network:osmocom:latest
feed, which in turn is only activated as part of the TTCN3 / TITAN
installation.

Change-Id: I66f1d5f7479a6d028abdac0cc6f4000faa78b135
2020-11-28 22:02:52 +01:00
Harald Welte cf9625379a ansible: add role for 'podman'
podman has one particularly interesting feature which docker
rejected years ago: The ability to run a container in a pre-existing
namespace.  This is quite useful for some of our test setups,
particularly those that involve hdlc net-devices

Change-Id: I4b14599f39c07ec83fa4222f9d040d443caae3e6
2020-11-18 22:57:07 +01:00
Pau Espin 8edbe0f134 ansible: ogt: Install new dependency python3-watchdog
As of osmo-gsm-tester.git Change-Id
I938602ee890712fda82fd3f812d8edb1bcd05e08, python3-watchdog is required.

Change-Id: I35026e82b255cd4e06b4211c4829cd12be59976e
2020-11-17 09:52:10 +00:00
Harald Welte 1f5faf85bd remove all debian8 slaves
we cannot even create new slaves as lxc-create doesn't support debian
jessie anymore.  Let's focus on Debian 9 + Debian 10

Change-Id: I2fddd460f7d1db755cc9b732b2be80494d26e091
2020-11-15 10:01:03 +01:00
Harald Welte e6c46260da ansible/docker: Use docker.io on architectures not supported by docker-ce
The docker company official builds are only present for amd64, but not
for other architectures.  Let's use the Debian docker.io package in such
situations.

Change-Id: Id85137fc0eb7fa5e2b31aa928c78bfdaadc281ac
2020-11-15 10:01:03 +01:00
Harald Welte 285c752569 ansible: ttcn3-slave: Use libfftranscode 0.3
0.3 is the first version for which we have i386 builds

Change-Id: I8bc20e43a82ff797a4f70db476cddaab39fdc2a2
2020-11-14 22:34:18 +01:00
Harald Welte a06cc0e3d1 ansible: ttcn3-slave: Fix for Debian 10 slaves
Change-Id: I943e4a8e49e4f95abf1e44dc297d30c950580680
2020-11-14 22:34:18 +01:00
Harald Welte f1fbfc95fe ansible: roles/docker: Add ca-certificates
If that package is not installed, adding the GPG key fails with SSL
certificate verification, at least on a debian10 i386 lxc:

TASK [add docker gpg key to apt keyring] *************************************************************
fatal: [gtp0-deb10build32-ansible]: FAILED! => {"changed": false, "msg": "Failed to download key at https://download.docker.com/linux/debian/gpg: Request failed: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>"}

Change-Id: I58259a99258c9c7471e1fcf03a3a7b62fa9c6a48
2020-11-14 22:34:18 +01:00
Harald Welte 7d83ceb25c README.md: Fixup ansible command line for docker login
We need to do a docker log in as 'osmocom-build' user, not as 'root'

Related: OS#4839
Change-Id: I8ea0338e49d332382c9d08b21ee247244f97d141
2020-11-02 17:54:05 +01:00
Harald Welte a01babb6e1 README.md: Add instructions on how to make jenkins slaves 'log in' private docker registry
Change-Id: Ibf7e8c6ef2b3efd5a0a193501de90797e2d6f080
2020-11-02 17:17:57 +01:00
Harald Welte 3d3a9c1651 ttcn3-slave.yml: don't hard-code amd64 as the dpkg architecture
We have no clue what the host architecture will be...

Change-Id: I32c0905c8e077faa02984ea67c452fcdb81440da
2020-11-02 17:08:48 +01:00
Alexander Couzens b12f989f86 ansible: osmocom-jenkins-slave: install SRS packages only on debian >= 9
Before debian 9 those packages aren't available. e.g. libczmq-dev or libsoapysdr-dev

Change-Id: I019ec01dc7e0a53fa4ce9e8579fbafb3916f559b
2020-08-31 15:57:56 +00:00
Alexander Couzens 1a72d50548 ansible: osmocom-jenkins-slave: osmocom_build_deps: install osmo-python-tests for py2 and py3
The osmo-python-tests will be used to test vty commands.
The old python2 package are only required to test old images because
the current master is only supporting python3.
Installing both doesn't conflict because the python site-packages are
different for the two version.

Change-Id: Ica279e83b763389ababb526336ebd95d45beee7a
2020-08-14 16:31:27 +02:00
Alexander Couzens 523aa0874e ansible/docker: set overlayfs2 and ipv6
overlayfs2 should be the default anyway. Enable it on all hosts,
since it was enabled on older hosts.

Change-Id: I20728d6017204c3978e23376baa89de6e91fed1e
2020-08-03 01:12:43 +00:00