Commit Graph

235 Commits

Author SHA1 Message Date
Oliver Smith 1fc7514296 ansible: fix updating /opt/coverity/current
Remove the previous /opt/coverity/current symlink, before creating the
new one. Otherwise the new symlink would be created inside "current"
pointing at the old version, instead of replacing "current" with the
new symlink.

  # ln -svf /opt/coverity/2023.6.2/* /opt/coverity/current
  '/opt/coverity/current/cov-analysis-linux64-2023.6.2' -> '/opt/coverity/2023.6.2/cov-analysis-linux64-2023.6.2'
  # readlink /opt/covervity/current
  /opt/coverity/2022.06/cov-analysis-linux64-2022.6.0

Add the verbosity flag to the command while at it.

(The glob is unrelated, why it is needed is described in
 I2b96d1e47f2697706a042937b2852f0fc5032a7b)

Related: SYS#6685
Change-Id: I4ecb4997829b3cc61c839d089bda44f821ca8b85
2023-12-08 13:40:08 +00:00
Oliver Smith 39dd96b390 coverity: upgrade to 2023.6.2
Change-Id: I91f087fa3b39e1e0bc48739f92c4c23b3c0aa34f
2023-12-08 09:34:32 +01:00
Harald Welte b301da1369 ansible: add raspbian11build container on lx2 machine
Change-Id: I739e40cf6ef8d77585dfeb067ed75e258d94bbb3
Related: OS#5873
2023-11-18 15:22:43 +01:00
Harald Welte a05542cf58 ansible/jenkins-slave/osmo-ci: Install 'make' required for osmo-ci
TASK [osmocom-jenkins-slave : rebuild osmocom docker images for jenkins build tests] ******************
fatal: [lx2-raspbian11build-ansible]: FAILED! => {
    "changed": true,
    "cmd": "./scripts/osmo-ci-docker-rebuild.sh",
    "delta": "0:00:04.129667",
    "end": "2023-11-18 14:09:52.142684",
    "rc": 1,
    "start": "2023-11-18 14:09:48.013017"
}

STDERR:

+ make -C ../debian-bookworm-build-arm BUILD_ARGS=--pull UPSTREAM_DISTRO=debian:bookworm DISTRO=debian-bookworm IMAGE=osmocom-build/debian-bookworm-build-arm OSMO_TTCN3_BRANCH=master
./scripts/osmo-ci-docker-rebuild.sh: 203: make: not found

Change-Id: I785e4224e729accec15beaa3aef8e01f12624d9d
2023-11-18 15:22:43 +01:00
Harald Welte 506e58eb5b ttcn3-slave: Install sudo
The sudo tool is required by the ansible 'become_user' used later in ttcn3-slave.yml

TASK [osmocom-jenkins-slave : ensure jenkins folder exists] *******************************************
fatal: [lx2-raspbian11build-ansible]: FAILED! => {
    "changed": false,
    "rc": 127
}

MODULE_STDOUT:
/bin/sh: 1: sudo: not found

Related: OS#5873
Change-Id: I2d50ef2dee8e060ff4f84810f2bba62d4801d638
2023-11-18 15:13:18 +01:00
Harald Welte f61843c0e8 hosts: Add build4-obsworker
Change-Id: I0c663fa281042aa98822d27a67add8ba07dca172
2023-11-18 11:38:46 +01:00
Harald Welte ced38b31bc obs-worker: support more recent opensuse versions
We don't want to install new builders with 15.3, and the playbook
should just detect the installed version rather than making assumptions

Change-Id: I96f9896c0eeab120633d5800198d8b93371f7a5a
2023-11-18 11:38:08 +01:00
Oliver Smith 4e3b7babda ansible: qemu: netcat -> netcat-openbsd
Make the ansible rule compatible with debian 12. The same package is
already in debian 11 and debian 10.

Fix for:
  E: Package 'netcat' has no installation candidate

Change-Id: Idcc90ad7a2a104a9b4f5a76c6c89d74604651530
2023-10-19 07:30:20 +00:00
Oliver Smith 367f550da9 ansible: jenkins: debian: add more packages
Add python3-setuptools, as it is needed to run the
update-osmo-python-on-slaves jenkins job.

Add bc, as it is needed in docker-playground.git's network_create() in
jenkins-common.sh.

Related: OS#6186
Change-Id: I1ff0711b7b29ba4dc78f51f257b986d7a0994a96
2023-10-19 07:30:20 +00:00
Oliver Smith 7edd780403 ansible: jenkins: debian: sort packages
Related: OS#6186
Change-Id: If4ce784821cc40a2e795afafa452c99bf0588253
2023-10-19 07:30:20 +00:00
Oliver Smith 119bc1f44a ansible: jenkins: use default-jre-headless
Use default-jre-headless instead of giving the openjdk version, so we
don't need to adjust it to whatever the current version in debian is.

The default-jre-headless package is at least available since debian 10:
https://packages.debian.org/search?suite=buster&arch=any&searchon=names&keywords=default-jre-headless

Related: OS#6186
Change-Id: I6c5c74ff3d75dca2f89765d95c4b4c6d67ace703
2023-10-19 07:30:20 +00:00
Oliver Smith 7d90d5b17f ansible: jenkins: debian: remove qemu package
This is a dummy package that isn't available anymore in debian 12.

https://packages.debian.org/buster/qemu says:
> This package can safely be removed.

We install qemu-user-static and qemu-system-arm, which contain qemu
binaries.

Related: OS#6186
Change-Id: I4d5bd376b1e370d9636b9a2e9e710f5f20b9ae98
2023-10-19 07:30:20 +00:00
Oliver Smith 89c0b50a08 ansible: hosts: add build4-deb12build-ansible
Related: OS#6186
Change-Id: Ia4ce2ab074941dda6b7c820673108d19d4136664
2023-10-19 07:30:20 +00:00
Oliver Smith 45de5a99d9 ansible: known_hosts: update
After deploying with ansible to the jenkins nodes, these entries were
added.

Change-Id: I3e3841d9f1f5581067b7dd90cdfcaf8337700e6f
2023-10-16 11:25:12 +02:00
Oliver Smith 81b714b636 ansible: hosts: remove gtp0-deb(10,11)build
Remove nodes that don't exist anymore according to:
https://jenkins.osmocom.org/jenkins/computer/

Change-Id: I8e8ae59d4e4c8be85bba527ac89cce04b15fcaea
2023-10-16 11:21:56 +02:00
Oliver Smith 975edb5099 ansible: ttcn3: install netcat-openbsd
Make nc available on jenkins nodes running ttcn3 tests, as it is now
used by ttcn3-hss-test-ogs.

Related: docker-playground Iebeefd9689ae2efa54678b99ff929ca255ec64d1
Change-Id: I397d4fe33813b083b61cdabb1d00ab4ea63a456b
2023-10-16 11:17:34 +02:00
Oliver Smith 25db4a639e ansible/util/known_hosts: new file
Add the known hosts file that ansible creates.

Change-Id: Ibef171ac2de5193c2335d4d28ef9347cc88b4c49
2023-09-20 11:09:14 +02:00
Oliver Smith 9aab3f54ad ansible: increase docker_max_image_space
The purpose of the limit is to remove images that are not being used in
a long time. With the current limit of 40 GB it leads to images being
removed just a few hours after they were last used, resulting in lots
of needless rebuilds. Also unexpected errors with images that we just
assume to be there (debian-bookworm-build).

Increases the limit for the nodes that run most of the jobs to 80 GB.

I have verified that the nodes have enough free space.

Related: OS#6183
Change-Id: I8cfb21a86510e43b8347c7f17159575b695ce9ba
2023-09-20 11:06:49 +02:00
Oliver Smith 228efc3f13 ansible: osmocom_build_deps: remove
The builds for almost all jenkins jobs are done in docker now, we
definitely don't need to install all of these dependencies on all
jenkins nodes anymore.

Remove the file for now, so it doesn't look like they still get used and
so we don't need to keep it up-to-date (e.g. inkscape was just replaced
with rsvg-convert).

When we deploy a new jenkins node and notice that we still do need a few
of these dependencies, we can bring them back later.

Change-Id: I7c87bd516d1bcdbcc21f65909dabbd3ccc38eede
2023-08-17 14:01:39 +02:00
Oliver Smith 82ff45e7f7 repo-install-test: add debian12
Related: OS#6057
Change-Id: I51fa3c5212b5e29f4c2804384781693e8de2c097
2023-08-03 11:03:58 +02:00
Oliver Smith 35bf872042 ansible/setup-simtest: don't install deps for docs
The part of the jenkins job that generates documentation doesn't run on
simtester anymore, so no need to install these dependencies anymore.

Related: pysim.git I5245c529db729e209d78a02ab9c917a90d0e0206
Related: OS#5497
Change-Id: I0dc1c9f9fc87ae1832d836d98f06e798b51c7e2e
2023-07-27 15:41:43 +02:00
Oliver Smith cbda91d500 ansible/setup-simtest: add tshark
As requested by Philipp, add tshark to simtest.

Change-Id: Ib330f08057888bab2253a9b38148eaa1a70f3800
2023-07-27 15:35:37 +02:00
Alexander Couzens 703be698bf ansible.cfg: set default remote_user = root
Change-Id: Iccd7dce77ac408d1a0d5782b4563a5dfcf112030
2023-03-25 08:07:42 +00:00
Alexander Couzens 68e1cddb3d ansible: add ssh configuration
Change-Id: Idbc7a51c20125182b1e484ae516e8d72239c1e00
2023-03-25 08:07:42 +00:00
Alexander Couzens 6e1210326b ansible: hosts: replace invalid char minus ("-") in host group names
Change-Id: I0c13ce4edb7f57c7ff77141b5511080bba639ec0
2023-03-23 08:52:06 +00:00
Alexander Couzens 607cedc308 ansible: install-coverity: ensure it's not installed twice
Check if the coverity path already exists. Even this wouldn't
detect if the coverity has been only installed half way.

Related: OS#5801
Change-Id: I95549983bb6bd47e04eb37c73afe5409637f87d3
2023-03-23 08:52:06 +00:00
Alexander Couzens f2e49f051c ansible: install-coverity: add missing tag coverity
Change-Id: I86be4c72ad3bce070ddf7f8270492b6cf8ea826e
2023-03-23 08:52:06 +00:00
Alexander Couzens 6d7432e774 ansible: install-poky-sdk: add tag poky to the apt install task
Change-Id: Id8e433a789aff8df241687f1f109a161e725a29a
2023-03-23 08:52:06 +00:00
Oliver Smith 733626688d ansible/osmocom-jenkins-slave: add libnftables-dev
Install libnftables-dev, so osmo-upf can build successfully in the
coverity jenkins job.

Change-Id: Id72ce8ed5b662085a16f56756f8f93627834d2e2
2023-02-01 10:58:04 +01:00
Alexander Couzens 113b295075 ansible: .gitignore: make poky regex more specific
Otherwise it would also prevent changes to the poky role

Change-Id: I3ac9a6129323f27ba387b44d8293d7b134716af8
2023-01-23 10:57:45 +00:00
Alexander Couzens e64f85a0ee ansible: install-poky-sdk: ensure it's not installed twice
Check if the poky path already exists. Even this wouldn't
detect if the poky has been only installed half way.

Related: OS#5801
Change-Id: I8ff8e72a5974378327d4692341b0a371c667b741
2023-01-23 10:57:30 +00:00
Alexander Couzens 6a290081c5 ansible: replace deprecated include by import_tasks
include_tasks and import_tasks have similar functionality.
Use import_tasks when importing non-variable depending files.
Further use import_tasks when not used within a loop.

Closes: OS#5797
Change-Id: I29766c65b9e1601a20aa8f20a438e8368c65f6a8
2023-01-23 10:57:30 +00:00
Harald Welte c0c48532be ansible inventory: add gtp0-deb11build
we're moving to debian 11 builders, gtp0 didn't have any

Change-Id: I1d387a460a9d71d537ab4790b4bb3177b1b96b88
2022-12-09 13:26:25 +01:00
Harald Welte 25a92ecb01 ansible inventory: We have recently killed gtp0-deb11build-i586
Change-Id: I15cb177e75f73fb8a06b17e42c99a6b573dc7574
2022-12-09 13:26:25 +01:00
Oliver Smith 47dde8559f ansible: add rules to install qemu images
Prepare to run repo-install-test inside qemu with kvm instead of
docker, see following patches.

Related: OS#5365
Change-Id: Ia7626fcd397d32e1180e320d4a76a65bcad01b7b
2022-12-09 10:55:15 +00:00
Oliver Smith 54a36bcb7d ansible/podman: install from debian repos
Podman is in the official debian repositories since debian 10, install
it from there instead of using a third-party repository.

Related: OS#5365
Change-Id: I01549915d2c8dfda5fa605a014a304229c3078f0
2022-12-08 13:04:31 +00:00
Oliver Smith 9bdd99b2ff ansible/hosts: remove debian 9 lxcs
Related: OS#5793
Change-Id: I465d93e5254ba2f65796fcfb62ea1063a0eb72c6
2022-12-06 11:17:16 +01:00
Oliver Smith 6bb5f4ce84 hosts: update coverity-slaves
Remove the deb9 lxcs as they are being retired, and add two deb11 lxcs.
Coverity is quite big with 1.2 GiB, so deploying and upgrading this
takes some time. Having it on two hosts should be enough.

Related: OS#5793
Change-Id: I02d43d7cbdbcba5581aa5da0a1effa3db7b445c0
2022-12-01 13:08:38 +01:00
Oliver Smith b0f36fe227 ansible/coverity: extract into subdirectory
The tarball provided by coverity doesn't have the same version in the
name as in the directory inside the tarball.

Tarball:
	cov-analysis-linux64-2022.06.tar.gz
Directory:
	cov-analysis-linux64-2022.6.0

Work around this by extracting it into a subdirectory and using glob to
match the directory name.

Related: OS#5793
Change-Id: I2b96d1e47f2697706a042937b2852f0fc5032a7b
2022-12-01 13:08:28 +01:00
Oliver Smith 8125b33b9a ansible: coverity: update version to 22.06
Coverity doesn't make the old version available for download and will
probably deprecate the previous version at some point. I'm about to
install it on new hosts, so upgrade to the latest version while at it.

Related: OS#5793
Change-Id: Id5700a9e5a46a9cfafc9959f5b77f9e21dd956b4
2022-12-01 13:08:07 +01:00
Oliver Smith 1a3ee20141 ansible/jenkins: add gerrit ssh key to known_hosts
Add the key to known hosts to prevent host key verification failed
errors when attempting to use the gerrit API from jenkins (to post
review comments) or when attempting to clone over ssh from gerrit (as
we do for some reason apparently).

Change-Id: I50981f201c5ca1425e30677b2578491f5111fc0c
2022-11-28 14:13:34 +01:00
Oliver Smith 9fbcd1c35b ansible/install-poky-sdk: install more depends
Install missing depends file and gcc, and sort the list alphabetically
while at it.

Fixes: OS#5794
Change-Id: Ia0530859d65818e55498bc97af139b8773445dcf
2022-11-28 14:10:50 +01:00
Harald Welte f04bfa8cc5 ansible: add {host2,build3}-deb11build-ansible
The playbook currently doesn't deploy, see https://osmocom.org/issues/5794

Change-Id: I15439bb85c1c03a96231ea195d6ad1679a0a1af1
2022-11-28 12:30:21 +01:00
Oliver Smith 1c7e7c21c3 ansible/setup-simtest: pre-install pysim depends
Avoid spending time to install the depends of pysim in each CI job.

Closes: OS#5497
Change-Id: I1e9ddb93f05108146a219bd28cf1e9332764bf8e
2022-10-20 09:18:25 +00:00
Oliver Smith f9c9069151 ansible/ttcn3-slave: fix adding apt key
Downloading the apt key from obs.osmocom.org returns a 404 error unless
an Accept HTTP header is sent. (Yes, really.) Also using apt-key is
deprecated, so change it to download the key with get_url as shown in
the ansible docs example.

Fix for:
  Failed to download key at https://obs.osmocom.org/projects/osmocom/public_key: HTTP Error 404: Not Found

Related: https://github.com/ansible/ansible/issues/50436#issuecomment-451205256
Related: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_key_module.html#examples
Change-Id: Idb23231d629ba8a75510deace0d1f130e889d535
2022-10-13 10:51:55 +02:00
Oliver Smith ed03bc78ed ansible/hosts: add build2-deb11build-ansible
Add it to the config. I've used this to enable ipv6 in docker, so
libosmocore builds don't fail in the socket test when building rpm/deb
packages on that host in docker.

Related: OS#2385
Change-Id: Ibe8e2a638fcbc7c0a69c211385a870b1a82b6fef
2022-10-13 10:51:55 +02:00
Oliver Smith 7fc58133d6 Replace most remaining git.osmocom.org urls
Replace the legacy git.osmocom.org clone URLs with either gerrit or
gitea in most places. Besides not using legacy urls anymore, the
advantage of using gerrit directly is that we don't have a delay of 10
min to several hours on the repository.

Not replaced:
* ansible/roles/osmocom-workstation: not sure if still used, would need
  a bit of a rewrite to use logic similar to osmo_git_clone_url().

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I90409c9e7f4db4e1765647021625f9acfccaa78b
2022-09-26 15:07:59 +02:00
Harald Welte 5455b42405 update git URLs (git -> https; gitea/gerrit)
Unencrypted git:// protocol offers no integrity or authentication,
making it subject to tampering.  Use https:// instead.

https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
https://blog.readthedocs.com/github-git-protocol-deprecation/

Change-Id: Ia44e4127ff87a3d4d747225d3258188a0a891f1b
2022-08-13 04:02:51 +07:00
Harald Welte d5b6cba4e9 ansible: Use downloads.osmocom.org instead of opensuse.org
Change-Id: Ib60eb87c7b0c17d20bbbfa8b94d8262f0c165b1c
Related: OS#5557
2022-07-13 07:41:55 +00:00
Harald Welte ac694eacb6 ansible/hosts: Replace failed OBS/qemu with real arm machine
We gave up on trying to fix OBS cross-building via qemu, and instead
bought a real ARM64 machine.

Change-Id: I5f3ffe451d029ba459575881bf88f2527fe9fd15
2022-07-13 07:41:55 +00:00