1
0
Fork 0
Commit Graph

1358 Commits

Author SHA1 Message Date
Oliver Smith 11f9724c39 OBS: add jenkins job to check for new distros
Add a new jenkins job to notify us about new versions of Debian,
Raspbian and Ubuntu becoming available in the openSUSE OBS. This allows
us to consistently follow along and make the repositories available in
the Osmocom OBS. I've decided to check the openSUSE OBS instead of
checking somewhere upstream, because it takes time between a new
release of a distribution and the version becoming available in
openSUSE OBS, and we have another job that syncs the configuration with
that OBS instance.

Related: OS#6163
Change-Id: I0abc49a95197da55f7d83ed4fd1c4ebb6bd14b1e
2023-12-06 06:10:46 +00:00
Oliver Smith 6b136f74d1 OBS: lib.osc.get_projects: print apiurl
Print the apiurl, to make more obvious what is going on when running the
function twice in a row with two different apiurls (as done in the next
patch).

Change-Id: I789d12ce59ff14f526ab0334e78a6b22a063ec06
2023-12-06 06:10:46 +00:00
Oliver Smith 5db0807a64 jobs/gerrit: osmo-upf: don't for build debian 10
osmo-upf currently can't build against debian 10, as the required
libnftables-dev version there is too low and we don't provide a backport
for it. As discussed, disable it for debian 10.

Adjust CI to build against debian 11 instead (in addition to the usual
almalinux 8, debian 12).

Change-Id: I63798d451b66bf728b58b02414c1a44f6156b356
2023-12-05 16:03:28 +01:00
Oliver Smith 046a284e83 jobs/osmocom-obs-sync: add newest Ubuntu, Raspbian
Related: OS#6163
Change-Id: I459bd48b50d80fcd07837377e19e89df21d6c328
2023-12-05 13:58:09 +01:00
Oliver Smith c4ee740d57 jobs: fix undefined parameter log warnings
Adjust all jobs in the gerrit verifications pipeline to use the same
parameters from a new include file, to prevent undefined parameter
warnings in the log.

Fixes: OS#6261
Change-Id: Iadc5cd8996eb4ed86634ceb35829a3e9239e598d
2023-12-04 08:55:28 +01:00
Oliver Smith 2031b3d747 repo-install-test: update package filtering logic
Instead of filtering with several blacklist_* files that must contain
the exact names of packages to be filtered, add a shell function that
uses fnmatch for filtering. Combine all lists into one.

This fixes the error we get with each limesuite release, without the
need to increase the version in the txt files every time. Currently the
repo-install-tests are failing for all debian versions because of this.

Change-Id: I6745b10804685119d68b089f129ec9b0cde8cdf5
2023-11-27 07:09:33 +00:00
Daniel Willmann de1651b1bf scripts/obs: Add -M/--no-meta option to avoid depending on osmocom-*
This is useful for testing one-off dev packages

Change-Id: Id18c75de559c9ba29efd38d8510f2db3206c4209
2023-11-24 10:00:57 +01:00
Oliver Smith a13ce691d1 scripts/docker-cleanup: buildkit cache too
In newer docker versions, a buildkit cache was introduced. It gets used
while building images. Clean it up as well.

Related: https://osmocom.org/projects/osmocom-servers/wiki/Docker_cache_clean_up
Change-Id: Icf5237def75d4bcef6b0065f3f1f1da2ff362322
2023-11-21 13:01:03 +00:00
Harald Welte b301da1369 ansible: add raspbian11build container on lx2 machine
Change-Id: I739e40cf6ef8d77585dfeb067ed75e258d94bbb3
Related: OS#5873
2023-11-18 15:22:43 +01:00
Harald Welte a05542cf58 ansible/jenkins-slave/osmo-ci: Install 'make' required for osmo-ci
TASK [osmocom-jenkins-slave : rebuild osmocom docker images for jenkins build tests] ******************
fatal: [lx2-raspbian11build-ansible]: FAILED! => {
    "changed": true,
    "cmd": "./scripts/osmo-ci-docker-rebuild.sh",
    "delta": "0:00:04.129667",
    "end": "2023-11-18 14:09:52.142684",
    "rc": 1,
    "start": "2023-11-18 14:09:48.013017"
}

STDERR:

+ make -C ../debian-bookworm-build-arm BUILD_ARGS=--pull UPSTREAM_DISTRO=debian:bookworm DISTRO=debian-bookworm IMAGE=osmocom-build/debian-bookworm-build-arm OSMO_TTCN3_BRANCH=master
./scripts/osmo-ci-docker-rebuild.sh: 203: make: not found

Change-Id: I785e4224e729accec15beaa3aef8e01f12624d9d
2023-11-18 15:22:43 +01:00
Harald Welte 506e58eb5b ttcn3-slave: Install sudo
The sudo tool is required by the ansible 'become_user' used later in ttcn3-slave.yml

TASK [osmocom-jenkins-slave : ensure jenkins folder exists] *******************************************
fatal: [lx2-raspbian11build-ansible]: FAILED! => {
    "changed": false,
    "rc": 127
}

MODULE_STDOUT:
/bin/sh: 1: sudo: not found

Related: OS#5873
Change-Id: I2d50ef2dee8e060ff4f84810f2bba62d4801d638
2023-11-18 15:13:18 +01:00
Harald Welte f61843c0e8 hosts: Add build4-obsworker
Change-Id: I0c663fa281042aa98822d27a67add8ba07dca172
2023-11-18 11:38:46 +01:00
Harald Welte ced38b31bc obs-worker: support more recent opensuse versions
We don't want to install new builders with 15.3, and the playbook
should just detect the installed version rather than making assumptions

Change-Id: I96f9896c0eeab120633d5800198d8b93371f7a5a
2023-11-18 11:38:08 +01:00
Harald Welte 26ae81f6bd obs-mirror: rsync over from remote server via rsync
We no longer run the archive on the same machine/lxc as the normal
downloads server

Change-Id: Id490690394db9eb773f945ad4d7e65fe7fd5e17a
Related: OS#6257
2023-11-15 15:17:07 +01:00
Harald Welte 0afdbcf8bd obs-mirror: keep nightly builds for two years
As we're moving the archive to a machine with sufficient storage,
let's extend the time we keep an archive of nightly builds from
one month to two years.

Change-Id: Ic8529ee5679f15c4e4940a5f095374605c2e5c79
Related: OS#6257
2023-11-14 21:23:59 +01:00
Oliver Smith 38707a473c tarballs: git archive: make proper tar.bz2
Compress the tarball from "git archive" with bzip2 to turn it into a
proper .tar.bz2. Previously the archive was named .tar.bz2, but it was
just a regular tarball as git archive didn't guess the type from the
filename (supposedly this works for .tar.gz and some others) and just
defaulted to tar.

Move the code that adds build_dep.tar.gz to the archive before the bzip2
call, because tar can't add files to the archive if it is compressed.

Change-Id: I5830f9123150540a6467c19c64461d2fa1972a52
2023-11-07 12:23:20 +01:00
Oliver Smith 0c79a15162 tarballs: git archive: put files in directory
Make the releases created from the "git archive" code path more similar
to the autotools releases, by having all files in a subdirectory with
the project name and tag name instead of having them directly in the
root of the tarball.

  $ tree osmo_dia2gsup-0.1.1 | head
  osmo_dia2gsup-0.1.1
  ├── build_dep.tar.gz
  ├── contrib
  │   ├── generate_build_dep.sh
  │   ├── jenkins.sh
  │   └── systemd
  │       └── osmo_dia2gsup.service
  ├── debian
  │   ├── changelog
  │   ├── compat

Change-Id: If37dcdc04278de0d4479cf72e0ca33c90b7deea8
2023-11-07 11:56:37 +01:00
Oliver Smith 88d62e3c69 tarballs: add osmo_dia2gsup
Change-Id: I9cbd3d3ce27a4b4c9c8e19b2876386d170d4f0a6
2023-11-07 11:40:06 +01:00
Oliver Smith a9e9c366b1 repo-install-test: ignore limesuite-images
Don't attempt to install limesuite-images. It runs a post-install script
that downloads files from an external server and fails currently, as
there are no images for 23.10. While we have limesuite-images packages
in our OBS repository, this is just a side-effect of building limesuite.
What we are really interested in is liblimesuite for osmo-trx, as I
understand.

Add --no-install-recommends to the apt-get install line in
install_repo_packages_debian, because the main limesuite package has
limesuite-images in recommends and would pull it in otherwise.

Change-Id: I237408c805977c831f352a57a301ea45753d1ac1
2023-11-02 21:42:19 +00:00
Oliver Smith 8ba48eb28a repo-install-test: fix ca-certificates-java error
The repo-install-test started to fail on debian 10 with the following
error. Apparently this happens when installing ca-certificates-java
after installing java. Add a workaround.

  Setting up ca-certificates-java (20190405) ...
  head: cannot open '/etc/ssl/certs/java/cacerts' for reading: No such file or directory
  Exception in thread "main" java.lang.InternalError: Error loading java.security file

Change-Id: I00b9c97d9d85fb37ba33a48caa732cd50de99683
2023-11-02 21:42:19 +00:00
Oliver Smith 25f74d3d51 Update git tag patterns for rtl-sdr, osmo-fl2k
With the v2.0.0 release of rtl-sdr, it was decided to include the v in
the tag pattern again, so adjust it in the related scripts. Adjust
osmo-fl2k too in the OBS scripts because there the tag pattern was not
set.

Change-Id: If1bb64722bd24b0079d2427ea3b0f5b5c87ef7ac
2023-11-02 13:53:22 +01:00
Oliver Smith 867f2d28fd OBS: add workaround for limesuite 23.10 release
The previous workaround for building on debian 12 is not needed anymore,
as the related patch has been merged. Replace it with a new workaround
needed for building the limesuite 23.10 release.

Change-Id: I569642ebb13ac3eb3e241642e299be0a0a2ab15d
2023-10-31 12:04:33 +01:00
Oliver Smith b365c1df68 scripts/tarballs: gapk: add install depends
Now that we run ./configure before building the tarball, make sure the
dependencies are installed.

Related: OS#6227
Change-Id: I82eb0a87f6e294fb258003ebe76f850a99952938
2023-10-26 12:31:52 +02:00
Oliver Smith b59c791d13 scripts/tarballs: gapk > v1.0: use make dist-bzip2
Use autotools logic to build tarballs for gapk above version v1.0.

Related: OS#6227
Change-Id: If9a04b458553e311cf8b891b204244db2b1ae2d8
2023-10-26 11:05:22 +02:00
Oliver Smith 3d2c3dd14c scripts/repo-install-test: fix sftp compat, part 2
Copy files inside repo-install-test, instead of copying the directory
again. Otherwise it creates a /repo-install-test/repo-install-test/ path
inside qemu and then fails to run the script inside the directory:

Fix for:
  sh: 0: cannot open /repo-install-test/run-inside-env.sh: No such file

Fixes: a7791217 ("scripts/repo-install-test: fix sftp compat")
Change-Id: I04d8c0e0fd31ee5d2a0912300dc93f5339bbef11
2023-10-23 12:57:54 +02:00
Oliver Smith a779121725 scripts/repo-install-test: fix sftp compat
When running scp, Debian 12 uses the SFTP protocol by default instead of
the legacy SCP protocol. For some reason, this means it doesn't create
the /repo-install-test directory inside QEMU:

  + … scp … -r scripts/repo-install-test root@127.0.0.1:/repo-install-test
  Warning: Permanently added '[127.0.0.1]:22793' (ED25519) to the list of known hosts.
  scp: realpath /repo-install-test: No such file
  scp: upload "/repo-install-test": path canonicalization failed
  scp: failed to upload directory scripts/repo-install-test to /repo-install-test

Changing to the legacy protocol with "scp -O" would make it work again,
but instead of doing that add the mkdir so it works with the new
protocol too.

Related: OS#6186
Change-Id: I740affad2ea794094c8faa89a89bcad094e0075d
2023-10-20 12:19:30 +02:00
Oliver Smith 48b3edb0cc scripts/tarballs: create release dir if missing
Fix for:
rsync: [Receiver] change_dir#3 "/downloads/home/releases/web-files/gapk" failed: No such file or directory (2)

Change-Id: I46310d633302028d486dd356147c9c4f35b24940
2023-10-19 11:37:53 +02:00
Oliver Smith 7fbb49f87d scripts/tarballs: add osmo-fl2k, rtl-sdr
Closes: OS#6226, OS#6225
Change-Id: Idff299c65902b6818e409a409a3d303e8fccc4f9
2023-10-19 11:09:43 +02:00
Oliver Smith 5597e82d55 scripts/tarballs: add gapk
Closes: OS#6227
Change-Id: I6f26b6f5953701b383acb3b93058b66a286f7989
2023-10-19 11:07:06 +02:00
Oliver Smith 4e0a34b26e scripts/common: osmo_git_last_commit…: add vars
Add variable names for $1, $2, $3 so the logic is easier to follow.

Related: OS#6227
Change-Id: I7abbcefc8d5bd4c5b84c30a7e38ca3dcb855dd56
2023-10-19 11:07:06 +02:00
Oliver Smith a498ca6f2a scripts/tarballs: add osmo-e1-recorder
Closes: OS#6228
Change-Id: I6bd8387f99c258226125b33705645c43aadd875f
2023-10-19 11:02:18 +02:00
Oliver Smith 4e3b7babda ansible: qemu: netcat -> netcat-openbsd
Make the ansible rule compatible with debian 12. The same package is
already in debian 11 and debian 10.

Fix for:
  E: Package 'netcat' has no installation candidate

Change-Id: Idcc90ad7a2a104a9b4f5a76c6c89d74604651530
2023-10-19 07:30:20 +00:00
Oliver Smith 367f550da9 ansible: jenkins: debian: add more packages
Add python3-setuptools, as it is needed to run the
update-osmo-python-on-slaves jenkins job.

Add bc, as it is needed in docker-playground.git's network_create() in
jenkins-common.sh.

Related: OS#6186
Change-Id: I1ff0711b7b29ba4dc78f51f257b986d7a0994a96
2023-10-19 07:30:20 +00:00
Oliver Smith 7edd780403 ansible: jenkins: debian: sort packages
Related: OS#6186
Change-Id: If4ce784821cc40a2e795afafa452c99bf0588253
2023-10-19 07:30:20 +00:00
Oliver Smith 6d4b0effa6 jobs/update-osmo-*: add build4-deb12build-ansible
Related: OS#6186
Change-Id: I1f732718df87ed3b850332dbf1a7f6b2f854f280
2023-10-19 07:30:20 +00:00
Oliver Smith 119bc1f44a ansible: jenkins: use default-jre-headless
Use default-jre-headless instead of giving the openjdk version, so we
don't need to adjust it to whatever the current version in debian is.

The default-jre-headless package is at least available since debian 10:
https://packages.debian.org/search?suite=buster&arch=any&searchon=names&keywords=default-jre-headless

Related: OS#6186
Change-Id: I6c5c74ff3d75dca2f89765d95c4b4c6d67ace703
2023-10-19 07:30:20 +00:00
Oliver Smith 7d90d5b17f ansible: jenkins: debian: remove qemu package
This is a dummy package that isn't available anymore in debian 12.

https://packages.debian.org/buster/qemu says:
> This package can safely be removed.

We install qemu-user-static and qemu-system-arm, which contain qemu
binaries.

Related: OS#6186
Change-Id: I4d5bd376b1e370d9636b9a2e9e710f5f20b9ae98
2023-10-19 07:30:20 +00:00
Oliver Smith 89c0b50a08 ansible: hosts: add build4-deb12build-ansible
Related: OS#6186
Change-Id: Ia4ce2ab074941dda6b7c820673108d19d4136664
2023-10-19 07:30:20 +00:00
Pau Espin 7be78df106 jobs/ttcn3-testsuites.yml: Add ttcn3-epdg-test
Related: OS#6204
Change-Id: I4b566c1e767c7a6d6cb474a257b0a2632a2afe92
2023-10-18 12:29:16 +02:00
Oliver Smith 45de5a99d9 ansible: known_hosts: update
After deploying with ansible to the jenkins nodes, these entries were
added.

Change-Id: I3e3841d9f1f5581067b7dd90cdfcaf8337700e6f
2023-10-16 11:25:12 +02:00
Oliver Smith 81b714b636 ansible: hosts: remove gtp0-deb(10,11)build
Remove nodes that don't exist anymore according to:
https://jenkins.osmocom.org/jenkins/computer/

Change-Id: I8e8ae59d4e4c8be85bba527ac89cce04b15fcaea
2023-10-16 11:21:56 +02:00
Oliver Smith 975edb5099 ansible: ttcn3: install netcat-openbsd
Make nc available on jenkins nodes running ttcn3 tests, as it is now
used by ttcn3-hss-test-ogs.

Related: docker-playground Iebeefd9689ae2efa54678b99ff929ca255ec64d1
Change-Id: I397d4fe33813b083b61cdabb1d00ab4ea63a456b
2023-10-16 11:17:34 +02:00
Pau Espin 35a1155a42 jobs/ttcn3-testsuites.yml: Add ttcn3-hss-test-ogs
Related: SYS#6588
Change-Id: Ic98a5a22b2c10b32052e613d0a46fbc7640c60d9
2023-10-10 19:26:08 +02:00
Oliver Smith 15777116dd obs: build_binpkg: no network during build
Simulate that we don't have network during package builds in gerrit
verifications, like it is the case when the packages will be built by
OBS.

I've also considered using network namespaces via "unshare -n" or
bubblewrap, but it wouldn't be possible to create a loopback device
inside the network namespace inside docker (maybe with --privileged, but
not worth it) and e.g. the osmo-mgw tests rely on lo.

Depends: osmo-epdg I2bb0b5f608cd5d9a24c2a367425c524447bd002c
Change-Id: I00282552e9988d66fdc269c6230f0e1432ace360
2023-10-06 14:05:51 +02:00
Oliver Smith 2e809f4637 jobs/master-builds: add erlang projects
Add the erlang projects found in gerrit-verifications to master-builds
too, as we have it with most other Osmocom projects.

Change-Id: I6cf5a3c1e52ee73ad63eb2d7d5b1af19a9809026
2023-10-06 08:21:48 +00:00
Oliver Smith 193e2a4ce6 jobs/master: use same url scheme as gerrit jobs
Make it easier to copy entries from gerrit-verifications.yml to
master-builds.yml by using the same url schema in both.

Change-Id: Ibe992dd8027a8b8df8623abc57a590972443449f
2023-10-06 08:21:48 +00:00
Oliver Smith f8e2ee24ed obs: srcpkg: support having no git tags
Instead of failing when a project has no git-version-gen script, and no
git tags, use 0.0.0 as version. This is needed for the new osmo-epdg
project, which doesn't have tags yet.

Change-Id: I2f0f409feb96611095dfa511a33ea730e5bbf3e4
2023-10-05 18:05:41 +02:00
Pau Espin 76a67bc8d7 obs: Fix typo in log line informing variable to be modified
Change-Id: I381f30ad2443bbd86a23d2808a5f8e3c650953b7
2023-10-05 17:36:42 +02:00
Pau Espin 325162a1fe obs: add erlang/osmo-epdg to projects_osmocom
Change-Id: I5c5537e2d96752cb68fb251552bd444a9abf8cd1
2023-10-05 17:36:42 +02:00
Oliver Smith 3f037f4a73 obs: srcpkg.py: Run generate_build_dep.sh for all projects
Adjust srcpkg.py to run a generate_build_dep.sh not only for
osmo_dia2gsup, but for any Osmocom project that has such a script. This
is in preparation for osmo-epdg, which will also have one.

Change-Id: Icd9dc734ec938398668640f77ec83aae4a1ee0f4
2023-10-05 17:35:02 +02:00