Commit Graph

1111 Commits

Author SHA1 Message Date
Oliver Smith 6bb5f4ce84 hosts: update coverity-slaves
Remove the deb9 lxcs as they are being retired, and add two deb11 lxcs.
Coverity is quite big with 1.2 GiB, so deploying and upgrading this
takes some time. Having it on two hosts should be enough.

Related: OS#5793
Change-Id: I02d43d7cbdbcba5581aa5da0a1effa3db7b445c0
2022-12-01 13:08:38 +01:00
Oliver Smith b0f36fe227 ansible/coverity: extract into subdirectory
The tarball provided by coverity doesn't have the same version in the
name as in the directory inside the tarball.

Tarball:
	cov-analysis-linux64-2022.06.tar.gz
Directory:
	cov-analysis-linux64-2022.6.0

Work around this by extracting it into a subdirectory and using glob to
match the directory name.

Related: OS#5793
Change-Id: I2b96d1e47f2697706a042937b2852f0fc5032a7b
2022-12-01 13:08:28 +01:00
Oliver Smith 8125b33b9a ansible: coverity: update version to 22.06
Coverity doesn't make the old version available for download and will
probably deprecate the previous version at some point. I'm about to
install it on new hosts, so upgrade to the latest version while at it.

Related: OS#5793
Change-Id: Id5700a9e5a46a9cfafc9959f5b77f9e21dd956b4
2022-12-01 13:08:07 +01:00
Oliver Smith fe8bc324d9 jobs/master-builds: more binaries ssh credentials
Make the binaries_ftp.osmocom.org ssh credentials available to more jobs
that need them.

Related: OS#5798
Change-Id: I4f719c6daba06dcfadd252a4030293bdc0e26337
2022-12-01 10:30:13 +01:00
Oliver Smith 2c6f249934 jobs/osmocom-release-tarballs: use any node
Prepare to retire build2-deb9build-ansible. With changes in the script,
it doesn't need a persistent workspace anymore, it skips building source
tarballs if they exist on the remote server.

Depends: docker-playground I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce
Related: OS#5793
Change-Id: If56614caaa9c0c44419af596451d48c1970e9014
2022-11-30 17:46:57 +01:00
Oliver Smith 4546478c64 jobs/osmocom-release-tarballs: update clone url
Use gerrit.osmocom.org so we don't have a delay.

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I4391016fda1584fa7d423521802458dca9b1b7ec
2022-11-30 17:46:57 +01:00
Oliver Smith 5f0876d674 jobs: don't limit to debian9
Related: OS#5793
Change-Id: Idcbbf766928964c4896aef7c6687ea7003203427
2022-11-30 17:46:57 +01:00
Oliver Smith 0aad1883f1 jobs: don't set node where it doesn't make sense
Remove the node attribute as these can run on any node. I've configured
the three deb11 nodes with "Usage: Use this node as much as possible"
instead of previously "Only build jobs with label expression matching
this node", so simply not specifying a node is now possible. Previously
it wouldn't have any node to run on as all of them were configured that
way.

Related: OS#5793
Change-Id: I2d67daf1cc7930649d097af23c18990cc3ae55fd
2022-11-30 17:46:57 +01:00
Oliver Smith 521b83ba3a scripts/osmocom-build-old…: skip openbsc 1.4.{0,1}
Don't attempt to build openbsc 1.4.0 and 1.4.1 in the
osmocom-build-old-tags-against-master job, as it fails to build with a
modern gcc with a potential null pointer dereference warning that gets
treated as error. This was fixed in 1.4.2 with openbsc patch
I93d816a20ba208e9fd32a1fc172a78ccd326e1ba.

A more modern gcc gets used, as this job gets changed to not only run on
jenkins nodes with debian 9 installed, but also more modern versions.
This job does not run in docker, and since it still passes it doesn't
seem worth changing that now.

Related: OS#5793
Change-Id: Ibf4b66195fbd70a68d63929b156a9aac54a699bc
2022-11-30 17:46:43 +01:00
Oliver Smith aa13c6e302 jobs: new naming scheme for ftp.osmocom.org keys
Use new ssh keys for ftp.osmocom.org with consistent naming scheme. It
was not possible to use @ in the ID name for new IDs, so I've used an
underscore instead.

Change-Id: Id0cf70e70944cf1a1963e0046b1470a0485c05f8
2022-11-30 13:03:09 +01:00
Oliver Smith 96f25bf519 jobs/master-builds: use binaries ssh credentials
Allow overriding the ssh agent credentials per repository, and use
binaries_ftp.osmocom.org where needed. Use docs_ftp.osmocom.org instead
of 97634b9d-a867-4f6d-825d-1da99813e93d so it's clear what the key is
for. I generated a new one with the new ID.

Fixes: OS#5798
Change-Id: I821968e09aaac3f6f6178de33fc909f81cb4abcb
2022-11-30 12:56:47 +01:00
Oliver Smith 989fb2ca58 jobs/update-osmo-ci: add build2-deb11build-ansible
Change-Id: Ief90b38d0a92f27c7ec37ffeb09dd3595b9d0baf
2022-11-29 11:42:11 +01:00
Oliver Smith e7c3edc144 jobs/update-osmo-ci: order nodes alphabetically
Make it easier to compare the list with:
https://jenkins.osmocom.org/jenkins/computer/

Change-Id: I6e132c5768bf09271ffcfe8eec8311ff5d7e28df
2022-11-29 11:40:41 +01:00
Oliver Smith 935d742837 jobs/master-builds: pysim: don't use deb9 lxc
Don't require the debian 9 lxc anymore for publishing docs, the ssh key
problem that made this necessary has been solved.

Closes: OS#5767
Change-Id: Ia4ee524db059a978bedc1d92804fb59d7513d2cb
2022-11-28 15:10:31 +01:00
Oliver Smith 310859b21a jobs/master-builds: don't only use debian 9 lxcs
Change-Id: I62bf0b968103724ec676900de6cee65fdb00b9e4
2022-11-28 15:09:33 +01:00
Oliver Smith 40ced5a1a0 jobs/master-builds: pass ssh agent sock to docker
Pass the ssh agent to docker for master builds, so we can publish
manuals and other files generated during the builds. This used to work
with the .ssh directory where each lxc had its own key. But we are
changing that to have the keys in jenkins and use the ssh agent, so it
works without additional configuration when adding new nodes/lxcs (such
as the two deb11 lxcs we just added).

Change-Id: I6dc1fd5d7e81ec135f805562608f06cf03a1baeb
2022-11-28 15:09:33 +01:00
Oliver Smith ffd4e39873 jobs/gerrit: don't only use debian 9 lxcs
Change-Id: I2d14bffb09439e031dec3ab3633a36710434a229
2022-11-28 14:38:52 +01:00
Oliver Smith 5d184fe869 jobs/update-osmo-ci: update list of nodes
* remove host2-deb9build
* add {host2,build3}-deb11build-ansible

Change-Id: I79bbb84cd19df7838a30a954ffa47a2a62517f77
2022-11-28 14:32:08 +01:00
Oliver Smith 1a3ee20141 ansible/jenkins: add gerrit ssh key to known_hosts
Add the key to known hosts to prevent host key verification failed
errors when attempting to use the gerrit API from jenkins (to post
review comments) or when attempting to clone over ssh from gerrit (as
we do for some reason apparently).

Change-Id: I50981f201c5ca1425e30677b2578491f5111fc0c
2022-11-28 14:13:34 +01:00
Oliver Smith 9fbcd1c35b ansible/install-poky-sdk: install more depends
Install missing depends file and gcc, and sort the list alphabetically
while at it.

Fixes: OS#5794
Change-Id: Ia0530859d65818e55498bc97af139b8773445dcf
2022-11-28 14:10:50 +01:00
Harald Welte f04bfa8cc5 ansible: add {host2,build3}-deb11build-ansible
The playbook currently doesn't deploy, see https://osmocom.org/issues/5794

Change-Id: I15439bb85c1c03a96231ea195d6ad1679a0a1af1
2022-11-28 12:30:21 +01:00
Oliver Smith 1263d6e0b9 obs/srcpkg: open5gs: download all subprojects
With recent changes in open5gs, building the source packages we
generated fails:
[   45s] ../lib/metrics/meson.build:52:4: ERROR: Automatic wrap-based subproject downloading is disabled
[   45s] dh_auto_configure: error: cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 meson .. --wrap-mode=nodownload --buildtype=plain --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=lib/x86_64-linux-gnu --libexecdir=lib/x86_64-linux-gnu returned exit code 1

Fix this by downloading all subprojects. This is now possible as I added
a meson.build file to the prometheus-client-c repository that open5gs
uses: https://github.com/open5gs/prometheus-client-c/pull/2

Change-Id: If3910b520382b177a77b216c93771ea88414723d
2022-11-21 16:48:35 +01:00
Oliver Smith b3fa89154b jobs/update-osmo-ci: remove gtp0-deb11build32
The node doesn't have a purpose, remove it (as discussed in related
issue).

Related: OS#5453
Change-Id: I241b6dc91af18be38edaff1bd17dfc9633ca1cec
2022-11-17 11:53:11 +01:00
Oliver Smith 1880077b13 jobs/ttcn3: notify Sukchan for open5gs testsuites
I've asked Sukchan, and he wants to be notified for the testsuites
running against open5gs.

Change-Id: I87650e1f1aa77953ae759b910be03242e3cf3aee
2022-11-17 11:42:29 +01:00
Oliver Smith 9e6c10cd3c jobs/ttcn3: allow overriding email per job
Change-Id: I4d9e08dfbc5c2ad442fc8b4da09009a047f67784
2022-11-17 11:37:10 +01:00
Oliver Smith 60d16b6af9 jobs/ttcn3: send mails to jenkins-notifications ML
Send the mails to the mailing list rather than to only Harald and me, so
it's less likely that failures go unnoticed in case we are both
unavailable.

Change-Id: I4fc554f0624e1ce98cafaf383132241365f1e4d9
2022-11-17 11:31:48 +01:00
Oliver Smith 3c2aac2324 jobs/coverity: remove display-name
Make it consistent with all other jobs and don't set a display name.
This way the job will be sorted properly in the list of jobs and won't
weirdly jump to the top when loading the page.

Change-Id: Idf1a8db45daf75ae110cfae3042627fbcdd14575
2022-11-16 13:32:05 +01:00
Oliver Smith c4924d2657 jobs/osmocom-api: add jenkins-job-builder config
Add a jenkins-job-builder configuration for the existing Osmocom_API job
and refactor it to not use the deprecated multi-scm plugin. Also update
the git clone urls.

Related: OS#5763
Change-Id: Ia518979307ce633ccd2c431a8e8eea866da2a50d
2022-11-16 13:24:51 +01:00
Oliver Smith f560ce08ab jenkins-gerrit: fix ambiguous use of review api
Fix an error when the same Change-Id is used on multiple branches or in
theory projects. This is actually allowed by gerrit, and we use this
e.g. when backporting patches from master.

Use the project, change number (e.g. 30147) and patchset number (e.g. 2)
instead of the Change-Id.

Fix for:
  + ssh -p 29418 -l jenkins gerrit.osmocom.org gerrit review 4835a62cd88f0d69db76fb3bfd2df02176a91a6d --json
  fatal: "4835a62cd88f0d69db76fb3bfd2df02176a91a6d" matches multiple patch sets

Related: https://gerrit-review.googlesource.com/Documentation/cmd-review.html
Change-Id: I2d627f8f3b400fa57a50a228d47df2194f60fd08
2022-11-15 11:38:08 +00:00
Oliver Smith 6b7741400d master-pysim: use specific node for docs upload
Related: OS#5767
Change-Id: I25d799ccd0609732f7f82034f97881f1c75dcdf8
2022-11-14 13:13:34 +01:00
Oliver Smith 27a9092b7c jobs/master…-dahdi: don't use multiple scms plugin
Related: OS#5763
Change-Id: If115116c0d6920c86a50dc43d57a8a925f6db064
2022-11-10 16:45:09 +01:00
Oliver Smith 28ace174fa jobs/gerrit…-dahdi: don't use multiple scms plugin
Don't use the deprecated plugin. Let jenkins clone the big linux
repository and clone the code from gerrit in the shell section of the
script. Since we can't directly clone $GERRIT_REFSPEC, create an empty
git repository and fetch it.

Related: OS#5763
Change-Id: I52314f55e7e55382b4e8fcd63d3aba306b401447
2022-11-10 16:45:09 +01:00
Oliver Smith 8763b1ed25 jobs/gerrit-binpkgs: don't use multiple-scm plugin
Change the default branch from origin/master to master, as the git clone
command expects just the branch name without a remote name.

Related: OS#5763
Change-Id: Ia5a17c6d25bf811a992ab495a2531ae613cb144f
2022-11-10 16:44:20 +01:00
Oliver Smith 8453ebc0a4 jobs/gerrit-lint: don't use multiple-scm plugin
Clone osmo-ci.git in the shell script part to avoid using the no-longer
maintained multiple-scm plugin.

Related: OS#5763
Change-Id: I3e9bdfbbf00a234cd559bbf142432a788cfe8f75
2022-11-10 13:11:56 +01:00
Oliver Smith e4a33ebda2 jenkins-gerrit: display JOB_TYPE infront of url
In the summary comment posted by jenkins to the patch on gerrit, display
the JOB_TYPE infront of the URL instead of "build", if the build job is
a matrix job that uses JOB_TYPE as variable.

For such jobs, it changes:
  [build] https://jenkins.osmocom.org/…
  [build] https://jenkins.osmocom.org/…
  [build] https://jenkins.osmocom.org/…
  [build] https://jenkins.osmocom.org/…
  [lint] https://jenkins.osmocom.org/…
to:
  [manuals] https://jenkins.osmocom.org/…
  [gateware] https://jenkins.osmocom.org/…
  [firmware] https://jenkins.osmocom.org/…
  [software] https://jenkins.osmocom.org/…
  [lint] https://jenkins.osmocom.org/…

JOB_TYPE is used by osmo-e1-hardware and pysim.

Change-Id: I51f49e4799961776dbddaedd76c14ed37a0e6c84
2022-11-07 08:16:23 +00:00
Oliver Smith ac5bae1efa jobs: pysim: split into job types
Split the test, pylint and docs job types in the matrix to run them in
parallel.

Depends: pysim I5245c529db729e209d78a02ab9c917a90d0e0206
Related: OS#5497
Change-Id: Iea4f15fd9c9f8f36cb8d638c48da000eafe746a4
2022-11-07 08:16:23 +00:00
Oliver Smith 288fa2d51e obs: check_builders: at least 10, fail on error
The number of builders connected to the OBS server seems to fluctuate a
bit (used to be 14, then 17 now 16). Make sure that we have at least 10
builders connected. Fix that the test didn't exit with 1 on error.

Related: https://obs.osmocom.org/monitor
Change-Id: Iedd506601a5450550e21bf701309b4ea79a3d897
2022-11-04 15:08:00 +00:00
Harald Welte c70aba5906 ggsn-kernel-gtp: Use my current SSH public key
Change-Id: I64b8c853fd34de4210da43165bebc6f8514b2e1a
2022-11-04 15:07:35 +00:00
Harald Welte 5a5a9ec6e5 ggsn-kernel-gtp: Use new OBS package feed, not old obs.opensuse.org
Change-Id: I1566daa23425973bb4155a40f1cdd6040b494420
2022-11-04 15:07:30 +00:00
Oliver Smith 288fe221e8 lint: enable FUNCTION_WITHOUT_ARGS
Complain if one writes func() instead of func(void).

Change-Id: Ic00041025ac2584f250b5a096eae8fd0d857d9fb
2022-11-04 13:24:10 +00:00
Oliver Smith ae69a1aa67 jobs: osmo-e1-hardware: split into job types
Split the job, so building the software can be done with the regular
docker image instead of the fpga-build one (that doesn't have the needed
dependencies).

Related: osmo-e1-hardware I26e839de419c6eb86bf2a6250a1adf06b761d1fa
Change-Id: I90fbe72c722e491f41b9607d1a6389964d7e43b4
2022-11-04 11:17:59 +00:00
Oliver Smith a3e83d6d40 jobs/master-builds: add timeout_cmd
Add the default timeout of 30 minutes to each job, just like in
gerrit-verifications.yml.

Change-Id: I2ed80258d5f2fbcd8cf27db74db125d9a52b9988
2022-11-04 11:17:59 +00:00
Oliver Smith c53223d047 jobs/obs: don't upload to build.opensuse.org
The transitional phase where we upload to both the old and new OBS
server is over, so just upload to the new server from now on.

Related: OS#5557
Change-Id: If6f87dc6dbb4bafa3abb1d840f8d43cce6432dfd
2022-11-04 08:03:07 +00:00
Oliver Smith 8d7596a311 obs: build_binpkg: add --run-shell-on-error arg
Allow easy inspection of the build dir inside docker on error.

Related: OS#5737
Change-Id: I218c3189d36d24c64cbd7b9eab379aff8ce2bee2
2022-11-03 13:01:43 +01:00
Oliver Smith 13b76cced1 obs: fix building rpms with man pages
Add the "%ext_man .gz" macro from OBS, so building rpms with man pages
does not fail with errors like:
  File not found: /home/user/rpmbuild/BUILDROOT/osmo-ggsn-1.9.0.3.0d3b-0.x86_64/usr/share/man/man8/osmo-ggsn.8

Fixes: OS#5737
Change-Id: Ib6950fb46e9f94aabae98021b215f69838557045
2022-11-03 12:22:45 +01:00
Oliver Smith c4df55dcfd jenkins-gerrit: send mail for V+1 for some users
Add a list of users that do not only get a notification mail when the
gerrit verification failed, but also on success.

Change-Id: I603b8a911c8f17aa726d9e3d5d644ad3262b42dd
2022-10-27 16:43:14 +02:00
Oliver Smith f52c26a695 jobs: add osmocom-obs-check-builders
Add a job that checks if the amount of connected builders to the OBS
server matches what we expect. This should prevent what we had today,
that it only was noticed by chance several days after all builders
failed to connect and the packages were outdated since then.

I'm not sure if there is a proper api to do this check, but I don't
expect the string we check for to change often so this should work fine.

Change-Id: I6e7c1f206551722d6bfe1631b9c1da8d34d85ba8
2022-10-26 08:35:05 +00:00
Oliver Smith 6b17805553 jobs/master-builds: add ice40-usbtrace
Related: OS#6169
Change-Id: I8872f20eabf2975aea602c9e95f66d144f74742d
2022-10-24 17:09:42 +02:00
Oliver Smith 1cd0685b55 jenkins-gerrit: add note for build fails/pkgs pass
One might be wondering what happened if the build job failed, but
building the binary packages succeeded. Since we run 'make check' in
all cases. Add this short explanation:

  The build job(s) failed, but deb/rpm jobs passed.
  We don't enable external/vty tests when building
  packages, so maybe those failed. Check the logs.

Related: OS#2385
Change-Id: I70027ec970a220c3ddfa766302faec7bd8752118
2022-10-21 08:34:15 +00:00
Oliver Smith 1259ab99c6 contrib/jenkins.sh: set-url for existing clones
Make sure that the recent change from git.osmocom.org ->
gerrit.osmocom.org for git clone urls is applied to the existing clones
as well.

Change-Id: I3d9c193bf353ecff0ee8bb4c507de59139fc0925
2022-10-20 14:44:36 +00:00