Commit Graph

448 Commits

Author SHA1 Message Date
Oliver Smith 30ccbd3f70 jobs/ttcn3: don't block on same testsuite running
Don't block jobs if other jobs with the same testsuite are already
running. This used to be necessary as we had subnets hardcoded per
testsuite, and therefore couldn't run it twice on the same host (e.g.
once against latest, once against nightly).

Related: OS#5802
Depends: docker-playground I57152b08ef0f38e17e7019a8df032189b03f56cf
Change-Id: I3159403e2ce2ec184ee48b4ff4f145e718e9b428
2023-01-16 15:23:57 +00:00
Oliver Smith b44f768b7b jobs/gerrit-verifications: osmo-trx: fix matrix
Add the missing comma between "--without-sse, --without-neon", so the
matrix works as expected. Without this patch, the two builds with these
flags didn't run.

Change-Id: I51e58765dd33b15df2e9f2eb09c1260724e4ec9c
2023-01-16 12:55:21 +01:00
Oliver Smith 48614942c6 jobs/master-builds: use osmo-iuh/master for asn1c
As discussed with Pau, let's build the "osmo-iuh/master" branch of our
asn1c.git repository in jenkins. This branch is what we actually use for
osmo-iuh and "master" should track upstream master.

Related: OS#2435
Change-Id: Ic97a23a175b301e6f42f7ca67deb84dcc3ee3f90
2022-12-27 17:02:38 +00:00
Oliver Smith b6d36134a7 jobs/update-osmo-ci-on-slaves-dp: fix node
This trigger job can run on any node, don't have it require a debian9
label that doesn't exist anymore.

Change-Id: I1a23a4c99fe007387f13e330b0b08f6941392829
2022-12-23 11:01:32 +00:00
Harald Welte 14772790d6 ttcn3-testsuites-kernel-git: Use normal e-mail address
There's no need for a hard-coded special list of e-mail addresses for
kernel related tests.

Change-Id: Ida4313997c9ecee13af22414e6a1799a2d9dd2af
2022-12-22 08:54:53 +01:00
Harald Welte d2bcdd73d1 don't report every unstable TTCN3 build
This saves deleting >= 80 e-mails every day.

Closes: OS#5841
Change-Id: I9593a3c3a94ab3f66ed4cd92906e7428a9d04bf5
2022-12-22 08:54:43 +01:00
Oliver Smith 50a54c923b jobs/update-osmo-ci-on-slaves: don't run on ogt
Don't run this job on the osmo-gsm-tester machines. Right now this job
takes quite some time to build various docker images which we don't even
need on osmo-gsm-tester. It also uploads osmo-ci.git and
osmo-gsm-manuals.git, which don't need to be updated on these if we
don't run the osmo-gsm-tester jenkins jobs on them (see previous
commit). According to Pau they would be needed for
osmo-gsm-tester_ttcn3.

Related: OS#5811
Change-Id: I39d72740d896981f069ad339013c1e1e3d4f3bba
2022-12-14 17:29:08 +01:00
Oliver Smith 9086b10d8d jobs/ogt: disable _run-prod, _run-rnd, _ttcn3
Disable these osmo-gsm-tester jobs, as they have been failing for a long
time and we currently won't spend time on fixing them.

Change-Id: I8db4fbe2c695e28eb9f7e738c5a848fb0071714a
2022-12-14 15:11:19 +01:00
Oliver Smith 4f1201ff06 jobs/master-builds,gerrit-verifications: fix node
Do not set a node for master-builds and gerrit-verifications to run on,
as this leads to needlessly blocking slots on the jenkins nodes.

Each of these jobs is a matrix job with a slave_axis. The matrix job
itself can run on the "builtin" node, as it practically doesn't cause
load - all it does is start other jobs in the matrix (even if there is
only one other job, which is the default). Jobs started in the matrix
job do run on a configured node already, as the slave_axis parameter is
set.

I've just noticed this problem because currently the raspberry pi nodes
are offline, and so the master-osmo-trx, master-osmo-pcu and other jobs
take up lots of build slots just to wait until the raspberry pi nodes
come online and an actual build can start.

Change-Id: I447dd152fcc942ec48d0bcfbadd496e879c882da
2022-12-12 12:23:47 +01:00
Oliver Smith d5106e3d25 jobs/repo-install-test: adjust for qemu
Related: OS#5365
Change-Id: Ie7675811d1a5d6b7cf7e4378c86d747ef6f7a249
2022-12-09 11:56:03 +01:00
Oliver Smith 5d3fb63b3d jobs/README: modernize
Remove outdated debian 9 related information, update section on labels.

Change-Id: I6616f4bdacc90c295db7996855e9aa2b48172b43
2022-12-06 11:17:16 +01:00
Oliver Smith 3b5464d0ed jobs: retire osmocom-gerrit-debian* labels
Let the gerrit-binpkgs and gerrit-pipeline-result jobs use the
"osmocom-gerrit" label. Having the -debian10/11 label previously was
useful to make sure this doesn't run on an ancient python version that
fails on f-strings. But since the debian9 builders were retired, this is
not needed anymore. These are the only jobs that used these, so now the
labels in the README can be simplified (following patch).

Related: OS#5793
Change-Id: Ieee95d25e036dcd4869d89b10c281179024672e7
2022-12-06 11:17:16 +01:00
Oliver Smith fd7c88e992 jobs/update-osmo-ci: remove deb9 nodes
Related: OS#5793
Change-Id: Id2bf3c273848c1de12a3dce3bc007dd205bf4af7
2022-12-06 11:17:16 +01:00
Oliver Smith 0428da0f6d jobs/osmo-gsm-tester virtual: fix permissions
The docker container creates the trial dir and some files inside of it
as root, therefore the rm -rf trial-* line fails. Add logic to fix the
permissions before attempting to remove old trial-* and when the script
exits.

Related: OS#5793
Change-Id: I91bd6656f33667f401247721bb34298a6c66fc62
2022-12-02 10:50:07 +01:00
Oliver Smith d5e0fe011d jobs/osmo-gsm-tester virtual: send mails on error
Change-Id: If91644a3dfea1af259f7a511a7784cf32d8d23ea
2022-12-01 16:41:10 +01:00
Oliver Smith 23cf9b4b05 jobs/osmo-gsm-tester: remove old test result early
The virtual osmo-gsm-tester has been failing for a long time, but the
test results were still green. This was because the job failed before it
removed old test results, and so those were used. Remove the old results
as early as possible to prevent this in the future.

Change-Id: I23f2f7b24fe41bbded2e05abbe6ea5e717b88e9e
2022-12-01 15:53:30 +01:00
Oliver Smith 4bd7da1c9d jobs/osmo-gsm-tester virtual: fix opts desciption
The default value is what it says in the line above, not empty.

Change-Id: I5ce8a47b43525dcca6bf54184a48f11b0c87c19e
2022-12-01 15:37:18 +01:00
Oliver Smith d3321f9434 jobs/osmo-gsm-tester virtual: don't limit to deb9
Related: OS#5793
Change-Id: I8e7ffa65b4ff6d1816ad167367b0f6ff8c00e64b
2022-12-01 15:04:38 +01:00
Oliver Smith ce5875fd07 jobs/osmo-gsm-tester virtual: fix docker img
Build the docker image right before using it, instead of building it in
another jenkins job update-osmo-ci-on-slaves via
osmo-ci-docker-rebuild.sh.

The logic in osmo-ci-docker-rebuild.sh was broken. I didn't realize at
the time that this image is only used for the virtual osmo-gsm-tester,
not the physical ones. But only the machines running the physical
osmo-gsm-tester have the /var/tmp/osmo-gsm-tester/state path. The
virtual osmo-gsm-tester isn't running on these machines but on generic
jenkins nodes.

Building the image right before using it makes sense for this job, as it
is the only user of the image. If it was already built from the same
Dockerfile, a cached version is used.

Fix for:
  Unable to find image 'osmocom-build/osmo-gsm-tester:latest' locally

Fixes: 9139e76b ("osmo-ci-docker-rebuild: don't always build osmo-gsm-tester")
Change-Id: Icad9459de1d3a3a4e65ecacf7f903433bb504cc9
2022-12-01 15:04:38 +01:00
Oliver Smith cc7af6cd5a jobs/osmo-gsm-tester virtual: use literal style
Use literal style instead of folded style in the shell argument of
osmo-gsm-tester_virtual as otherwise new lines behave unintuitively.

Related: https://yaml-multiline.info/
Change-Id: I862a0ed24bae2c713e857a459797e1a978df0560
2022-12-01 15:04:38 +01:00
Oliver Smith 3307a90691 jobs/osmo-gsm-tester virtual: fix default branch
Change it to "master" instead of "origin/master" to prepare for building
the docker image right before running it. The Makefile from
docker-playground.git picks up the OSMO_GSM_TESTER_BRANCH environment
variable and expects it to be in that format.

Change-Id: I4ad1ce71fd5b5b57a65267e0be51755f7495b07d
2022-12-01 15:04:38 +01:00
Oliver Smith 92b636d295 jobs/osmo-gsm-tester: clone from gerrit git url
Use the gerrit git url instead of the gitea one so we don't have a
sync delay.

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I586fdd55ded6c3776cacd6760d88aa455da22990
2022-12-01 15:04:38 +01:00
Oliver Smith fe8bc324d9 jobs/master-builds: more binaries ssh credentials
Make the binaries_ftp.osmocom.org ssh credentials available to more jobs
that need them.

Related: OS#5798
Change-Id: I4f719c6daba06dcfadd252a4030293bdc0e26337
2022-12-01 10:30:13 +01:00
Oliver Smith 2c6f249934 jobs/osmocom-release-tarballs: use any node
Prepare to retire build2-deb9build-ansible. With changes in the script,
it doesn't need a persistent workspace anymore, it skips building source
tarballs if they exist on the remote server.

Depends: docker-playground I4b8c149c9cdbe7c613eea5d9be15794de5e5ddce
Related: OS#5793
Change-Id: If56614caaa9c0c44419af596451d48c1970e9014
2022-11-30 17:46:57 +01:00
Oliver Smith 4546478c64 jobs/osmocom-release-tarballs: update clone url
Use gerrit.osmocom.org so we don't have a delay.

Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I4391016fda1584fa7d423521802458dca9b1b7ec
2022-11-30 17:46:57 +01:00
Oliver Smith 5f0876d674 jobs: don't limit to debian9
Related: OS#5793
Change-Id: Idcbbf766928964c4896aef7c6687ea7003203427
2022-11-30 17:46:57 +01:00
Oliver Smith 0aad1883f1 jobs: don't set node where it doesn't make sense
Remove the node attribute as these can run on any node. I've configured
the three deb11 nodes with "Usage: Use this node as much as possible"
instead of previously "Only build jobs with label expression matching
this node", so simply not specifying a node is now possible. Previously
it wouldn't have any node to run on as all of them were configured that
way.

Related: OS#5793
Change-Id: I2d67daf1cc7930649d097af23c18990cc3ae55fd
2022-11-30 17:46:57 +01:00
Oliver Smith aa13c6e302 jobs: new naming scheme for ftp.osmocom.org keys
Use new ssh keys for ftp.osmocom.org with consistent naming scheme. It
was not possible to use @ in the ID name for new IDs, so I've used an
underscore instead.

Change-Id: Id0cf70e70944cf1a1963e0046b1470a0485c05f8
2022-11-30 13:03:09 +01:00
Oliver Smith 96f25bf519 jobs/master-builds: use binaries ssh credentials
Allow overriding the ssh agent credentials per repository, and use
binaries_ftp.osmocom.org where needed. Use docs_ftp.osmocom.org instead
of 97634b9d-a867-4f6d-825d-1da99813e93d so it's clear what the key is
for. I generated a new one with the new ID.

Fixes: OS#5798
Change-Id: I821968e09aaac3f6f6178de33fc909f81cb4abcb
2022-11-30 12:56:47 +01:00
Oliver Smith 989fb2ca58 jobs/update-osmo-ci: add build2-deb11build-ansible
Change-Id: Ief90b38d0a92f27c7ec37ffeb09dd3595b9d0baf
2022-11-29 11:42:11 +01:00
Oliver Smith e7c3edc144 jobs/update-osmo-ci: order nodes alphabetically
Make it easier to compare the list with:
https://jenkins.osmocom.org/jenkins/computer/

Change-Id: I6e132c5768bf09271ffcfe8eec8311ff5d7e28df
2022-11-29 11:40:41 +01:00
Oliver Smith 935d742837 jobs/master-builds: pysim: don't use deb9 lxc
Don't require the debian 9 lxc anymore for publishing docs, the ssh key
problem that made this necessary has been solved.

Closes: OS#5767
Change-Id: Ia4ee524db059a978bedc1d92804fb59d7513d2cb
2022-11-28 15:10:31 +01:00
Oliver Smith 310859b21a jobs/master-builds: don't only use debian 9 lxcs
Change-Id: I62bf0b968103724ec676900de6cee65fdb00b9e4
2022-11-28 15:09:33 +01:00
Oliver Smith 40ced5a1a0 jobs/master-builds: pass ssh agent sock to docker
Pass the ssh agent to docker for master builds, so we can publish
manuals and other files generated during the builds. This used to work
with the .ssh directory where each lxc had its own key. But we are
changing that to have the keys in jenkins and use the ssh agent, so it
works without additional configuration when adding new nodes/lxcs (such
as the two deb11 lxcs we just added).

Change-Id: I6dc1fd5d7e81ec135f805562608f06cf03a1baeb
2022-11-28 15:09:33 +01:00
Oliver Smith ffd4e39873 jobs/gerrit: don't only use debian 9 lxcs
Change-Id: I2d14bffb09439e031dec3ab3633a36710434a229
2022-11-28 14:38:52 +01:00
Oliver Smith 5d184fe869 jobs/update-osmo-ci: update list of nodes
* remove host2-deb9build
* add {host2,build3}-deb11build-ansible

Change-Id: I79bbb84cd19df7838a30a954ffa47a2a62517f77
2022-11-28 14:32:08 +01:00
Oliver Smith b3fa89154b jobs/update-osmo-ci: remove gtp0-deb11build32
The node doesn't have a purpose, remove it (as discussed in related
issue).

Related: OS#5453
Change-Id: I241b6dc91af18be38edaff1bd17dfc9633ca1cec
2022-11-17 11:53:11 +01:00
Oliver Smith 1880077b13 jobs/ttcn3: notify Sukchan for open5gs testsuites
I've asked Sukchan, and he wants to be notified for the testsuites
running against open5gs.

Change-Id: I87650e1f1aa77953ae759b910be03242e3cf3aee
2022-11-17 11:42:29 +01:00
Oliver Smith 9e6c10cd3c jobs/ttcn3: allow overriding email per job
Change-Id: I4d9e08dfbc5c2ad442fc8b4da09009a047f67784
2022-11-17 11:37:10 +01:00
Oliver Smith 60d16b6af9 jobs/ttcn3: send mails to jenkins-notifications ML
Send the mails to the mailing list rather than to only Harald and me, so
it's less likely that failures go unnoticed in case we are both
unavailable.

Change-Id: I4fc554f0624e1ce98cafaf383132241365f1e4d9
2022-11-17 11:31:48 +01:00
Oliver Smith 3c2aac2324 jobs/coverity: remove display-name
Make it consistent with all other jobs and don't set a display name.
This way the job will be sorted properly in the list of jobs and won't
weirdly jump to the top when loading the page.

Change-Id: Idf1a8db45daf75ae110cfae3042627fbcdd14575
2022-11-16 13:32:05 +01:00
Oliver Smith c4924d2657 jobs/osmocom-api: add jenkins-job-builder config
Add a jenkins-job-builder configuration for the existing Osmocom_API job
and refactor it to not use the deprecated multi-scm plugin. Also update
the git clone urls.

Related: OS#5763
Change-Id: Ia518979307ce633ccd2c431a8e8eea866da2a50d
2022-11-16 13:24:51 +01:00
Oliver Smith f560ce08ab jenkins-gerrit: fix ambiguous use of review api
Fix an error when the same Change-Id is used on multiple branches or in
theory projects. This is actually allowed by gerrit, and we use this
e.g. when backporting patches from master.

Use the project, change number (e.g. 30147) and patchset number (e.g. 2)
instead of the Change-Id.

Fix for:
  + ssh -p 29418 -l jenkins gerrit.osmocom.org gerrit review 4835a62cd88f0d69db76fb3bfd2df02176a91a6d --json
  fatal: "4835a62cd88f0d69db76fb3bfd2df02176a91a6d" matches multiple patch sets

Related: https://gerrit-review.googlesource.com/Documentation/cmd-review.html
Change-Id: I2d627f8f3b400fa57a50a228d47df2194f60fd08
2022-11-15 11:38:08 +00:00
Oliver Smith 6b7741400d master-pysim: use specific node for docs upload
Related: OS#5767
Change-Id: I25d799ccd0609732f7f82034f97881f1c75dcdf8
2022-11-14 13:13:34 +01:00
Oliver Smith 27a9092b7c jobs/master…-dahdi: don't use multiple scms plugin
Related: OS#5763
Change-Id: If115116c0d6920c86a50dc43d57a8a925f6db064
2022-11-10 16:45:09 +01:00
Oliver Smith 28ace174fa jobs/gerrit…-dahdi: don't use multiple scms plugin
Don't use the deprecated plugin. Let jenkins clone the big linux
repository and clone the code from gerrit in the shell section of the
script. Since we can't directly clone $GERRIT_REFSPEC, create an empty
git repository and fetch it.

Related: OS#5763
Change-Id: I52314f55e7e55382b4e8fcd63d3aba306b401447
2022-11-10 16:45:09 +01:00
Oliver Smith 8763b1ed25 jobs/gerrit-binpkgs: don't use multiple-scm plugin
Change the default branch from origin/master to master, as the git clone
command expects just the branch name without a remote name.

Related: OS#5763
Change-Id: Ia5a17c6d25bf811a992ab495a2531ae613cb144f
2022-11-10 16:44:20 +01:00
Oliver Smith 8453ebc0a4 jobs/gerrit-lint: don't use multiple-scm plugin
Clone osmo-ci.git in the shell script part to avoid using the no-longer
maintained multiple-scm plugin.

Related: OS#5763
Change-Id: I3e9bdfbbf00a234cd559bbf142432a788cfe8f75
2022-11-10 13:11:56 +01:00
Oliver Smith ac5bae1efa jobs: pysim: split into job types
Split the test, pylint and docs job types in the matrix to run them in
parallel.

Depends: pysim I5245c529db729e209d78a02ab9c917a90d0e0206
Related: OS#5497
Change-Id: Iea4f15fd9c9f8f36cb8d638c48da000eafe746a4
2022-11-07 08:16:23 +00:00
Oliver Smith ae69a1aa67 jobs: osmo-e1-hardware: split into job types
Split the job, so building the software can be done with the regular
docker image instead of the fpga-build one (that doesn't have the needed
dependencies).

Related: osmo-e1-hardware I26e839de419c6eb86bf2a6250a1adf06b761d1fa
Change-Id: I90fbe72c722e491f41b9607d1a6389964d7e43b4
2022-11-04 11:17:59 +00:00