Commit Graph

1139 Commits

Author SHA1 Message Date
Oliver Smith b3fa89154b jobs/update-osmo-ci: remove gtp0-deb11build32
The node doesn't have a purpose, remove it (as discussed in related
issue).

Related: OS#5453
Change-Id: I241b6dc91af18be38edaff1bd17dfc9633ca1cec
2022-11-17 11:53:11 +01:00
Oliver Smith 1880077b13 jobs/ttcn3: notify Sukchan for open5gs testsuites
I've asked Sukchan, and he wants to be notified for the testsuites
running against open5gs.

Change-Id: I87650e1f1aa77953ae759b910be03242e3cf3aee
2022-11-17 11:42:29 +01:00
Oliver Smith 9e6c10cd3c jobs/ttcn3: allow overriding email per job
Change-Id: I4d9e08dfbc5c2ad442fc8b4da09009a047f67784
2022-11-17 11:37:10 +01:00
Oliver Smith 60d16b6af9 jobs/ttcn3: send mails to jenkins-notifications ML
Send the mails to the mailing list rather than to only Harald and me, so
it's less likely that failures go unnoticed in case we are both
unavailable.

Change-Id: I4fc554f0624e1ce98cafaf383132241365f1e4d9
2022-11-17 11:31:48 +01:00
Oliver Smith 3c2aac2324 jobs/coverity: remove display-name
Make it consistent with all other jobs and don't set a display name.
This way the job will be sorted properly in the list of jobs and won't
weirdly jump to the top when loading the page.

Change-Id: Idf1a8db45daf75ae110cfae3042627fbcdd14575
2022-11-16 13:32:05 +01:00
Oliver Smith c4924d2657 jobs/osmocom-api: add jenkins-job-builder config
Add a jenkins-job-builder configuration for the existing Osmocom_API job
and refactor it to not use the deprecated multi-scm plugin. Also update
the git clone urls.

Related: OS#5763
Change-Id: Ia518979307ce633ccd2c431a8e8eea866da2a50d
2022-11-16 13:24:51 +01:00
Oliver Smith f560ce08ab jenkins-gerrit: fix ambiguous use of review api
Fix an error when the same Change-Id is used on multiple branches or in
theory projects. This is actually allowed by gerrit, and we use this
e.g. when backporting patches from master.

Use the project, change number (e.g. 30147) and patchset number (e.g. 2)
instead of the Change-Id.

Fix for:
  + ssh -p 29418 -l jenkins gerrit.osmocom.org gerrit review 4835a62cd88f0d69db76fb3bfd2df02176a91a6d --json
  fatal: "4835a62cd88f0d69db76fb3bfd2df02176a91a6d" matches multiple patch sets

Related: https://gerrit-review.googlesource.com/Documentation/cmd-review.html
Change-Id: I2d627f8f3b400fa57a50a228d47df2194f60fd08
2022-11-15 11:38:08 +00:00
Oliver Smith 6b7741400d master-pysim: use specific node for docs upload
Related: OS#5767
Change-Id: I25d799ccd0609732f7f82034f97881f1c75dcdf8
2022-11-14 13:13:34 +01:00
Oliver Smith 27a9092b7c jobs/master…-dahdi: don't use multiple scms plugin
Related: OS#5763
Change-Id: If115116c0d6920c86a50dc43d57a8a925f6db064
2022-11-10 16:45:09 +01:00
Oliver Smith 28ace174fa jobs/gerrit…-dahdi: don't use multiple scms plugin
Don't use the deprecated plugin. Let jenkins clone the big linux
repository and clone the code from gerrit in the shell section of the
script. Since we can't directly clone $GERRIT_REFSPEC, create an empty
git repository and fetch it.

Related: OS#5763
Change-Id: I52314f55e7e55382b4e8fcd63d3aba306b401447
2022-11-10 16:45:09 +01:00
Oliver Smith 8763b1ed25 jobs/gerrit-binpkgs: don't use multiple-scm plugin
Change the default branch from origin/master to master, as the git clone
command expects just the branch name without a remote name.

Related: OS#5763
Change-Id: Ia5a17c6d25bf811a992ab495a2531ae613cb144f
2022-11-10 16:44:20 +01:00
Oliver Smith 8453ebc0a4 jobs/gerrit-lint: don't use multiple-scm plugin
Clone osmo-ci.git in the shell script part to avoid using the no-longer
maintained multiple-scm plugin.

Related: OS#5763
Change-Id: I3e9bdfbbf00a234cd559bbf142432a788cfe8f75
2022-11-10 13:11:56 +01:00
Oliver Smith e4a33ebda2 jenkins-gerrit: display JOB_TYPE infront of url
In the summary comment posted by jenkins to the patch on gerrit, display
the JOB_TYPE infront of the URL instead of "build", if the build job is
a matrix job that uses JOB_TYPE as variable.

For such jobs, it changes:
  [build] https://jenkins.osmocom.org/…
  [build] https://jenkins.osmocom.org/…
  [build] https://jenkins.osmocom.org/…
  [build] https://jenkins.osmocom.org/…
  [lint] https://jenkins.osmocom.org/…
to:
  [manuals] https://jenkins.osmocom.org/…
  [gateware] https://jenkins.osmocom.org/…
  [firmware] https://jenkins.osmocom.org/…
  [software] https://jenkins.osmocom.org/…
  [lint] https://jenkins.osmocom.org/…

JOB_TYPE is used by osmo-e1-hardware and pysim.

Change-Id: I51f49e4799961776dbddaedd76c14ed37a0e6c84
2022-11-07 08:16:23 +00:00
Oliver Smith ac5bae1efa jobs: pysim: split into job types
Split the test, pylint and docs job types in the matrix to run them in
parallel.

Depends: pysim I5245c529db729e209d78a02ab9c917a90d0e0206
Related: OS#5497
Change-Id: Iea4f15fd9c9f8f36cb8d638c48da000eafe746a4
2022-11-07 08:16:23 +00:00
Oliver Smith 288fa2d51e obs: check_builders: at least 10, fail on error
The number of builders connected to the OBS server seems to fluctuate a
bit (used to be 14, then 17 now 16). Make sure that we have at least 10
builders connected. Fix that the test didn't exit with 1 on error.

Related: https://obs.osmocom.org/monitor
Change-Id: Iedd506601a5450550e21bf701309b4ea79a3d897
2022-11-04 15:08:00 +00:00
Harald Welte c70aba5906 ggsn-kernel-gtp: Use my current SSH public key
Change-Id: I64b8c853fd34de4210da43165bebc6f8514b2e1a
2022-11-04 15:07:35 +00:00
Harald Welte 5a5a9ec6e5 ggsn-kernel-gtp: Use new OBS package feed, not old obs.opensuse.org
Change-Id: I1566daa23425973bb4155a40f1cdd6040b494420
2022-11-04 15:07:30 +00:00
Oliver Smith 288fe221e8 lint: enable FUNCTION_WITHOUT_ARGS
Complain if one writes func() instead of func(void).

Change-Id: Ic00041025ac2584f250b5a096eae8fd0d857d9fb
2022-11-04 13:24:10 +00:00
Oliver Smith ae69a1aa67 jobs: osmo-e1-hardware: split into job types
Split the job, so building the software can be done with the regular
docker image instead of the fpga-build one (that doesn't have the needed
dependencies).

Related: osmo-e1-hardware I26e839de419c6eb86bf2a6250a1adf06b761d1fa
Change-Id: I90fbe72c722e491f41b9607d1a6389964d7e43b4
2022-11-04 11:17:59 +00:00
Oliver Smith a3e83d6d40 jobs/master-builds: add timeout_cmd
Add the default timeout of 30 minutes to each job, just like in
gerrit-verifications.yml.

Change-Id: I2ed80258d5f2fbcd8cf27db74db125d9a52b9988
2022-11-04 11:17:59 +00:00
Oliver Smith c53223d047 jobs/obs: don't upload to build.opensuse.org
The transitional phase where we upload to both the old and new OBS
server is over, so just upload to the new server from now on.

Related: OS#5557
Change-Id: If6f87dc6dbb4bafa3abb1d840f8d43cce6432dfd
2022-11-04 08:03:07 +00:00
Oliver Smith 8d7596a311 obs: build_binpkg: add --run-shell-on-error arg
Allow easy inspection of the build dir inside docker on error.

Related: OS#5737
Change-Id: I218c3189d36d24c64cbd7b9eab379aff8ce2bee2
2022-11-03 13:01:43 +01:00
Oliver Smith 13b76cced1 obs: fix building rpms with man pages
Add the "%ext_man .gz" macro from OBS, so building rpms with man pages
does not fail with errors like:
  File not found: /home/user/rpmbuild/BUILDROOT/osmo-ggsn-1.9.0.3.0d3b-0.x86_64/usr/share/man/man8/osmo-ggsn.8

Fixes: OS#5737
Change-Id: Ib6950fb46e9f94aabae98021b215f69838557045
2022-11-03 12:22:45 +01:00
Oliver Smith c4df55dcfd jenkins-gerrit: send mail for V+1 for some users
Add a list of users that do not only get a notification mail when the
gerrit verification failed, but also on success.

Change-Id: I603b8a911c8f17aa726d9e3d5d644ad3262b42dd
2022-10-27 16:43:14 +02:00
Oliver Smith f52c26a695 jobs: add osmocom-obs-check-builders
Add a job that checks if the amount of connected builders to the OBS
server matches what we expect. This should prevent what we had today,
that it only was noticed by chance several days after all builders
failed to connect and the packages were outdated since then.

I'm not sure if there is a proper api to do this check, but I don't
expect the string we check for to change often so this should work fine.

Change-Id: I6e7c1f206551722d6bfe1631b9c1da8d34d85ba8
2022-10-26 08:35:05 +00:00
Oliver Smith 6b17805553 jobs/master-builds: add ice40-usbtrace
Related: OS#6169
Change-Id: I8872f20eabf2975aea602c9e95f66d144f74742d
2022-10-24 17:09:42 +02:00
Oliver Smith 1cd0685b55 jenkins-gerrit: add note for build fails/pkgs pass
One might be wondering what happened if the build job failed, but
building the binary packages succeeded. Since we run 'make check' in
all cases. Add this short explanation:

  The build job(s) failed, but deb/rpm jobs passed.
  We don't enable external/vty tests when building
  packages, so maybe those failed. Check the logs.

Related: OS#2385
Change-Id: I70027ec970a220c3ddfa766302faec7bd8752118
2022-10-21 08:34:15 +00:00
Oliver Smith 1259ab99c6 contrib/jenkins.sh: set-url for existing clones
Make sure that the recent change from git.osmocom.org ->
gerrit.osmocom.org for git clone urls is applied to the existing clones
as well.

Change-Id: I3d9c193bf353ecff0ee8bb4c507de59139fc0925
2022-10-20 14:44:36 +00:00
Oliver Smith 8a66dcd1f8 jobs/update-osmo-ci-on-slaves: set wipe-workspace
Wipe the workspace before running the job, so the _docker_playground
clone gets recreated (and e.g. doesn't have the old clone url).

Change-Id: I5626c4728ebd7f6b693484c7801963985832398d
2022-10-20 09:19:09 +00:00
Oliver Smith 5485ed3198 contrib/jenkins.sh: run …rebuild.sh from workspace
After cloning a repository, go back into the workspace directory. Run
scripts/osmo-ci-docker-rebuild.sh from there, instead of the version
cloned into ~/osmo-ci. This is the behavior one would expect.

Without this patch, a _docker_playground temporary git clone gets
created/used in ~/osmo-ci, although it would make much more sense to
have it in the workspace dir.

Change-Id: I998c0d47ee5ce6f9978d1a6c9e195d5ce40e4e85
2022-10-20 09:19:09 +00:00
Oliver Smith 1c7e7c21c3 ansible/setup-simtest: pre-install pysim depends
Avoid spending time to install the depends of pysim in each CI job.

Closes: OS#5497
Change-Id: I1e9ddb93f05108146a219bd28cf1e9332764bf8e
2022-10-20 09:18:25 +00:00
Oliver Smith 5be9a7442b obs: fix master rpm repository description
Instead of having the description of the nightly repository (copy paste
error), put in "osmocom:master". This gets printed a few times in the
log of the gerrit-binpkgs-rpm jenkins job and may lead to confusion -
the nightly repository is not used, it's a separate master repository.

Related: OS#2385
Change-Id: Ia5a88b064da66d7bf5cebe910961f752262b1bac
2022-10-19 10:46:11 +02:00
Oliver Smith 84866a7d8e jenkins-gerrit: add link to retrigger the build
It's useful to be able to retrigger the pipeline, for example if the
build failed because a depending patch was not merged yet and later it
is merged. Add the link where the "Retrigger" button is to make this
convenient.

Related: OS#2385
Change-Id: I647efac9e79d755282b78f14bd27244c99ef7f11
2022-10-14 16:35:14 +02:00
Oliver Smith 933804b5be jobs/gerrit-verifications: pipeline: agent none
Don't allocate a slot on one of the jenkins nodes for running the
pipeline. The pipeline job is lightweight, it just starts other jobs
and waits for their completion.

Without this patch we got into a situation where a lot of patches were
submitted to gerrit at the same time, pipelines were started for all of
them, taking up slots, and then all of the pipelines waiting until free
slots become available before starting the actual build jobs. Thus
blocking each other until I manually stopped them.

Related: OS#2385
Change-Id: I289fcf1431a91a38fc37270e6a2f43d744c79e4f
2022-10-14 09:48:13 +00:00
Oliver Smith f9c9069151 ansible/ttcn3-slave: fix adding apt key
Downloading the apt key from obs.osmocom.org returns a 404 error unless
an Accept HTTP header is sent. (Yes, really.) Also using apt-key is
deprecated, so change it to download the key with get_url as shown in
the ansible docs example.

Fix for:
  Failed to download key at https://obs.osmocom.org/projects/osmocom/public_key: HTTP Error 404: Not Found

Related: https://github.com/ansible/ansible/issues/50436#issuecomment-451205256
Related: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_key_module.html#examples
Change-Id: Idb23231d629ba8a75510deace0d1f130e889d535
2022-10-13 10:51:55 +02:00
Oliver Smith ed03bc78ed ansible/hosts: add build2-deb11build-ansible
Add it to the config. I've used this to enable ipv6 in docker, so
libosmocore builds don't fail in the socket test when building rpm/deb
packages on that host in docker.

Related: OS#2385
Change-Id: Ibe8e2a638fcbc7c0a69c211385a870b1a82b6fef
2022-10-13 10:51:55 +02:00
Oliver Smith 1449f02cf6 jobs/gerrit-verifications: set pipeline concurrent
Allow running multiple gerrit-verification pipelines of the same project
at the same time to use the jenkins nodes more efficiently.

Note that the concurrent setting of the build jobs are not changed. Some
projects have this set to false, and for those, there will be no more
than one build job running at the same time. The build job gets started
from the pipeline, along with other jobs like lint and deb/rpm package
building.

Related: OS#2385
Change-Id: Ie2a19013e73d96a937e8f83c815c6a82cfe0768d
2022-10-13 10:51:55 +02:00
Oliver Smith d602a298d4 jobs/gerrit-verifications: link to new wiki page
Related: OS#2385
Change-Id: Ia4eac0c39fd156db1fc30e86ef55c48c6105912f
2022-10-13 10:51:55 +02:00
Oliver Smith 05e836e722 jobs/gerrit-verifications: add osmo-ci
Run the linter again on osmo-ci. It used to run on osmo-ci.git with
gerrit-lint.yml before recent refactoring in
I5f1340248cd116c5fa5134eee3fae50baf7ef851. Adding it to
gerrit-verifications like this makes it run again, pipeline_lint: true
is the default.

Related: OS#2385
Change-Id: Ie6d8a99647846fb4008953b444bba642cbcf3b1e
2022-10-13 10:51:55 +02:00
Oliver Smith 191ecd07cf jobs/gerrit-verifications*: drop unsupported args
Fix warnings when updating the job with Jenkins Job Builder:
  WARNING:jenkins_jobs.modules.triggers:Gerrit Trigger property 'no-name-and-email' is deprecated
  WARNING:jenkins_jobs.modules.triggers:Gerrit Trigger property 'trigger-for-unreviewed-patches' is not supported in this plugin version

Related: OS#2385
Change-Id: I6a0057738955bc2507cb5d8c4d48edf652c3529f
2022-10-13 10:51:55 +02:00
Oliver Smith b4dca82f83 jobs/gerrit-verifications: add approvals comment
Related: OS#2385
Change-Id: I4dfc0f84ffe584523bbff4c41295968a07ba097d
2022-10-13 10:51:52 +02:00
Oliver Smith 5354547dff jenkins-gerrit: tweak notify setting
Instead of notifying everybody listed as reviewers and in CC:
* notify the owner if the build failed
* notify nobody if the build was successful

Related: OS#2385
Change-Id: I7c6c183b98624eb75d5dccd9766ee3ff5568b06e
2022-10-12 12:22:44 +02:00
Oliver Smith 348751bc05 jenkins-gerrit: add 0-9 to re_start_build
Let it match "Starting building: gerrit-osmo-ttcn3-hacks-build #5". It
failed on the 3 in ttcn3 before.

Related: OS#2385
Change-Id: I247af55e2c0e3a2dd088ab1c951d8535cfc93229
2022-10-12 11:07:42 +02:00
Oliver Smith a02b53fe4f jenkins-gerrit: meaningful re_start_build error
Related: OS#2385
Change-Id: Ib29b7ce69a27453b460f946ad67bd12faffc2c89
2022-10-12 11:07:09 +02:00
Oliver Smith 9c2e2575a9 obs: support Osmocom projects without prefix
Allow running build_srcpkg.py with "osmo_dia2gsup" instead of
"erlang/osmo_dia2gsup", because that's how it gets passed along from
jobs/gerrit-verifications.yml for the binary package build
verifications.

Related: OS#2385
Change-Id: I09304b219e7002495bd3fcce61bc68c34d5ffcd8
2022-10-11 12:07:40 +02:00
Oliver Smith 4e679c8f2e jobs/gerrit-verifications: write summary comment
Instead of having multiple mails from jenkins per submitted gerrit
patch, only have one at the end of the pipeline that writes a summary
with all relevant links and votes +1/-1 for verified.

This also resolves the problem we had for a long time, that the link for
the build job didn't directly point to the relevant console output.
Instead it pointed to a matrix job that didn't have a direct link to the
relevant job.

The summary looks like this:

1 failed:
  [build] https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-bsc-nat-build/a1=default,a2=default,a3=default,a4=default,label=osmocom-gerrit-debian9/11/consoleFull

3 passed:
  [rpm] https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs-rpm/5/consoleFull
  [deb] https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs-deb/5/consoleFull
  [lint] https://jenkins.osmocom.org/jenkins/job/gerrit-lint/11/consoleFull

Build Failed

Related: OS#2385
Change-Id: Idcab969e1b5ca4e0f1383bee8f36f2d1aac4f624
2022-10-11 12:07:40 +02:00
Oliver Smith d0ef24c3c6 obs: config: update note about testing packages
Related: OS#2385
Change-Id: If6d772052ca64ef93536bf8729a72271f67fbecb
2022-10-11 12:07:40 +02:00
Oliver Smith 38109f8429 obs/README: add overview with build_binpkg.py
Related: OS#2385
Change-Id: I1b741ad325e6541e6d59dfd50de66bee805e56fe
2022-10-11 12:07:40 +02:00
Oliver Smith 9e1f9ae039 jobs/gerrit-binpkgs: new job for deb, rpm packages
Build deb and rpm packages for each patch submitted to gerrit for the
projects in the list.

Example:
https://gerrit.osmocom.org/c/osmo-bsc-nat/+/29492

Related: OS#2385
Change-Id: I7ca8869c2e9f2e7c74a360933be12b5c2b47c2fc
2022-10-11 12:07:40 +02:00
Oliver Smith 7614916a72 obs: build_binpkg: use almalinux:8 naming
Instead of using "--docker centos:8" as argument and translating it
later on in the code to use "almalinux:8" as base distribution, use
"--docker almalinux:8" as argument. It was brought up in code review
that this makes it less confusing.

Related: OS#2385
Change-Id: Id8298e8bafe065010f7bc00f1ff261aa6431ed4c
2022-10-11 12:07:40 +02:00