Add a list of users that do not only get a notification mail when the
gerrit verification failed, but also on success.
Change-Id: I603b8a911c8f17aa726d9e3d5d644ad3262b42dd
Add a job that checks if the amount of connected builders to the OBS
server matches what we expect. This should prevent what we had today,
that it only was noticed by chance several days after all builders
failed to connect and the packages were outdated since then.
I'm not sure if there is a proper api to do this check, but I don't
expect the string we check for to change often so this should work fine.
Change-Id: I6e7c1f206551722d6bfe1631b9c1da8d34d85ba8
One might be wondering what happened if the build job failed, but
building the binary packages succeeded. Since we run 'make check' in
all cases. Add this short explanation:
The build job(s) failed, but deb/rpm jobs passed.
We don't enable external/vty tests when building
packages, so maybe those failed. Check the logs.
Related: OS#2385
Change-Id: I70027ec970a220c3ddfa766302faec7bd8752118
Make sure that the recent change from git.osmocom.org ->
gerrit.osmocom.org for git clone urls is applied to the existing clones
as well.
Change-Id: I3d9c193bf353ecff0ee8bb4c507de59139fc0925
Wipe the workspace before running the job, so the _docker_playground
clone gets recreated (and e.g. doesn't have the old clone url).
Change-Id: I5626c4728ebd7f6b693484c7801963985832398d
After cloning a repository, go back into the workspace directory. Run
scripts/osmo-ci-docker-rebuild.sh from there, instead of the version
cloned into ~/osmo-ci. This is the behavior one would expect.
Without this patch, a _docker_playground temporary git clone gets
created/used in ~/osmo-ci, although it would make much more sense to
have it in the workspace dir.
Change-Id: I998c0d47ee5ce6f9978d1a6c9e195d5ce40e4e85
Instead of having the description of the nightly repository (copy paste
error), put in "osmocom:master". This gets printed a few times in the
log of the gerrit-binpkgs-rpm jenkins job and may lead to confusion -
the nightly repository is not used, it's a separate master repository.
Related: OS#2385
Change-Id: Ia5a88b064da66d7bf5cebe910961f752262b1bac
It's useful to be able to retrigger the pipeline, for example if the
build failed because a depending patch was not merged yet and later it
is merged. Add the link where the "Retrigger" button is to make this
convenient.
Related: OS#2385
Change-Id: I647efac9e79d755282b78f14bd27244c99ef7f11
Don't allocate a slot on one of the jenkins nodes for running the
pipeline. The pipeline job is lightweight, it just starts other jobs
and waits for their completion.
Without this patch we got into a situation where a lot of patches were
submitted to gerrit at the same time, pipelines were started for all of
them, taking up slots, and then all of the pipelines waiting until free
slots become available before starting the actual build jobs. Thus
blocking each other until I manually stopped them.
Related: OS#2385
Change-Id: I289fcf1431a91a38fc37270e6a2f43d744c79e4f
Add it to the config. I've used this to enable ipv6 in docker, so
libosmocore builds don't fail in the socket test when building rpm/deb
packages on that host in docker.
Related: OS#2385
Change-Id: Ibe8e2a638fcbc7c0a69c211385a870b1a82b6fef
Allow running multiple gerrit-verification pipelines of the same project
at the same time to use the jenkins nodes more efficiently.
Note that the concurrent setting of the build jobs are not changed. Some
projects have this set to false, and for those, there will be no more
than one build job running at the same time. The build job gets started
from the pipeline, along with other jobs like lint and deb/rpm package
building.
Related: OS#2385
Change-Id: Ie2a19013e73d96a937e8f83c815c6a82cfe0768d
Run the linter again on osmo-ci. It used to run on osmo-ci.git with
gerrit-lint.yml before recent refactoring in
I5f1340248cd116c5fa5134eee3fae50baf7ef851. Adding it to
gerrit-verifications like this makes it run again, pipeline_lint: true
is the default.
Related: OS#2385
Change-Id: Ie6d8a99647846fb4008953b444bba642cbcf3b1e
Fix warnings when updating the job with Jenkins Job Builder:
WARNING:jenkins_jobs.modules.triggers:Gerrit Trigger property 'no-name-and-email' is deprecated
WARNING:jenkins_jobs.modules.triggers:Gerrit Trigger property 'trigger-for-unreviewed-patches' is not supported in this plugin version
Related: OS#2385
Change-Id: I6a0057738955bc2507cb5d8c4d48edf652c3529f
Instead of notifying everybody listed as reviewers and in CC:
* notify the owner if the build failed
* notify nobody if the build was successful
Related: OS#2385
Change-Id: I7c6c183b98624eb75d5dccd9766ee3ff5568b06e
Let it match "Starting building: gerrit-osmo-ttcn3-hacks-build #5". It
failed on the 3 in ttcn3 before.
Related: OS#2385
Change-Id: I247af55e2c0e3a2dd088ab1c951d8535cfc93229
Allow running build_srcpkg.py with "osmo_dia2gsup" instead of
"erlang/osmo_dia2gsup", because that's how it gets passed along from
jobs/gerrit-verifications.yml for the binary package build
verifications.
Related: OS#2385
Change-Id: I09304b219e7002495bd3fcce61bc68c34d5ffcd8
Build deb and rpm packages for each patch submitted to gerrit for the
projects in the list.
Example:
https://gerrit.osmocom.org/c/osmo-bsc-nat/+/29492
Related: OS#2385
Change-Id: I7ca8869c2e9f2e7c74a360933be12b5c2b47c2fc
Instead of using "--docker centos:8" as argument and translating it
later on in the code to use "almalinux:8" as base distribution, use
"--docker almalinux:8" as argument. It was brought up in code review
that this makes it less confusing.
Related: OS#2385
Change-Id: Id8298e8bafe065010f7bc00f1ff261aa6431ed4c
Reduce mails by running gerrit-lint as part of the gerrit-verifications
pipeline.
Related: OS#2385
Change-Id: I5f1340248cd116c5fa5134eee3fae50baf7ef851
Prepare to unify the list of projects in jobs/gerrit-verifications.yml
and jobs/gerrit-lint.yml by allowing to skip the build for some
repositories (e.g. docker-playground is linted but we don't run a
contrib/jenkins.sh there).
Change-Id: Ie6264d44120798c0ecbcaf9c63d9b95f74ae9e37
Prepare to reduce the amount of comments sent to gerrit from jenkins and
resulting mails. Instead of triggering the jobs directly from gerrit,
trigger a pipeline job that calls the original job (now called
gerrit-{repos}-build) and will in future patches call all other gerrit
verification jobs too (linter, building deb/rpm packages).
Change-Id: Ica208a5177739655cdd493d461da146d440cd542
Installing osmo-gsm-manuals-dev plus depends takes a long time. Don't do
this for every build, instead do it once when building a second docker
container and then use that.
Related: OS#2385
Change-Id: I8475bd954352b572197795ad4cd9461e39896d48
Configure dpkg to not extract man pages. Otherwise it will spend some
time regenerating the man page index whenever installing build
dependencies before starting to build a package.
Related: OS#2385
Change-Id: I1c9e3883b976e023c96dfd59eb147770f7ad99a7
Add a script to build deb/rpm packages, as it would be done on
obs.osmocom.org. This will be used by jenkins to verify deb and rpm
builds for each submitted gerrit patch.
I have attempted to use 'osc build' instead of directly calling
apt-get build-dep and dpkg-buildpackage (and rpm equivalents). Using
'osc build' would have the advantage that the build works as close to
the OBS build as possible. However it would try to install dependencies
with sudo, so we would need to have sudo available in the docker
container that builds the untrusted code from gerrit. Let's not do that.
Related: OS#2385
Change-Id: I4c6b5d61af35df98cbc70d9ddc8ad36d38a9ce18
Don't use os.path.basename on the script_path parameter passed to the
function. A future patch will pass a script that is inside the data
subdirectory to this function, therefore this is needed.
Related: OS#2385
Change-Id: Ide78d976f9af445c4c8d8748bc274d7289064769
Rename Dockerfile to build_srcpkg.Dockerfile and adjust related code to
allow using a different Dockerfile for building the binary packages in a
future patch.
Related: OS#2385
Change-Id: I8ef7944a4a81acd6c915998f37139eebad2b2d3e
No need to pass args here. It was only used for args.verbose, and that
isn't necessary since we don't need to restore the verbose mode.
Related: OS#2385
Change-Id: I620cdef46e18f4c66644f14003caf2183c89686f
Put it on an extra line like the other arguments to prevent it from
getting overlooked.
Related: OS#2385
Change-Id: Ie7dea2734e7c47766dfe64b7091bc4f5f72d53a5
Add one new job for building source packages and sending them to
obs.osmocom.org. Trigger it from all master-* jobs.
I've also considered adding one job per existing master job that would
only update one package at a time (master-libosmocore-obs,
master-osmo-bsc-obs, ...). With some additional development effort it
should be possible, and it would make each individual master OBS job
faster. But given that with the current implementation it only takes
20s to 30s for *all* packages if there are no changes, as it compares
git remote HEAD with the version currently on OBS before starting to
clone repositories and building the source packages (similar to
Osmocom_OBS_latest_obs.osmocom.org), it didn't seem worth optimizing.
Set concurrent to false as the triggers from master-builds will likely
cause it to run multiple times in parallel otherwise.
Related: https://jenkins.osmocom.org/jenkins/view/OBS/job/Osmocom_OBS_master_obs.osmocom.org/
Related: https://obs.osmocom.org/project/show/osmocom:master
Related: OS#2385
Change-Id: I53a494f13f81ae837f2d362c54e1bdf13f121db3
Add a new master feed, where packages are updated as soon as patches get
merged to master. Upload a commit_$COMMIT.txt file for each package in
this feed and use it to determine if the package needs to be updated or
not.
In most packages the commit is already part of the version in the dsc
file, e.g. "libosmocore_1.7.0.38.c3b90.dsc", but that's harder to parse
and is more likely to have a hash collision (just 5 characters).
Related: OS#2385
Change-Id: I3b0b4f4876b8c1eeb61f20d903a6f2cac6e99638
Make it slightly easier to read, before extending the latest branch with
master too in a follow-up patch.
Related: OS#2385
Change-Id: I4f1d4e250eb4d8163f1ded29106a44deb0d0feff
Refactor checkout_default_branch so the get_default_branch logic can be
used from another function in a future patch.
Related: OS#2385
Change-Id: If3f0ab500ff165d81f61b19ad41af90bd39dce8f
Since [1] was merged this script is executed with '-xe' making it
fail immediately if a command returns a non-zero code. grep may
return 1 if pattern is not found, so let's suspend it.
Change-Id: I21a01478b64015fd699ed31634bf2fb1dc8f6750
Fixes: [1] Icf6546271a01cd9a3dab1ce0d1bc02807d08ad85
Coverity job currently fails because it fails to fetch osmo-gmr.git.
Change-Id: I4d52dbf0f81332dc1856944a6a3c7376cb1e0638
Fixes: I17590d0b0b46274e2ceb007e20d7d1dc039f4e8d
Replace the legacy git.osmocom.org clone URLs with either gerrit or
gitea in most places. Besides not using legacy urls anymore, the
advantage of using gerrit directly is that we don't have a delay of 10
min to several hours on the repository.
Not replaced:
* ansible/roles/osmocom-workstation: not sure if still used, would need
a bit of a rewrite to use logic similar to osmo_git_clone_url().
Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I90409c9e7f4db4e1765647021625f9acfccaa78b
This had been added for debugging purposes and isn't used anymore. Also
one less hit for git grep git.osmocom.org.
Change-Id: I10cbb598f0d5d675034ca87568a8b6f1920f2ad3
Update the function to not use legacy URLs anymore and allow adding
future gitea URLs more easily. Update the description of the function to
reflect the new motivation for using this.
Change-Id: Ibf53ca2c444fae82af73e417410faa5b16f5de59
Oliver Smith
Vadim Yanitskiy