io_uring in the kernel memlocks a big amount of space with older
kernels, such as the kernel on host2-deb11build-ansible. It seemed to be
enough to increase it to 1 GiB. However now we see it failing again
with:
failure during io_uring_queue_init(): Cannot allocate memory
Aborted (core dumped)
Set it to unlimited to ensure that a low limit is not the reason we see
it failing. If it still happens after this, we probably need to make
sure that we don't run too many SUTs with io_uring in parallel on
jenkins nodes with older kernels.
Related: OS#6405
Change-Id: I39aaa2ed7d0310a12d0c4493a79ff46b2e7e2864
After most recent patches from Pau were merged, the io_uring testsuites
run on all jenkins nodes with the ttcn3 label again!
host2-deb11build-ansible has an older host kernel and needs ulimit
memlock increased to 1 GB. I've tried out e.g. 500 MB and it isn't
enough.
Closes: OS#6405
Change-Id: I59b9a7c3928281a0557c17629029adcbc5ae7410
The ttcn3-ggsn-test-kernel-* jobs should not wipe their workspace, so we
can avoid shallow-cloning the kernel if the commit did not change.
The way the scripts in docker-playground.git are written, this also
caused a permission error.
Change-Id: I4ac1b2c3f5ba63c131dd1d6a4888f32a27264eba
A change introducing osmo_io for RTP/RTCP of omo-mgw
is currently in review at https://gerrit.osmocom.org/c/osmo-mgw/+/36363
Let's make sure we have a related io_uring test job.
Change-Id: I672c72c855bebc1f895be51f3fa3cf0e368005ff
Related: OS#6387, SYS#5094
Rename it from ttcn3-io_uring to io_uring, as it will be used for
non-ttcn3 jobs too in the next patch.
Related: OS#6405
Change-Id: Icd23a4497771c939f6b988ea244c9dcabfd30e3f
io_uring_queue_init fails on some of the jenkins nodes running the TTCN3
testsuites with io_uring. It seems that the kernel is too old on these
nodes. Run the testsuites only on jenkins nodes with the new
ttcn3-io_uring label, which have a recent kernel.
Related: OS#6357
Change-Id: If917806f9056fdf99863f4132f44659b2bfd44c3
Set LIBOSMO_IO_BACKEND=IO_URING via the DOCKER_ARGS variable, which all
SUTs already use in docker-playground.git (see ttcn3-*/jenkins.sh).
The list of testsuites to test for is takend from:
https://osmocom.org/issues/6357#note-9
Closes: OS#6357
Change-Id: I8eb3f4fdfe54268d3b623d88c8746f5395ba509d
Test EPDG_Tests.TC_mt_ipv4_echo_req fails when running under debian11
(node build2-deb11build-ansible), most probably due to some kernel bug
fixed later on.
Since anyway osmo-epdg requires OTP from debian12, it makes no sense to
attempt to support older kernel version from debian11, so simply avoid
picking debian11 to run the tests.
Change-Id: I247853cbefa0864f54bad5e5b6597556c46a51e3
Ensure we trigger building new OBS packages first, then wait plenty of
time until the binary packages are available (and run jobs in the
meantime that don't need them), and only after that we run the jobs
that need the binary packages.
Otherwise TTCN3 jobs may test the packages from the previous day, and
some jobs may fail completely due to packages not being completely built
yet. For example, yesterday the new Osmocom CNI releases were tagged,
which means the :latest packages also need to be rebuilt (-> building
all OBS packages takes longer). The osmocom-release-manuals and
-tarballs jobs failed, because the new binary packages were not
available yet when they ran.
Change all timers to the format "H 20 * * *" to have a deterministic
hour and semi-random minute based on the job name.
Change-Id: Ib68f9a78bae27a63706a8c95715bf6a244b7bf1d
Remove the "artifact-days-to-keep: -1" and "artifact-num-to-keep: -1"
lines, as they don't have an effect. -1 is the default value, which
means "infinite" and causes the value from "days-to-keep" /
"num-to-keep" to get used instead.
This patch re-applies part of 719ff976 ("jobs: tweak build-discarder
values"), which had been reverted in a052c13c.
Related: https://jenkins-job-builder.readthedocs.io/en/latest/properties.html#properties.build-discarder
Related: OS#5980
Change-Id: Ic073643634e1814147d765dd7e8f3f02e81effc3
This reverts commit 719ff97608.
The disk consumption of jenkins has grown by almost 500GB
since we merged this patch. Clearly this is not expected, and we'd
have ran out of disk space in a few weeks.
I personally think the current allocation of 1.5TB of disk space to
jenkins should be more than sufficient; we just need to manage it
better.
Closes: OS#5980
Change-Id: I6b744a8b84a3e1255a8d51f73d1721ccfd028ac1
Remove num-to-keep from most jobs, as this leads to keeping the build
logs for a much shorter timeline than desired. For example the
gerrit-binpkgs-deb job that runs for most projects when pushing patches
to jenkins reaches the 120 limit in less than 24h - and so when clicking
the link on a failed build from yesterday it is already deleted.
Instead just keep the logs for the last 30 days, no matter how many were
submitted on one day. Storing logs doesn't take up much space.
Remove the artifact-days-to-keep and artifact-num-to-keep lines, as they
don't have an effect. For jobs that do have artifacts, the actual value
is min(days-to-keep,artifact-days-to-keep) and same with num-to-keep.
While at it, increase the ttcn3-testsuites build-discarder to 120 days
as this means more data will show up in the test result analyzer at
which we look frequently.
Change-Id: Iec5c22c7fcf6c1fd2db71611045f15dc6580ed86
Delete the part that was using the "blocking" variable too.
Fixes: 30ccbd3 ("jobs/ttcn3: don't block on same testsuite running")
Change-Id: I23bc6eb99d70447263b79bc4e33524a6932b43eb
Don't block jobs if other jobs with the same testsuite are already
running. This used to be necessary as we had subnets hardcoded per
testsuite, and therefore couldn't run it twice on the same host (e.g.
once against latest, once against nightly).
Related: OS#5802
Depends: docker-playground I57152b08ef0f38e17e7019a8df032189b03f56cf
Change-Id: I3159403e2ce2ec184ee48b4ff4f145e718e9b428
Send the mails to the mailing list rather than to only Harald and me, so
it's less likely that failures go unnoticed in case we are both
unavailable.
Change-Id: I4fc554f0624e1ce98cafaf383132241365f1e4d9
Replace the legacy git.osmocom.org clone URLs with either gerrit or
gitea in most places. Besides not using legacy urls anymore, the
advantage of using gerrit directly is that we don't have a delay of 10
min to several hours on the repository.
Not replaced:
* ansible/roles/osmocom-workstation: not sure if still used, would need
a bit of a rewrite to use logic similar to osmo_git_clone_url().
Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I90409c9e7f4db4e1765647021625f9acfccaa78b
Drop _MIRROR variables, as both the testsuite packages and the packages
to test are now on downloads.osmocom.org. Previously these variables
were necessary as the testsuite packages were only on
download.opensuse.org (before we set up obs.osmocom.org, which publishes
packages to downloads.osmocom.org).
Change-Id: I34050da1e2444cf684ff8bd07673e3268b4dd497
Get the centos8 packages directly from the package repository again
(https://downloads.osmocom.org/packages/) instead of getting them from
the delayed obs-mirror dir (https://downloads.osmocom.org/obs-mirror/).
Use the packages generated from obs.osmocom.org for the testsuite too.
This reverts 526f1e12d3 and
I8201f34a0f9747eea78c358a626cc3981d7407c4 which set this up to work
around sporadically missing metadata files from download.opensuse.org.
Change-Id: Ic7920ad71a0cb6362883f8b9ed3dbc9bbb992988
Remove reference to stretch (debian 9), as the jobs run on a newer
debian version now. Remove the reference altogether so we don't need to
update it here. The current version can be looked up in
docker-playground.git.
Change-Id: I06c74830b0f6fe0baed44c300f9e1023fb0522de
These two projects have been released recently as part of Osmocom CNI
202206. Let's enable testing them since version in latest is good enough
for testing.
Change-Id: I1587e8400eff2d8b1ec26ab2fff1fbfa772fe8f8
In previous patch Id114d55edc2ba92f7851243ab7df51db332079c4, only the
OSMOCOM_REPO_MIRROR should have been overridden. However
OSMOCOM_REPO_TESTSUITE_MIRROR defaults to OSMOCOM_REPO_MIRROR, so
explicitly set it in this patch to the opensuse mirror.
This fixes problems while building debian-bullseye-titan caused by the
different directory structure of both mirrors, and OSMOCOM_REPO_PATH
intentionally not taken into account while building the testsuite
docker image.
This is part 2/2 of fixing TTCN3-centos-bts-test, which has been failing
since OSMOCOM_REPO_MIRROR was changed in osmo-ci.git. Part 1 is in
docker-playground I78e0445a1b82e5cd72ecaa86e8fe25516b19f485.
Change-Id: I8201f34a0f9747eea78c358a626cc3981d7407c4
Lately we've been seeing a lot of build failures from the TTCN3-centos
jobs running against the nightly repository. It seems that the mirror we
get redirected to isn't syncing properly. Use the Osmocom mirror
instead.
Fix for:
Nightly packages of the Osmocom project (CentOS 190 kB/s | 1.7 kB 00:00
Nightly packages of the Osmocom project (CentOS 1.8 kB/s | 634 B 00:00
[91mErrors during downloading metadata for repository 'network_osmocom_nightly':
- Status code: 404 for 7062058d3e234003a6c3595331cf3fc100-filelists.xml.gz (IP: 134.76.12.6)
- Status code: 404 for ad91db8cb719947d9abc971661c588e8c6-primary.xml.gz (IP: 134.76.12.6)
[0m[91mError: Failed to download metadata for repo 'network_osmocom_nightly': Yum repo downloading error: Downloading error(s): repodata/ad91db8cb7273beaefca0e473d9c4dd12d12786819947d9abc971661c588e8c6-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/7062058d3e5395b4b02570496e41e19e8fd306f5234003a6c3595331cf3fc100-filelists.xml.gz - Cannot download, all mirrors were already tried without success
Change-Id: Id114d55edc2ba92f7851243ab7df51db332079c4
Fix sporadic errors related to the clean up script running while images
being built:
failed to get digest sha256:8ea2ba9a1...
Closes: OS#5447
Change-Id: If722fddce8de65a1ae012a46d6a202f430e0c69d
Now that mgw support is in osmo-hnbgw master and docker/ttcn3 testsuite
is working we can enable the test again.
Change-Id: I4c7d1ee876e90d52827b2d8cc57342298dd44061
Related: OS#5152
Add a configuration for the job to the yml file. It looks like this job
was created manually, or the configuration was just not pushed to
gerrit.
This makes e-mail notifications consistent and fixes missing colors in
build output.
Change-Id: I14995dea0a0d223b78e20b49953d5c814c1ad4a1
Vadim Yanitskiy
Oliver Smith
Pau Espin
Harald Welte
Neels Hofmeyr
Daniel Willmann