We started using git submodules with osmo-trx. Adjust the obs scripts
to actually make the git submodules part of the source packages.
Note that this didn't fail in jenkins before with the rpm build
verification, as jenkins updates git submodules on its own.
Fix for:
[ 165s] Makefile.am:32: error: required directory ./osmocom-bb/src/host/trxcon does not exist
Change-Id: I51b423f3885d6ead5c21a83bdf8ef6051dc34fe3
Apparently the output of repoquery isn't always sorted, it started
failing on jenkins after moving the test inside qemu. Add an explicit
sort.
Fix for:
+ comm -23 osmocom_packages_all.txt blacklist.txt
comm: file 1 is not in sorted order
Related: OS#5365
Change-Id: Icb00df102555e06b66b1c2597488b625e3c77f1c
While I'm developing this, Jenkins is currently failing here. Make it
easier to debug this by printing the file contents.
Related: OS#5365
Change-Id: Ifbf4ca7f49c1f4441f84695aea0936515e01ffd4
The DISTRO variable is either debian10 or debian11, fix the broken
check. This condition is there in the first place, because we don't
build the usrp1 backend for centos8.
Change-Id: I987f27db257961faf06824df2dcc8f9db1fedccf
Related: OS#5365
Add services from new projects and enable previously disabled services,
now that this test runs in qemu and services have more permissions like
setting realtime priorities.
Related: OS#5365
Change-Id: Iec7db433cac4c77226e0f1ae2ba502de0d1a8a2b
Change repo-install-test to run inside of qemu instead of docker. This
job needs to run systemd to verify that the systemd services start up
properly. Running systemd inside docker was never officially supported,
it worked with cgroups1 but does not work anymore with cgroups2.
An alternative approach was running inside podman instead of docker
(running systemd inside of podman is officially supported). However we
would have needed various workarounds with podman and wouldn't be able
to test all Osmocom systemd services in the end, due to lack of
permissions (see review of I394918fc61de36acce65ffb33defcb8fc21801c4).
By running with a separate kernel inside qemu we can run all Osmocom
services.
Related: OS#5365
Change-Id: Ie7f1bccb05779cb3614016c0b965b810bbb1471b
Remove the fallback clean up code, as it also may lead to images getting
removed right before we need to use them. Besides that, it should be
dead code by now since docuum should be running on all our jenkins nodes
to clean up old images based on last use date.
Change-Id: I9ca0c2ba245bdd75d9fb8eaf341055e8c2ab1b55
Don't delete images while they are being used, to fix these errors we
see from time to time in the middle of "docker build" on jenkins:
unknown parent image ID sha256:1b072e35048cd8b680eddabdc641ac678edb1184d222d5e7b3fbe0b3c333129a
This happens because "docker build" creates so-called dangling images
for each step processed of a Dockerfile. The "docker system prune" call
deletes these dangling images (among other things).
Remove the "docker system prune" call. We already have the docuum daemon
to deal with unused images (dangling and not dangling), it removes them
based on last use date so that the used space is always below a
configured limit. As it deletes images that haven't been used the
longest when it reaches the limit, it will not result in the problem
explained above.
Besides images, "docker system prune" also removes unused containers
(instances of images created with 'docker run' without --rm) and
networks. Add "docker container prune" and "docker network prune"
commands to remove them from now on.
Also remove the redundant container removal logic (previous it was
redundant with "docker system prune", now redundant with "docker
container prune").
Related: https://docs.docker.com/config/pruning/
Change-Id: Ia1b466eea43dd135373949e8e3e6b005c169ea0c
After building packages with the OBS scripts, show their contents. This
allows easy checking if config files were correctly packaged etc. By
adding it here, it will also show up at the end of the related jenkins
jobs for gerrit verifications.
Related: OS#5817
Change-Id: Ie30b07f35f7e41990fa352523427d86458291d4d
Build the docker image right before using it, instead of building it in
another jenkins job update-osmo-ci-on-slaves via
osmo-ci-docker-rebuild.sh.
The logic in osmo-ci-docker-rebuild.sh was broken. I didn't realize at
the time that this image is only used for the virtual osmo-gsm-tester,
not the physical ones. But only the machines running the physical
osmo-gsm-tester have the /var/tmp/osmo-gsm-tester/state path. The
virtual osmo-gsm-tester isn't running on these machines but on generic
jenkins nodes.
Building the image right before using it makes sense for this job, as it
is the only user of the image. If it was already built from the same
Dockerfile, a cached version is used.
Fix for:
Unable to find image 'osmocom-build/osmo-gsm-tester:latest' locally
Fixes: 9139e76b ("osmo-ci-docker-rebuild: don't always build osmo-gsm-tester")
Change-Id: Icad9459de1d3a3a4e65ecacf7f903433bb504cc9
Don't attempt to build openbsc 1.4.0 and 1.4.1 in the
osmocom-build-old-tags-against-master job, as it fails to build with a
modern gcc with a potential null pointer dereference warning that gets
treated as error. This was fixed in 1.4.2 with openbsc patch
I93d816a20ba208e9fd32a1fc172a78ccd326e1ba.
A more modern gcc gets used, as this job gets changed to not only run on
jenkins nodes with debian 9 installed, but also more modern versions.
This job does not run in docker, and since it still passes it doesn't
seem worth changing that now.
Related: OS#5793
Change-Id: Ibf4b66195fbd70a68d63929b156a9aac54a699bc
With recent changes in open5gs, building the source packages we
generated fails:
[ 45s] ../lib/metrics/meson.build:52:4: ERROR: Automatic wrap-based subproject downloading is disabled
[ 45s] dh_auto_configure: error: cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 meson .. --wrap-mode=nodownload --buildtype=plain --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=lib/x86_64-linux-gnu --libexecdir=lib/x86_64-linux-gnu returned exit code 1
Fix this by downloading all subprojects. This is now possible as I added
a meson.build file to the prometheus-client-c repository that open5gs
uses: https://github.com/open5gs/prometheus-client-c/pull/2
Change-Id: If3910b520382b177a77b216c93771ea88414723d
Fix an error when the same Change-Id is used on multiple branches or in
theory projects. This is actually allowed by gerrit, and we use this
e.g. when backporting patches from master.
Use the project, change number (e.g. 30147) and patchset number (e.g. 2)
instead of the Change-Id.
Fix for:
+ ssh -p 29418 -l jenkins gerrit.osmocom.org gerrit review 4835a62cd88f0d69db76fb3bfd2df02176a91a6d --json
fatal: "4835a62cd88f0d69db76fb3bfd2df02176a91a6d" matches multiple patch sets
Related: https://gerrit-review.googlesource.com/Documentation/cmd-review.html
Change-Id: I2d627f8f3b400fa57a50a228d47df2194f60fd08
The number of builders connected to the OBS server seems to fluctuate a
bit (used to be 14, then 17 now 16). Make sure that we have at least 10
builders connected. Fix that the test didn't exit with 1 on error.
Related: https://obs.osmocom.org/monitor
Change-Id: Iedd506601a5450550e21bf701309b4ea79a3d897
Add the "%ext_man .gz" macro from OBS, so building rpms with man pages
does not fail with errors like:
File not found: /home/user/rpmbuild/BUILDROOT/osmo-ggsn-1.9.0.3.0d3b-0.x86_64/usr/share/man/man8/osmo-ggsn.8
Fixes: OS#5737
Change-Id: Ib6950fb46e9f94aabae98021b215f69838557045
Add a list of users that do not only get a notification mail when the
gerrit verification failed, but also on success.
Change-Id: I603b8a911c8f17aa726d9e3d5d644ad3262b42dd
Add a job that checks if the amount of connected builders to the OBS
server matches what we expect. This should prevent what we had today,
that it only was noticed by chance several days after all builders
failed to connect and the packages were outdated since then.
I'm not sure if there is a proper api to do this check, but I don't
expect the string we check for to change often so this should work fine.
Change-Id: I6e7c1f206551722d6bfe1631b9c1da8d34d85ba8
One might be wondering what happened if the build job failed, but
building the binary packages succeeded. Since we run 'make check' in
all cases. Add this short explanation:
The build job(s) failed, but deb/rpm jobs passed.
We don't enable external/vty tests when building
packages, so maybe those failed. Check the logs.
Related: OS#2385
Change-Id: I70027ec970a220c3ddfa766302faec7bd8752118
Instead of having the description of the nightly repository (copy paste
error), put in "osmocom:master". This gets printed a few times in the
log of the gerrit-binpkgs-rpm jenkins job and may lead to confusion -
the nightly repository is not used, it's a separate master repository.
Related: OS#2385
Change-Id: Ia5a88b064da66d7bf5cebe910961f752262b1bac
It's useful to be able to retrigger the pipeline, for example if the
build failed because a depending patch was not merged yet and later it
is merged. Add the link where the "Retrigger" button is to make this
convenient.
Related: OS#2385
Change-Id: I647efac9e79d755282b78f14bd27244c99ef7f11
Instead of notifying everybody listed as reviewers and in CC:
* notify the owner if the build failed
* notify nobody if the build was successful
Related: OS#2385
Change-Id: I7c6c183b98624eb75d5dccd9766ee3ff5568b06e
Let it match "Starting building: gerrit-osmo-ttcn3-hacks-build #5". It
failed on the 3 in ttcn3 before.
Related: OS#2385
Change-Id: I247af55e2c0e3a2dd088ab1c951d8535cfc93229
Allow running build_srcpkg.py with "osmo_dia2gsup" instead of
"erlang/osmo_dia2gsup", because that's how it gets passed along from
jobs/gerrit-verifications.yml for the binary package build
verifications.
Related: OS#2385
Change-Id: I09304b219e7002495bd3fcce61bc68c34d5ffcd8
Instead of using "--docker centos:8" as argument and translating it
later on in the code to use "almalinux:8" as base distribution, use
"--docker almalinux:8" as argument. It was brought up in code review
that this makes it less confusing.
Related: OS#2385
Change-Id: Id8298e8bafe065010f7bc00f1ff261aa6431ed4c
Installing osmo-gsm-manuals-dev plus depends takes a long time. Don't do
this for every build, instead do it once when building a second docker
container and then use that.
Related: OS#2385
Change-Id: I8475bd954352b572197795ad4cd9461e39896d48
Configure dpkg to not extract man pages. Otherwise it will spend some
time regenerating the man page index whenever installing build
dependencies before starting to build a package.
Related: OS#2385
Change-Id: I1c9e3883b976e023c96dfd59eb147770f7ad99a7
Add a script to build deb/rpm packages, as it would be done on
obs.osmocom.org. This will be used by jenkins to verify deb and rpm
builds for each submitted gerrit patch.
I have attempted to use 'osc build' instead of directly calling
apt-get build-dep and dpkg-buildpackage (and rpm equivalents). Using
'osc build' would have the advantage that the build works as close to
the OBS build as possible. However it would try to install dependencies
with sudo, so we would need to have sudo available in the docker
container that builds the untrusted code from gerrit. Let's not do that.
Related: OS#2385
Change-Id: I4c6b5d61af35df98cbc70d9ddc8ad36d38a9ce18
Don't use os.path.basename on the script_path parameter passed to the
function. A future patch will pass a script that is inside the data
subdirectory to this function, therefore this is needed.
Related: OS#2385
Change-Id: Ide78d976f9af445c4c8d8748bc274d7289064769
Rename Dockerfile to build_srcpkg.Dockerfile and adjust related code to
allow using a different Dockerfile for building the binary packages in a
future patch.
Related: OS#2385
Change-Id: I8ef7944a4a81acd6c915998f37139eebad2b2d3e
No need to pass args here. It was only used for args.verbose, and that
isn't necessary since we don't need to restore the verbose mode.
Related: OS#2385
Change-Id: I620cdef46e18f4c66644f14003caf2183c89686f
Put it on an extra line like the other arguments to prevent it from
getting overlooked.
Related: OS#2385
Change-Id: Ie7dea2734e7c47766dfe64b7091bc4f5f72d53a5
Add one new job for building source packages and sending them to
obs.osmocom.org. Trigger it from all master-* jobs.
I've also considered adding one job per existing master job that would
only update one package at a time (master-libosmocore-obs,
master-osmo-bsc-obs, ...). With some additional development effort it
should be possible, and it would make each individual master OBS job
faster. But given that with the current implementation it only takes
20s to 30s for *all* packages if there are no changes, as it compares
git remote HEAD with the version currently on OBS before starting to
clone repositories and building the source packages (similar to
Osmocom_OBS_latest_obs.osmocom.org), it didn't seem worth optimizing.
Set concurrent to false as the triggers from master-builds will likely
cause it to run multiple times in parallel otherwise.
Related: https://jenkins.osmocom.org/jenkins/view/OBS/job/Osmocom_OBS_master_obs.osmocom.org/
Related: https://obs.osmocom.org/project/show/osmocom:master
Related: OS#2385
Change-Id: I53a494f13f81ae837f2d362c54e1bdf13f121db3
Add a new master feed, where packages are updated as soon as patches get
merged to master. Upload a commit_$COMMIT.txt file for each package in
this feed and use it to determine if the package needs to be updated or
not.
In most packages the commit is already part of the version in the dsc
file, e.g. "libosmocore_1.7.0.38.c3b90.dsc", but that's harder to parse
and is more likely to have a hash collision (just 5 characters).
Related: OS#2385
Change-Id: I3b0b4f4876b8c1eeb61f20d903a6f2cac6e99638
Make it slightly easier to read, before extending the latest branch with
master too in a follow-up patch.
Related: OS#2385
Change-Id: I4f1d4e250eb4d8163f1ded29106a44deb0d0feff
Refactor checkout_default_branch so the get_default_branch logic can be
used from another function in a future patch.
Related: OS#2385
Change-Id: If3f0ab500ff165d81f61b19ad41af90bd39dce8f
Coverity job currently fails because it fails to fetch osmo-gmr.git.
Change-Id: I4d52dbf0f81332dc1856944a6a3c7376cb1e0638
Fixes: I17590d0b0b46274e2ceb007e20d7d1dc039f4e8d
Replace the legacy git.osmocom.org clone URLs with either gerrit or
gitea in most places. Besides not using legacy urls anymore, the
advantage of using gerrit directly is that we don't have a delay of 10
min to several hours on the repository.
Not replaced:
* ansible/roles/osmocom-workstation: not sure if still used, would need
a bit of a rewrite to use logic similar to osmo_git_clone_url().
Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
Change-Id: I90409c9e7f4db4e1765647021625f9acfccaa78b
This had been added for debugging purposes and isn't used anymore. Also
one less hit for git grep git.osmocom.org.
Change-Id: I10cbb598f0d5d675034ca87568a8b6f1920f2ad3
Update the function to not use legacy URLs anymore and allow adding
future gitea URLs more easily. Update the description of the function to
reflect the new motivation for using this.
Change-Id: Ibf53ca2c444fae82af73e417410faa5b16f5de59
While generating srcpkg we're not really interested in entire
project history (which can be quite heavy) - let's only get
the bare minimum from git.
Change-Id: I71629c244d2c29728a88871e83b20493341321c8
Having an option to explicitly checkout sources for package from gerrit
using numeric ID comes in handy for testing changes.
Related: OS#4107
Change-Id: I403c2083ef8c3f6f966c5b376fb13ea46d8a7a2b
Not all the required programs are actually required to build subset of packages.
Add option to ignore the default check.
Change-Id: I8b5eaedab87b586fb1ca4c22746be4dfbb3c2176
Don't pass stdin to the programs, as we expect them to run
non-interactively and also don't show the program's output unless -v is
used or the exit code is not 0.
Change-Id: I7e893101c2a3e7b005659ec72aa44fa932b7ccd9
Check if the oscrc exists, as otherwise the osc command will
interactively prompt for the credentials. If running without verbose
mode (the default), the command output is hidden and so it would just
appear to be hanging without this patch if the config doesn't exist.
Change-Id: I6f140b2c1d171d3019fd259399e6a46da9c6451f
When running in verbose mode, it's enough that the output is displayed
while the command is running. Don't print it again on error.
Change-Id: I4fcf2421a6f82bd94ba098b39f58e5d3f18aa8fd
Don't attempt to install nftables-devel or python3-nftables in centos8
in the repo-install-test. What happens is, that dnf tries to install the
versions from the centos repositories, which conflict with the ones from
the OBS repositories. This fixes the failing
Osmocom-repo-install-centos8 jenkins job.
Change-Id: I57670069cc09a5493667971997f0245e1d6a7259
Instead of hardcoding https://git.osmocom.org for all clones, use
the osmo_git_clone_url function. This clones via https from gerrit where
possible, it should fix the many errors we are currently seeing from
the master-openbsc job:
+ osmo-deps.sh libosmo-sccp
+ project=libosmo-sccp
+ branch=master
+ git branch -a
+ grep -c remotes/origin/master$
+ [ x1 != x0 ]
+ branch=origin/master
+ test -d libosmo-sccp
+ git clone https://git.osmocom.org/libosmo-sccp libosmo-sccp
Cloning into 'libosmo-sccp'...
error: HTTP/2 stream 0 was closed cleanly, but before getting all response header fields, treated as error (curl_result = 92, http_code = 0, sha1 = d23e38020fdbb685570145acd3a35e22a5a91344)
error: HTTP/2 stream 0 was closed cleanly, but before getting all response header fields, treated as error (curl_result = 92, http_code = 0, sha1 = 68b450098714e3015a65b6628a7e61ac09dd4d47)
error: Unable to find 68b450098714e3015a65b6628a7e61ac09dd4d47 under https://git.osmocom.org/libosmo-sccp
Cannot obtain needed tree 68b450098714e3015a65b6628a7e61ac09dd4d47
while processing commit d23e38020fdbb685570145acd3a35e22a5a91344.
error: fetch failed.
Related: https://lists.osmocom.org/hyperkitty/list/openbsc@lists.osmocom.org/thread/RHMXTPTKU2SPGCVXF55RFJQ5GYBZ46CO/
Change-Id: I700d608ff74eca3981ed41f04ee9ced9629436aa
Deb and rpm packages don't build for osmo-upf so disable it again until
it is fixed.
Related: OS#5655, OS#5654
Change-Id: Idf18159b786c51154fda9b40758b08a8656fc215
For the conflict test, don't attempt to install packages from the next
feed for centos8. This was useful when we didn't have binary packages
for centos8 for latest yet, but we have them now. The next feed existed
temporarily and isn't updated anymore for a long time. We did not add it
to obs.osmocom.org.
This is in preparation for adjusting repo-install-test to use packages
from obs.osmocom.org, where we don't have a next feed.
Change-Id: I73a6c051c423eb30ddeab1688fc78fe959e60cf4
If there's an empty debian/changelog, packaging needs to be fixed in the
git repository of the project. Print a meaningful error instead of:
File "/obs/lib/srcpkg.py", line 68, in get_version_for_feed
if ":" in ret:
TypeError: argument of type 'NoneType' is not iterable
Change-Id: I294d844ccccfa12599b6ba1def087e4a32d289e7
As Neels pointed out, with the current code the 2022q1 and similar feeds
do not get checked out properly if:
* the repository was not freshly cloned (like jenkins would do)
* the repository was fetched (--git-fetch)
Instead of the last revision of that branch from origin, we get the
revision that was previously checked out on the branch. Fix this by
prepending origin/.
Change-Id: I2e265fc84abd4596c386bb56cbabb2af121dc7ca
Fix the Osmocom-list-commits jenkins job that is currently failing with:
fatal: repository 'https://git.osmocom.org/osmo-upf/' not found
Change-Id: I2acb9880be57411f11805a2195076b514156aaf3
Looks like I copied the list of packages from the old latest script, so
these two that were only in nightly are missing. Once added, they will
be built for nightly again and not for latest as there's no tag matching
the patterin in the git repositories.
Change-Id: I4b96f15e95c70f1dca12a292964e6f12b04e4a04
Apparently a \n at the end of debian/source/format is not required, so
don't fix it if it is missing. This avoids printing this confusing line:
osmo-trx: fixing debian/source/format (3.0 (native) => 3.0 (native))
Change-Id: I7f9bb22a389a2109109f7fecd3b7ae0413fe6f5b
Harald requested that the OBS scripts should not stop if building one
specific source package fails, instead it should keep going and report
at the end a non-success exit code.
Given that the shell script code has historically grown and became hard
to maintain, I decided to rewrite the scripts for implementing this
feature. This rewrite solves additional problems:
* No full checkout of an OBS project like network:osmocom:latest
anymore, with lots of packages that won't get updated (e.g. the uhd
package has a uhd-images_3.14.1.1.tar.xz file that is 108 MB). With
the old code, developers had to wait minutes during the checkout
before the script reaches code that is currently being developed. Now
only single packages get checked out right before they get updated.
* No need to clone git repositories over and over. With the new code,
git repos only get cloned if needed (for latest it is not needed if
the remote git tag is the same as the version in OBS). During
development, the cloned git repositories are cached.
* Output from commands like "git tag -l" is not written to the log
unless they failed. This makes the log more readable, which is
especially important when a package fails to build, we keep going and
need to spot the build error in the middle of the log later on.
* No more duplicated code for nightly and latest scripts that worked
similar but had slight differences. Also the list of packages is not
duplicated for nightly and latest anymore; nightly uses all packages
and latest uses packages that have at least one git tag.
* Building source packages is decoupled from uploading them. A separate
script build_srcpkg.py can be used to just build the deb + rpm spec
source packages, without interacting with the OBS server.
* The scripts can optionally run in docker with a command-line switch,
and this is used by jenkins. This way we don't need to install
more dependencies on the host such as rebar3 which is now needed for
erlang/osmo_dia2gsup.
* Add erlang/osmo_dia2gsup and run its generate_build_dep.sh (SYS#6006)
I have done the new implementation in python to make use of argparse
and to be able to use try/except and print a trace when building one
package fails.
Example output:
* https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_nightly_obs.osmocom.org/48/console
* https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_latest_obs.osmocom.org/46/console
Change-Id: I45a555d05a9da808c0fe0145aae665f583cb80d9
Fix the pattern that gets the source tarball with ls in order to update
the "Source:" line in the spec file. The previous assumption was that it
would always match:
ls -1 "${name}_"*".tar."*
But this isn't the case anymore since a spec file was added to
libosmo-dsp where the package name ($name) is libosmo-dsp, but the
tarball doesn't have the minus character:
libosmodsp_0.4.0.1.d7d9.202207150002.tar.xz
Therefore just change the pattern to:
ls -1 *_*.tar.*
We can do this because there is always only one tarball in the directory
and we wouldn't use ".tar." in a package name.
Fixes: OS#5619
Change-Id: Ic704f480102ab6f90f3936a3be30e3020402e750
Make the variable available, as it's needed by the scripts that
osmocom-packages-docker.sh calls since
Ib271e16e93ebc58ed1ecb0ea881e1f1370e40928.
Change-Id: I8e53523b9b1ceafa45fe7b3dca2731d4e1c74959
Build debian-bullseye-erlang on x86_64 nodes only, as the image doesn't
get used on nodes with other arches. The image is only used by
gerrit-verifications jobs for osmo_dia2gsum and osmo_gsup, which are
restriected to run on these nodes:
https://jenkins.osmocom.org/jenkins/label/osmocom-gerrit-debian9/
Remove the check and comments for not building for debian 10 x32, as
that node has been replaced with a debian 11 one.
Related: OS#5453
Change-Id: I5636dfc9e06a44b096f9dc55528d677ea2dfc352
osmo-python-tests was added to latest just like it was in nightly, by
adding the following two lines to build_osmocom():
checkout python/osmo-python-tests
...
build osmo-python-tests
But the build fails, because the "checkout" function clones into a
"python/osmo-python-tests" directory in latest, and not into a
"osmo-python-tests" directory like in nightly. Make it behave the same
by adjusting the "checkout" function in nightly.
(This code has historically grown and should be refactored imho, but
let's fix this bug first.)
Fix for:
====> Building osmo-python-tests
+ cd /home/osmocom-build/jenkins/workspace/Osmocom_OBS_latest_obs.osmocom.org/osmo-python-tests
./scripts/osmocom-latest-packages.sh: 99: cd: can't cd to .../osmo-python-tests
Change-Id: I97074ee953cadac476fe2e4b686689c26bac1ea1
Move the long shell command into a jenkins.sh in docker-playground, and
move the osmocom-release-tarballs.sh script there as well. This makes it
easier to test the script, and only patching one repository is needed
from now on when changing to a newer debian version.
Depends: docker-playground I4cf66097f508c029f688ec8af1cab2238d42f84f
Change-Id: I15f6c08b807f1462f91e95b624cdc1aad0592dd6
Currently there is only one jenkins node labeled as
osmo-gsm-tester-build. Build the osmo-gsm-tester container only there,
not on all x86_64 nodes.
The /var/tmp/osmo-gsm-tester/state directory is set up by
ansible/roles/gsm-tester/tasks/main.yml.
Related: https://jenkins.osmocom.org/jenkins/label/osmo-gsm-tester-build/
Change-Id: I9191d6d54d516ce75f3f576f34463dcaad22c85a
Split the OBS related jenkins jobs into two, one that uploads to the
new obs.osmocom.org and one that uploads to build.opensuse.org.
Set an alias for "osc" that sets the right server, and set the
expand_aliases shopt so the bash scripts that source common-obs.sh will
not ignore the alias.
Related: OS#5557
Change-Id: Ib271e16e93ebc58ed1ecb0ea881e1f1370e40928
The public OBS key expired on 2022-05-22 and was replaced with a key
that was only shortly valid until 2022-06-08. Shortly after it was
replaced with a key that is valid longer, until 2024-08-02.
On 2022-06-09, one of the osmocom-repo-install tests started failing.
For some reason, the key in the latest/Debian_10 directory was not
updated to the latest one:
https://download.opensuse.org/repositories/network:/osmocom:/latest/Debian_10/Release.key
Since the key is the same for all of network:osmocom, adjust the
function to download it from a place that the OBS web UI links to when
attempting to download the public GPG key.
I guess the latest/Debian_10/Release.key will get updated once making a
new release and updating the packages in the repository. But sinc
there's a lot of other tasks to do, just use this practical solution for
now.
Change-Id: Idd0fb6e07cba959a36269244b0c7b5c62aaffeee
Don't attempt to build debian-bullseye (11) in docker on debian 10 x86,
as it results in:
GPG error: http://security.debian.org/debian-security bullseye-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://security.debian.org/debian-security bullseye-security InRelease' is not signed.
This is a temporary measure until we've upgraded the only jenkins node
that runs debian 10 on x86, gtp0-deb10build32.
Related: OS#5453
Change-Id: Iba7679c027ca89a29393c220d01b195cfce4fec3
For a long time I was using '{}' to indicate a nulled out struct. But
apparently '{0}' is the favored way to write that. Let's allow using
this notation to terminate a value_string[].
Change-Id: Id2f5ba897ec83f34f8d3c4425353c0baf309bb6d
Let's use newer debian to avoid problems with older erlang. Furthermore,
the new image builds rebar3 to avoid issues with unmatching erlang
versions.
Change-Id: I7b1956c515daccf6ab1ff87031c6fec649cadb4b
Only run the simple image clean code if docuum is not running. It works
well enough in most cases, but has the drawbacks that it never deletes
"latest" images or images not matching "^osmocom-build", and may delete
images that are still being used (OS#5447). With the other tool, all
images are considered for removal, and the ones that have not been used
the longest time are removed first.
Related: OS#5477, OS#5066, SYS#5827
Change-Id: I1cef0833c096de0fa5acf77156bb5dd362e2ef9c
Do not only clean up dangling images, but also containers, volumes and
networks.
Related: SYS#5827
Change-Id: If441b251de50063f0229d36fb1bc260a4cb1dd87
Clone simtrace2.git before trying to create the tarball with git.
Fix for:
simtrace2
simtrace2-0.1.tar.bz2 (creating)
+ cd /osmo-ci/_temp/repos/simtrace2
/osmo-ci/scripts/osmocom-release-tarballs.sh: 195: cd: can't cd to /osmo-ci/_temp/repos/simtrace2
Related: OS#5347
Fixes: 0221a0 ("OSMO_RELEASE_REPOS: add simtrace2, osmo-remsim")
Change-Id: I0a845549ba1fe9f0d9ab55a5c5c7bf5b8f57caae
Adjust to simtrace2's directory structure, which does not have a
configure.ac in the main directory like all other repositories. The main
directory has a regular Makefile without autotools, only the host dir
has a configure.ac file (and only in newer versions). Deal with this by
creating two tarballs, one with "git archive" for the whole directory,
and one for the host dir only with the usual "autoreconf -fi;
./configure; make dist-bzip2". The latter one has the files created by
autoreconf ("configure" script and others).
simtrace2
├── simtrace2-0.1.tar.bz2
├── simtrace2-0.2.tar.bz2
├── simtrace2-0.3.tar.bz2
├── simtrace2-0.4.tar.bz2
├── simtrace2-0.5.1.tar.bz2
├── simtrace2-0.5.tar.bz2
├── simtrace2-0.6.1.tar.bz2
├── simtrace2-0.6.tar.bz2
├── simtrace2-0.7.0.tar.bz2
├── simtrace2-0.7.1.tar.bz2
├── simtrace2-0.8.0.tar.bz2
├── simtrace2-0.8.1.tar.bz2
├── simtrace2-host-0.6.1.tar.bz2
├── simtrace2-host-0.6.tar.bz2
├── simtrace2-host-0.7.0.tar.bz2
├── simtrace2-host-0.7.1.tar.bz2
├── simtrace2-host-0.8.0.tar.bz2
└── simtrace2-host-0.8.1.tar.bz2
Closes: OS#5347
Change-Id: Ib52a23a2a7d6ea64bfa539b1d026f035fdb3af57
Old osmo-msc releases failed to build because logging output of
libosmo-mgcp-client has changed. I'm backporting the fix as 1.7.1 and
1.6.4. The script builds the last 3 releases (1.6.4, 1.7.0, 1.7.1), so
mark 1.7.0 as known error.
Related: osmo-msc Id197e4ab9ba12e284299ef520edee9c362513bf1
Change-Id: I86f8252d450165f4be3d7c97fa70235638f7dd96
Drop these workarounds, as we are not building binary packages for
debian 8 anymore.
Related: OS#5223
Change-Id: Ibe7ba124557969df62798ba49c4489e9606c2341
Disable osmo-pcap-server for latest again, as the port is still
conflicting there with osmo-bts.
Fixes: 7ca9c4 ("repo-install-test: clear SERVICES_NIGHTLY list")
Related: OS#5203
Change-Id: I711e0e13c3e3af30407b85fd10aca9446f2b94ba
All services that start up in nightly should also start in the latest
release now, as new versions have been released of related programs.
Change-Id: Idc94270978ec5ca67d6f7e20e009f905e972f984
Use the nightly script instead of latest, so all packages get
upgraded when upgrading osmocom-2021q1.
Related: SYS#5370
Change-Id: If8de585652997aae1edb586c948c181f564f6994
Determine the package name of the conflict package itself and all
conflicting packages automatically, so we don't need to have it
duplicated in the OBS latest and OBS nightly scripts.
This is in preparation to move osmocom-2021q1 from latest to nightly.
With the current logic in nightly for the conflict package, it would not
be possible.
Related: SYS#5370
Change-Id: I183b9040250e66e0d7d17ef4b95af9e7d4a26f04
Allow overriding OSMO_OBS_CONFLICT_PKGVER, so we can increase the
version of the osmocom-2021q1 package whenever something changes on one
of the 2021q1 branches.
Related: SYS#5370
Change-Id: Iae45d7462aab8227ed3756e6cccfa3e64cb04211
Add a simple helper script to run osmocom-*-packages.sh in docker to
avoid installing dependencies on the host system.
Related: SYS#5370
Depends: docker-playground Ibb55ad18d2ccf4313f52fa3e3c10d4420c84dced
Change-Id: Icc89e20950c2aaa67b209340d1d797b76fce32d2
Move get_commit_version to common-obs.sh and call it in
osmocom-latest-packages.sh, if the feed is not "latest". This way, the
packages don't have the latest tag as version anymore, and the version
changes if commits get pushed to the feed's branch.
Related: SYS#5370
Change-Id: I4a4fa3b8f66652ef36a7fe62047a88a69c473f19
Move git_version_gen calls into an own function and add some of the
description from I76e3713f0b01a6110091ff90e8e53aa79533c374 where this
code was added.
Don't call it inside get_commit_version anymore, but call it before.
Don't try to cat the resulting .tarball-version there if it doesn't
exist.
Related: SYS#5370
Change-Id: I9a1b6ae4b4311abb77dc6390733c5e330e3d489e
Skip checking out and building source packages of all other packages, if
the environment variable is set.
Related: SYS#5370
Change-Id: I83c3744713fd6abda4b832460f30eb2e79ebeed8
Uploading to network:osmocom:* should only be done when these scripts
are running in the Osmocom jenkins. Remove the default and require users
of the script to explicitly set PROJ.
Related: SYS#5370
Change-Id: If49ce217e77716b63dfde9139e869672a54b66a2
Instead of only appending the date to non-Osmocom packages in
get_commit_version, append it to all packages in build(). This ensures
we increase the version of Osmocom packages even if the commit did not
change.
Fixes: OS#5135
Change-Id: I04d84f39f4093c8edfe21a94c10ecb8d3c7b5b64
Skip the logic to generate a new debian/changelog version for
osmocom-nightly and osmocom-next packages in build(). Use
$OSMO_OBS_CONFLICT_PKGVER instead, as it gets written to
debian/changelog in common-obs-conflict.sh already, and append the date
to the variable.
Related: OS#5135
Change-Id: I85f0bcb633c16c7b5a81104f198d9561f53c0c01
Depending on the service, error messages are not shown with the
systemctl command. Run journalctl for failed services so we get the
reason for the failure in the jenkins log.
Related: OS#5130
Change-Id: Ib454424d7867137246fadd73255d4dbff63731a6
Fix osmo_obs_add_rpm_spec() to not assume to be in the $oscdir.
This caused the following error when being called from
osmocom-latest-packages.sh in the code path for adding a new package:
ls: cannot access 'osmo-gbproxy_*.tar.*': No such file or directory
Related: OS#5051
Change-Id: I467e332b69accfabba53332fdb9cd785991855fc
Replace the current logic, that would only run osmo_obs_add_rpm_spec
when adding a new package, or when the version of a package has changed,
with running it every time.
Running the command when it is not needed does not hurt, as it does not
take significant time, and osc does not attempt to upload the file when
it did not change.
The advantage is, that we can update/upload the spec file without
tagging a new version, if a bug prevented it from getting uploaded
before (as it just was the case for all Osmocom packages).
Related: OS#5054
Change-Id: Ie067c97b5f54ec5b3309ddbd2bfb7f846cd0ccd3
Pass the path to the project's git repository to osmo_obs_add_rpm_spec,
instead of $output (has the output of "gbp buildpackage").
Related: OS#5054
Change-Id: I799398120ab0cbdb74b2d74a3fb139395d66d449
Make a separate commit for the distro specific patch, instead of using
"git commit --amend". Otherwise, if HEAD was pointing to the latest tag
before the amend, git-version-gen will use the previous tag instead of
the latest one after the amend.
Fixes: OS#5053
Change-Id: I67770a19ee60101df989f98673a22705ad50beed
Harald Welte
Oliver Smith
Vadim Yanitskiy
Max
Neels Hofmeyr
Pau Espin