Commit Graph

1302 Commits

Author SHA1 Message Date
Oliver Smith 62693be148 OBS: fix generating wrong limesuite version
Instead of using git-version-gen in all git repositories, only use it
where it is part of the repository. Use git directly to build the
version string otherwise. To fix generating the wrong version string for
limesuite, use "git describe --tags" instead of "git describe" as in
git-version-gen. This is needed because limesuite releases > 20.01 only
have lightweight (non-annotated) tags.

Fix for:
  limesuite: WARNING: version from changelog (22.09.1-1) is higher than
  version based on git tag (20.01.0.182-4828.202309140026), using
  version from changelog (git tag not pushed yet?)

Change-Id: I00d97d5cd12b2938a002d3e5c8ada6503f976e36
2023-09-14 15:40:53 +02:00
Oliver Smith 774347c25b jobs/master-builds: add osmo-python-tests
Run contrib/jenkins.sh of osmo-python-tests once a day from master too,
not only from gerrit verifications.

Change-Id: I164553d0948549d60c45b8840716c608463dc486
2023-09-13 16:11:29 +00:00
Oliver Smith 03cb833746 jobs: update trigger timers
Ensure we trigger building new OBS packages first, then wait plenty of
time until the binary packages are available (and run jobs in the
meantime that don't need them), and only after that we run the jobs
that need the binary packages.

Otherwise TTCN3 jobs may test the packages from the previous day, and
some jobs may fail completely due to packages not being completely built
yet. For example, yesterday the new Osmocom CNI releases were tagged,
which means the :latest packages also need to be rebuilt (-> building
all OBS packages takes longer). The osmocom-release-manuals and
-tarballs jobs failed, because the new binary packages were not
available yet when they ran.

Change all timers to the format "H 20 * * *" to have a deterministic
hour and semi-random minute based on the job name.

Change-Id: Ib68f9a78bae27a63706a8c95715bf6a244b7bf1d
2023-09-13 16:11:29 +00:00
Oliver Smith 757d58710b OBS: keep going if version can't be parsed
Don't crash in Osmocom_OBS_wireshark because the version can't be
parsed.

Fix for:
  packaging.version.InvalidVersion: Invalid version: '4.1.1rc0.238'

Change-Id: I0cf6c7c2d5939dccc14107f0a8c15593ea8bc4c2
2023-09-13 15:00:38 +02:00
Oliver Smith f453e35022 scripts: wireshark: remove laforge/doc-fixes
The two patches from this branch have been upstreamed:
15ed53005b

And with recent changes to docs/README.dissector, git fails to detect
that these are already applied.

Fix for:
  CONFLICT (content): Merge conflict in doc/README.dissector

Change-Id: I17017ba396996e781250d30406193c4f5ec44f82
2023-09-13 14:38:04 +02:00
Oliver Smith 3970487882 OBS: don't downgrade version in debian/changelog
Fix the bug that the version in debian/changelog could get lowered if a
release was made and the release git tag was not pushed yet.

Fixes: OS#6173
Change-Id: I550ed10a60c863626d870e35034028f0bd066211
2023-09-12 17:31:45 +02:00
Oliver Smith d27ba27a55 OBS: exit() -> sys.exit()
As pointed out in code review, sys.exit() should be used instead of
exit() from site.

https://docs.python.org/3/library/constants.html#exit says:
> The site module (which is imported automatically during startup,
> except if the -S command-line option is given) adds several
> constants to the built-in namespace. They are useful for the
> interactive interpreter shell and should not be used in programs.

Change-Id: Ia9d355e660afd1ad3c60064900d8be66e8e1d385
2023-09-08 11:09:39 +02:00
Oliver Smith 10d7aedfeb OBS: build_binpkg: support debian:testing
Change-Id: I71de94ef34c3ccc3610643bd8861b78cccd266a5
2023-09-08 11:09:39 +02:00
Oliver Smith 1b66903ddb jobs/osmocom-obs-sync: new job
Related: OS#6165
Change-Id: Ibcf02eb5feee2b314ff28861835040c21aea4294
2023-09-08 09:06:57 +00:00
Oliver Smith 4cfe5d42b5 OBS: add sync_obs_projects.py
Add a new script that syncronizes base distributions from openSUSE's
OBS, like Debian:12. This is done by getting  "meta" and "prjconf" from
the source OBS, adjusting it and writing it to our OBS instance.

The motivation for this is making our OBS more resilient, by using these
synchronized projects we can build packages even if the openSUSE OBS API
is not reachable.

Related: OS#6165
Change-Id: I00122f7819ba682cb791c370d781377cfb302629
2023-09-08 09:06:57 +00:00
Oliver Smith aa2e3cc5c0 OBS: lib/osc: add private proj variable
Adjust lib/osc.py to take the OBS project from a global variable that
can be set with lib.osc.set_apiurl(), instead of using
lib.args.obs_project. Move the sanity check to set_apiurl().

This is in preparation for adding a new sync script, which will operate
on multiple OBS instances.

Related: OS#6165
Change-Id: Iabb871bcb432f2fbdaec9cbcab3d663ccf969901
2023-09-08 09:06:57 +00:00
Oliver Smith 18c730fc10 OBS: build_srcpkg.Dockerfile: use bookworm
rebar3 is in debian bookworm's repositories, hence it is not needed
anymore to use osmocom:latest here.

Change-Id: Iddd0295ee928fc9a47d4d206e414b78a2561d13d
2023-09-08 09:06:57 +00:00
Pau Espin 4f538b554a lint: Add osmo_ss7_asp_rx_unknown_cb to typedefs_osmo.txt
Change-Id: Ifeb2ed66f32b2c3802ea1aab4857f334c3748da3
2023-09-07 13:20:57 +02:00
Oliver Smith eac517c1af jobs/master-builds: xgoldmon: remove osmo-ci dir
Fix running the job multiple times in a row. As the master jobs don't
wipe the git repositories, we need to remove the osmo-ci dir first
before attempting another shallow clone.

Fixes: 451cbe7d ("jobs/master-builds: add xgoldmon")
Change-Id: I877944dbca9d1c3ca57b05e947ba9b3506971bdc
2023-08-30 12:25:52 +02:00
Oliver Smith 9ba26aa8fb jobs/master-builds: add osmo-ir77
Replace the legacy osmo-ir77 job (not done with jenkins-job-builder)
with a new master-osmo-ir77 job that works just like the other master
jobs:
- build in docker
- have build commands in contrib/jenkins.sh

Depends: satellite/osmo-ir77#1
Change-Id: I45034e4ed9ed8ad5683ac2de24521649f138b41c
2023-08-29 13:02:33 +02:00
Oliver Smith 1b6970edac jobs/master-builds: add simtrace
Replace the legacy SIMtrace job (not done with jenkins-job-builder) with
a new master-simtrace job that works just like the other master jobs:
- build in docker
- have build commands in contrib/jenkins.sh

Set the same notification mails as for simtrace2.

Depends: sim-card/simtrace#1
Change-Id: I0980ceafa4d1187630b75b45b01b538c750021bb
2023-08-29 13:00:33 +02:00
Oliver Smith 451cbe7d86 jobs/master-builds: add xgoldmon
Replace the legacy xgoldmon job (not done with jenkins-job-builder) with
a new master-xgoldmon job that works more similar to other master jobs:
- build in docker
- have build commands in a jenkins.sh script

Put the jenkins.sh script into osmo-ci, as the upstream repository is
outside of Osmocom infrastructure.

The motivation for this change is, that the current xgoldmon job is
failing since libosmocore depends on liburing by default. This uncovered
that the job is still running outside of docker, where the dependency
has already been added. The following patches will modernize other jobs
which have the same problem.

Change-Id: Ice5704eb12f3c3a777961bc18a55fac63df80fd6
2023-08-29 12:56:34 +02:00
Pau Espin 896bb61d1b jobs: Add nightly job for ttcn3-dia2gsup-test
Related: OS#5757
Change-Id: Icb1a18da0daea1b481b4633eed2de817bfd5a15a
2023-08-28 12:34:07 +00:00
Oliver Smith baa1c11915 jobs/coverity: fix path to home dir
Fix for:
./jenkins.sh: line 35: /home/build/osmo-ci/coverity/get_token.sh: No such file or directory

Fixes: 56bc906e ("coverity: run inside docker")
Change-Id: I87fadd2dffcfaa04eaa942dfb8a496334cb722d5
2023-08-28 12:39:20 +02:00
Oliver Smith 56bc906eed coverity: run inside docker
Run the coverity job inside docker, so all depends are available (fixes
that it currently fails because liburing isn't available for
libosmocore).

Depends: docker-playground I25862a7e3c8a73e13fd4a9237ab57500d8dfc95c
Change-Id: I5cfdb6b2e12e176ff6d6ed6c1b8505d7694993f9
2023-08-25 13:32:59 +02:00
Oliver Smith e9f537dd76 jobs/osmocom-build-tags: run inside docker
Run this job inside docker, so all depends are available (fixes that
it currently fails because liburing isn't available for libosmocore).

Change-Id: I5a8243b3096dba8f94f715413c84683c7495777c
2023-08-25 13:32:59 +02:00
Oliver Smith bb6db80a2c scripts/osmocom-api-doxygen: check ssh early
Check for working SSH right at the start of the job, so we don't need to
wait 2 minutes until it has built everything to find it out.

Change-Id: Id93c15d38a49859dc63d39f70f051a7df871c398
2023-08-25 13:31:24 +02:00
Oliver Smith 53bd6b8ade jobs/osmocom-api: run inside docker
Run the osmocom-api job inside docker, so all depends are available
(fixes that it currently fails because liburing isn't available for
libosmocore).

Use the contrib/known_hosts file, instead of writing an own copy during
the job.

Change-Id: I6e831c71c4c88772c3e4232fcb1a9e2c1c73d997
2023-08-25 13:31:24 +02:00
Oliver Smith e4ac3ae199 jobs/gerrit: osmo-dia2gsup: test debian 11 and 12
Don't try to build the debian package for debian 10. It fails as rebar3
is not in debian 10. Test the build for debian 11 and 12, which is what
we build the package for on obs.osmocom.org.

Change-Id: Id01b466f1bacc9cbb8e835f69da765f5fdccfdc2
2023-08-23 16:59:28 +02:00
Oliver Smith 228efc3f13 ansible: osmocom_build_deps: remove
The builds for almost all jenkins jobs are done in docker now, we
definitely don't need to install all of these dependencies on all
jenkins nodes anymore.

Remove the file for now, so it doesn't look like they still get used and
so we don't need to keep it up-to-date (e.g. inkscape was just replaced
with rsvg-convert).

When we deploy a new jenkins node and notice that we still do need a few
of these dependencies, we can bring them back later.

Change-Id: I7c87bd516d1bcdbcc21f65909dabbd3ccc38eede
2023-08-17 14:01:39 +02:00
Oliver Smith 8920d73730 jobs/osmocom-release-tarballs: modernize
Rewrite the osmocom-release-manuals script (previous version is in
docker-playground Ic35a28a386170b85d32aab8f2bd33e48e6d45392):

* Instead of using a separate docker container for this, that also lists
  all dependencies for all packages (as needed to pass ./configure), use
  debian-bookworm-build and install missing packages at time of
  generating the tarballs with "apt-get build-dep". Missing dependencies
  are typically other Osmocom libraries.

* This allows removing the debian 11 based release-tarball-build-dist
  container. As the script doesn't depend on a separate docker container
  anymore, move it to osmo-ci.git.

* Make it similar to scripts/manuals/publish-manuals-for-tags.sh, so it
  is easier to maintain both.

Related: OS#6057
Change-Id: I9f8b671b9780da500637a64fc4dbc72b450f9d11
2023-08-17 14:01:32 +02:00
Oliver Smith b65409baa9 scripts/manuals: adjust patch for removing draft
Adjust the patch, to not remove the "NOTE: TO REMOVE DRAFT MODE" as it
isn't in osmo-gsm-manuals anymore.

Related: osmo-gsm-manuals I23d11bab274793eb8bcedc3fd9290901823e32d2
Change-Id: Ia27cb8e4dd0204c0a91c322bed65bf8c9847a7be
2023-08-17 13:58:35 +02:00
Oliver Smith 688dcc1f6a jobs/registry-update-base-images: add bookworm
Related: OS#6057
Change-Id: I55ff43f1764dd5f7de46360264e0afbb4659910f
2023-08-14 09:28:40 +00:00
Oliver Smith 6ff5153042 scripts/osmo-ci-docker-rebuild: drop debian-bullseye-jenkins
It is not used anymore, stop building it.

Related: OS#6057
Change-Id: I8e11e4c87fea9a2cfc5e72f0427cf4ca82030554
2023-08-14 09:28:40 +00:00
Oliver Smith c749c28c3c scripts/manuals/publish-manuals-for-tags: use bookworm
Related: OS#6057
Change-Id: Ifa56e7f035fde5fb750376c64572277de2417d65
2023-08-14 09:28:40 +00:00
Oliver Smith bec3289f8e jobs/gerrit-verifications: use debian-bookworm-titan
Related: OS#6057
Change-Id: Ifd7fc7a2360fc64be78c6a06f42d0f4d46af01a0
2023-08-14 09:28:40 +00:00
Oliver Smith a5be55376b jobs/registry-rebuild-upload-titan: use bookworm
Related: OS#6057
Depends: docker-playground I4415fcde6518bd26d1e2cb5785ddaf84ac2116b3
Change-Id: Ibc25a62c2d3a2f6cd3e8136311ce2b37688c71a7
2023-08-14 09:28:40 +00:00
Harald Welte 3a74117ead update obs-mirror include list
* add Debian 12 for latest and nightly package archive
* replace nightly Ubuntu 22.04 with 23.04
* stop nightly Debian 10 archive (11 + 12 is sufficient)

Change-Id: I4f4103732a6dc3d73ada85482c6a2f806b4100aa
2023-08-11 08:05:45 +02:00
Oliver Smith 82ff45e7f7 repo-install-test: add debian12
Related: OS#6057
Change-Id: I51fa3c5212b5e29f4c2804384781693e8de2c097
2023-08-03 11:03:58 +02:00
Oliver Smith 2f5fa9edf8 jobs: remove endianness check
Related: OS#6122
Related: https://lists.osmocom.org/hyperkitty/list/openbsc@lists.osmocom.org/thread/ORBO2FWVVPCHTAXSPZTQLSSM4YB76ITB/
Change-Id: I2358e6bc8189f9c5bdca10aac28032cb7a4667fa
2023-08-02 12:25:25 +00:00
Oliver Smith 2a506ccad1 jobs/osmo-gsm-tester-runner: build debian-buster-jenkins
Fix the failing osmo-gsm-tester_virtual job by building the docker image
it depends on first, debian-buster-jenkins.

We are migrating most of the CI infrastructure away from old debian
versions to debian 12, but this is not possible here as explained in
OS#6126.

Related: OS#6126
Change-Id: I5f7468a402d82e3b6ee03b4f792ae7e3aae3942b
2023-08-01 11:35:00 +02:00
Oliver Smith 30d42ea961 jobs/osmocom-release-{manuals,tarballs}: set node
Set node to osmocom-master, so these jobs don't run on any node.

Change-Id: I3142b0463f7c61d47991822bf592f49ab4a421f7
2023-07-31 11:12:25 +02:00
Oliver Smith 7c15e261a9 jobs/master,gerrit: don't use bullseye for python2
openbsc, osmo-e1-recorder, osmo-smlc have been adjusted to not require
python2 anymore. Let them use debian 12 for building.

Closes: OS#5950
Change-Id: I1d9204b5b59866fa79839221bb47ec4f7206a982
2023-07-31 08:22:09 +00:00
Oliver Smith 592c48ac76 scripts/jenkins-gerrit: sort jobs list
Having two debian entries in there in random order looks weird so sort
the job lists.

Change-Id: I795f821783b52a6a9cd1967d33f67b656efccf22
2023-07-31 08:22:09 +00:00
Oliver Smith 56b1633d8b scripts/jenkins-gerrit: make assert more verbose
Change-Id: I0ea40663220f7eb8ee72e43f3f6b776be5835db2
2023-07-31 08:22:09 +00:00
Oliver Smith 7277e4399f jobs/gerrit-verifications: build for debian 10, 12
Build packages in CI for the oldest and newest debian release, for which
we provide binary packages in the Osmocom OBS repositories.

See "jobs: master/gerrit: use debian bookworm (12)" for reasoning:
I079e55a1325083714c8d39f922b2563e843fc0bc

Related: OS#6057
Change-Id: I959b466865bd327cc72cde4a1763ac13c2c2d797
2023-07-31 08:22:09 +00:00
Oliver Smith 6d9f0843a9 jobs/gerrit-verifications: fix binpkgs for libusrp
Don't build libusrp for centos8/almalinux. libusrp does have a .spec.in
file, but it needs SDCC for the build and SDCC is not available there.

In theory we could build the rpm in CI for opensuse tumbleweed, but it
requires adding support for building for opensuse in
scripts/obs/data/build_binpkg.Dockerfile and
scripts/obs/data/build_rpm.sh first, and since it works quite
differently than centos8 and libusrp changes rarely (last code change in
2021) I've decided to not do that now.

Fixes: OS#5898
Change-Id: If61765fe628321cae004307f4845d8927a1c7019
2023-07-31 08:22:09 +00:00
Oliver Smith 1dcb49a811 jobs/gerrit-binpkgs: don't hardcode distros
Make it possible to set a list of distributions to check in
gerrit-verifications.yml instead of only having one boolean for testing
two hardcoded rpm and deb distributions.

Change-Id: I59487e3dc2f55057de1b6a322f088fff0d18654c
2023-07-31 08:22:09 +00:00
Oliver Smith bd763747ad jobs/gerrit: refactor pipeline script
* Deduplicate the code to run a job by moving it to a function.
* Print the status of jobs right after they finished, instead of waiting
  until all jobs are done
* Make the status print messages more readable

Change-Id: I641a5b483721ce2bbf21bd61d8f4e83faf94ac24
2023-07-31 08:22:09 +00:00
Oliver Smith a4d780c6d5 Cosmetic: jobs/gerrit: update pipeline comment
Change-Id: If06af0e955240e1b9d678f1020767bbfb70b1d96
2023-07-31 08:22:09 +00:00
Oliver Smith 35bf872042 ansible/setup-simtest: don't install deps for docs
The part of the jenkins job that generates documentation doesn't run on
simtester anymore, so no need to install these dependencies anymore.

Related: pysim.git I5245c529db729e209d78a02ab9c917a90d0e0206
Related: OS#5497
Change-Id: I0dc1c9f9fc87ae1832d836d98f06e798b51c7e2e
2023-07-27 15:41:43 +02:00
Oliver Smith cbda91d500 ansible/setup-simtest: add tshark
As requested by Philipp, add tshark to simtest.

Change-Id: Ib330f08057888bab2253a9b38148eaa1a70f3800
2023-07-27 15:35:37 +02:00
Oliver Smith 36df36fa74 jobs/gerrit-verifications: disable lint for asn1c
The asn1c code does not follow our coding guidelines, and in the
interest of keeping it closer to upstream it does not make sense to
reformat it.

Change-Id: Iae97d8997b576e43c9a73dfcc61a9260f875310f
2023-07-17 11:58:44 +02:00
Oliver Smith 8ec747db10 jobs/update-osmo-ci: add ansicolor wrapper
Change-Id: Ic94f2cc14b2af3a3711ac3eadec8c92b62006587
2023-07-13 14:57:16 +02:00
Oliver Smith 127e79f171 jobs/update-osmo-ci: add branch parameters
Add parameters to set the osmo-ci and docker-playground branch. I'm
using this to test the debian 12 based containers before merging the
changes to master.

Related: OS#6057
Change-Id: I62265300048031cbb65e997b921373894500233f
2023-07-13 14:57:16 +02:00