Commit Graph

1221 Commits (a052c13c12a73ae195e9367aed43c30f7a0837c1)

Author SHA1 Message Date
Harald Welte a052c13c12 Revert "jobs: tweak build-discarder values"
This reverts commit 719ff97608.

The disk consumption of jenkins has grown by almost 500GB
since we merged this patch.  Clearly this is not expected, and we'd
have ran out of disk space in a few weeks.

I personally think the current allocation of 1.5TB of disk space to
jenkins should be more than sufficient; we just need to manage it
better.

Closes: OS#5980
Change-Id: I6b744a8b84a3e1255a8d51f73d1721ccfd028ac1
2023-03-28 13:35:33 +02:00
Alexander Couzens 703be698bf ansible.cfg: set default remote_user = root
Change-Id: Iccd7dce77ac408d1a0d5782b4563a5dfcf112030
2023-03-25 08:07:42 +00:00
Alexander Couzens 68e1cddb3d ansible: add ssh configuration
Change-Id: Idbc7a51c20125182b1e484ae516e8d72239c1e00
2023-03-25 08:07:42 +00:00
Oliver Smith 4427bd1b42 scripts/manuals: fix missing UserKnownHostsFile
Fix for:
  :: Listing files on server: /docs
  Host key verification failed.

Related: OS#5902
Change-Id: I485e2268eec50f46ed0436d4b3a5cedfab20af96
2023-03-24 10:43:17 +01:00
Oliver Smith f9a7f1e893 jobs/osmocom-release-manuals: add new job
Add a jenkins job that automatically builds manuals for release tags, if
they don't exist on the server already.

Closes: OS#5902
Related: https://downloads.osmocom.org/docs/
Change-Id: I0ecb238660553c3c857e1b310873eca8a8d09dab
2023-03-23 10:48:30 +00:00
Alexander Couzens 6e1210326b ansible: hosts: replace invalid char minus ("-") in host group names
Change-Id: I0c13ce4edb7f57c7ff77141b5511080bba639ec0
2023-03-23 08:52:06 +00:00
Alexander Couzens 607cedc308 ansible: install-coverity: ensure it's not installed twice
Check if the coverity path already exists. Even this wouldn't
detect if the coverity has been only installed half way.

Related: OS#5801
Change-Id: I95549983bb6bd47e04eb37c73afe5409637f87d3
2023-03-23 08:52:06 +00:00
Alexander Couzens f2e49f051c ansible: install-coverity: add missing tag coverity
Change-Id: I86be4c72ad3bce070ddf7f8270492b6cf8ea826e
2023-03-23 08:52:06 +00:00
Alexander Couzens 6d7432e774 ansible: install-poky-sdk: add tag poky to the apt install task
Change-Id: Id8e433a789aff8df241687f1f109a161e725a29a
2023-03-23 08:52:06 +00:00
Oliver Smith 937425179d obs: wireshark: remove laforge/amr-2023 branch
Remove it, as the AMR patches have been upstreamed \o/

Related: https://gitlab.com/wireshark/wireshark/-/merge_requests/10025
Change-Id: I7f3f06bc5390ffac9b3760035831ae17d28b744e
2023-03-22 12:02:12 +01:00
Oliver Smith b55b11d671 obs: build_binpkg: add ubuntu 22.04
Add it for debugging a build failure that we only saw on Ubuntu.

Related: OS#5946
Change-Id: I4a927447d8f8b89411b93c563fbc6aa8d4998ffa
2023-03-22 09:30:53 +00:00
Oliver Smith d6eed85d44 jobs/master-builds: use debian-bullseye-jenkins
I've verified that all master-builds jobs still work with this change.

Related: OS#5949
Depends: docker-playground I51925d0ab9e5a779379efab59c381ef12fb60929
Change-Id: I7d5bc7bb4c1457d4e05fd6e0d27668382c39973a
2023-03-21 16:49:22 +01:00
Oliver Smith 24f3e088dc scripts/docker-rebuild: add debian-bullseye-jenkins-arm
Prepare to let all master-builds jobs use debian-bullseye instead of
debian-buster.

Depends: docker-playground I8e1741f86ffb8abd658d1e4e0415dfd11fb1a8a1
Related: OS#5949
Change-Id: I55bd5869ecd753549b8f2f2822e825623d940acd
2023-03-21 16:48:50 +01:00
Oliver Smith 22f3e07636 jobs: master-osmo-e1-hardware: fix manuals
Build the manuals with the regular docker image, instead of the
fpga-build one.

Fix for:
  build/shrink-pdfs.sh: 8: build/shrink-pdfs.sh: ps2pdf: not found

Related: SYS#6380
Change-Id: I6e9f832dc449af0ca7def29ef5a9161285b01736
2023-03-20 10:46:30 +01:00
Oliver Smith e7eba15315 jobs: add Osmocom_OBS_wireshark
Add a nightly jenkins job that does the following:
* Clone Wireshark from upstream
* Merge several Osmocom branches on top
* Build a source package
* Submit it to the osmocom:wireshark OBS project

Related: OS#2537
Change-Id: Ifb49c5cb22a4de0da30a920e5450a27172b11d73
2023-03-17 11:33:55 +01:00
Oliver Smith 1ab4ce32f9 obs: add --version-append argument
Related: OS#2537
Change-Id: Iff5a530bebd645be13e0206ce0a670e1dfe6e2f9
2023-03-17 11:33:55 +01:00
Oliver Smith dc1415959f obs: add --allow-unknown-package
Prepare to add a wireshark package. It will be in its own
osmocom:wireshark repository, and not in osmocom:nightly, :latest,
:master, so it's not added to lib/config.py.

Move lib.set_args() before parse_packages(), so the latter is able to
use lib.args.allow_unknown_package.

Related: OS#2537
Change-Id: I007f937ccb629e0593ec9253541ed05df42fac66
2023-03-17 11:33:55 +01:00
Oliver Smith 3ccebc4514 obs: srcpkg.build: fix conflict_version regression
During refactoring, conflict_version was changed to version here, which
was wrong. The packages need to depend on the conflict_version of the
meta package, version is the version of the package itself.

Fix for:
  The following packages have unmet dependencies:
   libosmo-netif-dev : Depends: osmocom-nightly (= 202303160009) but it is not going to be installed
                       Depends: libosmocore but it is not going to be installed
                       Depends: libosmonetif11 (= 1.3.0.8.9e65.202303160009) but it is not going to be installed
   libosmocore-dev : Depends: osmocom-nightly (= 1.8.0.79-b394d.202303170006) but it is not going to be installed
                     Depends: libosmocore but it is not going to be installed

Fixes: 0ed0e464 ("obs: don't pass conflict_version through functions")
Change-Id: I2326e3817c6f6887ef1196e603c3877768119a66
2023-03-17 11:33:45 +01:00
Oliver Smith 8d137c66a7 obs: add --git-skip-checkout argument
Prepare to use this for wireshark, to build a branch on-the-fly that
does not get pushed to origin, with Osmocom patches on top of upstream
master.

Related: OS#2537
Change-Id: Ifc963daf51ba3542f67420daaf7b29745404a92e
2023-03-15 18:25:52 +01:00
Oliver Smith f7ff90d64b obs: don't pass skip_up_to_date through functions
Change-Id: Ide0c1f5452f5ce965541b45039345abc52b3d140
2023-03-15 18:25:52 +01:00
Oliver Smith 78cd3a3c75 obs: don't pass meta through functions
Change-Id: Ieb49e5f8630e3e918b11a7a225febbbcee2e82b3
2023-03-15 18:25:52 +01:00
Oliver Smith 372b33a582 obs: don't pass fetch through functions
Change-Id: I0689868db6fadaaaca470cdefbf60ceedc2eb225
2023-03-15 18:25:52 +01:00
Oliver Smith 0ed0e46421 obs: don't pass conflict_version through functions
Change-Id: I3d712e94d56269c49ee5baecfee1fa9f35101488
2023-03-15 18:25:52 +01:00
Oliver Smith 06f7482a7e obs: don't pass branch through functions
Change-Id: I9ec6d04c51bc795dd9cd4738313e60a556791eb2
2023-03-15 18:25:52 +01:00
Oliver Smith dd36e9f489 obs: don't pass feed through functions
Change-Id: I9124d33c45ef3fef48666b485fd169eec0eb393a
2023-03-15 18:25:52 +01:00
Oliver Smith e4313bc6b7 obs: don't pass proj through functions
Change-Id: Id213ae155f34c74529845ff86468b68ffa6ffb1d
2023-03-15 18:25:52 +01:00
Oliver Smith 9337c6c035 obs: lib.args: store argparse result here
Prepare to add more arguments to the OBS scripts, so they can be used
for building the wireshark package in its own OBS project.

It was not so clear to me when writing this initially, but with more and
more arguments getting added: it's not very maintainable to pass each
variable from the argparse result several functions down to where it
gets used... so unfortunately this means the code needs to be refactored
to just use libs.args in the few lines of code where the arguments
actually get used.

This patch adds lib.args, and the next patches refactor existing code to
use it instead of passing parameters, where it makes sense. After that
follow the patches related to adding the wireshark package.

Related: OS#2537
Change-Id: I48853444f1386dfb842c174ebc45f9decc8bec0f
2023-03-15 18:20:21 +01:00
Oliver Smith 719ff97608 jobs: tweak build-discarder values
Remove num-to-keep from most jobs, as this leads to keeping the build
logs for a much shorter timeline than desired. For example the
gerrit-binpkgs-deb job that runs for most projects when pushing patches
to jenkins reaches the 120 limit in less than 24h - and so when clicking
the link on a failed build from yesterday it is already deleted.

Instead just keep the logs for the last 30 days, no matter how many were
submitted on one day. Storing logs doesn't take up much space.

Remove the artifact-days-to-keep and artifact-num-to-keep lines, as they
don't have an effect. For jobs that do have artifacts, the actual value
is min(days-to-keep,artifact-days-to-keep) and same with num-to-keep.

While at it, increase the ttcn3-testsuites build-discarder to 120 days
as this means more data will show up in the test result analyzer at
which we look frequently.

Change-Id: Iec5c22c7fcf6c1fd2db71611045f15dc6580ed86
2023-03-07 12:05:29 +01:00
Oliver Smith fdf3336f6c contrib/obs-upgrade-eclipse-titan.sh: new script
Add helper script for upgrading the eclipse-titan package in OBS.

Change-Id: Ia42adf5ddb469b18588c659adc9ded951e68e44b
2023-03-02 10:18:25 +01:00
Oliver Smith 2ede9dcd1d jobs/gerrit-pipeline-endianness: ignore submodules
Let the check only run on the main repository, not on any submodules.
This fixes that it would currently fail in osmo-trx: the osmocom-bb
submodule has a copy of libosmocore where the endianness check is known
to be failing (see https://gerrit.osmocom.org/c/osmocom-bb/+/31403).

Change-Id: I795a64a66b4a2e316a99e6b523cc33a9ed364272
2023-02-27 09:17:31 +01:00
Oliver Smith 3e5c5d4a75 build-old-tags: tags_to_ignore: add osmo-hlr
Add the osmo-hlr releases where tests are not foward-compatible to the
related libosmocore change. Fixup releases 1.6.1 and 1.5.1 have been
made that are foward compatible.

Related: libosmocore I446e54d0ddf4a18c46ee022b1249af73552e3ce1
Change-Id: Ia772c9e823e494fdfce179da58735d3e6cee8e95
2023-02-23 12:23:42 +01:00
Oliver Smith 175deaf6b8 jobs/ttcn3-testsuites: replace 2022q2 with 2023q1
Related: SYS#6283
Change-Id: I1e1e47c3a040bc5b4aa79a5584517ff5b42cb317
2023-02-21 17:23:38 +00:00
Oliver Smith 4170c2b71e jobs/gerrit: start endianness job in pipeline
Related: OS#5884
Change-Id: Ib84a51aa8465cf513cb2e4b31121a04760f688e9
2023-02-20 17:27:03 +01:00
Oliver Smith 95d8649f5c jobs/gerrit-pipeline-endianness: new job
Add a new job that runs struct_endianness.py from libosmocore.git and
complains if there is a diff. The next patch adds it to the pipeline.

Related: OS#5884
Change-Id: Ie12ef58905e5cdf1fb6816b59efb65c0e56b5e8d
2023-02-20 17:16:45 +01:00
Oliver Smith 51748a77a7 jenkins-gerrit: make links clickable again
After gerrit was upgraded, the comment formatting changed. Links inside
blocks indented with two spaces cannot be clicked anymore, so change it
to a listing.

Change-Id: I4dc338c22e9afe4b9d9df845c6f5d9b3a91f93d9
2023-02-20 11:27:37 +01:00
Oliver Smith 6dcd1360ff repo-install-test: run uname -a
Print the kernel version running inside the VM.

Change-Id: Ic111ac8eda11c5609ec6ce7cf8a22584d32ed567
2023-02-13 12:30:58 +01:00
Oliver Smith 378abe1844 repo-install-test: add DOMAIN variable
Allow overriding the downloads.osmocom.org domain of the url where
packages are downloaded from, so we can download them from
people.osmocom.org instead while developing / debugging, e.g.:
  https://people.osmocom.org/packages/home:/osmith:/nightly/

Change-Id: I36bc0eae9fdee75512c1dbdca84cd6224b8c192a
2023-02-13 12:30:58 +01:00
Oliver Smith 6e945b50c3 repo-install-test: update tests for e1d/ggsn/pfcp
Disable workarounds for OS#5817, where packaging bugs have been fixed.

Change-Id: I2452993abc1c0fe78bef464f1a62fe578b48016b
2023-02-13 12:30:55 +01:00
Oliver Smith b31d480ed6 repo-install-test: adjust to 2023-02 releases
Change-Id: Ic1d912efbe504b6e35b59e31d12b983b9d924355
2023-02-13 11:56:12 +01:00
Oliver Smith e3f0badc7d repo-install-test: disable osmo-upf service check
Don't check the osmo-upf service until it's fixed, so we don't miss
other errors that repo-install-test may find.

Related: OS#5905
Change-Id: I970cad1bdb4586afa8ba5b4dac31bb9ac02b7b2d
2023-02-13 11:56:12 +01:00
Oliver Smith 7f0060ae70 obs: git.py: fix f-string
Change-Id: Iaaedf019012d77e4a385054532d1d10659ef8e76
2023-02-08 16:25:22 +01:00
Oliver Smith 6dde3dcd38 coverity: build osmo-ggsn with --enable-gtp-linux
Change-Id: I5b240dbfe2369a5a54ab6115a8780cf9cc36b537
2023-02-01 12:06:16 +01:00
Oliver Smith 56939cc0a7 coverity/local_test: use all CPU cores
Change-Id: I018fda7cb85bb7e480dbc55d296c2e216098fc5e
2023-02-01 11:30:11 +01:00
Oliver Smith 6a6b19ec6f coverity: check if sdcc is installed
Check at the start if sdcc is installed, as it's required to build
libusrp. It might not be installed when testing locally.

Change-Id: I226f439f7eff657d01c667aae464c080b91d5a82
2023-02-01 10:58:04 +01:00
Oliver Smith 7bd02257c9 coverity/local_test: fix copy error
coverity/common.sh is a symlink to ../scripts/common.sh in this git
repository. The local_test.sh already adjusts the symlink so it points
to the right place after copying related scripts to /tmp/coverity.
However when running the script again, cp would fail with:

cp: '/home/user/code/osmo-dev/src/osmo-ci/coverity/common.sh' and '/tmp/coverity/common.sh' are the same file

Remove the symlink to avoid this error.

Change-Id: I081871e6e06547a90879bc4aa08ac2b5c5ddaeef
2023-02-01 10:58:04 +01:00
Oliver Smith 2f0de239e6 coverity: disable building osmo-isdntap
As it doesn't seem to build properly with default dahdi-linux, disable
it for now.

Related: OS#5886
Change-Id: Icb5cf7517a2ba0f3abb1d44337677f9502069ec6
2023-02-01 10:58:04 +01:00
Oliver Smith 9a637bc596 coverity: osmo-remsim: fix install errors
Set usbdropdir and serialconfdir to fix:
  /usr/bin/mkdir -p '/usr/lib/pcsc/drivers/libifd-osmo-remsim-client.bundle/Contents'
  /usr/bin/mkdir: cannot create directory ‘/usr/lib/pcsc’: Permission denied

Change-Id: I08e72bb06a823752154fe3e2011a42c144c657c4
2023-02-01 10:58:04 +01:00
Oliver Smith 2584debad5 coverity: add libgtpnl
Fix for:
  configure: error: Package requirements (libgtpnl >= 1.2.0) were not met

Change-Id: I534b8a550061e52d4c5ab95ad98e146a77f386bc
2023-02-01 10:58:04 +01:00
Oliver Smith 733626688d ansible/osmocom-jenkins-slave: add libnftables-dev
Install libnftables-dev, so osmo-upf can build successfully in the
coverity jenkins job.

Change-Id: Id72ce8ed5b662085a16f56756f8f93627834d2e2
2023-02-01 10:58:04 +01:00
Harald Welte 8abf46760a coverity: Add many missing projects
Add the following projects to our coverity test job:
  osmo-pcap
  osmo-e1d
  osmo-e1-recorder
  osmo-upf
  osmo-isdntap
  osmo-uecups
  osmo-remsim

Change-Id: Idee49aa4c15fcfb1a469db9e4978523af5608d70
2023-01-31 13:29:48 +01:00