Commit Graph

1212 Commits

Author SHA1 Message Date
Oliver Smith 118cdef4c5 jobs/osmocom-release-manuals: add new job
Related: OS#5902
Change-Id: I0ecb238660553c3c857e1b310873eca8a8d09dab
2023-03-22 13:59:32 +01:00
Oliver Smith b55b11d671 obs: build_binpkg: add ubuntu 22.04
Add it for debugging a build failure that we only saw on Ubuntu.

Related: OS#5946
Change-Id: I4a927447d8f8b89411b93c563fbc6aa8d4998ffa
2023-03-22 09:30:53 +00:00
Oliver Smith d6eed85d44 jobs/master-builds: use debian-bullseye-jenkins
I've verified that all master-builds jobs still work with this change.

Related: OS#5949
Depends: docker-playground I51925d0ab9e5a779379efab59c381ef12fb60929
Change-Id: I7d5bc7bb4c1457d4e05fd6e0d27668382c39973a
2023-03-21 16:49:22 +01:00
Oliver Smith 24f3e088dc scripts/docker-rebuild: add debian-bullseye-jenkins-arm
Prepare to let all master-builds jobs use debian-bullseye instead of
debian-buster.

Depends: docker-playground I8e1741f86ffb8abd658d1e4e0415dfd11fb1a8a1
Related: OS#5949
Change-Id: I55bd5869ecd753549b8f2f2822e825623d940acd
2023-03-21 16:48:50 +01:00
Oliver Smith 22f3e07636 jobs: master-osmo-e1-hardware: fix manuals
Build the manuals with the regular docker image, instead of the
fpga-build one.

Fix for:
  build/shrink-pdfs.sh: 8: build/shrink-pdfs.sh: ps2pdf: not found

Related: SYS#6380
Change-Id: I6e9f832dc449af0ca7def29ef5a9161285b01736
2023-03-20 10:46:30 +01:00
Oliver Smith e7eba15315 jobs: add Osmocom_OBS_wireshark
Add a nightly jenkins job that does the following:
* Clone Wireshark from upstream
* Merge several Osmocom branches on top
* Build a source package
* Submit it to the osmocom:wireshark OBS project

Related: OS#2537
Change-Id: Ifb49c5cb22a4de0da30a920e5450a27172b11d73
2023-03-17 11:33:55 +01:00
Oliver Smith 1ab4ce32f9 obs: add --version-append argument
Related: OS#2537
Change-Id: Iff5a530bebd645be13e0206ce0a670e1dfe6e2f9
2023-03-17 11:33:55 +01:00
Oliver Smith dc1415959f obs: add --allow-unknown-package
Prepare to add a wireshark package. It will be in its own
osmocom:wireshark repository, and not in osmocom:nightly, :latest,
:master, so it's not added to lib/config.py.

Move lib.set_args() before parse_packages(), so the latter is able to
use lib.args.allow_unknown_package.

Related: OS#2537
Change-Id: I007f937ccb629e0593ec9253541ed05df42fac66
2023-03-17 11:33:55 +01:00
Oliver Smith 3ccebc4514 obs: srcpkg.build: fix conflict_version regression
During refactoring, conflict_version was changed to version here, which
was wrong. The packages need to depend on the conflict_version of the
meta package, version is the version of the package itself.

Fix for:
  The following packages have unmet dependencies:
   libosmo-netif-dev : Depends: osmocom-nightly (= 202303160009) but it is not going to be installed
                       Depends: libosmocore but it is not going to be installed
                       Depends: libosmonetif11 (= 1.3.0.8.9e65.202303160009) but it is not going to be installed
   libosmocore-dev : Depends: osmocom-nightly (= 1.8.0.79-b394d.202303170006) but it is not going to be installed
                     Depends: libosmocore but it is not going to be installed

Fixes: 0ed0e464 ("obs: don't pass conflict_version through functions")
Change-Id: I2326e3817c6f6887ef1196e603c3877768119a66
2023-03-17 11:33:45 +01:00
Oliver Smith 8d137c66a7 obs: add --git-skip-checkout argument
Prepare to use this for wireshark, to build a branch on-the-fly that
does not get pushed to origin, with Osmocom patches on top of upstream
master.

Related: OS#2537
Change-Id: Ifc963daf51ba3542f67420daaf7b29745404a92e
2023-03-15 18:25:52 +01:00
Oliver Smith f7ff90d64b obs: don't pass skip_up_to_date through functions
Change-Id: Ide0c1f5452f5ce965541b45039345abc52b3d140
2023-03-15 18:25:52 +01:00
Oliver Smith 78cd3a3c75 obs: don't pass meta through functions
Change-Id: Ieb49e5f8630e3e918b11a7a225febbbcee2e82b3
2023-03-15 18:25:52 +01:00
Oliver Smith 372b33a582 obs: don't pass fetch through functions
Change-Id: I0689868db6fadaaaca470cdefbf60ceedc2eb225
2023-03-15 18:25:52 +01:00
Oliver Smith 0ed0e46421 obs: don't pass conflict_version through functions
Change-Id: I3d712e94d56269c49ee5baecfee1fa9f35101488
2023-03-15 18:25:52 +01:00
Oliver Smith 06f7482a7e obs: don't pass branch through functions
Change-Id: I9ec6d04c51bc795dd9cd4738313e60a556791eb2
2023-03-15 18:25:52 +01:00
Oliver Smith dd36e9f489 obs: don't pass feed through functions
Change-Id: I9124d33c45ef3fef48666b485fd169eec0eb393a
2023-03-15 18:25:52 +01:00
Oliver Smith e4313bc6b7 obs: don't pass proj through functions
Change-Id: Id213ae155f34c74529845ff86468b68ffa6ffb1d
2023-03-15 18:25:52 +01:00
Oliver Smith 9337c6c035 obs: lib.args: store argparse result here
Prepare to add more arguments to the OBS scripts, so they can be used
for building the wireshark package in its own OBS project.

It was not so clear to me when writing this initially, but with more and
more arguments getting added: it's not very maintainable to pass each
variable from the argparse result several functions down to where it
gets used... so unfortunately this means the code needs to be refactored
to just use libs.args in the few lines of code where the arguments
actually get used.

This patch adds lib.args, and the next patches refactor existing code to
use it instead of passing parameters, where it makes sense. After that
follow the patches related to adding the wireshark package.

Related: OS#2537
Change-Id: I48853444f1386dfb842c174ebc45f9decc8bec0f
2023-03-15 18:20:21 +01:00
Oliver Smith 719ff97608 jobs: tweak build-discarder values
Remove num-to-keep from most jobs, as this leads to keeping the build
logs for a much shorter timeline than desired. For example the
gerrit-binpkgs-deb job that runs for most projects when pushing patches
to jenkins reaches the 120 limit in less than 24h - and so when clicking
the link on a failed build from yesterday it is already deleted.

Instead just keep the logs for the last 30 days, no matter how many were
submitted on one day. Storing logs doesn't take up much space.

Remove the artifact-days-to-keep and artifact-num-to-keep lines, as they
don't have an effect. For jobs that do have artifacts, the actual value
is min(days-to-keep,artifact-days-to-keep) and same with num-to-keep.

While at it, increase the ttcn3-testsuites build-discarder to 120 days
as this means more data will show up in the test result analyzer at
which we look frequently.

Change-Id: Iec5c22c7fcf6c1fd2db71611045f15dc6580ed86
2023-03-07 12:05:29 +01:00
Oliver Smith fdf3336f6c contrib/obs-upgrade-eclipse-titan.sh: new script
Add helper script for upgrading the eclipse-titan package in OBS.

Change-Id: Ia42adf5ddb469b18588c659adc9ded951e68e44b
2023-03-02 10:18:25 +01:00
Oliver Smith 2ede9dcd1d jobs/gerrit-pipeline-endianness: ignore submodules
Let the check only run on the main repository, not on any submodules.
This fixes that it would currently fail in osmo-trx: the osmocom-bb
submodule has a copy of libosmocore where the endianness check is known
to be failing (see https://gerrit.osmocom.org/c/osmocom-bb/+/31403).

Change-Id: I795a64a66b4a2e316a99e6b523cc33a9ed364272
2023-02-27 09:17:31 +01:00
Oliver Smith 3e5c5d4a75 build-old-tags: tags_to_ignore: add osmo-hlr
Add the osmo-hlr releases where tests are not foward-compatible to the
related libosmocore change. Fixup releases 1.6.1 and 1.5.1 have been
made that are foward compatible.

Related: libosmocore I446e54d0ddf4a18c46ee022b1249af73552e3ce1
Change-Id: Ia772c9e823e494fdfce179da58735d3e6cee8e95
2023-02-23 12:23:42 +01:00
Oliver Smith 175deaf6b8 jobs/ttcn3-testsuites: replace 2022q2 with 2023q1
Related: SYS#6283
Change-Id: I1e1e47c3a040bc5b4aa79a5584517ff5b42cb317
2023-02-21 17:23:38 +00:00
Oliver Smith 4170c2b71e jobs/gerrit: start endianness job in pipeline
Related: OS#5884
Change-Id: Ib84a51aa8465cf513cb2e4b31121a04760f688e9
2023-02-20 17:27:03 +01:00
Oliver Smith 95d8649f5c jobs/gerrit-pipeline-endianness: new job
Add a new job that runs struct_endianness.py from libosmocore.git and
complains if there is a diff. The next patch adds it to the pipeline.

Related: OS#5884
Change-Id: Ie12ef58905e5cdf1fb6816b59efb65c0e56b5e8d
2023-02-20 17:16:45 +01:00
Oliver Smith 51748a77a7 jenkins-gerrit: make links clickable again
After gerrit was upgraded, the comment formatting changed. Links inside
blocks indented with two spaces cannot be clicked anymore, so change it
to a listing.

Change-Id: I4dc338c22e9afe4b9d9df845c6f5d9b3a91f93d9
2023-02-20 11:27:37 +01:00
Oliver Smith 6dcd1360ff repo-install-test: run uname -a
Print the kernel version running inside the VM.

Change-Id: Ic111ac8eda11c5609ec6ce7cf8a22584d32ed567
2023-02-13 12:30:58 +01:00
Oliver Smith 378abe1844 repo-install-test: add DOMAIN variable
Allow overriding the downloads.osmocom.org domain of the url where
packages are downloaded from, so we can download them from
people.osmocom.org instead while developing / debugging, e.g.:
  https://people.osmocom.org/packages/home:/osmith:/nightly/

Change-Id: I36bc0eae9fdee75512c1dbdca84cd6224b8c192a
2023-02-13 12:30:58 +01:00
Oliver Smith 6e945b50c3 repo-install-test: update tests for e1d/ggsn/pfcp
Disable workarounds for OS#5817, where packaging bugs have been fixed.

Change-Id: I2452993abc1c0fe78bef464f1a62fe578b48016b
2023-02-13 12:30:55 +01:00
Oliver Smith b31d480ed6 repo-install-test: adjust to 2023-02 releases
Change-Id: Ic1d912efbe504b6e35b59e31d12b983b9d924355
2023-02-13 11:56:12 +01:00
Oliver Smith e3f0badc7d repo-install-test: disable osmo-upf service check
Don't check the osmo-upf service until it's fixed, so we don't miss
other errors that repo-install-test may find.

Related: OS#5905
Change-Id: I970cad1bdb4586afa8ba5b4dac31bb9ac02b7b2d
2023-02-13 11:56:12 +01:00
Oliver Smith 7f0060ae70 obs: git.py: fix f-string
Change-Id: Iaaedf019012d77e4a385054532d1d10659ef8e76
2023-02-08 16:25:22 +01:00
Oliver Smith 6dde3dcd38 coverity: build osmo-ggsn with --enable-gtp-linux
Change-Id: I5b240dbfe2369a5a54ab6115a8780cf9cc36b537
2023-02-01 12:06:16 +01:00
Oliver Smith 56939cc0a7 coverity/local_test: use all CPU cores
Change-Id: I018fda7cb85bb7e480dbc55d296c2e216098fc5e
2023-02-01 11:30:11 +01:00
Oliver Smith 6a6b19ec6f coverity: check if sdcc is installed
Check at the start if sdcc is installed, as it's required to build
libusrp. It might not be installed when testing locally.

Change-Id: I226f439f7eff657d01c667aae464c080b91d5a82
2023-02-01 10:58:04 +01:00
Oliver Smith 7bd02257c9 coverity/local_test: fix copy error
coverity/common.sh is a symlink to ../scripts/common.sh in this git
repository. The local_test.sh already adjusts the symlink so it points
to the right place after copying related scripts to /tmp/coverity.
However when running the script again, cp would fail with:

cp: '/home/user/code/osmo-dev/src/osmo-ci/coverity/common.sh' and '/tmp/coverity/common.sh' are the same file

Remove the symlink to avoid this error.

Change-Id: I081871e6e06547a90879bc4aa08ac2b5c5ddaeef
2023-02-01 10:58:04 +01:00
Oliver Smith 2f0de239e6 coverity: disable building osmo-isdntap
As it doesn't seem to build properly with default dahdi-linux, disable
it for now.

Related: OS#5886
Change-Id: Icb5cf7517a2ba0f3abb1d44337677f9502069ec6
2023-02-01 10:58:04 +01:00
Oliver Smith 9a637bc596 coverity: osmo-remsim: fix install errors
Set usbdropdir and serialconfdir to fix:
  /usr/bin/mkdir -p '/usr/lib/pcsc/drivers/libifd-osmo-remsim-client.bundle/Contents'
  /usr/bin/mkdir: cannot create directory ‘/usr/lib/pcsc’: Permission denied

Change-Id: I08e72bb06a823752154fe3e2011a42c144c657c4
2023-02-01 10:58:04 +01:00
Oliver Smith 2584debad5 coverity: add libgtpnl
Fix for:
  configure: error: Package requirements (libgtpnl >= 1.2.0) were not met

Change-Id: I534b8a550061e52d4c5ab95ad98e146a77f386bc
2023-02-01 10:58:04 +01:00
Oliver Smith 733626688d ansible/osmocom-jenkins-slave: add libnftables-dev
Install libnftables-dev, so osmo-upf can build successfully in the
coverity jenkins job.

Change-Id: Id72ce8ed5b662085a16f56756f8f93627834d2e2
2023-02-01 10:58:04 +01:00
Harald Welte 8abf46760a coverity: Add many missing projects
Add the following projects to our coverity test job:
  osmo-pcap
  osmo-e1d
  osmo-e1-recorder
  osmo-upf
  osmo-isdntap
  osmo-uecups
  osmo-remsim

Change-Id: Idee49aa4c15fcfb1a469db9e4978523af5608d70
2023-01-31 13:29:48 +01:00
Oliver Smith 4773f26d2d jobs/master,gerrit: fix pysim, sysmo-usim-tool
Don't attempt to create the CCACHE_DIR and chown it to osmocom-build for
the simtester jenkins node, and more generally all that were not
created with the ansible provisioning scripts. In case of simtester,
there is no osmocom-build user and we don't have the space there to use
ccache.

Fix for:
  chown: invalid user: ‘osmocom-build:osmocom-build’

Related: OS#5848
Change-Id: I339d9ba4ad0c959d7325820eb53bfa1f0e04c164
2023-01-25 12:59:30 +00:00
Oliver Smith a788a8b14f jobs/master,gerrit: add docker_run_ccache
Put arguments for ccache into a new docker_run_ccache variable, and use
it with all projects where it makes sense to use ccache.

Other projects use docker_run without the ccache parameters again:
* osmo_ss7, osmo_dia2gsup, pysim:
  Erlang/Python projects
* osmo-e1-hardware, ice40-usbtrace, no2bootloader, osmo-opencm3-projects:
  Don't use the default docker image
* osmo-gsm-tester:
  Only builds manuals (jenkins-build-manuals.sh)

Jenkins job builder's variable substitution doesn't replace {docker_run}
inside docker_run_ccache when used in cmd, so just duplicate the lines.

Related: OS#5848
Change-Id: Ib6d7f309b972979d3a3bc87f8fcf71f9e930c0da
2023-01-25 12:59:18 +00:00
Oliver Smith d909ad2997 jobs/gerrit: openbsc: set pipeline_binpkgs: false
Don't attempt to build deb packages for openbsc in gerrit-verifications.

Fix for:
  ERROR: unknown package: openbsc
  See packages_osmocom and packages_other in obs/lib/config.py

Change-Id: I3cd810366a6d711c1fdbf3d5ff03d507441307bd
2023-01-25 09:17:23 +00:00
Oliver Smith 945687b030 jobs/master,gerrit: use ccache to speed up builds
Related: OS#5848
Change-Id: I81f244ca09e660698df617549d70476fc96255e2
2023-01-24 10:56:03 +00:00
Oliver Smith c914967c53 jobs/master,gerrit: use minimal image for arm
Building debian-buster-jenkins on the raspberry pis may take > 1h,
depending on what changed. It has a lot of things we don't use, so build
and use a minimal image with the rpis instead.

Closes: OS#5863
Depends: docker-playground I4fb4b48b233acaef815c4c27ec6f17f12cfe836b
Change-Id: I73e62fbbf93824c0d37963039de4e00f26a43cbe
2023-01-24 08:04:54 +00:00
Oliver Smith 989c47bc91 scripts/osmo-build.sh / ARTIFACT_STORE: remove
This script was supposed to cache already built Osmocom libraries. But
it was only used with openbsc, didn't work as expected anymore (looking
at openbsc-master, it just builds all deps from source without caching)
and will be replaced with logic that allows using ccache in a future
patch.

Using ccache has the big advantage that we can cache all build objects
where the source files did not change, and not just do it on a
dependency level. Oftentimes only few source files change in our
patches, so we can really cache everything else that is not affected
within any git repository and not just a dependency that didn't change.

ccache also does automatic cache size management to ensure a cache stays
within a given size limit.

Related: OS#5848
Depends: openbsc Ib3272feec76b30412ca60dec204255b64e33831b
Change-Id: Id94d6126b476077d57839e4a884621b8c034f0c6
2023-01-23 13:31:08 +00:00
Alexander Couzens 113b295075 ansible: .gitignore: make poky regex more specific
Otherwise it would also prevent changes to the poky role

Change-Id: I3ac9a6129323f27ba387b44d8293d7b134716af8
2023-01-23 10:57:45 +00:00
Alexander Couzens e64f85a0ee ansible: install-poky-sdk: ensure it's not installed twice
Check if the poky path already exists. Even this wouldn't
detect if the poky has been only installed half way.

Related: OS#5801
Change-Id: I8ff8e72a5974378327d4692341b0a371c667b741
2023-01-23 10:57:30 +00:00
Alexander Couzens 6a290081c5 ansible: replace deprecated include by import_tasks
include_tasks and import_tasks have similar functionality.
Use import_tasks when importing non-variable depending files.
Further use import_tasks when not used within a loop.

Closes: OS#5797
Change-Id: I29766c65b9e1601a20aa8f20a438e8368c65f6a8
2023-01-23 10:57:30 +00:00