Commit Graph

952 Commits

Author SHA1 Message Date
Oliver Smith 6754a89e4c docker-rebuild: add debian-bullseye-jenkins
Related: OS#6041
Depends: docker-playground I849568826b1993fd9fb652779e631188af4a319e
Change-Id: I22f95de0bd131f0f0cfd5e2a24e17bed22e5d248
2022-07-12 14:33:30 +00:00
Oliver Smith fa8a74dfde docker-rebuild: build erlang img on x86_64 only
Build debian-bullseye-erlang on x86_64 nodes only, as the image doesn't
get used on nodes with other arches. The image is only used by
gerrit-verifications jobs for osmo_dia2gsum and osmo_gsup, which are
restriected to run on these nodes:
https://jenkins.osmocom.org/jenkins/label/osmocom-gerrit-debian9/

Remove the check and comments for not building for debian 10 x32, as
that node has been replaced with a debian 11 one.

Related: OS#5453
Change-Id: I5636dfc9e06a44b096f9dc55528d677ea2dfc352
2022-07-12 14:33:30 +00:00
Oliver Smith e1540eee9e jobs/coverity.yml: add parameter for git branch
Allow configuring a different git branch before starting the build. This
is especially useful to reproduce a failure that only happens when using
coverity tools during a build.

Change-Id: I5f7c615ff5af4bf1f34ccef62767b5ca71cd2d8d
2022-07-12 14:45:09 +02:00
Oliver Smith ff94d1def5 jobs/coverity: don't cd ~/osmo-ci/coverity
Run the coverity/jenkins.sh from the osmo-ci.git repository that jenkins
checks out at the beginning of the build, instead of using a copy in ~/
that is updated whenever pushing to master. This makes it less
confusing, I configured the job to use a WIP branch and didn't realize
that it still used the code from master.

Change-Id: Ie992d71d2afec6982447b8fe3626c821a9e48d19
2022-07-12 14:44:52 +02:00
Oliver Smith e58c208706 jobs/coverity: modernize
* no need to mention "(jenkins-job-builder)" in the display-name
* set mail to jenkins-notifications ML like most other jobs

Change-Id: I0284dd7354157e057903f53cde149ce652e76550
2022-07-12 14:03:05 +02:00
Oliver Smith 451d687036 Revert "WIP: coverity: Make sure curl is installed, as our coverity upload job needs it"
Revert 3038c19155, which adds a second
osmocom-coverity-deps job that apparently was never deployed (or was
deployed and then deleted). The mentioned curl dependency shouldn't go
here either.

Change-Id: I5d4709c0019ef4bae6ccbd11c28eeb23d75692c1
2022-07-12 14:02:59 +02:00
Oliver Smith c42a20a8f1 coverity: don't build tests for all
The coverity jenkins job has been failing since we tried to build tests
for all projects (and in an earlier attempt, build and run tests for all
projects). Make the coverity job work again by skipping build of tests
for asn1c. I looked into it earlier and the cause wasn't obvious to me,
and it's currently not worth spending time on.

Fix for:
libtool:   error: cannot find the library '../../skeletons/libasn1cskeletons.la' or unhandled argument '../../skeletons/libasn1cskeletons.la'

Change-Id: Iba0bfe0e753812058436eaf515391eb1f4b09951
2022-07-12 13:44:06 +02:00
Oliver Smith 56d1a172d6 scripts/repo-install-test: add debian 11, drop 9
Change-Id: I3e154def9051076bd8091b3f027ee76bbb94cb18
2022-07-08 15:35:14 +02:00
Oliver Smith 6135bf1bc4 scripts/OBS latest: fix checkout with dir in path
osmo-python-tests was added to latest just like it was in nightly, by
adding the following two lines to build_osmocom():

  checkout python/osmo-python-tests
  ...
  build osmo-python-tests

But the build fails, because the "checkout" function clones into a
"python/osmo-python-tests" directory in latest, and not into a
"osmo-python-tests" directory like in nightly. Make it behave the same
by adjusting the "checkout" function in nightly.

(This code has historically grown and should be refactored imho, but
let's fix this bug first.)

Fix for:
  ====> Building osmo-python-tests
  + cd /home/osmocom-build/jenkins/workspace/Osmocom_OBS_latest_obs.osmocom.org/osmo-python-tests
  ./scripts/osmocom-latest-packages.sh: 99: cd: can't cd to .../osmo-python-tests

Change-Id: I97074ee953cadac476fe2e4b686689c26bac1ea1
2022-07-08 14:44:43 +02:00
Oliver Smith 6116c0dfe2 scripts/osmocom-latest-packages.sh: build osmo-python-tests
Build the source package too, don't just checkout the sources.

Related: OS#5604
Change-Id: I5b7c6b6849edbca44f45041a6e506f10933b5d43
2022-07-07 09:19:32 +00:00
Oliver Smith 2b272e0e83 jobs/registry-update-base-images: drop old images
No need to mirror these images if we don't use them anymore.
Remove debian 8 and 9, and ubuntu zesty. The latter is the codename for
ubuntu 17.04, which is a non-LTS and was unsupported since 2018-01-13.

Related: OS#5601
Change-Id: If1c198038b0fd5faf0763172447962969448940e
2022-07-06 15:20:22 +00:00
Oliver Smith dc00f325c7 ci-docker-rebuild: drop debian-stretch-jenkins
Related: OS#5601
Change-Id: I572ff9624fba25d7d96af79d1229b614be1c11ba
2022-07-06 15:20:22 +00:00
Oliver Smith 095a110506 jobs/repo-install-test: update debian versions
Run against debian 10 and 11, and not against debian 9 anymore.

Related: OS#5601
Change-Id: Iaf7c596e0910ca2ce0fafd2819e79d4659b92514
2022-07-06 15:20:22 +00:00
Oliver Smith 8424c5986f jobs: stretch -> buster
Related: OS#5601
Change-Id: Ieb2bfc426563e48b9b8cd3910efbebdf09b41f46
2022-07-06 15:20:22 +00:00
Oliver Smith 7bc57b7ef1 scripts/osmocom-latest-packages: add osmo-python-tests
Add it in the same place as in osmocom-nightly-packages.sh.

Closes: OS#5604
Change-Id: I187c73f8dff6800abaaeb6126ace13142a8ae20d
2022-07-06 16:08:41 +02:00
Vadim Yanitskiy 2be279a9e0 fixup: coverity: compile tests, but not execute them
In change 82265bc46d [1] I attempted
to fix aaeab8c976 [2], however both
pushd and popd commands are not available in plain shell.

Change-Id: Ib63ded66781c774e16773537c3b943237904b03e
Fixes: [1] I92b648cd1e80399561067eae99426faed1ff1e25
Fixes: [2] Idf0714a59356633021be21acc47f9471d059b7fe
2022-07-04 16:57:33 +07:00
Vadim Yanitskiy 82265bc46d fixup: coverity: compile tests, but not execute them
Change-Id: I92b648cd1e80399561067eae99426faed1ff1e25
Fixes: Idf0714a59356633021be21acc47f9471d059b7fe
2022-07-04 15:42:11 +07:00
Vadim Yanitskiy aaeab8c976 coverity: compile tests, but not execute them
Compiling unit tests would allow Coverity to analyze them too and thus
find more bugs.  The problem is that binaries listed in check_PROGRAMS
are not compiled during 'make all', but only during 'make check'.

One approach [1] is to run 'make check' in do_build(), however it breaks
'Coverity-Upload' job because some unit tests are failing.  This is why
I had to revert bf84ba9f69 together with
ae41942214 that was supposed to fix it.

Even if all unit tests were passing, we still don't want to execute them
because running tests is beyond the scope of 'Coverity-Upload' job.

Another approach is to patch Makefile.am files replacing check_PROGRAMS
with noinst_PROGRAMS, so that the test binaries are compiled during
'make all' but not executed.

Change-Id: Idf0714a59356633021be21acc47f9471d059b7fe
Related: [1] I3f3149a61df96646798c72d0ad31c41b97e6b81f
2022-07-04 14:39:30 +07:00
Vadim Yanitskiy 618f171cb1 Revert "coverity/build_Osmocom.sh: run 'make check' in do_build()"
This reverts commit bf84ba9f69.

Change-Id: I722f8cbb1934a4f5ce2969e243f223019f123777
2022-07-04 14:38:45 +07:00
Vadim Yanitskiy 9baa805a74 Revert "coverity/build_Osmocom.sh: don't do check for all"
This reverts commit ae41942214.

Change-Id: I5018e0da020f32e3a46a2772e79794e5a3a686ca
2022-07-04 14:38:45 +07:00
Vadim Yanitskiy 4f8a4954b9 lint: simplify find command in exclude_paths_common_asn1c()
The purpose of "cut -d / -f 2-" is to strip off leading "./" in
the paths.  The same can be achieved by doing "-printf '%P\n'".

Matching "*.c" and "*.h" files can be expressed as "-name *.[hc]".

Change-Id: Iae3fc5c8842df6926e6ff16a41be5663f1dedd1b
2022-07-03 04:55:54 +07:00
Vadim Yanitskiy e6ad3a2b2f lint: ignore symlinks (no newline at end of file)
Change-Id: I6db6ebb51cdfd54e8f9c2c5cc7affef60a7dec32
2022-07-03 04:49:13 +07:00
Oliver Smith 2bf20f60ed jobs/osmocom-release-tarballs: move to dp
Move the long shell command into a jenkins.sh in docker-playground, and
move the osmocom-release-tarballs.sh script there as well. This makes it
easier to test the script, and only patching one repository is needed
from now on when changing to a newer debian version.

Depends: docker-playground I4cf66097f508c029f688ec8af1cab2238d42f84f
Change-Id: I15f6c08b807f1462f91e95b624cdc1aad0592dd6
2022-07-01 15:50:37 +00:00
Pau Espin 0d92cd5985 jobs: ttcn3-testsuites: Enable hnodeb,hnbgw,sccp,gbproxy 'latest' after new project releases
These two projects have been released recently as part of Osmocom CNI
202206. Let's enable testing them since version in latest is good enough
for testing.

Change-Id: I1587e8400eff2d8b1ec26ab2fff1fbfa772fe8f8
2022-07-01 14:24:35 +00:00
Oliver Smith ee5920f95d coverity/local_test.sh: new file
Add the script I've used to locally reproduce the build failure and fix
it.

Change-Id: I12241499a0669559bc40eb3e76f967a7c6a2dc70
2022-06-30 15:31:37 +02:00
Oliver Smith 26a02acdf0 coverity/prepare_source: add SRC_SKIP_FETCH/CLEAN
When running these scripts locally while developing, it's more useful to
not fetch the sources and clean the git repository instead. Add
environment variables that allow doing this.

Change-Id: Id6d3cdabb776435df36c0145d93e514693062fca
2022-06-30 15:30:00 +02:00
Oliver Smith 12630c87ab coverity/prepare_source: fix running twice
Don't attempt to clone layer1-api if it exists already. Jenkins deletes
the sources for every run, but for development it is useful to be able
to run this script multiple times without getting errors.

Change-Id: I17bd76a5ff6abafd4bd0189073930f45de20afe8
2022-06-30 12:27:07 +02:00
Oliver Smith ae41942214 coverity/build_Osmocom.sh: don't do check for all
'make check' does not pass for asn1c, it fails with the following. I
looked into it and the cause is not obvious to me, so in order to make
the jenkins job pass again I've changed it back to just run 'make'
instead.

Fix for:
  ...
  fatal: making test-suite.log: failed to create ../tests/98-attribute-class-OK.asn1.log
  fatal: making test-suite.log: failed to create ../tests/99-class-sample-OK.asn1.trs
  fatal: making test-suite.log: failed to create ../tests/99-class-sample-OK.asn1.log
  Makefile:735: recipe for target 'test-suite.log' failed
  make[3]: *** [test-suite.log] Error 1
  make[3]: Leaving directory '/home/osmocom-build/osmo-ci/coverity/source-Osmocom/asn1c/libasn1fix'
  Makefile:841: recipe for target 'check-TESTS' failed

Disable 'make check' for libusrp too as it also didn't pass:

  Making check in usrp2
  make[4]: Entering directory '/tmp/coverity/source-Osmocom/libusrp/firmware/src/usrp2'
  test -f `basename 'eeprom_boot.a51'` || ln -s 'eeprom_boot.a51' .
  test -f ../common/`basename 'eeprom_boot.a51'` -o \
          \! -f `dirname 'eeprom_boot.a51'`/../common/`basename 'eeprom_boot.a51'` \
          || ln -s `dirname 'eeprom_boot.a51'`/../common/`basename 'eeprom_boot.a51'` ../common/`basename 'eeprom_boot.a51'`
  sdas8051 -plosgff `basename 'eeprom_boot.a51'`
  sdcc -mmcs51 --no-xinit-opt -I../../../firmware/include -I../../../firmware/src/usrp2 -I../../../firmware/src/common -I../../../firmware/src/common -DHAVE_USRP2 \
          -c -o eeprom_init.rel `test -f 'eeprom_init.c' || echo './'`eeprom_init.c
  test -f `basename '_startup.a51'` || ln -s '_startup.a51' .
  test -f ../common/`basename '_startup.a51'` -o \
          \! -f `dirname '_startup.a51'`/../common/`basename '_startup.a51'` \
          || ln -s `dirname '_startup.a51'`/../common/`basename '_startup.a51'` ../common/`basename '_startup.a51'`
  sdas8051 -plosgff `basename '_startup.a51'`
  make[4]: *** No rule to make target '../../lib/libfx2.lib', needed by 'eeprom_boot.ihx'.  Stop.

All other 'make check' calls do work as expected, the build script runs
through again with this patch.

Fixes: bf84ba9f ("coverity/build_Osmocom.sh: run 'make check' in do_build()")
Change-Id: I6b9eb8e73c51d925897579fb51bc0a1e86e77270
2022-06-30 12:25:56 +02:00
Oliver Smith ea1172056f coverity/prepare_source_Osmocom.sh: fix typo
Osm_c_ocom -> Osmocom

Change-Id: Ib5a8f35eb9ef3bb8488b29ed9a9b7f43e6b6f088
2022-06-30 11:26:24 +02:00
Vadim Yanitskiy bf84ba9f69 coverity/build_Osmocom.sh: run 'make check' in do_build()
This enables building and running tests (check_PROGRAMS), so that
Coverity will be able to find bugs in their code too.

Change-Id: I3f3149a61df96646798c72d0ad31c41b97e6b81f
2022-06-29 13:40:16 +00:00
Oliver Smith 2b7b90e555 jobs/update-osmo-*: change mail recipients
Use jenkins-notifications@lists.osmocom.org like in most other jobs to
not clutter the other ML.

Change-Id: I95c22215dbe09d06d9458a980d15370440f67fd6
2022-06-29 09:33:30 +00:00
Oliver Smith 9139e76baa osmo-ci-docker-rebuild: don't always build osmo-gsm-tester
Currently there is only one jenkins node labeled as
osmo-gsm-tester-build. Build the osmo-gsm-tester container only there,
not on all x86_64 nodes.

The /var/tmp/osmo-gsm-tester/state directory is set up by
ansible/roles/gsm-tester/tasks/main.yml.

Related: https://jenkins.osmocom.org/jenkins/label/osmo-gsm-tester-build/
Change-Id: I9191d6d54d516ce75f3f576f34463dcaad22c85a
2022-06-29 09:33:30 +00:00
Oliver Smith 4f8cf4049f ansible/gsm-tester: use debian's soapysdr-module-lms7
Same reasoning as in docker-playground
Ie2408258eda407d9db6c2f0f5fbe475861756778.

Change-Id: I957dc42df640e513554f54e597a12df05e0fd740
2022-06-29 09:33:30 +00:00
Harald Welte bab323da52 install-coverity: update to 2021.12
Starting in February 2022, coverity stopped to support the old
2019.03 version that we were using.  Unfortunately that didn't
generate any e-mail notification to the project admins so it
remained undetected until now :(

What sucks is that the filename states 2021.12 but then the
actual directory created upon extraction is 2021.12.1, meaning
our ansible rule breaks.  We have to rename the tarball to 2021.12.1
before running the playbook...

Closes: OS#5594
Change-Id: I950834cc2ad70979a53a563610bbc9feef1f5f20
2022-06-28 15:14:24 +00:00
Oliver Smith 8637f74ced coverity: add osmo-bsc-nat
Fixes: OS#5596
Change-Id: Id7cf497ea15c3ccdf76c047c6a67f17fac94e0dd
2022-06-28 12:11:15 +02:00
Oliver Smith fc4579cf76 jobs/osmocom-obs: run against obs.osmocom.org too
Split the OBS related jenkins jobs into two, one that uploads to the
new obs.osmocom.org and one that uploads to build.opensuse.org.

Set an alias for "osc" that sets the right server, and set the
expand_aliases shopt so the bash scripts that source common-obs.sh will
not ignore the alias.

Related: OS#5557
Change-Id: Ib271e16e93ebc58ed1ecb0ea881e1f1370e40928
2022-06-27 13:31:44 +02:00
Oliver Smith 6e7afd94d3 jobs/ttcn3-testsuites: drop 2021q1, 2021q4
Not needed anymore, as discussed in related issue.

Related: SYS#6007
Change-Id: Id8f7d7c198647df60b684fd5767e77cadff60a76
2022-06-21 14:40:27 +02:00
Oliver Smith 51ed5421f1 jobs, scripts: add 2022q2
Related: SYS#6007
Change-Id: I59198f38ba817f56f6b24eae163297393587ec0b
2022-06-21 13:22:05 +02:00
Oliver Smith 3577608180 jobs/master-builds: disable master-osmo-opencm3-projects
As discussed in the team meeting, disable it for now.

Change-Id: I4a79a146042d5cf586f5273960a9a2b92b45b1be
2022-06-21 11:39:32 +02:00
Oliver Smith 6dccbdbe92 lint: ignore test output in .ctrl, .vty files
Change-Id: I7611bb1b26c4abbfb572182a490f289216e7b708
2022-06-20 15:29:00 +02:00
Neels Hofmeyr 38d569d45f add libosmo-pfcp
Change-Id: Icb75adec886cf1053b80c968fea59e597ed289a7
2022-06-17 16:33:10 +02:00
Oliver Smith cd9c983b61 ansible: get OBS pub key from proper URL
Same reasoning as in previous patch
Idd0fb6e07cba959a36269244b0c7b5c62aaffeee.

Change-Id: I630654ade0edddd23f9fe7c578d1f329f5f49603
2022-06-10 15:35:46 +02:00
Oliver Smith a1b4e8bb55 repo-install-test: get OBS pub key from proper URL
The public OBS key expired on 2022-05-22 and was replaced with a key
that was only shortly valid until 2022-06-08. Shortly after it was
replaced with a key that is valid longer, until 2024-08-02.

On 2022-06-09, one of the osmocom-repo-install tests started failing.
For some reason, the key in the latest/Debian_10 directory was not
updated to the latest one:
https://download.opensuse.org/repositories/network:/osmocom:/latest/Debian_10/Release.key

Since the key is the same for all of network:osmocom, adjust the
function to download it from a place that the OBS web UI links to when
attempting to download the public GPG key.

I guess the latest/Debian_10/Release.key will get updated once making a
new release and updating the packages in the repository. But sinc
there's a lot of other tasks to do, just use this practical solution for
now.

Change-Id: Idd0fb6e07cba959a36269244b0c7b5c62aaffeee
2022-06-10 15:15:37 +02:00
Oliver Smith cfcd2f8bec ansible/gsm-tester-bts/Release.key: update
The old one exipired, this one is valid until 2024-08-02.

Related: docker-playground I56a8708ae55cc4cc5928cd80bb57d63f32e22423
Change-Id: I9526481aaee8b84348de399bf2f58f4bb0d137e8
2022-06-10 11:17:32 +02:00
Oliver Smith 185317396b ansible: rpis: enable docuum
Now that all rpis have been upgraded to debian 11, enable docuum like
it's already done for x86_64.

Change-Id: I7ab54f03819704cf60a69a417de6d14cb5cd7a56
2022-05-30 16:54:51 +02:00
Oliver Smith f6e305928a jobs/update-osmo-ci: gtp0-deb{10 => 11}build32
Related: OS#5453
Change-Id: I4b10550f4274995e70bf32f013bffecc1039d8a0
2022-05-18 16:43:01 +02:00
Oliver Smith 9ecd2280d8 ansible/jenkins/osmocom_build_deps: drop py2 pkgs
Drop python2 packages. It's deprecated, packages don't exist in
debian 11 and our jenkins nodes don't need it anymore.

Related: OS#5453
Change-Id: Id2da7adab83f240e7e0b45c018f91c9e639fc80d
2022-05-18 15:15:03 +02:00
Oliver Smith f511264bdd ansible/jenkins/osmocom_build_deps: drop dh-systemd
This package is no longer available in debian 11. Even in debian 9 the
description says:
  "This package is for transitional purposes and can be removed safely."

Related: OS#5453
Change-Id: I6f72de4d7fb0aab4b153dfc0affbc6f11d20ee4c
2022-05-18 15:05:31 +02:00
Oliver Smith e2ef4fed0e ansible/hosts: add gtp0-deb11build-i586
Related: OS#5453
Change-Id: I107dfb543090989c8b7c746be381aa19f0afa366
2022-05-18 15:00:47 +02:00
Harald Welte e50ad03b46 docker: Fix vfs storage cleanup task
The 'grep' logic of Change-Id I58f3f4a26ad6fa4698d87475cefb6ab21e66b15a
seems to be inverted.  if "overlay2" is found (grep successful) the
exit code of grep is '0'.

Change-Id: I334b0c71ef626ffd2c84097e902a5164faa3fa8b
Related: OS#5827
2022-05-17 22:15:16 +02:00