Commit Graph

364 Commits

Author SHA1 Message Date
Harald Welte 942f5ef96c master-opencm3-projects: Build from gitea, not git
git.osmocom.org is a mirror of gitea.osmocom.org, let's build from
the actual master repo, not from the (possibly slightly outdated) mirror

Change-Id: I12d090a9ee7fde09892567ada739826484a4a35b
2022-08-12 12:11:05 +02:00
Vadim Yanitskiy ac0b663951 {jobs,coverity}: add a new project - libosmo-gprs
Change-Id: I83e86084e9f7a7431bf5a70c372da1f9ccd28436
2022-08-10 04:39:36 +07:00
Vadim Yanitskiy 33c09f4344 jobs/master-builds: fix libosmo-pfcp related trigger mess
* master-libosmo-pfcp is triggered directly by libosmocore, not
  by master-libosmo-netif, nor master-libosmo-abis, so
** fix comment explaining the hierarchy.

* master-osmo-upf is triggered by master-libosmo-pfcp, so
** it should not be triggered directly by libosmocore.

Change-Id: Ie4a5cb47464e9abfc9d04e63e9e71619f33ffd42
Fixes: Icb75adec886cf1053b80c968fea59e597ed289a7
2022-08-10 04:08:30 +07:00
Oliver Smith c1f64b40d1 Cosmetic: jobs/ttcn3: debian-stretch -> debian
Remove reference to stretch (debian 9), as the jobs run on a newer
debian version now. Remove the reference altogether so we don't need to
update it here. The current version can be looked up in
docker-playground.git.

Change-Id: I06c74830b0f6fe0baed44c300f9e1023fb0522de
2022-08-04 12:43:15 +00:00
Oliver Smith 35030b7618 scripts/obs: rewrite pushing source pkgs to OBS
Harald requested that the OBS scripts should not stop if building one
specific source package fails, instead it should keep going and report
at the end a non-success exit code.

Given that the shell script code has historically grown and became hard
to maintain, I decided to rewrite the scripts for implementing this
feature. This rewrite solves additional problems:

* No full checkout of an OBS project like network:osmocom:latest
  anymore, with lots of packages that won't get updated (e.g. the uhd
  package has a uhd-images_3.14.1.1.tar.xz file that is 108 MB). With
  the old code, developers had to wait minutes during the checkout
  before the script reaches code that is currently being developed. Now
  only single packages get checked out right before they get updated.

* No need to clone git repositories over and over. With the new code,
  git repos only get cloned if needed (for latest it is not needed if
  the remote git tag is the same as the version in OBS). During
  development, the cloned git repositories are cached.

* Output from commands like "git tag -l" is not written to the log
  unless they failed. This makes the log more readable, which is
  especially important when a package fails to build, we keep going and
  need to spot the build error in the middle of the log later on.

* No more duplicated code for nightly and latest scripts that worked
  similar but had slight differences. Also the list of packages is not
  duplicated for nightly and latest anymore; nightly uses all packages
  and latest uses packages that have at least one git tag.

* Building source packages is decoupled from uploading them. A separate
  script build_srcpkg.py can be used to just build the deb + rpm spec
  source packages, without interacting with the OBS server.

* The scripts can optionally run in docker with a command-line switch,
  and this is used by jenkins. This way we don't need to install
  more dependencies on the host such as rebar3 which is now needed for
  erlang/osmo_dia2gsup.

* Add erlang/osmo_dia2gsup and run its generate_build_dep.sh (SYS#6006)

I have done the new implementation in python to make use of argparse
and to be able to use try/except and print a trace when building one
package fails.

Example output:
* https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_nightly_obs.osmocom.org/48/console
* https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_latest_obs.osmocom.org/46/console

Change-Id: I45a555d05a9da808c0fe0145aae665f583cb80d9
2022-08-01 16:29:20 +02:00
Oliver Smith 444ad3677e jobs/ttcn3-testsuites: remove 2022q1
Related: SYS#6007
Change-Id: Ic4cfd2dbf85fb08394166605c37f1778bf7dcb44
2022-07-15 13:39:54 +00:00
Oliver Smith 2bb0a98756 jobs/gerrit-verifications: add osmo-opencm3-projects
Related: SYS#6041
Change-Id: Ic514e96ff515c5703f39f5a31f419ff344179368
2022-07-12 14:33:30 +00:00
Oliver Smith 86c77d3958 jobs/master-builds: enable osmo-opencm3-projects
Use debian-bullseye-jenkins to have picolibc-arm-none-eabi available.

Related: OS#6041
Change-Id: Iefff6e3c3b18f4af99079635a45320faaac54e50
2022-07-12 14:33:30 +00:00
Oliver Smith e1540eee9e jobs/coverity.yml: add parameter for git branch
Allow configuring a different git branch before starting the build. This
is especially useful to reproduce a failure that only happens when using
coverity tools during a build.

Change-Id: I5f7c615ff5af4bf1f34ccef62767b5ca71cd2d8d
2022-07-12 14:45:09 +02:00
Oliver Smith ff94d1def5 jobs/coverity: don't cd ~/osmo-ci/coverity
Run the coverity/jenkins.sh from the osmo-ci.git repository that jenkins
checks out at the beginning of the build, instead of using a copy in ~/
that is updated whenever pushing to master. This makes it less
confusing, I configured the job to use a WIP branch and didn't realize
that it still used the code from master.

Change-Id: Ie992d71d2afec6982447b8fe3626c821a9e48d19
2022-07-12 14:44:52 +02:00
Oliver Smith e58c208706 jobs/coverity: modernize
* no need to mention "(jenkins-job-builder)" in the display-name
* set mail to jenkins-notifications ML like most other jobs

Change-Id: I0284dd7354157e057903f53cde149ce652e76550
2022-07-12 14:03:05 +02:00
Oliver Smith 451d687036 Revert "WIP: coverity: Make sure curl is installed, as our coverity upload job needs it"
Revert 3038c19155, which adds a second
osmocom-coverity-deps job that apparently was never deployed (or was
deployed and then deleted). The mentioned curl dependency shouldn't go
here either.

Change-Id: I5d4709c0019ef4bae6ccbd11c28eeb23d75692c1
2022-07-12 14:02:59 +02:00
Oliver Smith 2b272e0e83 jobs/registry-update-base-images: drop old images
No need to mirror these images if we don't use them anymore.
Remove debian 8 and 9, and ubuntu zesty. The latter is the codename for
ubuntu 17.04, which is a non-LTS and was unsupported since 2018-01-13.

Related: OS#5601
Change-Id: If1c198038b0fd5faf0763172447962969448940e
2022-07-06 15:20:22 +00:00
Oliver Smith 095a110506 jobs/repo-install-test: update debian versions
Run against debian 10 and 11, and not against debian 9 anymore.

Related: OS#5601
Change-Id: Iaf7c596e0910ca2ce0fafd2819e79d4659b92514
2022-07-06 15:20:22 +00:00
Oliver Smith 8424c5986f jobs: stretch -> buster
Related: OS#5601
Change-Id: Ieb2bfc426563e48b9b8cd3910efbebdf09b41f46
2022-07-06 15:20:22 +00:00
Oliver Smith 2bf20f60ed jobs/osmocom-release-tarballs: move to dp
Move the long shell command into a jenkins.sh in docker-playground, and
move the osmocom-release-tarballs.sh script there as well. This makes it
easier to test the script, and only patching one repository is needed
from now on when changing to a newer debian version.

Depends: docker-playground I4cf66097f508c029f688ec8af1cab2238d42f84f
Change-Id: I15f6c08b807f1462f91e95b624cdc1aad0592dd6
2022-07-01 15:50:37 +00:00
Pau Espin 0d92cd5985 jobs: ttcn3-testsuites: Enable hnodeb,hnbgw,sccp,gbproxy 'latest' after new project releases
These two projects have been released recently as part of Osmocom CNI
202206. Let's enable testing them since version in latest is good enough
for testing.

Change-Id: I1587e8400eff2d8b1ec26ab2fff1fbfa772fe8f8
2022-07-01 14:24:35 +00:00
Oliver Smith 2b7b90e555 jobs/update-osmo-*: change mail recipients
Use jenkins-notifications@lists.osmocom.org like in most other jobs to
not clutter the other ML.

Change-Id: I95c22215dbe09d06d9458a980d15370440f67fd6
2022-06-29 09:33:30 +00:00
Oliver Smith fc4579cf76 jobs/osmocom-obs: run against obs.osmocom.org too
Split the OBS related jenkins jobs into two, one that uploads to the
new obs.osmocom.org and one that uploads to build.opensuse.org.

Set an alias for "osc" that sets the right server, and set the
expand_aliases shopt so the bash scripts that source common-obs.sh will
not ignore the alias.

Related: OS#5557
Change-Id: Ib271e16e93ebc58ed1ecb0ea881e1f1370e40928
2022-06-27 13:31:44 +02:00
Oliver Smith 6e7afd94d3 jobs/ttcn3-testsuites: drop 2021q1, 2021q4
Not needed anymore, as discussed in related issue.

Related: SYS#6007
Change-Id: Id8f7d7c198647df60b684fd5767e77cadff60a76
2022-06-21 14:40:27 +02:00
Oliver Smith 51ed5421f1 jobs, scripts: add 2022q2
Related: SYS#6007
Change-Id: I59198f38ba817f56f6b24eae163297393587ec0b
2022-06-21 13:22:05 +02:00
Oliver Smith 3577608180 jobs/master-builds: disable master-osmo-opencm3-projects
As discussed in the team meeting, disable it for now.

Change-Id: I4a79a146042d5cf586f5273960a9a2b92b45b1be
2022-06-21 11:39:32 +02:00
Neels Hofmeyr 38d569d45f add libosmo-pfcp
Change-Id: Icb75adec886cf1053b80c968fea59e597ed289a7
2022-06-17 16:33:10 +02:00
Oliver Smith f6e305928a jobs/update-osmo-ci: gtp0-deb{10 => 11}build32
Related: OS#5453
Change-Id: I4b10550f4274995e70bf32f013bffecc1039d8a0
2022-05-18 16:43:01 +02:00
Oliver Smith c825551ca8 jobs: TTCN3-centos: set OSMOCOM_REPO_TESTSUITE_MIRROR
In previous patch Id114d55edc2ba92f7851243ab7df51db332079c4, only the
OSMOCOM_REPO_MIRROR should have been overridden. However
OSMOCOM_REPO_TESTSUITE_MIRROR defaults to OSMOCOM_REPO_MIRROR, so
explicitly set it in this patch to the opensuse mirror.

This fixes problems while building debian-bullseye-titan caused by the
different directory structure of both mirrors, and OSMOCOM_REPO_PATH
intentionally not taken into account while building the testsuite
docker image.

This is part 2/2 of fixing TTCN3-centos-bts-test, which has been failing
since OSMOCOM_REPO_MIRROR was changed in osmo-ci.git. Part 1 is in
docker-playground I78e0445a1b82e5cd72ecaa86e8fe25516b19f485.

Change-Id: I8201f34a0f9747eea78c358a626cc3981d7407c4
2022-05-13 11:15:00 +02:00
Oliver Smith 526f1e12d3 jobs/ttcn3-testsuites: centos: use osmocom mirror
Lately we've been seeing a lot of build failures from the TTCN3-centos
jobs running against the nightly repository. It seems that the mirror we
get redirected to isn't syncing properly. Use the Osmocom mirror
instead.

Fix for:
  Nightly packages of the Osmocom project (CentOS 190 kB/s | 1.7 kB     00:00
  Nightly packages of the Osmocom project (CentOS 1.8 kB/s | 634  B     00:00
  [91mErrors during downloading metadata for repository 'network_osmocom_nightly':
    - Status code: 404 for 7062058d3e234003a6c3595331cf3fc100-filelists.xml.gz (IP: 134.76.12.6)
    - Status code: 404 for ad91db8cb719947d9abc971661c588e8c6-primary.xml.gz (IP: 134.76.12.6)
  [0m[91mError: Failed to download metadata for repo 'network_osmocom_nightly': Yum repo downloading error: Downloading error(s): repodata/ad91db8cb7273beaefca0e473d9c4dd12d12786819947d9abc971661c588e8c6-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/7062058d3e5395b4b02570496e41e19e8fd306f5234003a6c3595331cf3fc100-filelists.xml.gz - Cannot download, all mirrors were already tried without success

Change-Id: Id114d55edc2ba92f7851243ab7df51db332079c4
2022-05-05 15:05:50 +02:00
Oliver Smith 382aebd040 treewide: rename rpi4-raspbian10 to rpi4-raspbian11
The raspberries have been upgraded to the debian 11 based raspberry pi
OS. Adjust the name to avoid confusion.

Related: OS#5453
Change-Id: I0b2e08b11d83243e81b2bb4412ffcc9bc59821ff
2022-05-02 13:29:50 +00:00
Harald Welte 032c0de4b8 dahdi-linux: Add gerrit build verification against master
Now that master-dahdi-linux-master is passing, we should verify any
future patches also against master.

Change-Id: I86cfd7e868dcaeeb6c6dbc6f08046a70b4582cc7
2022-04-24 19:48:47 +02:00
Harald Welte 5b933d8aea dahdi: use buster, not stretch
stretch has too old gcc which doesn't support -Wno-format-truncation
which is used in dahdi-linux since 2018.

Related: OS#5407
Change-Id: I4fca722e67b2c649dea4f9799d277cd568bcc493
2022-04-24 18:20:58 +02:00
Oliver Smith 7abb4ca810 OBS, ttcn3-testsuites: add 2022q1
Related: SYS#5887
Change-Id: Idadb541b49e57b4403a6a6e4c966bcba9c6651db
2022-04-13 19:38:18 +02:00
Pau Espin 99578e8653 jobs: Add ttcn3-pgw-test jenkins job
Change-Id: I363a7a8eb524abfeb45c81603c29cea96f86a787
2022-04-07 17:55:39 +02:00
Oliver Smith f5ab1346db jobs/ttcn3-testsuites: adjust nplab-m3ua timers
Fix sporadic errors related to the clean up script running while images
being built:

  failed to get digest sha256:8ea2ba9a1...

Closes: OS#5447
Change-Id: If722fddce8de65a1ae012a46d6a202f430e0c69d
2022-02-28 10:42:24 +01:00
Daniel Willmann 5ecc602efe Reenable ttcn3-hnbgw-test
Now that mgw support is in osmo-hnbgw master and docker/ttcn3 testsuite
is working we can enable the test again.

Change-Id: I4c7d1ee876e90d52827b2d8cc57342298dd44061
Related: OS#5152
2022-02-24 11:43:39 +01:00
Oliver Smith 8922843ec2 jobs/ttcn3-testsuites.yml: add ttcn3-fr-test
Add a configuration for the job to the yml file. It looks like this job
was created manually, or the configuration was just not pushed to
gerrit.

This makes e-mail notifications consistent and fixes missing colors in
build output.

Change-Id: I14995dea0a0d223b78e20b49953d5c814c1ad4a1
2022-02-22 11:55:13 +00:00
Oliver Smith f271bea63d jobs/gerrit-lint: don't verify dahdi-linux
Closes: OS#5465
Change-Id: I41ede04849da033a497daf39b34828bed00ab724
2022-02-22 11:33:25 +01:00
Harald Welte 8349494c9e jenkins-jobs: Add master + gerrit for dahdi-tools
Change-Id: I9c007e070e168bfe6509f50dbc3b4e0846c31356
2022-02-21 21:23:58 +01:00
Pau Espin ef0e042ae8 jobs/ttcn3-testsuites.yml: Add ttcn3-ggsn-test-ogs
Change-Id: I61950f4fc613faae3a731ace394b76c8754bd3c4
2022-02-21 13:08:48 +00:00
Oliver Smith b712491a56 jobs/gerrit-verifications: build ttcn3 in docker
Build the testsuites in the same docker container during gerrit
verification, as it will be the case when building and running the
testsuites in the various ttcn3-* jenkins jobs.

This avoids the need to install eclipse-titan and libfftranscode on the
jenkins nodes.

Related: OS#5452
Change-Id: I6a667255bb55a945b4c6279d711142aa252c6996
2022-02-21 08:08:00 +00:00
Pau Espin ef9b2c4062 Use docker image debian-bullseye-erlang instead of debian-buster-erlang
Let's use newer debian to avoid problems with older erlang. Furthermore,
the new image builds rebar3 to avoid issues with unmatching erlang
versions.

Change-Id: I7b1956c515daccf6ab1ff87031c6fec649cadb4b
2022-02-15 20:36:10 +01:00
Oliver Smith 5bf4022a5b jobs/gerrit-lint: run on docker-playground too
Change-Id: I7213155ccd61a006231609b1d7ae1eec187b3e7b
2022-02-11 15:44:54 +01:00
Harald Welte 5cff7924ec jobs/master-builds.yml: Add master-pysim to publish manual
We only had gerrit build verification for pysim so far, but no 'master'
job that periodically builds/tests master (and publishes the manual).

As a reult, the pySim manual was still the original version from April
2021, while the code has of course progressed quite a bit in git since.

Change-Id: I4dce05cdb5f2c574256378d5fb66cff1c329c141
Closes: OS#5271
2022-02-09 17:48:04 +01:00
Pau Espin 1982f52f8e Deploy osmo-ci to ogt nodes and update it periodically
We need osmo-ci in those nodes in order for the docker old images to be
cleaned through cron entry installed by ansible docker role, which uses
/home/jenkins/osmo-ci/scripts/docker-cleanup.sh.

Related: OS#5441
Change-Id: Ieb6db0b24dce6099cfa24871a957471ca2c8bcee
2022-02-07 11:29:26 +01:00
Oliver Smith 7ad596e4d9 jobs/master-builds-dahdi.yml: new file
Related: OS#5407
Depends: dahdi-linux I904ab66a1ecd72492642ac2cc4cb102c7283c590
Change-Id: I8a479d0a214ff56c5b66f468a1c2b1b2f674e091
2022-02-04 15:05:04 +01:00
Oliver Smith b3cef938c9 jobs/gerrit-verifications-dahdi.yml: new file
Add gerrit verifications for the out-of-tree dahdi-linux kernel module.
Put this in a separate file (not gerrit-verifications.yml), because:
* not only gerrit git repo gets cloned, but also linux.git, and caching
  needs to be done properly for it
* so we can have one separate job per linux version that gets tested

Test v4.19 and v5.10 for now, as suggested in the issue.

Related: OS#5407
Depends: docker-playground Id72d19ad08681cd7cb3194de2226292f19e96df5
Change-Id: I65c7663a6b8045796536a727e4adf5554f16b2a9
2022-02-04 14:40:17 +01:00
Oliver Smith b7bf1b735d jobs/gerrit-lint: add dahdi-linux
Related: OS#5407
Change-Id: I7fdb40e40fb8cf8c9072a4e898aeffa43d1d7ae0
2022-02-04 12:36:28 +01:00
Oliver Smith 68d8fab771 jobs/registry-update-base-images: centos8 -> alma
Follow up on the change of using almalinux instead of centos8 in
docker-playground, with providing the almalinux base images in the
Osmocom registry.

Related: SYS#5818
Related: docker-playground I30e1a773b901b1d2187214445116c7f2aecc4e36
Change-Id: I93a8892da628b8d2f9cbb2a6453e8ab3153a9c4a
2022-02-03 09:31:58 +01:00
Oliver Smith c04fb387ea jobs/{master-builds,gerrit-*}: add osmo-bsc-nat
Related: SYS#5560
Change-Id: I240cb614bf3905d04bccc6a00c72567a2ad722d8
2022-01-31 14:05:34 +01:00
Daniel Willmann a96eeaade9 Disable ttcn3-hnbgw-test for now
Change-Id: Ia385c4155ffed761c5887fc4d6c76163ca732527
2022-01-26 11:44:46 +01:00
Harald Welte 8a3cd203f8 jenkins-jobs: add master build for opencm3 projects
Change-Id: I95f84079478ce57d8d5c0cffd71bc67e1605e68e
2022-01-18 14:58:41 +00:00
Oliver Smith 6350b2f537 Revert "remove linter jobs for osmo-e1d and osmo-e1-firmware"
Enable the linter again, as we decided to use a repo-specific checkpatch
config in these.

This reverts commit 2c6aa82c9a.

Related: OS#5399
Change-Id: Ib79a20658bfae64d9b718e53e6a9393708a4e694
2022-01-18 09:47:10 +01:00