Commit Graph

507 Commits

Author SHA1 Message Date
Pau Espin a264f9fcdd gsm-tester: Manual build and install of patchelf v0.10
Due to [1], we require both debian9 patchelf v0.9 and v0.10 to be
available and used by osmo-gsm-teser depending on the binary to be
patched.

[1] https://github.com/NixOS/patchelf/issues/192
Related: OS#4389
Change-Id: I28825d723b85900fb51cc5b8a9d14c6ef346e667
2020-02-11 14:00:32 +01:00
Harald Welte fcad862f86 coverity: Our coverity job needs curl; install it
curl \
 --form token="$token" \
 --form email=holger@freyther.de --form file=@Osmocom.tgz \
 --form version=Version --form description=AutoUpload \
 https://scan.coverity.com/builds?project=Osmocom
./jenkins.sh: line 39: curl: command not found

Change-Id: Ibf4a40860fe6c8424a60672dc4758b9f17c532a3
2020-02-08 13:28:54 +01:00
Harald Welte a42c3c1900 upgrade coverity to 2019.03
Change-Id: I0425b24a2afd979eb2a44dee512512746929770e
2020-02-08 13:13:04 +01:00
Pau Espin e94da921d7 ansible: gsm-tester: Add missing dep python3-setuptools
That's required when building python modules with pip.

Change-Id: I80785a4061e91126092a04080a2c9fdb288be49a
2020-02-03 16:03:47 +01:00
Pau Espin c17216723b ansible: mdbus2 is not nowadays available on Debian10
Change-Id: I0eb2ed28d9e3bc9054ffeb81c9e6fb5428af5bfb
2020-01-29 19:57:06 +01:00
Pau Espin f57eb28d13 Install srsLTE build dependencies in jenkins buildhosts
Change-Id: Ib42987416ee90868ab088bfa2f92d92c764088b0
2020-01-29 19:54:51 +01:00
Pau Espin 5f4e887c5f jobs: Introduce osmo-gsm-tester_build-srslte
Change-Id: I08a59c6065ca4fb6fbbed5b12e73abf4024f1dcb
2020-01-29 19:38:19 +01:00
Pau Espin 3456d3a2a5 ansible: gsm-tester: Install srsLTE dependencies
Change-Id: Ia6804d0d0c40ad2ebe8a09b0861f14f15578a834
2020-01-29 14:33:44 +01:00
Harald Welte 84cbb6fa13 Fix ansible playbooks after docker directory was removed here
In I30a61aebcadef5536e74edd35e1c75ef77a2da9f, we moved the Dockerfiles
from here to docker-playground.git, but failed to adjust the ansible
playbook that wants to rebuild those containers during setting up of
a build slave.

Change-Id: Ife64fdaf24107b0b31b8a6d4c78a1df420662ca0
Related: OS#4345
Closes: OS#4377
2020-01-26 14:03:51 +01:00
Harald Welte 8eb4fdb591 ansible: Install poky-sdk only on x86_64 architecture
The SDK is pre-compiled for a x86_64 host architecture, we cannot
use it e.g. on armhf (raspi).

Change-Id: Ia565d2dba427c00aa08a4eca62ac7410448ada04
2020-01-26 13:35:44 +01:00
Harald Welte 0bd157f0bd ansible/docker: ensure gnupg + crontab are installed
Both are required by docker, but seem to be missing from explicit
dpkg 'Requires' :/

TASK [docker : add docker gpg key to apt keyring] ********************************************************
fatal: [rpi4-deb9build-ansible]: FAILED! => {"changed": false, "msg": "Failed to find required executable gpg in paths: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"}

TASK [docker : cleanup old docker images] ****************************************************************
[WARNING]: The value 0 (type int) in a string field was converted to '0' (type string). If this does not
look like what you expect, quote the entire value to ensure it does not change.

fatal: [rpi4-deb9build-ansible]: FAILED! => {"changed": false, "msg": "Failed to find required executable crontab in paths: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"}

Change-Id: I2d342f85f6bc1b80eaafc7075552019fea147894
2020-01-26 13:35:33 +01:00
Pau Espin e0f4129b63 ttcn3-testsuites.yml: Add job for ttcn3-sccp-test
Only for nightly, not latest, since example/sccp_demo_user is not
installed by autotools and debian packages, so only in nightly where we
build from source we are able to copy it to docker image manually.

Related: OS#4343
Change-Id: Ia0a68d74215bdc1bc6319241d60f858312fa14e9
2020-01-17 18:13:47 +01:00
Oliver Smith fb3dc0b775 gerrit-verifications: fix erlang/osmo_ss7 name
Fixes: aadc60f403 ("jobs/gerrit-verifications.yml: add erlang repos")
Related: OS#4345
Change-Id: Id78f7e62e12c7595329ef8a265e522f27a087604
2020-01-17 16:21:44 +01:00
Oliver Smith 2c13d633ec gerrit-verification: fix erlang cmd
Jenkins job builder only does one pass of variable substitution. Replace
the cmd_erlang variable with its contents, so the jobs don't fail with:

[osmocom-gerrit-debian9] $ /bin/sh -xe /tmp/jenkins7189809471369692535.sh
+ {docker_run} {docker_img_erlang} /build/contrib/jenkins.sh
/tmp/jenkins7189809471369692535.sh: 2: /tmp/jenkins7189809471369692535.sh: {docker_run}: not found

Fixes: aadc60f403 ("jobs/gerrit-verifications.yml: add erlang repos")
Related: OS#4345
Change-Id: I6e121c79ecf872a7058de3a6dc0a940fa8091f68
2020-01-17 15:03:08 +00:00
Oliver Smith 5434eb477f osmocom-release-tarballs: publish with rsync
Related: OS#3870
Change-Id: I5f88ae0c483ed03c42c47d1cd207ac2722cfa083
2020-01-17 15:52:16 +01:00
Oliver Smith d93fd48da9 OBS: clone from gerrit.osmocom.org
Work around "garbage at end of loose object" errors that occasionally
cause these jenkins jobs to fail.

A few repositories are not hosted on gerrit, so they still get cloned
from git.osmocom.org. However, having almost all repositories cloned from
gerrit should improve the situation a lot.

Related: OS#4083
Change-Id: Id8f08a1bc10d6c81be9ad44c60646e2ea9f6cf4e
2020-01-16 14:42:02 +00:00
Oliver Smith aadc60f403 jobs/gerrit-verifications.yml: add erlang repos
Run contrib/jenkins.sh in erlang repositories, that were recently
updated (osmo_gsup, osmo_dia2gsup, osmo_ss7).

Depends: docker-playground Ia3eaec6090c9652549b2850de74ee21730374bbd
Related: OS#4345
Change-Id: I05d152de6b7a04dee935d79b9987c511351eca95
2020-01-15 10:26:23 +01:00
Oliver Smith b79e06367b docker: move to docker-playground.git
Instead of building "osmocom:deb9_amd64" from this repository, build
"$USER/debian-stretch-jenkins" from docker-playground.git (same
Dockerfile). Adjust all jobs to use the new image name.

Add a new "update-osmo-ci-on-slaves-dp" jenkins job, which triggers
the existing "update-osmo-ci-on-slaves" job whenever
docker-playground.git changes.

Replace docker/rebuild_osmocom_jenkins_image.sh with
scripts/osmo-ci-docker-rebuild.sh, so we can get rid of the docker dir.
I thought about dropping the script completely, and directly writing the
two lines into contrib/jenkins.sh. But I kept the extra script for
convenience, when testing locally.

Related: OS#4345
Depends: docker-playground I125ae8a6bcabbd1f485028c79b0abacda0622c3a
Change-Id: I30a61aebcadef5536e74edd35e1c75ef77a2da9f
2020-01-15 10:24:38 +01:00
Oliver Smith d0418d0f92 docker_images_require: add symlink code path
Make development easier by skipping fetch, checkout and reset --hard if
_docker_playground is a symlink. Document _docker_playground in
README.adoc and explain how to set up the symlink.

Change-Id: If6209ff71488d39e590f5f8506b9d73ad0314846
2020-01-15 10:19:13 +01:00
Oliver Smith 51e91cd9bb gerrit-verifications: introduce "gerrit_url" var
Prepare for erlang repositories, which will use this variable.

Related: OS#4345
Change-Id: Ia91764797745b48ae832cfbc16976e561b288a50
2020-01-14 14:47:53 +01:00
Oliver Smith c18ab84b10 docker: drop python2 (again)
Re-apply patch Iacace670bdb949e67227e0faa1765e36bba42018, after
providing additional patches to make all Osmocom repositories, that use
external tests, work with python3 (after learning the hard way, that
they were not really python3 compatible yet, although they looked like
it):

https://gerrit.osmocom.org/q/topic:drop-py2

This patch should not be merged, before all other patches from that
topic are merged.

Depends: osmo-python-tests I3ffc3519bf6c22536a49dad7a966188ddad351a7
Related: OS#2819
Change-Id: I7b5385416d5c48a2f4e07449192b3f74223cb094
2020-01-13 09:29:27 +00:00
Oliver Smith c31ee44bd8 jobs/update-osmo-ci-on-slaves: origin/master only
Don't run, when different branches were pushed.

Change-Id: I2cf55391066b60cf6a594f528ed0eaf59a886391
2020-01-08 15:04:25 +01:00
Harald Welte a47e13ee49 osmo-depcheck: Skip comment lines containing PKG_CHECK_CONFIG
This is a problem e.g. with current osmo-hlr containing
"\t# FIXME: PKG_CHECK_MODULES() may return cached result here!"

Change-Id: I30d539a895bf39aaabe907be9eb52d7e4b3977a7
2019-12-17 14:38:22 +01:00
Harald Welte 8d0605c4fb verify_*.py: Ignore UTF-8 decoding errors
Some of our source files are inherited from other sources, particularly
for microcontroller firmware projects.  We cannot assume they're all
clean UTF-8.  Let's ignore any decoder errors when verifying log
statements and value_string arrays.

Closes: OS#4334
Change-Id: I1e19f4bc6bee46481c6ea743e8334bd4485909be
2019-12-17 13:19:12 +01:00
Harald Welte bcd5a7b927 implement osmo_source_subdir() for simtrace2
Change-Id: I3afad11991d2e8bd72f44bed05f7204653e63133
2019-12-17 02:34:58 +01:00
Oliver Smith 853f1ba318 osmocom_build_deps.yml: add inkscape
osmo-gsm-manuals needs inkscape to handle svgs. Not all osmo-* projects
are building in docker, so make sure it is also installed on the build
slaves directly.

Related: OS#4314
Change-Id: I63460336d2c4a7594ec9c5c9b60a3d43f0d157aa
2019-12-09 09:59:40 +01:00
Oliver Smith bf034999f1 Revert "docker: drop python2"
This reverts commit 9097082be5.

I was under the impression, that all previous scripts in
osmo-python-tests were already working with python 3. But as it turns
out, this isn't true. Reverting, so I can properly post follow-up
patches, that fix the issues before we apply this "drop python2" patch
again.

Related: OS#2819
Change-Id: Iab7768a77a55fd0465244460bd7488858dd69b60
2019-12-06 15:05:55 +00:00
Oliver Smith 9e69cb84d4 Revert "docker: drop python2"
This reverts commit 9097082be5.

I was under the impression, that all previous scripts in
osmo-python-tests were already working with python 3. But as it turns
out, this isn't true. Reverting, so I can properly post follow-up
patches, that fix the issues before we apply this "drop python2" patch
again.

Related: OS#2819
Change-Id: I68e99ed3fb2eba86eaccec7a3cab75658e9543b1
2019-12-06 16:01:01 +01:00
Oliver Smith 9097082be5 docker: drop python2
Depends: osmo-python-tests Iabda95073faa2191fd117e9637e0858c589e9d9e
Depends: osmo-gsm-manuals I57522e57e39682d6d123af4584785d327361e5a3
Related: OS#2819
Change-Id: Iacace670bdb949e67227e0faa1765e36bba42018
2019-12-05 17:53:27 +01:00
Oliver Smith d52cb3cc6b docker: install inkscape
Needed by osmo-gsm-manuals to add svg graphics to the manuals.

Change-Id: I754b133d77743582bd84c33c74ecc9eb9ca4c0ef
2019-12-05 16:51:49 +01:00
Harald Welte 0bc2af048b master-builds: add osmo-{cbc,e1d,e1-recorder}
Change-Id: Idfede8e9e79019b67b9aff4881d5cea97514aeba
2019-12-04 12:32:15 +01:00
Harald Welte 71c642a0a5 add gerrit-verifications for osmo-e1-recorder and osmo-cbc
Change-Id: I1df28b4e7a7cbcf9151cd57a77001d88f1875415
2019-12-04 11:06:05 +00:00
Oliver Smith 02bb4049ba Cosmetic: gerrit/master jobs: link to wiki
Place a link to the wiki page, that now documents all available
environment variables.

Related: OS#4243
Change-Id: I770360b671c64369cf6cdb4fb4222921cc6af58e
2019-12-04 10:52:21 +01:00
Pau Espin 2d8903c363 ansible: gsm-tester: Enable ip forwarding and masquerading on boot
Some nodes in the internal network, such as LimeNetMicro, will need
Internet access to upgrade osmo-trx-lms from OBS repos.
It also makes it easier to update manually other nodes inside the
internal network.

Change-Id: I2c89cf9cfcb55b3153e7be212c68ffa8db0f6927
2019-12-02 11:15:14 +00:00
Oliver Smith 13eaaa3644 jobs/master-builds.yml: fix osmo-fl2k
Don't use the old "osmocom:amd64" image. Remove the entire cmd line, so
it uses the default docker command with the default docker image
"osmocom:deb9_amd64". Also remove concurrent: true, because that default
nowadays too.

Related: OS#3203
Change-Id: I93db12d3652f26db059e0ede52d2b2fdd3e89d34
2019-12-02 09:54:56 +01:00
Oliver Smith f8d85c9f18 jobs/master-builds.yml: add osmo-fl2k
Related: OS#3203
Depends: https://lists.osmocom.org/pipermail/osmocom-sdr/2019-July/001951.html
Change-Id: Iac411a5d5e312b8db0b5388f58571141fb2f32df
2019-12-01 10:40:16 +00:00
Harald Welte 45ce0417d4 add '--publish' to osmo-asf4-dfu and osmo-ccid-firmware builds
Change-Id: Ie45b9fcabc35f6fd9aff33af0f0587c8ccd88ebd
2019-12-01 10:38:16 +00:00
Pau Espin d8e2f0a682 ansible: Fix several file permission mask format
From ansible documentation [1]:
"""
For those used to /usr/bin/chmod remember that modes are actually octal
numbers. You must either add a leading zero so that Ansible's YAML
parser knows it is an octal number (like 0644 or 01777) or quote it
(like '644' or '1777') so Ansible receives a string and can do its own
conversion from string into number.
Giving Ansible a number without following one of these rules will end up
with a decimal number which will have unexpected results.
"""

[1] https://docs.ansible.com/ansible/latest/modules/template_module.html?highlight=template

Change-Id: I91008927c50016a837420c4249b5f69b9153572e
2019-11-27 09:30:28 +01:00
Pau Espin 55c6811513 ansible: gsm-tester-prod: Set DNS server to a working service
10.42.42.2 is simply one of the IP addresses assigned to osmo-gsm-tester
main unit on its own internal network (the one used between itself and
the connected BTS or helper hosts). There's no DNS server running there.
The main unit already runs lots of services, and there's really no need
for a DNS server running there, so simply point it to a working DNS
server.

Change-Id: I443cc560553c07635763a13bc73ead4835e6a009
2019-11-26 13:30:44 +01:00
Harald Welte 15e2204a9f osmocom_build_deps.yml: Move from python-nwdiag to python3-nwdiag
Change-Id: I9e3c65342d1bb62d7baa9d589930aa042fb5f7a9
Closes: OS#4279
Related: OS#4246
Depends: osmo-gsm-manuals Ia2c0c833329f8ecab1c427e0d2c002f17ab49a94
2019-11-23 09:20:22 +01:00
Harald Welte 3038c19155 WIP: coverity: Make sure curl is installed, as our coverity upload job needs it
This is untested. I'm on the train and have very limited understanding
of Ansible

Change-Id: I9918b89971bf6c2e200f1d96d993d5fbd6741b71
2019-11-23 09:19:10 +01:00
Harald Welte 803216f1fd dockerfile: Add 'dahdi-source' for libosmo-abis
Change-Id: Icfd244268958be71f538d27307cdf0fabeecd956
2019-11-23 09:19:10 +01:00
Pau Espin dffe2e859f ansible: gsm-tester-prod: Set dhcp ip addr for LimeNet-micro board
Change-Id: Ibcdf14ba09ab46240cd6bccac82f7b21f99123ef
2019-11-22 14:55:12 +00:00
Oliver Smith 0777e3b292 docker: remove python-nwdiag (using nwdiag3 now)
Related: OS#4246
Depends: osmo-gsm-manuals Ia2c0c833329f8ecab1c427e0d2c002f17ab49a94
Change-Id: Ib53e1fee56e27bd88fc1a7b9593bacf53328a097
2019-11-22 11:26:34 +01:00
Oliver Smith 1cb0aa92c9 docker: install python3-nwdiag
Related: OS#4246
Change-Id: I0ebee6ac11df779ab954bd6648a0bebefc3350a2
2019-11-22 11:22:50 +01:00
Harald Welte bee13a5f53 osmocom_build_deps.yml: Add dahdi-source for libosmo-abis
Change-Id: Ic314c992f83d9fe1ea6035d67070d7c6004cc9e3
2019-11-11 17:13:00 +01:00
Harald Welte 5d627e65ea osmocom_build_deps: Add libaio-dev for osmo-ccid-client
Change-Id: I6c3b536bee3d1bbd6f526bc3765fc4ad9aa4a0dd
2019-11-11 17:13:00 +01:00
Pau Espin 95d4544544 jobs: Introduce ttcn3-stp-test(-latest) jenkins job
Change-Id: I7a82e1fa82cd28433cdb7a9128c7ab3da6cabf47
2019-10-24 16:52:04 +02:00
Oliver Smith 7c987a83b3 docker: only one apt/pip cmd for installing deps
Make the retry logic in rebuild_osmocom_jenkins_image.sh obsolete by
merging all apt commands for installing dependencies into one. This
makes development easier, because the docker image will not get rebuilt
again when any error happens (whether that error was related to the apt
cache or not, during development it is mostly not related to the apt
cache).

This also makes it easier to read through the list of dependencies, and
removes duplicate (or even conflicting, pkg-config vs pkgconf) entries.
While at it, also merge the pip commands into one.

Change-Id: I190e62dd39533b284c93107607dac24ece6d7be6
2019-10-18 10:44:11 +02:00
Oliver Smith f58d2231a9 gerrit, master builds: default to concurrent: true
Now that almost everything runs in docker, we don't need to worry about
problems with running VTY/CTRL tests in parallel for these projects.

Related: OS#3726
Change-Id: I5fdf759faf47ec7e4f9cb2657bd004585a667add
2019-10-18 10:44:10 +02:00