Commit Graph

543 Commits

Author SHA1 Message Date
Oliver Smith 15c1f80dcc Add osmocom-build-old-tags-against-master.sh
Build old releases of Osmocom programs and libraries against
"master of the day" to detect breakage.

Redirect each build's output to its own log file, so it is easy to see
what is currently getting build, and what failed. On error, print the
end of the new failing build logs, along with a note to find the full
logs in the jenkins artifacts.

This initial configuration builds the last three release tags of the
Osmocom repositories against master. The configuration can be changed
easily.

Indicate known failures with "err" instead of "ERR" in the output, do
not cause them to fail the build and do not print the beginning of the
error log (it is still in the artifacts). This way, new errors stand out
and don't get overlooked among the known errors.

Related: OS#3765
Change-Id: I7cb45cc40c9930840a3d4e6a86f39e1400478ed3
2019-03-26 15:54:38 +01:00
Oliver Smith e81f90f7e1 osmo-build-dep.sh: run 'make check' when CHECK=1
Prepare for osmocom-build-old-tags-against-master.sh.

Related: OS#3765
Change-Id: I248b53d7a6fe1b64f2a994c568c0466dc93858de
2019-03-26 10:39:13 +01:00
Oliver Smith f8653decf5 osmo-build-dep.sh: cd into subdir before building
Support building the legacy openbsc source with osmo-build.dep.sh. This
will be used by the upcoming osmocom-build-old-tags-against-master.sh
script.

Related: OS#3765
Change-Id: I852e103e80bf295f692cf13c4cb38e80fbc19eca
2019-03-26 10:39:13 +01:00
Oliver Smith 3b3a250f6b osmocom-list-commits.sh: move functions to common.sh
Prepare for the upcoming osmocom-build-old-tags-against-master.sh
script, which will benefit from querying the last n git tags in advance.

Related: OS#3765
Change-Id: I61be4cffb9275cabc1b253f0b298503ad0d3aea4
2019-03-26 10:38:58 +01:00
Oliver Smith bb714ca042 osmocom-list-commits.sh: fix sorting release tags
Properly sort "git ls-remote" output, which looks like:
352f32c9cffef2e5df99cd2f03368dc56c99ee23	refs/tags/0.4.0

Use / as field separator and field no 3, instead of assuming that the
string gets split automatically after the tab character (which it does
not).

Use -V (version sort) instead of numeric sort.

Change-Id: I334e684ac5109fc289b68af78165862148074ea7
2019-03-22 15:57:38 +00:00
Pau Espin b3b2e37246 scripts: Enable osmo-sysmon debian package OBS build
Change-Id: I638c58be1ddf5e7fcf5a51de532ae8c4c06dcdbb
2019-03-22 12:51:14 +00:00
Oliver Smith 60a4fb2062 jobs: add osmocom-list-commits.yml
Related: OS#3840
Change-Id: I9a65cb750bbe6c278b9f67eb70bdb36759cb775b
2019-03-20 11:41:28 +01:00
Oliver Smith 391bdf542f Add scripts/osmocom-list-commits.sh
Generate a table of Osmocom CNI repositories and their latest tag,
related commit, and last commit on master.

Related: OS#3840
Change-Id: I91cab0139229e6c1c67e889d33b3d984025bc9da
2019-03-20 11:11:32 +01:00
Harald Welte 915a2f76aa build_deps: add libulfius-dev and libjansson-dev for osmo-remsim
Change-Id: I8d490babb4f1f47e94d15c74a8fba439522e1e18
2019-03-15 08:21:00 +01:00
Harald Welte cb944d35f3 ttcn3-testsuites: Split ttcn3-pcu-test-sns from ttcn3-pcu-test
The SNS variant of the tests require the PCU to be started with
different config file.

Change-Id: I42785b68f136198152ad79072346faccfe4a9424
2019-03-14 21:56:21 +01:00
Holger Hans Peter Freyther 4f5c162c6d python: Add pyflakes for some static analysis
Be able to run pyflakes as part of the CI process.

Change-Id: I59e6673df5b8d915dee1f745169fc5d0426c6aed
2019-03-05 06:35:52 +00:00
Harald Welte b1ec95f3c7 jenkins: Send build failures to jenkins-notifications mailing list
Change-Id: I5609a0f4b8f176ffa692db732abb4694acaaf927
2019-03-03 16:04:41 +01:00
Pau Espin d7a11ab977 jobs: osmo-gsm-tester: Add oc2g jobs
Change-Id: I5ea9b783a2a2017cb134e72087e217a4184184b2
2019-02-21 18:50:40 +01:00
Holger Hans Peter Freyther e197c12f81 jobs: Introduce a tester using "virtual" tests
With mobile, virtphy, osmo-bts-virtual we can run e2e tests without
the need of any hardware. Create a dedicated job for it. This avoids
having to share resources (e.g. lock state/ip allocations) with other
builds.

Change-Id: I19993b287cc948dbdfba9d375f80a7904f96ff4a
2019-02-21 16:04:55 +00:00
Harald Welte 38ce864f63 ttcn3-testsuites: Add PCU tests
Change-Id: Ifd8113c8db2b6e84d6e01539a6df031dc6f4ee80
Closes: OS#2890
2019-02-20 22:41:44 +01:00
Holger Hans Peter Freyther 51a13b7882 docker: Add lua-socket for the OsmocomBB<->GSM Tester communication
Change-Id: I9156c63c64dd32a91136240913baa7e4a5cdc893
2019-02-20 08:14:39 +00:00
Holger Hans Peter Freyther f8a085347a docker: Use the same Dockerfile for jessie and stretch
The binaries built on the Jenkins to be used by the osmo-gsm-tester
require libgnutls.so.30. This is provided by the libgnutls30 package
which is not available on jessie (Debian 8) and a backport requires
some effort.

Re-use the existing Dockerfile for jessie and stretch (Debian 9) using
an argument. Refactor the shell script to pass the right argument. Add
the debian version into the tag name to make space for buster (Debian
10).

Change-Id: I44c8e49a548cb64defd2eba868ec4bd435de90ba
2019-02-19 05:06:38 +00:00
Holger Hans Peter Freyther a6d9db32c1 docker: Remove unused files to remove the clutter
We have been updating some of these files besides no container being
built or used from them. Let's clean this up by removing them.

Change-Id: I7c5b559975e5f9a50368a5bb5bd961bdb9f85270
2019-02-19 05:06:38 +00:00
Holger Hans Peter Freyther 707ebb1b0a docker: Set language environment to UTF-8 (and en_US)
The osmo-gsm-tester requires a UTF-8 locale to generate the log
messages. Most of our users will have a UTF-8 and it seems okay
to change the default from C to en_US.UTF-8.

Change-Id: I79082a57fd5c9122dbd66997dd48e3ed3e89acb9
2019-02-18 14:29:15 +00:00
Holger Hans Peter Freyther 92f5198df4 docker: Add missing gsm-tester dependency
Add one more dependency and its dependency for the osmo-gsm-tester
(a separate patch is coming to update the docs).

Change-Id: Icd6a93a8a91c5443b310fdd2123183a8059d31d1
2019-02-18 14:28:27 +00:00
Holger Hans Peter Freyther 02e90c90eb docker: Add dependencies needed for the gsm tester as well
This way we can execute the tester with a virtual environment.

Change-Id: I1e6432d77442f590453a328023c9f09cd52787a8
2019-02-18 08:57:31 +00:00
Holger Hans Peter Freyther 28e7e9c44a Fix copy and paste in the script.
Change-Id: I51d01a8799f09ce7d813a0ef8353a4233d5c8bb5
2019-02-15 15:34:52 +00:00
Daniel Willmann 74eade85ae jobs/osmo-bts: Build osmo-bts-oc2g as well
Build osmo-bts with support for oc2g as well. Uses origin/nrw/oc2g as
firmware.

Fixes: OS#3745
Change-Id: Iaaeb44b949cf6403601f069800c381fe5eae2f6e
2019-02-05 16:56:04 +01:00
Harald Welte 7963507fdd gerrit-verifications: Add osmo-ccid-firmware
The repository currently doesn't yet have a contrib/jenkins.sh
script, so all verifications will fail.  This is intentional,
as it will force the addition of a related script in the first merged
commit.

Change-Id: I713d345240dec319ee07d99e9381fc1af6a2d86d
2019-02-04 14:43:39 +01:00
Harald Welte 225dbeefe1 osmocom-jenkins-slave: Deploy osmo-ci and build docker images
This is periodically performed by https://jenkins.osmocom.org/jenkins/job/update-osmo-ci-on-slaves
but in order to get to the point of updating it, we must at least
have done one initial deployment of the repository before.  This is
done by the takss in the new osmo-ci.yml

Closes: OS#3773
Change-Id: I581b8713cfdb3ca2ef7f0925730974056981e448
2019-01-31 20:27:50 +01:00
Harald Welte d2300f83b1 ansible/install-poky-sdk: Use "-y" command line argument
At least with current ansible, this patch is required to avoid
the poky SDK installer waiting indefinitely to somebody answering
his prompts on stdin.

Change-Id: I084a5a152f00bbd484f5616d417fa1486cb79413
2019-01-29 22:53:44 +01:00
Max 844f25dfbf Add liboping dependency for osmo-sysmon
Related: SYS#2655
Change-Id: I6198cd14ff86c5d8203d57674a17aa388f21c19d
2019-01-29 12:35:26 +00:00
Oliver Smith 4ae4c5403d ttcn3: block latest jobs while master is running
Jenkins job builder does not support string replacing in variables,
so the only way to make each ttcn3-foo-test-latest job get blocked by
their ttcn3-foo-test counterpart was specifying them each time.

Related: OS#3767
Change-Id: I124286d4b996e79a7b77603c7b59473b016c3bd5
2019-01-28 15:03:47 +01:00
Oliver Smith 0c11075541 ttcn3-bsc-test-sccplite-latest: enable job
This job does not get stuck anymore since the latest Osmocom release.

Change-Id: Ie87aeb52832b4ce0fbec96d2cff9dd04da0c11fc
2019-01-28 14:48:25 +01:00
Harald Welte 2dfb2c6fa1 osmocom-*-packages: clone via https:// not via unencrypted git://
Change-Id: Ibfb52cbac93451ba402edb6b59350620954b0d81
2019-01-26 18:29:01 +01:00
Harald Welte 65353904bc latest-packages: Add libosmo-dsp
Now that libosmo-dsp has a 0.4.0 tag that includes debian packaging
information, we can add it to the latest feed.

Change-Id: Ic082a4ad6b8582dfe44df5bdaafa525e52da7246
2019-01-26 18:29:01 +01:00
Harald Welte 789c5a9c3f nightly packages: Build libosmo-dsp
Change-Id: I989c7961336b4dacbc83e97d1c098f93f56ef6ab
2019-01-17 17:55:10 +01:00
Harald Welte 292aa1c872 add osmo-asf4-dfu master and gerrit jobs
Change-Id: I5bc5ca30809af8d8b472a75a9e02c4964761de38
2019-01-13 13:43:09 +01:00
Kevin Redon a6db661afb add maintainer email to simtrace project
Change-Id: Id0a6740749ea1e3d1c06396485744426b0a74f79
2019-01-12 10:25:19 +00:00
Harald Welte 326221b87c ansible: Install 'sdcc' compiler for libusrp firmware on build slaves
Change-Id: I2fa8fa1a195079a0d8beda6a0aa4fc090ca37970
2019-01-11 18:54:12 +01:00
Holger Hans Peter Freyther c184523221 gsm_tester: Add lua-socket dependency used by the scripts
Change-Id: Iea51362a592c5b42413cb6d72677d01a0e357555
2018-12-28 14:54:19 +00:00
Holger Hans Peter Freyther 7af1fef220 gsm-tester: Install liblua5.3-dev as mobile dependency
Change-Id: I188361cceacde60b9b0b954549d63a34ba81646a
2018-12-28 13:48:24 +00:00
Holger Hans Peter Freyther 245ddd1cec Reserve an ip address for the osmo-bts-virtual
Change-Id: Ia3b52d6fc66efc032b7de4769ad603da177e13da
2018-12-13 09:40:03 +00:00
Oliver Smith fd9ae9737d master-builds: openbsc: make SSH keys available
Mount ~/.ssh in openbsc's docker container, just like it is done with
all other Osmocom projects that build in docker (see Change-Id
Icfff3b6a620de79d1bab7ff6a35309f828581c09 ("master-builds: make SSH keys
available in Docker")).

Related: OS#3385
Change-Id: Ib31ab4033c1cc71c10a96445e8bcc3b9f02e2a1f
2018-12-12 17:03:33 +01:00
Oliver Smith 6d6938fa60 Build verification: manuals for the rest
Build manuals for osmocom-bb and openbsc in gerrit-verifications.yml.
Build and publish manuals for osmocom-bb, openbsc and osmo-gsm-tester
in master-builds.yml.

Following recommendations on the ML, building manuals for
osmo-gsm-tester was not done with a new job in gerrit-verifications.yml.
It will be done with the existing osmo-gsm-tester_gerrit job (which is
generated by osmo-gsm-tester-runner.yml). No changes are necessary for
that in osmo-ci.git, see I4ebfe12a164f807b63bc897aff44db83fc0705bd
("contrib/jenkins*: build and publish manuals") in osmo-gsm-tester.git.

Related: OS#3385
Change-Id: I2845d14b656f11e3a07107b4f16e8d7a0e3c7bbf
2018-12-12 14:02:33 +01:00
Oliver Smith 298747c735 jobs/ttcn3-testsuites.yml: add build-discarder
Add the same build discarder as we have in master-builds.yml. This
should help with the disk space problems.

Change-Id: Ib7f01751ca5f6fb3e29a436982e2891d9cab0488
2018-12-05 16:50:58 +01:00
Oliver Smith d77968c3d6 gerrit-verifications/master-builds: fix errors
Fix three errors that prevented Jenkins from accepting the new
configs from Ie0643b17a6be6e86e281b50d78dd7a6858cf99ee
("gerrit-verifications/master-builds: WITH_MANUALS"):

* master-builds: fourth axis needs to be in the template or else the
  server will throw a 500 error. gerrit-verifications already has four
  template axes.
* gerrit-verifications/osmo-pcu: || at the end of combination_filter
* gerrit-verifications/osmo-sip-connector: missing ":"

Change-Id: I4ad63a3167e59b0837ffd1576fb7c7cf2958ff32
2018-12-05 14:49:18 +01:00
Oliver Smith e5e3773d52 gerrit-verifications/master-builds: WITH_MANUALS
Add WITH_MANUALS=1 to generate (and publish in case of master-builds)
PDF manuals. Jobs with multiple axes generate the manuals only in the
fastest axis.

Related: OS#3385
Change-Id: Ie0643b17a6be6e86e281b50d78dd7a6858cf99ee
2018-12-05 13:41:52 +01:00
Oliver Smith 6599da6d31 master-builds: use PUBLISH=1 instead of --publish
Pass all inputs to contrib/jenkins.sh as environment variables instead
of mixing with commandline arguments. We decided on this method instead
of I3cc01dbc57213b6d6cdc9b394a516510d045f1ae ("use jenkins.sh
--publish"), to make the contrib/jenkins.sh scripts less confusing.

Related: OS#3385
Change-Id: Ifc88af086368ce8eea1ea3a375ba7d5603a5d7e5
2018-12-04 10:39:33 +01:00
Oliver Smith bef4e9cb5d master-builds/gerrit: remove redundant GTP="$GTP"
Variables for axes are already exported as environment variables.
Change 'GTP="$GTP" ./contrib/jenkins.sh' to './contrib/jenkins.sh' to
avoid confusion when introducing other variables.

Change-Id: I9238da3dd3c4461f82dc7c792f886177399be3dc
2018-12-04 09:21:50 +00:00
Oliver Smith ce105c6aa1 osmo-mgw: don't try to build with transcoding
Do not build osmo-mgw twice anymore (once with "--enable-mgcp-transcoding" and
once with "--disable-mgcp-transcoding"). configure.ac does not know
these options.

Change-Id: I080cd331caec31c53a43090e6c19d23e31abdec6
2018-12-03 16:45:41 +01:00
Oliver Smith 6f8339b5d0 master-builds: make SSH keys available in Docker
At runtime, mount the Jenkins host's ~/.ssh folder inside the Docker
containers, from which manual PDFs will be published.

To my surprise, local testing showed that ssh is looking for the keys
in /home/build/.ssh, although the HOME variable inside the image is set
to /build, not /home/build.

Related: OS#3385
Change-Id: Icfff3b6a620de79d1bab7ff6a35309f828581c09
2018-12-03 13:39:02 +00:00
Oliver Smith 03d5f4c71c cosmetic: gerrit-verifications: format docker cmd
Format the docker command lines just like they are formatted in
master-builds.yml. Most notably, this puts every environment variable on
its own line. This makes it much easier to compare both files side by
side.

Change-Id: I0adcb028be24c2e05189c15b85811777d6287387
2018-11-30 15:15:16 +01:00
Oliver Smith ffb3fa6286 master-builds: use jenkins.sh --publish
Add --publish to the contrib/jenkins.sh calls for each Osmocom project,
where PDF manuals should be published. As this parameter is only passed
to jenkins.sh from the master-builds job, not from the
gerrit-verification job, we can add logic to upload the generated PDFs
when getting triggered because of changes in master.

Related: OS#3385
Change-Id: I3cc01dbc57213b6d6cdc9b394a516510d045f1ae
2018-11-29 16:55:59 +01:00
Oliver Smith 0b978d5afe Dockerfile: add dependencies for osmo-gsm-manuals
Install all dependencies required for building PDF manuals inside the
Docker container, so we can build projects with their manuals in the
Gerrit build verification job.

I ran the Docker container locally to confirm that this works, as
described here:
https://osmocom.org/projects/osmocom-servers/wiki/Jenkins_build_verification_jobs

Related: OS#3385
Change-Id: I379ce02857353c097f79cf91af2c4c9dc14754ec
2018-11-29 14:23:07 +00:00