osmo-ci/jobs
Oliver Smith 4773f26d2d jobs/master,gerrit: fix pysim, sysmo-usim-tool
Don't attempt to create the CCACHE_DIR and chown it to osmocom-build for
the simtester jenkins node, and more generally all that were not
created with the ansible provisioning scripts. In case of simtester,
there is no osmocom-build user and we don't have the space there to use
ccache.

Fix for:
  chown: invalid user: ‘osmocom-build:osmocom-build’

Related: OS#5848
Change-Id: I339d9ba4ad0c959d7325820eb53bfa1f0e04c164
2023-01-25 12:59:30 +00:00
..
README.adoc jobs/master,gerrit: use ccache to speed up builds 2023-01-24 10:56:03 +00:00
coverity.yml jobs/coverity: remove display-name 2022-11-16 13:32:05 +01:00
gerrit-binpkgs.yml jobs: retire osmocom-gerrit-debian* labels 2022-12-06 11:17:16 +01:00
gerrit-lint.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
gerrit-verifications-comment.yml jobs/gerrit-verifications: post comment on start 2023-01-18 12:35:34 +00:00
gerrit-verifications-dahdi.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
gerrit-verifications.yml jobs/master,gerrit: fix pysim, sysmo-usim-tool 2023-01-25 12:59:30 +00:00
master-builds-dahdi.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
master-builds.yml jobs/master,gerrit: fix pysim, sysmo-usim-tool 2023-01-25 12:59:30 +00:00
octsim_osmo-ccid-firmware.yml update git URLs (git -> https; gitea/gerrit) 2022-08-13 04:02:51 +07:00
osmo-gsm-manuals-trigger.yml jobs: don't set node where it doesn't make sense 2022-11-30 17:46:57 +01:00
osmo-gsm-tester-builder.yml update git URLs (git -> https; gitea/gerrit) 2022-08-13 04:02:51 +07:00
osmo-gsm-tester-runner.yml jobs/ogt: disable _run-prod, _run-rnd, _ttcn3 2022-12-14 15:11:19 +01:00
osmo-gsm-tester-trigger.yml jobs: Add OGT open5gs build jobs 2021-03-15 19:13:07 +01:00
osmo-gsm-tester_run-gerrit.sh jobs: osmo-gsm-tester: Drop use of OSMO_GSM_TESTER_CONF for default conf dir 2020-05-07 19:16:15 +02:00
osmo-gsm-tester_run-prod.sh jobs: osmo-gsm-tester: Drop use of OSMO_GSM_TESTER_CONF for default conf dir 2020-05-07 19:16:15 +02:00
osmo-gsm-tester_run-rnd.sh jobs: osmo-gsm-tester: Drop use of OSMO_GSM_TESTER_CONF for default conf dir 2020-05-07 19:16:15 +02:00
osmo-gsm-tester_ttcn3.sh jobs: osmo-gsm-tester_ttcn3: Fix path to jenkins script 2020-06-09 14:10:46 +02:00
osmocom-api.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
osmocom-build-tags-against-master.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
osmocom-depcheck.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
osmocom-list-commits.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
osmocom-obs-check-builders.yml jobs: add osmocom-obs-check-builders 2022-10-26 08:35:05 +00:00
osmocom-obs.yml jobs/obs: don't upload to build.opensuse.org 2022-11-04 08:03:07 +00:00
osmocom-release-tarballs.yml jobs/osmocom-release-tarballs: use any node 2022-11-30 17:46:57 +01:00
registry-rebuild-upload-fpga-build.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
registry-rebuild-upload-titan.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
registry-triggers.yml jobs: don't set node where it doesn't make sense 2022-11-30 17:46:57 +01:00
registry-update-base-images.yml jobs: don't limit to debian9 2022-11-30 17:46:57 +01:00
repo-install-test.yml jobs/repo-install-test: adjust for qemu 2022-12-09 11:56:03 +01:00
ttcn3-testsuites-kernel-git.yml jobs/ttcn3: fixup for blocker remove in testsuites 2023-01-16 16:26:00 +01:00
ttcn3-testsuites.yml jobs/ttcn3: fixup for blocker remove in testsuites 2023-01-16 16:26:00 +01:00
update-osmo-ci-on-slaves.yml jobs/update-osmo-ci-on-slaves-dp: fix node 2022-12-23 11:01:32 +00:00
update-osmo-python-on-slaves.yml jobs/update-osmo-*: change mail recipients 2022-06-29 09:33:30 +00:00

README.adoc

These jenkins.osmocom.org job definitions, managed by
https://docs.openstack.org/infra/jenkins-job-builder/index.html[Jenkins Job Builder]

*Prepare:*

Install jenkins-job-builder:

  apt-get install jenkins-job-builder

Have a jenkins-job-builder.ini file. One of

    ~/.config/jenkins_jobs/jenkins_jobs.ini
    /etc/jenkins_jobs/jenkins_jobs.ini

or place one in here and pass it to jenkins-jobs using the --conf file.

Make sure the file not world readable to minimally safeguard your jenkins password.
Instead of using your jenkins password, use an *API Token*. To retrieve your token go
to Jenkins via a Webbrowser, click on your Username in the right corner, click on configure,
click on *Show API Toke...*.

jenkins_jobs.ini:

    [jenkins]
    user=my_user_name
    password=my_api_token
    url=https://jenkins.osmocom.org/jenkins

and

    chmod go-rwx jenkins_jobs.ini

*Update a single job on jenkins.osmocom.org:*

    jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml gerrit-osmo-msc

NOTE: when you supply a name not defined in that yml file, you will not get an
error message, just nothing will happen.

*Update all jobs of one file:*

    jenkins-jobs --conf jenkins_jobs.ini update gerrit-verifications.yml

*Update all jobs in all files:*

    jenkins-jobs --conf jenkins_jobs.ini update ./

*Troubleshooting:*

- 'jenkins.JenkinsException: create[gerrit-osmo-msc] failed'

jenkins.osmocom.org is not reachable, or URL in the config file is erratic.
Make sure it is exactly

    url=https://jenkins.osmocom.org/jenkins

- newlines:

Use 'key: |' to keep new lines in multiline values, e.g.:
  - shell: |
      echo hello
      echo world

See also:
* https://yaml-multiline.info/
* https://stackoverflow.com/a/21699210

- jobs named on cmdline are not updated:

Make sure the job name is correct, or just issue an entire yml file without
individual job names.

Also be aware that jobs are only actually updated when anything changed.

*Jenkins labels*

Most jenkins jobs should run a docker container and install all required
dependencies inside that, so we don't need to install them on the jenkins node.
These jobs don't need to set a label, they can just run on any generic jenkins
node that has docker available. So if you add a new job, you probably don't
need a label at all.

Existing jobs typically have a label set by the topic they belong to, e.g.:
- osmocom-master
- osmocom-gerrit
- ttcn3

Other labels indicate specific software/hardware works here, e.g.:
- coverity
- hdlc
- osmo-gsm-tester
- podman

*ccache*

The jobs from master-builds and gerrit-verifications use ccache. View the
statistics with SSH on the build nodes with:

 $ CCACHE_DIR=~/ccache/gerrit-verifications ccache -s
 $ CCACHE_DIR=~/ccache/master-builds ccache -s

Note that running multiple jobs in parallel influence the ccache statistics,
and it's impossible to tell which job caused which change in the stats (that's
why they are not printed at the end of each job, it would be confusing).