Commit Graph

303 Commits

Author SHA1 Message Date
Oliver Smith f4ecc8d670 net/templates/osmo-bts.cfg: modernize
Move logging section to the bottom, include common_logging and
common_template_warning.

Change-Id: I7b29c644b53e4f1907f33664d6cde6ec76cc8c04
2022-02-25 12:55:21 +01:00
Oliver Smith e76581931f net/templates: support running a second bsc
New config variables: BSC_COUNT, BSC1_*

Change-Id: If0fa5c5b4c76ae037d109211bbb27132689b4f80
2022-02-25 12:55:21 +01:00
Oliver Smith 97f58d4c2e net/templates: move bsc cfg/vars to bsc0
Prepare support for a second BSC, in order to test OsmoBSCNAT.

* Rename BSC_* variables to BSC0_*
* Rename MGW4BSC_* variables to MGW4BSC0_*
* Rename osmo-bsc.cfg to osmo-bsc-0.cfg
* Rename osmo-mgw-for-bsc.cfg to osmo-mgw-for-bsc-0.cfg

Change-Id: Iaba2e3143b7bb6451a0104042e69d47ee5f48d3c
2022-02-25 12:55:21 +01:00
Oliver Smith 19b27b3f98 net/templates: split off common osmo-bsc.cfg part
Prepare to add support running with two osmo-bsc's in order to test
osmo-bsc-nat.

Change-Id: Icb220657888b23ca19546cca78e10ae914f37751
2022-02-25 12:55:21 +01:00
Oliver Smith b2a83f7a17 net/README: update tmux documentation
Change-Id: I5561dae3898bf86176d7693c08732a68a1158a14
2022-02-25 12:55:21 +01:00
Oliver Smith b252b4a043 net: get missing config variables from config_2g3g
Don't ask the user to copy config_2g3g and manually keep it in sync with
config_2g3g with each change anymore. Instead, let the user config be a
diff on top of config_2g3g.

This is in preparation to adding new variables for use with OsmoBSCNAT.

Change-Id: I5870d39aee8be84c2024f8a2a71ee4a734a4fc7c
2022-02-25 12:55:16 +01:00
Oliver Smith 02f2254b23 net/fill_config.py: refactor print 'using config...'
Put each line in a separate, aligned print statement and use f-strings
(Python 3.6 feature, even debian oldstable has > 3.6 by now).
This is in preparation to add a new line about the original config in
the next patch.

Change-Id: Iad39a7889c107ceb8c16325bb545cb426eb9b6e2
2022-02-25 12:51:49 +01:00
Oliver Smith f34a9443eb net/templates/run.sh: drop 'fg'
This was related to attempting to fix the infinite "q Enter to close"
msgs bug, which has been properly fixed in
Iebb799493f76fd57f24b15c998ded2bd8e284e6b.

Remove fg to get rid of this error:
  zsh:fg:1: no job control in this shell.

Change-Id: Iba7a1adbc4f48b0477a53590ffec8af518fdf467
2022-02-23 12:31:42 +01:00
Oliver Smith 2cfd792312 net/templates/run.sh: put log name in trace.pcap
It's hard to distinguish multiple wireshark windows with trace.pcap
files, as only the file name and not the path is shown in the title.
Make it easier to distinguish them by adding the log name to the file
name.

Change-Id: I7007d7c1e74aef46f1942f4f6f675adebb890955
2022-02-21 11:36:48 +01:00
Oliver Smith e33f85f193 net: run.sh: fix infinite "q Enter to close" msgs
Fix the logic that is supposed to display a "q Enter to close" message
once and wait for user input, in case one of the Osmocom programs
crashed.

Explicitly read from /dev/tty, as it may not be connected as default
stdin anymore at this point. This happens when using the new code paths
with tmux as terminal, but also happened before recent refactorings for
me with all terminals.

Related: https://media.ccc.de/v/osmodevcall-20211210-osmith-osmo-dev-ttcn3#t=801
Change-Id: Iebb799493f76fd57f24b15c998ded2bd8e284e6b
2022-02-21 11:36:48 +01:00
Oliver Smith cd472bdbf7 net: run.sh: support tmux as terminal
Create one tmux session, with each Osmocom program in one window, if
tmux is chosen as "terminal". This allows using osmo-dev's run.sh over
ssh without X forwarding. Also the existing supported terminals
automatically scroll to the bottom as soon as there is new output,
making it hard to read logs.

Change-Id: Iaef08fb3cb3c6f58cd8490a631e177a03af9c974
2022-02-21 11:36:38 +01:00
Oliver Smith 6327371e4c net: run.sh: write launcher command to shell script
Instead of passing a whole script to the terminal via command-line
argument, write it to a file. This is in preparation to use the same
script with tmux, see next patch.

Change-Id: I15760d706b20628421c10a16514120f442d5477f
2022-02-21 10:45:25 +01:00
Oliver Smith 669f1dbe5c net: run.sh: store all spawned pids in files
Replace the previous approach of only storing the PIDs of the spawned
terminals, and then attempting to kill all other processes (the ones
running inside the spawned terminals and tcpdump) with "killall".
Instead, store PIDs of all spawned programs:
* Terminals
* Osmocom programs (new)
* Tcpdump (new)

Kill old processes at the start of run.sh as well, so even if run.sh was
killed before it could clean up, the Osmocom programs don't fail to
start anymore (as it was the case before).

Change-Id: I379ef71de2f1ab0ac4a28064e5cf9e275c4c61e9
2022-02-21 10:42:24 +01:00
Oliver Smith b28a9100ed net: require terminal to be configured
Prepare to add "tmux" as new terminal. As discussed in code review, we
decided to drop support for auto-detection.

Change-Id: I0afb6b0242c399334a2c37f18a5d26d5beeabedc
2022-02-18 18:58:37 +01:00
Oliver Smith 3ffaca828f net/templates/.editorconfig: new file
Change-Id: Ieeb67a10c7b7c27a5e0e9dbc8399fe2add9f34d4
2022-02-08 10:05:34 +00:00
Oliver Smith b6d58c0918 net/templates/osmo-bsc-nat.cfg: add addressbooks
Add MSC and BSC point-codes, as it is currently needed in the initial
message forwarding implementation.

Related: SYS#5560
Related: osmo-bsc-nat I4af398bb433341a98f818822e6c3af28b6d9dacd
Change-Id: Ia396cc677ceffa3fa86da5460f6c890b5f714059
2022-02-08 10:05:34 +00:00
Neels Hofmeyr fd10baa790 add osmo-upf, libgtpnl 2022-02-01 02:53:24 +01:00
Vadim Yanitskiy 3414a48340 Add tmux scripts for ttcn3-{msc,bsc,bts}-test
Change-Id: I201b77c013cbeef9d1a07b9422b27f375811ed69
2022-01-17 15:42:45 +00:00
Oliver Smith a3a1a0d94a net: add osmo-bsc-nat
Make STP_CN_IP and STP_RAN_IP configurable. If they are not set to the
same IP, run a second OsmoSTP and OsmoBSCNAT to connect both OsmoSTP.

Related: OS#2545
Change-Id: I8bfbd00bfeeebc6e1b87a8cd6333212db4380653
2022-01-17 09:11:55 +00:00
Oliver Smith 8fd5098f3f net/templates/run.sh: refactor kill logic
Put the terminal PIDs into a variable instead of running kill %1 %2 etc.
This makes it easier to conditionally spawn some of the programs.

Change-Id: Iaf6c175de2edf720009af13eace2b0ef83b0735b
2022-01-17 09:11:55 +00:00
Oliver Smith 5065107a60 all.deps, werror.opts: add osmo-bsc-nat
Change-Id: Ica90826c8bfbf4f295c49702d36e7c88e43f80cd
2022-01-17 09:11:55 +00:00
Oliver Smith 84df01adb8 gen_makefile.py: fix find errors
Don't print find errors if not all dirs were cloned. Found this also
while running osmo-dev on a fresh install.

Change-Id: I7955fedc96484e5703073b63906a03477452db6e
2021-12-10 17:22:58 +01:00
Oliver Smith aa015b1205 README: update deps and recommended opts
In preparation for the OsmoDevCall#19 talk, I'm installing this on a
fresh system and made some adjustments to README so it works without
complaining about missing dependencies.

Related: https://osmocom.org/news/155
Change-Id: I7522d8c9346e9823f3977b9190210eb5d5bcfe2d
2021-12-10 17:19:43 +01:00
Oliver Smith 6c7fed8a32 ttcn3.sh: use --auto-distclean
Change-Id: I7d61405dc989eed4935bc6de00e36c1f3b18a428
2021-11-12 15:51:38 +01:00
Oliver Smith adfa0c0cf1 gen_makefile.py: add --auto-distclean
Add a new option that avoids stopping the build with:
  configure: error: source directory already configured; run "make distclean" there first

I run into this frequently, when e.g. running './configure; make release'
inside Osmocom repositories and then attempting to build them with
osmo-dev. It is annoying as it does not fail immediatelly, and I need to
manually go to the source directories where it failed, run 'make
distclean' and then attempt to build again. Then it may fail on another
directory with the same problem.

Check for config.status, as that is also what ./configure checks for
before printing the error.

Change-Id: Iac71333e8c5da8dee4e8f34e2af8db12d2c08021
2021-11-12 15:51:38 +01:00
Oliver Smith f7f8c96e7b gen_makefile.py: proj_files: ignore config.h
Do not consider config.h inside the source tree as modified source file.
This file gets generated when running ./configure to build in-tree,
whereas osmo-dev always builds out-of-tree.

Having config.h inside proj_files causes problems with the next patch,
where conflicting in-tree builds will be automatically be cleaned with
"make distclean". For example:
* "make libosmocore" runs with a previously configured in-tree build
* proj_files contains config.h.
* Before attempting to build libosmocore out-of-tree, with the next
  patch, "make distclean" gets executed.
* config.h is now removed
* .make.libosmocore.build target expects all proj_files to exist and
   fails with:
   make: *** No rule to make target '../src/libosmocore/config.h', needed by '.make.libosmocore.build'.  Stop.

Change-Id: I10d970f2cadacb5071dcde975c76b15404038a3c
2021-11-12 15:51:38 +01:00
Oliver Smith dfd92fadab all.deps: remove duplicate osmo-pcu, osmo-trx
Fix warnings about duplicate make targets.

Change-Id: I7c6023f1c8bc23beff3bb512f059ce1b692d2adf
2021-11-12 15:51:27 +01:00
Neels Hofmeyr 03f42edb2c add osmo-bts.cfg 2021-10-23 20:45:12 +02:00
Neels Hofmeyr 8bb707b6cd add trx.opts to build osmo-bts-trx and osmo-trx with UHD 2021-10-23 20:45:12 +02:00
Neels Hofmeyr 5ce778062b werror.opts: add osmo-bts 2021-10-23 20:45:12 +02:00
Neels Hofmeyr 0c83295236 osmo-bsc.cfg: use 'type osmo-bts', not 'sysmobts' 2021-10-23 20:45:12 +02:00
Neels Hofmeyr 2c97908329 add osmo-trx 2021-10-23 20:45:12 +02:00
Neels Hofmeyr 2325e2e3a9 gen_makefile: add 'CFLAGS=-g' option --build-debug 2021-10-23 20:45:12 +02:00
Neels Hofmeyr c84b3e5862 add osmo-pcu to deps, sanitize.opts 2021-10-23 20:45:12 +02:00
Oliver Smith 505f60de3d gen_makefile.py: reformat find cmds in Makefile
Change-Id: Ic2fad22fdbf0fec4abfaeae60b4ebd74563a9392
2021-10-11 08:11:58 +02:00
Oliver Smith 04f805ebe8 all-tests.opts: add osmo-pcu
Change-Id: I6d9b8553482868a7fe4ee127dd9e722b748b55a9
2021-09-21 16:52:06 +02:00
Oliver Smith 82f278993e all.deps: add asn1c, osmo-gbproxy, osmo-pcap
Change-Id: Ic9f57d454d245631b24c9c8c14a8ea806587b33b
2021-09-21 16:52:06 +02:00
Oliver Smith 0a4d8ea698 gen_makefile.py: drop argument for deps file
Rename 3G+2G.deps to all.deps and remove the parameter from
gen_makefile.py. Create new Makefile targets for the various use cases,
usually we build either 'cn-bsc' or 'usrp'. The next patch will add more
projects to all.deps.

This change brings the Makefile logic closer to how it is actually used,
one build directory that is building most Osmocom projects. New projects
can be added to all.deps on demand, but then we have it in the
repository and not just a local change that we add temporarily to the
3G+2G.deps file when we need to build a specific project.

Extend all.deps with projects needed by ttcn3.sh, so it doesn't need to
write its custom .deps file anymore.

Change-Id: I6394882f67eecda3d2c03a97d3119657f7d3873f
2021-09-21 16:52:02 +02:00
Oliver Smith 5f611c76b3 ttcn3.sh: clone_repo: avoid make if possible
Change-Id: I54ad32c9fc51dc91c70b61fe91e68257985f310f
2021-08-31 17:44:30 +02:00
Oliver Smith 8d8ddf4124 ttcn3.sh: call osmo-dev's make less often
Merge build_osmo_program_osmodev into build_osmo_programs and change the
logic, so make only needs to run once. This saves a few seconds for each
test iteration.

Change-Id: Ie2cf7a93395f5ad04159f523d0d0f091303aaa57
2021-08-31 17:44:30 +02:00
Oliver Smith d3271069c4 ttcn3.sh: build with werror
Change-Id: I0b5f4b805d230a5c8157853b4b37b7fe62c4d539
2021-08-31 17:44:30 +02:00
Oliver Smith 7ece249e61 ttcn3.sh: run gen_makefile.py with --no-make-check
On my system, when building all components in docker, some of the C tests
don't pass (but they do pass on jenkins.osmocom.org, which runs a
similar setup). Do the easy workaround and just disable C tests, the
point of ttcn3.sh is to have fast ttcn-3 test iterations anyway, running
the C tests additionally slow each iteration down. One can run the C
tests before submitting changes, e.g. by building with osmo-dev without
ttcn3.sh.

Change-Id: I946635f1ec09f23428965f3db514027f64921fe4
2021-08-23 17:49:12 +02:00
Oliver Smith 855d66ee91 ttcn3.sh: don't build virtphy/trxcon/fake_trx
Remove leftovers from before refactoring in
Iecba5391dafc353058d369deb513b156534face1. No need to build these with
osmo-dev, as they are part of the osmocom-bb docker container that gets
built via ttcn3-bts-test/jenkins.sh.

Change-Id: I99ec4ff9caf34d1ddd7efde2f5ae2342a6984689
2021-08-13 09:17:40 +02:00
Oliver Smith 836bf568ef ttcn3/wrappers: remove
Leftover from before the script was refactored in
Iecba5391dafc353058d369deb513b156534face1.

Change-Id: Ic0890f71fe37a136580e12ebade979bcf14574c0
2021-08-13 09:17:40 +02:00
Oliver Smith 0fa204e179 ttcn3.sh: build everything inside docker
Replace the previous approach of building outside of docker and mounting
everything inside docker while running tests, with also building
everything inside docker. This prevents incompatibilities between host
system and docker (e.g. different glibc). As nice side-effect,
/usr/local is not filled up anymore.

Change-Id: Ib6db8ffd916c788c9de0b3d51c82e1d7bb3f6828
2021-08-13 09:17:40 +02:00
Oliver Smith c47eafbe0a gen_makefile.py: add --docker-cmd argument
To be used by ttcn3.sh to run the actual build inside docker. ttcn3.sh
runs the programs built by osmo-dev inside docker with the exact
configs that we use on jenkins.osmocom.org, but so far it builds the
component outside of docker. But building inside docker is necessary to
avoid incompatibilities between the host system and the docker
containers (e.g. different glibc).

Change-Id: Ibd20f0e0e2a5105085066a9f04b5868c68691024
2021-08-13 09:17:39 +02:00
Oliver Smith bfe59294ad ttcn3.sh: set LD_LIBRARY_PATH=/usr/local/lib
Fix for Osmocom programs using system's shared libraries, instead of the
ones mounted into /usr/local.

For example, for osmo-bsc the libosmocore installed in the
osmo-bsc-master docker container would be used (installed via apt from
OBS nightly) instead of the version that osmo-dev built from source.

Change-Id: I0dd7fb544ca59cb1df19fc3d73cfd07f3374e123
2021-08-13 09:17:39 +02:00
Oliver Smith f3eb0ba445 ttcn3.sh: support IMAGE_SUFFIX=latest
Add the image suffix to the marker in run_docker() which indicates if
the testsuite docker container was already built (with all its
dependencies), so the script doesn't assume that everything is already
built if using without IMAGE_SUFFIX first and then switching to
IMAGE_SUFFIX=latest.

Add an usage example to README for IMAGE_SUFFIX=latest.

Change-Id: I867740831810324a7f5906a0548f0d04ecb9a0a1
2021-08-13 09:17:39 +02:00
Oliver Smith f03dfa30a2 ttcn3.sh: get_program_repo: add osmo-pcap-*
Make "ttcn3.sh osmo-pcap-client" work.

Change-Id: I61f1b9f8ce9157f34cefad093b7ae7d445834961
2021-08-13 09:17:39 +02:00
Oliver Smith 6b84b46198 ttcn3.sh: set NO_LIST_OSMO_PACKAGES=1
Speed up tests by not running a separate docker instances first that
lists installed packages.

Depends: docker-playground Iaafb81e9ee94850ae3b6318574ad5370d74a2838
Change-Id: I3d9976d07f088e2d106d23259889a4bffe1abbcb
2021-07-12 08:35:49 +00:00