Only support for SMpSDU mode is introduced in this commit.
Not supported explicit list:
- Transparent mode
- ATM/AAL2 based Transport layer
- GTP-U based Transport Layer
- Iu Rate Control procedure
- Time Alignment procedure
APIs are provided to allocate the primitives properly inside the related
msgb. This way primitives can be placed in the headroom, leaving the
data part of the msgb for the IuUP payload, hence allowing re-use of the
msgb and 0 copy of IuUP payload when forwarding data over RNL<->TNL.
Since RNL and TNL primitives relu struct osmo_prim_header, which is not
packed, they cannot be set to packed, and hence proper memory alignment
in the msgb must be done to avoid misaligned accesses (Asan errors about
it otherwise).
Related: SYS#5516
Change-Id: Ibe356fa7b1abaca0091e368db8478e79c09c6cb0
It's the usual naming for unit test binaries. Without the '_test' endig,
the tdef_vty_test_{config_root,config_subnode,dynamic} binaries do not
match the 'tests/*/*_test' pattern and appear as untracked files in git.
Change-Id: I828fa45132e11a41c527d4b25df850c19871cb75
Intead of attempting to store all distinct values of a reporting period,
just store min, max, last as well as a sum and N of each reporting
period.
This gets rid of error messages like
DLSTATS ERROR stat_item.c:285 num_bts:oml_connected: 44 stats values skipped
while at the same time more accurately reporting the max value for each
reporting period. (So far stats_item only reports the max value; keep
that part unchanged, as shown in stats_test.c.)
With the other so far unused values (min, sum), we are ready to also
report the minimum value as well as an average value per reporting
period in the future, if/when our stats reporter allows for it.
Store the complete record of the previous reporting period. So far we
only compare the 'max' value, but like this we are ready to also see
changes in min, last and average value between reporting periods.
This patch breaks API by removing:
- struct members osmo_stats_item.stats_next_id, .last_offs and .values[]
- struct osmo_stats_item_value
- osmo_stat_item_get_next()
- osmo_stat_item_discard()
- osmo_stat_item_discard_all()
and by making struct osmo_stats_item opaque.
In libosmocore, we do have a policy of never breaking API. But since the
above should never be accessed by users of the osmo_stats_item API -- or
if they are, would no longer yield useful results, we decided to make an
exception in this case. The alternative would be to introduce a new
osmo_stats_item2 API and maintaining an unused legacy osmo_stats_item
forever, but we decided that the effort is not worth it. There are no
known users of the removed items.
Related: SYS#5542
Change-Id: I137992a5479fc39bbceb6c6c2af9c227bd33b39b
Move test output from stdout to stderr and enable logging to stderr.
This is in preparation for the next patch, which will add a new log
message when osmo_stat_item_get_next() skips a value.
Related: SYS#4877
Change-Id: Ie0eaec2f93ac6859397a6bfca45039fdcc27cb9e
BSSGP RIM uses a number of nested containers to signal RIM application
specific payload information in a generic way. Lets add the container
structurs required for NACC.
Depends: libosmocore If48f412c32e8e5a3e604a78d12b74787a4786374
Change-Id: Ibbc7fd67658e3040c12abb5706fe9d1f31894352
Related: SYS#5103
This adds an inter-thread queue "it_q" to libosmocore. With it_q,
one can perform thread-safe enqueing of messages to another thread,
who will receive the related messages triggered via an eventfd
handled in the usual libosmocore select loop abstraction.
Change-Id: Ie7d0c5fec715a2a577fae014b0b8a0e9c38418ef
A dummy client to do integration tests of the ns2 layer.
It drop all unit data. But allows vty tests.
Change-Id: I127c178426bc1a3da8de251740eda93853030d6d
The RIM Routing Information IE (see also 3GPP TS 48.018, section
11.3.70) is used to control the flow of BSSGP rim messages at the SGSN.
Change-Id: I6f88a9aeeb50a612d32e9efd23040c9740bc4f11
Related: SYS#5103
BSSLAP: there are APDUs transferred in BSSMAP-LE Connection Oriented
Information messages on Lb between BSC and SMLC.
Add BSSLAP coding for these APDU messages:
- TA Layer3
- TA Request
- TA Response, possibly containing Location Estimate coded in GAD
- Reject
- Reset (for intra-BSS handover during TA Request)
- Abort (for inter-BSS handover)
Add encoding and decoding tests.
Change-Id: I6409c4bcac402dc7626a3afce9081c59cd715fe8
GAD, Universal Geographical Area Description:
- raw coding for all GAD elements.
- SI-units encoding and decoding for Ellipsoid point with uncertainty circle,
which I presume is the typical "at most N meters away from cell tower located
at X,Y", which corresponds to the TA positioning currently being implemented.
- other SI-units GAD element encodings are so far not implemented.
Add encoding and decoding tests.
In gsm/protocol/gsm_23_032.h are the raw coding structs as defined in 3GPP TS
23.032.
In gsm/gad.h are structs carrying consistent units based on meters and degrees,
for convenient / less error prone handling of GAD data, and for human readable
representations of the GAD data.
The separation of the two is desirable because OsmoBSC will receive GAD data
from OsmoSMLC on the Lb interface, and pass on this data to the MSC via the A
interface. It is better to pass the GAD data as-is without de/encoding.
Change-Id: I7a9dd805a91b1ebb6353bde0cd169218acbf223c
The autogenerated bitXXgen.h headers for osmo_load16le_ext() thru
osmo_store64_be() are not actually tested at all. Add a test.
The test output shows that the osmo_load*be_ext for a shorter len do not return
nicely matching results. A practical example showing the difficulty in storing
and loading 24bit integer values as/from big-endian:
uint8_t buf[4];
memset(buf, 0, sizeof(buf));
osmo_store32be_ext(0x00112233, buf, 3); // stores 11 22 33
printf("%s\n", osmo_hexdump(buf, 4));
uint32_t r = osmo_load32be_ext(buf, 3); // returns 0x11223300, not 0x00112233
printf("0x%x\n", r);
output is:
11 22 33 00
0x11223300
In contrast, the little-endian variant properly aligns the loaded bytes on the
least significant octet:
uint8_t buf[4];
memset(buf, 0, sizeof(buf));
osmo_store32le_ext(0x00112233, buf, 3); // stores 33 22 11
printf("%s\n", osmo_hexdump(buf, 4));
uint32_t r = osmo_load32le_ext(buf, 3); // returns 0x00112233 as expected
printf("0x%x\n", r);
output for le is:
33 22 11 00
0x112233
Change-Id: I5542ace54376a206aa8574812d4c742c86c293b4
Some systmes (like the ones available in OBS) don't support creating
SCTP sockets, so we need to skip those tests there.
Change-Id: I1d16280674625877ec22cc60cbc5deb67868a656
These utilities will be used by osmo-bsc to determine the Network Resource
Indicator seen in the TMSI, and (potentially) by osmo-msc to compose a TMSI
with a specific NRI, for osmo-bsc's load balancing between several MSCs.
Add utility functions to:
- extract an NRI value from a TMSI.
- overwrite the NRI value in a TMSI.
- limit an NRI in a (random) TMSI to a given list of ranges.
- add NRI value ranges to a list.
- remove them from a list.
- match NRI value (range) to a list.
- parse NRI values from string, for VTY.
- common VTY functionality of adding/removing NRI values from argv.
Add C tests for the above.
Why we need public API for NRI ranges: In osmo-bsc alone, we need the same NRI
API twice, 1: to manage/list NRI value ranges per-MSC, and 2: to manage/list
NULL-NRI values. If we also consider (potentially) adding NRI support to
osmo-msc, we need the same API twice again there. Hence it is useful to define
re-used API up here in libosmocore.
Related: OS#3682
Change-Id: Icb57a2dd9323c7ea11b34003eccc7e68a0247bf5
This implements a multiplexer and de-multiplexer for the ITU-T I.460
standard. The latter covers the transmission of sub-slots of 32/16/8k
inside 64k timeslots.
Change-Id: Id522f06e73b77332b437b7a27e4966872da70eda
gsm0503_coding contains AMR decoder functions for HR and FR. Those can
only decode AMR payload frames but not amr DTX frames. Lets add
functionality to detect DTX frames. Also lets add decoding for SID_UPDATE
frames as well as error checking for the SID frame recognition patterns.
Related: OS#2978
Change-Id: I2bbdb39ea20461ca08b2e6f1a33532cb55cd5195
In some situations, we want to execute an external shell command
in a non-blocking way. Similar to 'system', but without waiting for
the child to complete. We also want to close all file descriptors
ahead of the exec() and filter + modify the environment.
Change-Id: Ib24ac8a083db32e55402ce496a5eabd8749cc888
Related: OS#4332
Make build and external tests work with python3, so we can drop
the python2 dependency.
This should be merged shortly after osmo-python-tests was migrated to
python3, and the jenkins build slaves were (automatically) updated to
have the new osmo-python-tests installed.
Related: OS#2819
Depends: osmo-python-tests I3ffc3519bf6c22536a49dad7a966188ddad351a7
Change-Id: I84ef43f700e125c7a65f92347f12844e07e65655
We shall not prevent programs from starting if their configuration
files contain deprecated 'logging level ...' commands. Just print
a warning and return CMD_SUCCESS instead of CMD_WARNING.
While writing a unit test, another funny bug has been uncovered.
Parsing of a deprecated command indeed triggers a deprecation
warning, originated from libosmovty's log_deprecated_func().
This function simply calls vty_out(), but...
Since the invocation of the vty_out() happens _before_ the VTY
is initialized, the process is actually writing that warning
to its own stdin! Most likely, because we use talloc_zero()
to allocate a new instance of struct 'vty'.
As a side effect, the evil warning magically appears in the output
of 'make check', breaking the test statistics. Let's work around
this bug for now by redirecting stdin to /dev/null.
Change-Id: Ia934581410cd41594791d4e14ee74c16abe1009a
Fixes: Ic9c1b566ec4a459f03e6319cf369691903cf9d00
The calculation of the beginning of a block for TCH/F, TCH/H and FACCH
can be challenging since those channels are affected by the diagonal
interleaving of the TCH channels. However, GSM 05.02 Section 7 Table 1
of 5 specifies how the blocks are distributed over the TDMA frame
interval. Lets add a mapping function that is based on that table
Related: OS#3803
Change-Id: I3d71c66f8c401f5afbad9b1c86c24580dab9e0ce
log_enable_multithread() enables use of locks inside the
implementation. Lock use is disabled by default, this way only
multi-thread processes need to enable it and suffer related
complexity/performance penalties.
Locks are required around osmo_log_target_list and items inside it,
since targets can be used, modified and deleted by different threads
concurrently (for instance, user writing "logging disable" in VTY while
another thread is willing to write into that target).
Multithread apps and libraries aiming at being used in multithread apps
should update their code to use the locks introduced here when
containing code iterating over osmo_log_target_list explictly or
implicitly by obtaining a log_target (eg. osmo_log_vty2tgt()).
Related: OS#4088
Change-Id: Id7711893b34263baacac6caf4d489467053131bb
Rather than having applications maintain their own talloc cotexts,
let's offer some root talloc contexts in libosmocore. Let's also
make them per thread right from the beginning. This will help
some multi-threaded applications to use talloc in a thread-safe
way.
Change-Id: Iae39cd57274bf6753ecaf186f229e582b42662e3
Add global flag osmo_fsm_term_safely() -- if set to true, enable the following
behavior:
Detect osmo_fsm_inst_term() occuring within osmo_fsm_inst_term():
- collect deallocations until the outermost osmo_fsm_inst_term() is done.
- call osmo_fsm_inst_free() *after* dispatching the parent event.
If a struct osmo_fsm_inst enters osmo_fsm_inst_term() while another is already
within osmo_fsm_inst_term(), do not directly deallocate it, but talloc-reparent
it to a separate talloc context, to be deallocated with the outermost FSM inst.
The effect is that all osmo_fsm_inst freed within an osmo_fsm_inst_term()
cascade will stay allocated until all osmo_fsm_inst_term() are complete and all
of them will be deallocated at the same time.
Mark the deferred deallocation state as __thread in an attempt to make cascaded
deallocation handling threadsafe. Keep the enable/disable flag separate, so
that it is global and not per-thread.
The feature is showcased by fsm_dealloc_test.c: with this feature, all of those
wild deallocation scenarios succeed.
Make fsm_dealloc_test a normal regression test in testsuite.at.
Rationale:
It is difficult to gracefully handle deallocations of groups of FSM instances
that reference each other. As soon as one child dispatching a cleanup event
causes its parent to deallocate before fsm.c was ready for it, deallocation
will hit a use-after-free. Before this patch, by using parent_term events and
distinct "terminating" FSM states, parent/child FSMs can be taught to wait for
all children to deallocate before deallocating the parent. But as soon as a
non-child / non-parent FSM instance is involved, or actually any other
cleanup() action that triggers parent FSMs or parent talloc contexts to become
unused, it is near impossible to think of all possible deallocation events
ricocheting, and to avoid running into freeing FSM instances that were still in
the middle of osmo_fsm_inst_term(), or FSM instances to enter
osmo_fsm_inst_term() more than once. This patch makes deallocation of "all
possible" setups of complex cross referencing FSM instances easy to handle
correctly, without running into use-after-free or double free situations, and,
notably, without changing calling code.
Change-Id: I8eda67540a1cd444491beb7856b9fcd0a3143b18
Despite efforts to properly handle "GONE" events and entering a ST_DESTROYING
only once, so far this test runs straight into a heap use-after-free. With
current fsm.c, it is hard to resolve the situation with the objects named
"other" also causing deallocations besides the FSM instance parent/child
relations.
For illustration, add an "expected" test output file fsm_dealloc_test.err,
making this pass will follow in a subsequent patch.
Change-Id: If801907c541bca9f524c9e5fd22ac280ca16979a
Provide a common implementation of use counting that supports naming each user
as well as counting more than just one use per user, depending on the rules the
caller implies.
In osmo-msc, we were originally using a simple int counter to see whether a
connection is still in use or should be discarded. For clarity, we later added
names to each user in the form of a bitmask of flags, to figure out exactly
which users are still active: for logging and to debug double get / double put
bugs. This however is still not adequate, since there may be more than one CM
Service Request pending. Also, it is a specialized implementation that is not
re-usable.
With this generalized implementation, we can:
- fix the problem of inadequate counting of multiple concurrent CM Service
Requests (more than one use count per user category),
- directly use arbitrary names for uses like __func__ or "foo" (no need to
define enums and value_string[]s),
- re-use the same code for e.g. vlr_subscr and get fairly detailed VLR
susbscriber usage logging for free.
Change-Id: Ife31e6798b4e728a23913179e346552a7dd338c0
For handling RTP IP addresses and ports, osmo-mgw, osmo-bsc and osmo-msc
so far have their own separate shims and code duplication around
inet_ntoa(), htons(), sockaddr conversions etc. Unify and standardize
with this common API.
In the MGW endpoint FSM that was introduced in osmo-bsc and which I
would like to re-use for osmo-msc (upcoming patch moving that to
osmo-mgw), it has turned out that using char* IP address and uint16_t
port number types are a convenient common denominator for logging,
MGCP message composition and GSM48. Ongoing osmo-msc work also uses this
for MNCC.
This is of course potentially useful for any other IP+port combinations
besides RTP stream handling.
Needless to say that most current implementations will probably stay
with their current own conversion code for a long time; for current
osmo-{bsc,msc,mgw} work (MGW endpoint FSM) though, I would like to move
to this API here.
Change-Id: Id617265337f09dfb6ddfe111ef5e578cd3dc9f63
This ensures that the rpath of the generated binaries is set to
use only the just-compiled libosmo{core,gsm,vty}.so and not any
system-wide installed libraries while avoiding the ugly shell script
wrapper.
Change-Id: I9b9ae0ed277ba71519661a66a70b7f86971e4511
Run INT_MAX and ULONG_MAX related tests only manually, remove from automatic
testing. This will hopefully fix recent build failures on various platforms.
Add a 64 bit output example for expected results when invoking
`./tdef_test range'. This is not checked automatically and merely serves for
manual reference.
For vty tests, use 32bit max values instead of INT_MAX and ULONG_MAX.
Change-Id: I6242243bde1d7ddebb858512a1f0b07f4ec3e5c2
Move T_def from osmo-bsc to libosmocore as osmo_tdef. Adjust naming to be more
consistent. Upgrade to first class API:
- add timer grouping
- add generic vty support
- add mising API doc
- add C test
- add VTY transcript tests, also as examples for using the API
From osmo_fsm_inst_state_chg() API doc, cross reference to osmo_tdef API.
The root reason for moving to libosmocore is that I want to use the
mgw_endpoint_fsm in osmo-msc for inter-MSC handover, and hence want to move the
FSM to libosmo-mgcp-client. This FSM uses the T_def from osmo-bsc. Though the
mgw_endpoint_fsm's use of T_def is minimal, I intend to use the osmo_tdef API
in osmo-msc (and probably elsewhere) as well. libosmocore is the most sensible
place for this.
osmo_tdef provides:
- a list of Tnnnn (GSM) timers with description, unit and default value.
- vty UI to allow users to configure non-default timeouts.
- API to tie T timers to osmo_fsm states and set them on state transitions.
- a few standard units (minute, second, millisecond) as well as a custom unit
(which relies on the timer's human readable description to indicate the
meaning of the value).
- conversion for standard units: for example, some GSM timers are defined in
minutes, while our FSM definitions need timeouts in seconds. Conversion is
for convenience only and can be easily avoided via the custom unit.
By keeping separate osmo_tdef arrays, several groups of timers can be kept
separately. The VTY tests in tests/tdef/ showcase different schemes:
- tests/vty/tdef_vty_test_config_root.c:
Keep several timer definitions in separately named groups: showcase the
osmo_tdef_vty_groups*() API. Each timer group exists exactly once.
- tests/vty/tdef_vty_test_config_subnode.c:
Keep a single list of timers without separate grouping.
Put this list on a specific subnode below the CONFIG_NODE.
There could be several separate subnodes with timers like this, i.e.
continuing from this example, sets timers could be separated by placing
timers in specific config subnodes instead of using the global group name.
- tests/vty/tdef_vty_test_dynamic.c:
Dynamically allocate timer definitions per each new created object.
Thus there can be an arbitrary number of independent timer definitions, one
per allocated object.
T_def was introduced during the recent osmo-bsc refactoring for inter-BSC
handover, and has proven useful:
- without osmo_tdef, each invocation of osmo_fsm_inst_state_chg() needs to be
programmed with the right timeout value, for all code paths that invoke this
state change. It is a likely source of errors to get one of them wrong. By
defining a T timer exactly for an FSM state, the caller can merely invoke the
state change and trust on the original state definition to apply the correct
timeout.
- it is helpful to have a standardized config file UI to provide user
configurable timeouts, instead of inventing new VTY commands for each
separate application of T timer numbers.
Change-Id: Ibd6b1ed7f1bd6e1f2e0fde53352055a4468f23e5
I want to tweak general VTY features and need to cover with a transcript test
to show the differences. Start by showing the current situation of optional
and multi-choice arguments.
Change-Id: I5a79c83fabd02aba6406b6e0d620969c4bd0cc1d
It was introduced and forgotten to add to EXTRA_DIST in:
"logging vty: add VTY transcript test"
commit 3a9ff11e57
change-Id I948e832a33131f8eab98651d6010ceb0ccbc9a9c
Change-Id: I1bcedf3097f02b2adc679560d1cbceb27dbc345e
Add functions to encode and decode Global Call Reference as per
3GPP TS 29.205 Table B 2.1.9.1 add corresponding tests.
Change-Id: Iee95aa4e5c056645b6cb5667e4a067097d52dfbf
Related: OS#2487
I am setting out to refactor various details about logging. To show the effect,
I am first adding this new test to illustrate the exact effects on the various
osmo programs.
Add logging_vty_test.c as a standalone program that simply defines a few
logging categories and opens a telnet vty to play with.
Add logging_vty_test.vty, as an osmo_verify_transcript_vty.py test script.
Add --enable-external-tests to configure.ac, to enable running
logging_vty_test.vty during 'make check'.
Also allow running 'make vty-test' without the need to first configure with
--enable-external-tests (a flexibility I've missed many times over in the other
osmo source trees).
Add a Makefile.am stub for external CTRL tests, basically a copy-paste from
osmo-msc.git. I doubt that libosmocore will get python driven CTRL interface
testing any time soon, but if so we will know to not run it concurrently.
Change-Id: I948e832a33131f8eab98651d6010ceb0ccbc9a9c
Do not link against the system-wide installed libosmo* libs when building the
regression test programs. Always use the locally built ones.
Linking some libosmo libraries causes libtool to pull in other libosmo libs
even though they were not explicitly named. For example, ctrl_test explicitly
links libosmoctrl, but this also has dependencies to libosmovty and libosmogsm:
ldd src/ctrl/.libs/libosmoctrl.so | grep osmo
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007f26c26d4000)
libosmogsm.so.10 => /usr/local/lib/libosmogsm.so.10 (0x00007f26c22bb000)
libosmovty.so.4 => /usr/local/lib/libosmovty.so.4 (0x00007f26c2171000)
If we omit explicit LDADD of these dependencies in the Makefile.am, libtool
will take the first canonical place to find them, which may just be the already
installed older versions of the same libs, which may or may not be compatible
with the current build. In any case, it is never intended to link installed
libs.
All library dependencies are listed by this quick script:
cd libosmocore
for l in $(find . -name "*.so") ; do echo; echo "$l"; ldd $l | grep libosmo; done
./.libs/libosmocore.so
./coding/.libs/libosmocoding.so
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007f25fc3c2000)
libosmogsm.so.10 => /usr/local/lib/libosmogsm.so.10 (0x00007f25fbfa9000)
libosmocodec.so.0 => /usr/local/lib/libosmocodec.so.0 (0x00007f25fbf9b000)
./codec/.libs/libosmocodec.so
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007fb4c900d000)
./ctrl/.libs/libosmoctrl.so
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007f5df5129000)
libosmogsm.so.10 => /usr/local/lib/libosmogsm.so.10 (0x00007f5df4d10000)
libosmovty.so.4 => /usr/local/lib/libosmovty.so.4 (0x00007f5df4bc6000)
./gb/.libs/libosmogb.so
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007f788e536000)
libosmovty.so.4 => /usr/local/lib/libosmovty.so.4 (0x00007f788e3ec000)
libosmogsm.so.10 => /usr/local/lib/libosmogsm.so.10 (0x00007f788dfd3000)
./vty/.libs/libosmovty.so
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007f3b7ed21000)
./gsm/.libs/libosmogsm.so
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007fc69472e000)
./sim/.libs/libosmosim.so
libosmocore.so.11 => /usr/local/lib/libosmocore.so.11 (0x00007f2f6412d000)
libosmogsm.so.10 => /usr/local/lib/libosmogsm.so.10 (0x00007f2f63d14000)
Add all explicit linking of all required library dependencies in all regression
test programs, as shown by above listing.
Example for reproducing a problem:
In libosmocore.a, introduce a new function, and call that from libosmovty code.
For example, I made loglevel_strs non-static in logging.c, and used that in
logging_vty.c. Build and install this in a place where libtool can find it.
Then go back to before this change and rebuild. You will see that linking
ctrl_test (before this patch) then complains about libosmovty requiring the
loglevel_strs symbol which it cannot find in libosmocore.so.
Change-Id: Id084e6e6efd25cd62b1bd7a4fc7c5985c39130c6
As oap_client has moved from osmo-sgsn to libosmogsm, it is only fair
that the related unit test shall also be moved here.
Change-Id: I9d64e10b4bacac9b530cf077841bad762fc6d558
Recent OS#3407 shows that we should verify stderr to catch sanitizer failures.
(They might not always be ignorable like that one.)
Change-Id: Ic9e437a1cc96ae081e0fd6a9b6e3156987e14c0c
The conv_gen.py utility was tested against both Python 2 and 3,
so there is no need to enforce Python 2. Also, having:
#!/usr/local/bin/python{2|3}
is a bad idea, because Python may be installed in a different location.
Change-Id: I6007d481047b584db13d6eda70fb99f11f9ddaa1
Vadim Yanitskiy
Harald Welte
Neels Hofmeyr
Oliver Smith
Pau Espin Pedrol
Philipp Maier
Pau Espin