Let's flag the API as deprecated so that people start using
log_set_print_filename2() API instead, which has less ackward
behavior implications like changing the print status of category-hex.
Related: OS#5034
Change-Id: If9b6b322989536a12094e6105c3aabc84d8be24a
This API wraps conventional gettid() linux-specific API, which even in
Linux itself is sometimes not properly supported/announced.
This API also allows future porting to other platforms if needed, and so
far falls back to getpid() if no gettid(9 can be found.
Code ported from osmo-trx.git, see commit 7a07de1efd4eb7cc11c33d3ad25cb2df70aa1ef1.
Related: OS#5027
Change-Id: Id7534beeb22fcd50813dab76dd68818e2ff87ec2
This adds an inter-thread queue "it_q" to libosmocore. With it_q,
one can perform thread-safe enqueing of messages to another thread,
who will receive the related messages triggered via an eventfd
handled in the usual libosmocore select loop abstraction.
Change-Id: Ie7d0c5fec715a2a577fae014b0b8a0e9c38418ef
In 'struct gsmtap_hdr' field 'snr_db' is defined as a signed integer,
however all functions that fill this structure accept an unsigned
integer. This is wrong, because SNR can be negative.
Let's use 'int8_t' instead of 'uint8_t'. Changing from unsigned
to signed should be relatively safe compared to the opposite.
Most of the callers I am aware of always do pass 0 anyway.
Change-Id: I9f432be5c346d563bf518111c14ff04d4a63f592
Related: SYS#5073
This function is called automatically on the main thread, but needs to
ba called explicitly in order to run the select loop on another thread.
Make it available for applications through talloc.h
Change-Id: Ie710ca9ad01d3fadb9f4ff344a55d6c01004727b
This is a helper function to broadcast an event to all of the
siblings of a specified FSM instance.
Change-Id: I2ce398741a8672d7b7c4058d056f46e2fe7353c1
Historically, BSSGP uses a non-constant, user-configurable integer
varieable for the logging sub-system. Let's replace this with a
statically-allocated library logging constant.
This is required if we want to use the subsystem number in e.g.
static initialized for osmo_fsm.log_subsys.
Change-Id: I506190aae9217c0956e4b5764d1a0c0772268e93
When importing the hashtable code in I8ef73a62fe9846ce45058eb21cf999dd3eed5741
I didn't import actual implementations of the fls() and fls64()
implementations, as at least gcc-10 was smart enough to detect
we only use it on constant types and hence the computation can happen
at build time via const_ilog2()
However, in our jenkins build verification' this doesn't appear to
happen, as we get below errors:
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h: In function ‘__ilog2_u32’:
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h:20:9: error: implicit declaration of function ‘fls’ [-Werror=implicit-function-declaration]
return fls(n) - 1;
^~~
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h: In function ‘__ilog2_u64’:
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h:28:9: error: implicit declaration of function ‘fls64’ [-Werror=implicit-function-declaration]
return fls64(n) - 1;
^~~~~
Let's provide some generic implementations for this case. If needed
one could also introduce architecture-specific assembly implementations
like in the Linux kernel, but so far we managed to keep libosmocore free
of any assembly tweaks.
Change-Id: Ifa4898eb66c8d949618edd47961b7a0330ed35b5
'new' is a reserved keyword in C++, so including this header from
a C++ project (like osmo-pcu) breaks compilation. Let's rename
it in the same way as it's already done in this file: add '_'.
Change-Id: I7f7d9143edca75ce932601386a8766b0a62c0e24
Fixes: I8ef73a62fe9846ce45058eb21cf999dd3eed5741
For more than a decade we've used the linuxlist.h for double-linked
lists. Let's also add the hlist (double-linked lists with single
pointer sized head, and the hashtable that builds on top of it.
This reflects the versions included in Linux 5.8 with some modifications
to make them build in userspace (remove RCU versions, adjust for
userspace include files and types, convert to doxygen).
Change-Id: I8ef73a62fe9846ce45058eb21cf999dd3eed5741
NSVC filtering was only implemented on sending messages, this also adds
log_set_context() calls to ns2_recv_vc()
Filtering by NSE is implemented similar to NSVC.
Change-Id: I63c0e85f82f5d08c5a6f535da94b8648498439d2
Related: SYS#5232
This adds an easy way to listen to netlink events form the Linux kernel
from within libosmocore applications.
The new dependency can be disabled via the "--disable-lbimnl" configure flag.
Change-Id: I4f787ee68f0d6d04f0a5655eb57d55b3b326a42f
This allows usual integer parsing at app level and calling this function
to make sure correct values will be passed to
osmo_serial_set_baudrate().
Change-Id: I41415c99d26128b33a8bf5ef7b38948bd1fe5d50
Some applications may need submillisecond timers, such as those
interacting with modbus serial lines (RS-485, RTU), which require
timers of values around 1.5 char-time (T1.5), where a data char is
composed of 11 bits sent on the line: 1 start bit, 8 data bits,
1 stop bit, and and parity bit (or 2nd stop bits if no parity).
For instance, for a baudrate of 9600:
1.5 * 11 / 9600 = 1.718 ms = 1718 us
So having a granularity of MS is not enough here.
Change-Id: I71848d7c1ee0649929ce07680ee7320bb2a42f0e
select is an ancient interface with weird restrictions, such as
the fact that it cannot be used for file descriptor values > 1024.
This may have been sufficient 40 years ago, but certainly is not in
2020. I wanted to migrate to epoll(), but unfortunately it doesn't
work well with the fact that existing programs simply set osmo_fd.flags
without making any API calls at the time they change those flags.
So let's do the migration to poll() as a first step, and then consider
epoll() as a second step further down the road, after introducing new
APIs and porting applications over.
The poll() code introduced in this patch is not extremely efficient,
as it needs to do extensive linked list iterations after poll() returns
in order to find the osmo_fd from the fd. Optimization is possible,
but let's postpone that to a follow-up patch.
At compile time, a new --enable-force-io-select argument can be given
to configure, forcing the use of the old select() backend instead of the
new poll() based backend.
Change-Id: I9e80da68a144b36926066610d0d3df06abe09bca
This change implements 'systemd-journal' logging target, that is
similar to the existing 'syslog' target. The key difference is
that 'systemd-journal' allows us to offload rendering of the meta
information, such as location (file name, line number), subsystem,
and logging level, to systemd. Moreover, we can attach arbitrary,
user-specific fields [1] to the logging messages, so they can be
used for advanced log filtering (e.g. by IMSI/TMSI/TLLI):
$ journalctl OSMO_SUBSYS=DMSC -f
Since we don't want to make libsystemd a required dependency, this
feature is optional, and needs to be enabled at build-time:
$ ./configure --enable-systemd-logging
The new logging target can be configured in the same way as any
other one - via the VTY interface, or using the configuration file:
log systemd-journal [raw]
logging level set-all notice
logging filter all 1
Two logging handlers are available: generic and raw. The first one
behaves similarly to both 'syslog' and 'stderr', i.e. all the meta
information is rendered by libosmocore itself, and then passed to
systemd together with the logging message. The later is more like
the 'gsmtap' target, so all available meta information is handed
over to systemd in form of fields [1]:
- CODE_FILE / CODE_LINE - location info,
- PRIORITY - syslog-compatible logging level,
- OSMO_SUBSYS - Osmocom-specific sub-system (e.g. DMSC),
- OSMO_SUBSYS_HEX - same as OSMO_SUBSYS, but encoded in hex,
- MESSAGE - the logging message itself,
and then can be rendered in any supported format (e.g. JSON).
More details about the API can be found in [2].
[1] https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html
[2] https://www.freedesktop.org/software/systemd/man/sd-journal.html
Change-Id: I609f5cf438e6ad9038d8fc95f00add6aac29fb23
If we watn to migrate to something like epoll(), user application
code must call a function of the libosmocore API whenever it changes
its read/write interest in a file descriptor.
Let's introduce API so applications can be ported to this API,
before making direct 'ofd->when' manipulations illegal as a second step.
Change-Id: Idb89ba7bc7c129a6304a76900d17f47daf54d17d
This will be useful to handle latitude and longitude numbers for GAD, which is
the location estimate representation used for LCS (Location Services).
The OsmoSMLC VTY user interface will provide floating-point strings like
"23.456" while GAD stores them as micro-degress 23456000. The osmo_gad_to_str*
will also convert latitude and longitude to floating-point string.
There was code review concerns against adding this API, upon which I tried to
use floating point string formats. But I encountered various problems with
accuracy and trailing zeros. For global positioning data (latitude and
longitude), even inaccuracy on the sixth significant decimal digit causes
noticeable positional shift. To achieve sufficient accuracy on the least
significant end, I need to use double instead of float. To remove trailing
zeros, the idea was to use '%.6g' format, but that can cause rounding. '%.6f'
on a double looks ok, but always includes trailing zeros. A test program shows:
%.6g of ((double)(int32_t)23230100)/1e6 = "23.2301" <-- good
%.6g of ((double)(int32_t)42419993)/1e6 = "42.42" <-- bad rounding
%.6g of ((double)(int32_t)23230199)/1e6 = "23.2302" <-- bad rounding
%.6f of ((double)(int32_t)23230100)/1e6 = "23.230100" <-- trailing zeros
%.6f of ((double)(int32_t)42419993)/1e6 = "42.419993" <-- good
%.6f of ((double)(int32_t)23230199)/1e6 = "23.230199" <-- good
It looks like when accepting that there will be trailing zeros, using double
with '%.6f' would work out, but in the end I am not certain enough that there
aren't more hidden rounding / precision glitches. Hence I decided to reinforce
the need to add this API: it is glitch free in sufficient precision for
latitude and longitude data, because it is based on integer arithmetic.
The need for this precision is particular to the (new) OsmoSMLC vty
configuration, where reading and writing back user config must not modify the
values the user entered. Considering to add these functions to osmo-smlc.git,
we might as well add them here to libosmocore utils, and also use them in
osmo_gad_to_str_*() functions.
Change-Id: Ib9aee749cd331712a4dcdadfb6a2dfa4c26da957
So far there is only osmo_use_count_name_buf(). Also provide a use count to
string using a talloc context, allowing to use OTC_SELECT.
- instead of foo_name(), rather use foo_to_str().
- osmo_use_count_name_buf() returns the buf and not the chars_needed. So add
osmo_use_count_to_str_buf() with a signature that is usable by
OSMO_NAME_C_IMPL().
- provide osmo_use_count_to_str_c() using OSMO_NAME_C_IMPL().
Change-Id: I1d2e7ee979f8c316ef99f7c65675b36d092ddfca
As shown in the recently added bitgen_test.c, using osmo_loadXXbe_ext() with a
smaller n produces results aligned on the most significant bytes, which is
cumbersome, since it does not return a previously stored value. This problem
exists only for the big-endian functions, the little-endian osmo_loadXXle_ext()
properly return values adjusted on the least significant octets.
Add osmo_loadXXbe_ext_2() variants that properly right-adjust the returned
value. Prominently highlight this behavior in API doc. Test the new functions
in bitgen_test.c.
For example, this eases handling of 24bit integers (e.g. loaded from buffer to
uint32_t, and stored into buffer from uint32_t). Also explicitly show this 24
bit case in bitgen_test.c
Change-Id: I2806df6f0f7bf1ad705d52fa386d4525b892b928
Add OSMO_ASSERT()s to ensure bounds checking.
For example, for osmo_store32le_ext(), passing n > 5 would read past the end of
the uint32_t. Similarly, osmo_load32le_ext() for n > 4 would write past the
uint32_t's end.
Change-Id: I2dc21582cd8a679b6624cefbc0c1678b093a3d08
In case the port isn't known at the time osmo_sockaddr_str_from_str2()
parse only the ip and don't touch the port.
This is the case when a user has different vty commands for ip and port.
Change-Id: Ifd4e282586b8bd40b912a9f1c25f9e8208420106
When dealing with IPv4 and IPv6 address, the struct sockaddr
allows to hold IPv4 and IPv6.
However even when struct sockaddr is being used, a cast to the
IPv4 or IPv6 family must happen. To work around this additional code,
use a union for the most common types.
Change-Id: If80172373735193401af872b18e1ff00c93880e7
Implement better API around 3GPP TS 24.008 Mobile Identity coding.
struct osmo_mobile_identity is a decoded representation of the raw Mobile
Identity, with a string representation as well as dedicated raw uint32_t TMSI.
The aim is to remove all uncertainty about decoded buffer sizes / data types.
I have patches ready for current osmo CNI programs, replacing the Mobile
Identity coding with this new API. Deprecate the old MI API.
osmo-bsc: I71c3b4c65dbfdfa51409e09d4868aea83225338a
osmo-msc: Ic3f969e739654c1e8c387aedeeba5cce07fe2307
osmo-sgsn: I4cacb10bac419633ca0c14f244f9903f7f517b49
Note that some GPRS and SGs related coding is done here in libosmocore and
hence currently remains using the old implementation (see previous version of
this patch: Ic3f969e739654c1e8c387aedeeba5cce07fe2307).
New API functions provide properly size-checking implementations of:
- decoding a raw MI from a bunch of MI octets;
- locating and decoding MI from a full 3GPP TS 24.008 Complete Layer 3 msgb;
- encoding to a buffer;
- encoding to the end of a msgb.
Other than the old gsm48_generate_mid(), omit a TLV tag and length from
encoding. Many callers manually stripped the tag and value after calling
gsm48_generate_mid(). The aim is to leave writing a TL to the caller entirely,
especially since some callers need to use a TvL, i.e. support a variable-size
length of 8 or 16 bit.
New validity checks so far not implemented anywhere else:
- stricter validation of number of digits of IMSI, IMEI, IMEI-SV MI.
- stricter on filler nibbles to be 0xf.
As a result, applications using osmo_mobile_identity will be stricter in
rejecting coding mistakes (some of which we currently have in our test suites,
and which we'll need to fix).
Rationale:
While implementing osmo-bsc's MSC pooling feature in osmo-bsc, this API will be
used to reduce the number of times a Mobile Identity is extracted from a raw
RSL message.
Extracting the Mobile Identity from messages has numerous duplicate
implementations across our code with various levels of specialization.
https://xkcd.com/927/
To name a few:
- libosmocore: gsm48_mi_to_string(), osmo_mi_name_buf()
- osmo-bsc: extract_sub()
- osmo-msc: mm_rx_loc_upd_req(), cm_serv_reuse_conn(), gsm48_rx_mm_serv_req(),
vlr_proc_acc_req()
We have existing functions to produce a human readable string from a Mobile
Identity, more or less awkward:
- gsm48_mi_to_string() decodes a TMSI as a decimal number. These days we use
hexadecimal TMSI everywhere.
- osmo_mi_name_buf() decodes the BCD digits from a raw MI every time, so we'd
need to pass around the raw message bytes. Also, osmo_mi_name_buf() has the
wrong signature, it should return a length like snprintf().
- osmo-bsc's extract_sub() first uses gsm48_mi_to_string() which encodes the
raw uint32_t TMSI to a string, and then calls strtoul() via
tmsi_from_string() to code those back to a raw uint32_t.
Each of the above implementations employ their own size overflow checks, each
invoke osmo_bcd2str() and implement their own TMSI osmo_load32be() handling.
Too much code dup, let's hope that each and every one is correct.
In osmo-bsc, I am now implementing MSC pooling, and need to extract NRI bits
from a TMSI Mobile Identity. Since none of the above functions are general
enough to be re-used, I found myself again copy-pasting Mobile Identity code:
locating the MI in a 24.008 message with proper size checks, decoding MI
octets.
This time I would like it to become a generally re-usable API.
This patch was first merged as Ic3f969e739654c1e8c387aedeeba5cce07fe2307 and
caused test fallout, because it re-implemented old API with the new stricter
decoding. In this patch version, old API remains 1:1 unchanged to avoid such
fallout. Applications will soon switch to the new osmo_mobile_identity API and
become stricter on MI coding when that happens, not implicitly by a new
libosmocore version.
Change-Id: If4f7be606e54cfa1c59084cf169785b1cbda5cf5
This reverts commit d1ceca9d48, as it
introduces regressions in both osmo-msc and osmo-nitb which have been
causing failing builds for several days now.
Change-Id: I4bd958d0cd2ab4b0c4725e6d114f4404d725fcf7
Implement better API around 3GPP TS 24.008 Mobile Identity coding.
struct osmo_mobile_identity is a decoded representation of the raw Mobile
Identity, with a string representation as well as dedicated raw uint32_t TMSI.
The aim is to remove all uncertainty about decoded buffer sizes / data types.
I have patches ready for all osmo programs, completely replacing the Mobile
Identity coding with this new API. Hence deprecate the old MI API.
New API functions provide properly size-checking implementations of:
- decoding a raw MI from a bunch of MI octets;
- locating and decoding MI from a full 3GPP TS 24.008 Complete Layer 3 msgb;
- encoding to a buffer;
- encoding to the end of a msgb.
Other than the old gsm48_generate_mid(), omit a TLV tag and length from
encoding. Many callers manually stripped the tag and value after calling
gsm48_generate_mid(). The aim is to leave writing a TL to the caller entirely,
especially since some callers need to use a TvL, i.e. support a variable-size
length of 8 or 16 bit.
New validity checks so far not implemented anywhere else:
- stricter validation of number of digits of IMSI, IMEI, IMEI-SV MI.
- stricter on filler nibbles to be 0xf.
Rationale:
While implementing osmo-bsc's MSC pooling feature in osmo-bsc, this API will be
used to reduce the number of times a Mobile Identity is extracted from a raw
RSL message.
Extracting the Mobile Identity from messages has numerous duplicate
implementations across our code with various levels of specialization.
https://xkcd.com/927/
To name a few:
- libosmocore: gsm48_mi_to_string(), osmo_mi_name_buf()
- osmo-bsc: extract_sub()
- osmo-msc: mm_rx_loc_upd_req(), cm_serv_reuse_conn(), gsm48_rx_mm_serv_req(),
vlr_proc_acc_req()
We have existing functions to produce a human readable string from a Mobile
Identity, more or less awkward:
- gsm48_mi_to_string() decodes a TMSI as a decimal number. These days we use
hexadecimal TMSI everywhere.
- osmo_mi_name_buf() decodes the BCD digits from a raw MI every time, so we'd
need to pass around the raw message bytes. Also, osmo_mi_name_buf() has the
wrong signature, it should return a length like snprintf().
- osmo-bsc's extract_sub() first uses gsm48_mi_to_string() which encodes the
raw uint32_t TMSI to a string, and then calls strtoul() via
tmsi_from_string() to code those back to a raw uint32_t.
Each of the above implementations employ their own size overflow checks, each
invoke osmo_bcd2str() and implement their own TMSI osmo_load32be() handling.
Too much code dup, let's hope that each and every one is correct.
In osmo-bsc, I am now implementing MSC pooling, and need to extract NRI bits
from a TMSI Mobile Identity. Since none of the above functions are general
enough to be re-used, I found myself again copy-pasting Mobile Identity code:
locating the MI in a 24.008 message with proper size checks, decoding MI
octets.
This time I would like it to become a generally re-usable API.
Change-Id: Ic3f969e739654c1e8c387aedeeba5cce07fe2307
Reliable monitoring requires regular flush of all stat values, even
if they have not changed. Otherwise (1) the monitoring app has to
maintain state and (2) can go out of sync if it's restarted while
the app is still running.
Change-Id: I04f1e7bdf0d6f20e4f15571e94191de61c47ddad
The signalfd(2) mechanism of Linux allows signals to be delivered
and processed via normal file descriptor I/O. This avoids any of the
usual problems about re-entrancy of signal processing, as signals can
be processed from the osmocom select() loop abstraction just like any
other event.
Change-Id: If8d89dd1f6989e1cd9b9367fad954d65f91ada30
For a process running as root, it may be desirable to drop privileges
down to a normal user before executing an external command. Let's
add a new API function for that.
Change-Id: If1431f930f72a8d6c1d102426874a11b7a2debd9
Pau Espin
Harald Welte
Vadim Yanitskiy
Daniel Willmann
Daniel Willmann