This adds a --enable-systemtap configure option, which will then
add static tracepoints to the generated libosmocore binary.
At this point, only two tracepoints are supported: log_start
and log_done. They can be used to trace the amount of time
a libosmocore-using application spends in potentiall blocking calls to
log to stderr or to files.
Related: OS#4311
Change-Id: I7e1ab664241deb524c9582cbd1bec31af46c747e
Instead of removing AC_DEFINE(), I should have used AS_IF().
Change-Id: I20e256bd6fdb0256c95ab7073e07b7437af6a12f
Fixes: I761a7afaeda9d232ac26edff47949e911f8f1f0c
This adds an inter-thread queue "it_q" to libosmocore. With it_q,
one can perform thread-safe enqueing of messages to another thread,
who will receive the related messages triggered via an eventfd
handled in the usual libosmocore select loop abstraction.
Change-Id: Ie7d0c5fec715a2a577fae014b0b8a0e9c38418ef
This adds an easy way to listen to netlink events form the Linux kernel
from within libosmocore applications.
The new dependency can be disabled via the "--disable-lbimnl" configure flag.
Change-Id: I4f787ee68f0d6d04f0a5655eb57d55b3b326a42f
select is an ancient interface with weird restrictions, such as
the fact that it cannot be used for file descriptor values > 1024.
This may have been sufficient 40 years ago, but certainly is not in
2020. I wanted to migrate to epoll(), but unfortunately it doesn't
work well with the fact that existing programs simply set osmo_fd.flags
without making any API calls at the time they change those flags.
So let's do the migration to poll() as a first step, and then consider
epoll() as a second step further down the road, after introducing new
APIs and porting applications over.
The poll() code introduced in this patch is not extremely efficient,
as it needs to do extensive linked list iterations after poll() returns
in order to find the osmo_fd from the fd. Optimization is possible,
but let's postpone that to a follow-up patch.
At compile time, a new --enable-force-io-select argument can be given
to configure, forcing the use of the old select() backend instead of the
new poll() based backend.
Change-Id: I9e80da68a144b36926066610d0d3df06abe09bca
This change implements 'systemd-journal' logging target, that is
similar to the existing 'syslog' target. The key difference is
that 'systemd-journal' allows us to offload rendering of the meta
information, such as location (file name, line number), subsystem,
and logging level, to systemd. Moreover, we can attach arbitrary,
user-specific fields [1] to the logging messages, so they can be
used for advanced log filtering (e.g. by IMSI/TMSI/TLLI):
$ journalctl OSMO_SUBSYS=DMSC -f
Since we don't want to make libsystemd a required dependency, this
feature is optional, and needs to be enabled at build-time:
$ ./configure --enable-systemd-logging
The new logging target can be configured in the same way as any
other one - via the VTY interface, or using the configuration file:
log systemd-journal [raw]
logging level set-all notice
logging filter all 1
Two logging handlers are available: generic and raw. The first one
behaves similarly to both 'syslog' and 'stderr', i.e. all the meta
information is rendered by libosmocore itself, and then passed to
systemd together with the logging message. The later is more like
the 'gsmtap' target, so all available meta information is handed
over to systemd in form of fields [1]:
- CODE_FILE / CODE_LINE - location info,
- PRIORITY - syslog-compatible logging level,
- OSMO_SUBSYS - Osmocom-specific sub-system (e.g. DMSC),
- OSMO_SUBSYS_HEX - same as OSMO_SUBSYS, but encoded in hex,
- MESSAGE - the logging message itself,
and then can be rendered in any supported format (e.g. JSON).
More details about the API can be found in [2].
[1] https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html
[2] https://www.freedesktop.org/software/systemd/man/sd-journal.html
Change-Id: I609f5cf438e6ad9038d8fc95f00add6aac29fb23
Some systmes (like the ones available in OBS) don't support creating
SCTP sockets, so we need to skip those tests there.
Change-Id: I1d16280674625877ec22cc60cbc5deb67868a656
configure flag required to enable this: --enable-neon
Although autodetection according to __ARM_NEON would work because this
is only defined if the fpu is neon neon-fp16 neon-vfpv3 neon-vfpv4
neon-fp-armv8 crypto-neon-fp-armv8 doing that would lead to a unknown
performance impact, so it needs to be enabled manually.
Speedup is about ~1.3-1.5 on a unspecified single core Cortex A9. This
requires handling a special case for RACH with len 14 which is far too
short for neon and would actually incur a performance penalty of 25%.
Related: OS#4585
Change-Id: I58ff2cb4ce3514f43390ff0a2121f81e6a4983b5
Remove OpenSUSE bug report link, set version to @VERSION@, make it build with
CentOS 8 etc.
Related: OS#4550
Change-Id: I59255889740195ec811a947a7130ae0918ea4b4d
AC_SEARCH_LIBS was finding the function correctly, but later on
AC_CHECK_FUNCS was not including the found LIBRARY_RT so the function
was not found, and hence HAVE_CLOCK_GETTIME ended up undefined (which in
turns disables support for osmo_clock_gettime() API).
This happened in systems like the soekris where the clock_gettime sybmol
is available in external lib -lrt.
Let's avoid double-checking for the function twice, and simply define
HAVE_CLOCK_GETTIME when AC_SEARCH_LIBS succeeds (the success action is
guaranteed to be called even if there's no extra lib required).
Change-Id: Iced1e0542cee6beb9f08f5299aad49fab142cfb4
The signalfd(2) mechanism of Linux allows signals to be delivered
and processed via normal file descriptor I/O. This avoids any of the
usual problems about re-entrancy of signal processing, as signals can
be processed from the osmocom select() loop abstraction just like any
other event.
Change-Id: If8d89dd1f6989e1cd9b9367fad954d65f91ada30
As pointed out at https://github.com/libexpat/libexpat/issues/312
libtool does not play nice with clang sanitizer builds at all.
For those builds LD shoud be set to clang too (and LDFLAGS needs the
sanitizer flags as well), because the clang compiler driver knows how
linking to the sanitizer libs works, but then at a later stage libtool
fails to actually produce the shared libraries and the build fails. This
is fixed by this patch.
Addtionally LD_LIBRARY_PATH has no effect on conftest runs during
configure time, so the rpath needs to be set to the asan library path to
ensure the configure run does not fail due to a missing asan library,
i.e.:
SANS='-fsanitize=memory -fsanitize-recover=all -shared-libsan'
export CC=clang-10
ASANPATH=$(dirname `$CC -print-file-name=libclang_rt.asan-x86_64.so`)
export LDFLAGS="-Wl,-rpath,$ASANPATH $SANS $LDFLAGS"
Change-Id: I8ebd9c6d4efda41c7c8196f963d1f04d65160754
libosmosim contains a variety of definitions and utility fuinctions
useful when working with SIM card [protocol]. They can not only
be used with PC/SC readers but also in other contexts.
Change-Id: I741940d3dc2a5653c760e9d1597d7f08afb3b631
In some situations, we want to execute an external shell command
in a non-blocking way. Similar to 'system', but without waiting for
the child to complete. We also want to close all file descriptors
ahead of the exec() and filter + modify the environment.
Change-Id: Ib24ac8a083db32e55402ce496a5eabd8749cc888
Related: OS#4332
Osmocom applications typically use libosmocore select.[ch] event loop
code as their main event dispatch mechanism. When they want to deal
with libusb in a non-blocking/asynchronous way, they need to integrate
libusb into that select().
The new libosmousb is doing exactly that: Providing a shared utility
library for Osmocom programs that wish to use libusb. This is useful
for example in simtrace2 host utilitie as well as osmo-e1d.
Change-Id: I656a1a38cbb5b1f3a9145d2869d3b4d0adefcae3
Closes: OS#4299
The coverage report shows what code is covered by tests and what is not
and the ratio could be tracked over time. These reports will allow us
to identify code that is not being tested and improve the test suites.
To enable the reports configure with --enable-code-coverage and execute
"make check-code-coverage". The HTML report will be generated in a
subdirectory with name libosmocore-$(PACKAGE_VERSION)-coverage/index.html
The report is generated using gcov, lcov and lcov_cobertura tools and
the OSMO_AC_CODE_COVERAGE macro. The osmo_ax_code_coverage.m4 is a copy of
ax_code_coverage.m4 taken from autoconf-archive v2018.03.13. It was
copied to avoid the additional external dependency and renamed to avoid
overwriting it in case autoconf-archive is already installed as we are
going to install it in $(datadir)/aclocal in order to be reused in other
osmocom's projects.
Closes: OS#1987
Change-Id: I6f4ffb91bd7f3dd070aa09dd16d5ad1faf130a4c
This way libosmocore build fails during configuring phase if expected
default behavior (building with libsctp support enabled and providing
osmo_sock_init2_multiaddr() API) fails. User is still provided with
--disable-libsctp option in case he doesn't need those features or his
environment doesn't provide required libsctp APIs.
Change-Id: I710c9cb1c6da0e5fc94b792df8bf60194a72208f
This API will be used by libosmo-netif's osmo_stream for SCTP sockets,
which in turn will be used by libosmo-sccp to support multi-homed
connections.
Related: OS#3608
Change-Id: Ic8681d9e093216c99c6bca4be81c31ef83688ed1
log_enable_multithread() enables use of locks inside the
implementation. Lock use is disabled by default, this way only
multi-thread processes need to enable it and suffer related
complexity/performance penalties.
Locks are required around osmo_log_target_list and items inside it,
since targets can be used, modified and deleted by different threads
concurrently (for instance, user writing "logging disable" in VTY while
another thread is willing to write into that target).
Multithread apps and libraries aiming at being used in multithread apps
should update their code to use the locks introduced here when
containing code iterating over osmo_log_target_list explictly or
implicitly by obtaining a log_target (eg. osmo_log_vty2tgt()).
Related: OS#4088
Change-Id: Id7711893b34263baacac6caf4d489467053131bb
Some toolchains (such as sysmobts 201705 one) containing the TLS bug on
old ARM gcc versions (<7.3.0) also crash if the initial workaround found
is aplied (CFLAGS="-mtls-dialect=gnu2"). In that scenario, let's provide
a way to disable the workaround (to avoid "ld" crashing) and warn the
user about requirement to build with -O0 to avoid runtime crashes.
Related: OS#4062
Related: SYS#4628
Change-Id: I04ff8c702eabcf4f6e7b59e11aece2744267cefe
Check if compiler being used contains the bug. GCC 7.3.0 is the oldest
version containing the fix, and version 6.3.0 is known to contain the
bug. Bug is only known to appear so far only on ARM32. If the bug is
present, gcc will generate a wrong binary which wil lend up segfaulting
when accessing TLS (__thread) variables under certain conditions.
Related: OS#4062
Related: SYS#4628
Change-Id: I8acc2cf41b73da0c3290f1cefd79f2bc68b0e77d
For handling RTP IP addresses and ports, osmo-mgw, osmo-bsc and osmo-msc
so far have their own separate shims and code duplication around
inet_ntoa(), htons(), sockaddr conversions etc. Unify and standardize
with this common API.
In the MGW endpoint FSM that was introduced in osmo-bsc and which I
would like to re-use for osmo-msc (upcoming patch moving that to
osmo-mgw), it has turned out that using char* IP address and uint16_t
port number types are a convenient common denominator for logging,
MGCP message composition and GSM48. Ongoing osmo-msc work also uses this
for MNCC.
This is of course potentially useful for any other IP+port combinations
besides RTP stream handling.
Needless to say that most current implementations will probably stay
with their current own conversion code for a long time; for current
osmo-{bsc,msc,mgw} work (MGW endpoint FSM) though, I would like to move
to this API here.
Change-Id: Id617265337f09dfb6ddfe111ef5e578cd3dc9f63
This reverts commit 6dd00d876e.
Unfortunately, it seems older a2x versions don't support "-D" for
manpage generation:
All the osmocom master builds started to fail with:
make[2]: Entering directory '/build/deps/libosmocore/man'
a2x --doctype manpage --format manpage -D . osmo-config-merge.adoc
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
a2x: ERROR: "xmllint" --nonet --noout --valid "/build/deps/libosmocore/man/osmo-config-merge.xml" returned
non-zero exit status 127
Makefile:545: recipe for target 'osmo-config-merge.8' failed
Change-Id: I0f45362d3e978c328d962a5c0d883eade27b875c
I am setting out to refactor various details about logging. To show the effect,
I am first adding this new test to illustrate the exact effects on the various
osmo programs.
Add logging_vty_test.c as a standalone program that simply defines a few
logging categories and opens a telnet vty to play with.
Add logging_vty_test.vty, as an osmo_verify_transcript_vty.py test script.
Add --enable-external-tests to configure.ac, to enable running
logging_vty_test.vty during 'make check'.
Also allow running 'make vty-test' without the need to first configure with
--enable-external-tests (a flexibility I've missed many times over in the other
osmo source trees).
Add a Makefile.am stub for external CTRL tests, basically a copy-paste from
osmo-msc.git. I doubt that libosmocore will get python driven CTRL interface
testing any time soon, but if so we will know to not run it concurrently.
Change-Id: I948e832a33131f8eab98651d6010ceb0ccbc9a9c
Due to OS#3360, build testing for arm-none-eabi was unfortunately
skipped for a long time. This is a number of fixes that make the
compile test pass again.
Related: OS#3360
Change-Id: I88e3c8e1a8786ca2a6a023b0d27c74be200a8588
There are some symbols for use between control_cmd.c and control_if.c,
which are not supposed to be exported publicly. Let's make sure we
keep those symbols local.
Change-Id: Ia85f36a9c4b2ebf4003718e0a230959638370320
Linux offers file descriptor based periodic (interval) timers,
which can achieve a higher precision than our userspace based
timers and which can be slave'd to CLOCK_MONOTINIC or other clock
sources. Let's add some code for osmo_fd wrapped versions that
integrate well with our select() abstraction.
The code has been used in osmo-bts-trx since June 2017 (change-id
I51b19adde14ebb7ef3bb863d45e06243c323e22e), and I'm just renaming
and moving it to libosmocore here. After a merge, the osmo-bts
implementations can be removed in favor if this one.
Change-Id: Ibeffba7c997252c003723bcd5d14122c4ded2fe7
Sometimes the library probiding dlopen is not the same one providing
dlsym.
This is the case when compiling with AddressSanitizer enabled. In this
case, AC_SEARCH_LIBS([dlopen]...) reports no lib is required, but tests
using dlsym still require to link against -ldl.
Change-Id: Ic619b0885688066b60c97caf1e2c7e5402c1d9f7
Provide a sane means of adding the -Werror compiler flag.
Currently, some of our jenkins.sh add -Werror by passing 'CFLAGS="-Werror"',
but that actually *overwrites* all the other CFLAGS we might want to have set.
Maintain these exceptions from -Werror:
a) deprecation (allow upstream to mark deprecation without breaking builds);
b) "#warning" pragmas (allow to remind ourselves of errors without breaking
builds)
As a last configure step before generating the output files, print the complete
CFLAGS and CPPFLAGS by means of AC_MSG_RESULT.
Change-Id: Ic5c8e68b64cd890b3309b4b26c7f22bde1edba83
The sercomm functions are unavailable in case of embedded build. Add
stub and link the tests against it.
Change-Id: I9bc5cb2f822b1a3ffdc6ec29f46b6bac8288314e
As of 67bdd80a96 the stats.c is
effectively disable so we should disable the corresponding tests as
well.
Change-Id: I42ff7a6619c0a5926fdc2ec779cf04689c567e15
The accelerated convolutional decoder uses SSSE3 instructions such
as PSIGNW (via _mm_sign_epi16) which go beyond what SSE3 offers. So
let's make sure we use the right compiler flag (-mssse3) and also the
right runtime check.
Without this patch, we would use illegal instructions e.g. on Opteron
Gen3 such as Opteron 2427, which are also used as build.opensuse.org
build hosts (build31 through build36) where we wouldn't pass "make
check" as a result.
Change-Id: I2754164384109f2821fd98ffb48f625893f2923d
Fixes: OS#2386
On systems with GNU/Linux kernel older than 3.17 (Debian 8 "jessie" for
example) the osmo_get_rand_id() would always return failure due to
missing getrandom() syscall.
To support such systems, let's add fallback code which uses GnuTLS
library. It can be disabled explicitly via '--disable-gnutls' option at
compile-time, otherwise ./configure will fail if both getrandom() and
GnuTLS are not available. When building with '--enable-embedded' the
fallback is disabled automatically.
Related: OS#1694
Change-Id: Ic77866ce65acf524b768882c751a4f9c0635740b
The function is a wrapper on top of getrandom() (if available via glibc) or
corresponding syscall. If neither is available than failure is always
returned.
It's intended to generate small random data good enough for session
identifiers and keys. To generate long-term cryptographic keys it's
better to use special crypto libraries (like GnuTLS for example)
instead.
As an example it's used to replace old insecure random number generator
in osmo-auc-gen utility.
Change-Id: I0241b814ea4c4ce1458f7ad76e31d390383c2048
Related: OS#1694
Despite the libosmocoding.map is preset since the library release,
one was not used in a proper way. The LTLDFLAGS were missing, so
let's add them.
Change-Id: Idf677825ff642d50bea43c7f970810783e864fdd
Add simple helper target to automate basic release steps:
* version bump
* prepare release commit
* git commit, tag and sign
For library projects:
* update debian/changelog from TODO-RELEASE
* cleanup TODO-RELEASE
For non-library projects:
* update debian/changelog from git log
Note: it requires bumpversion package to be installed, debian/control is
adjusted accordingly. The helper itself is installed to facilitate reuse
by other libraries.
N. B: you still have to manually adjust LIBVERSION in previous commit -
see TODO-RELEASE header for details.
Use it as follows:
make REL=minor release
The REL parameter defines which component of the version [1] to bump and
can be any of { major, minor, patch }.
[1] http://semver.org/
Change-Id: I790ceb958195b9f6cbabfe8c977dc30e2bd7414b
Related: OS#1861
This flag, when set, allows to unconditionally disable doxygen
documentation generation, even if doxygen command is found.
Change-Id: Iba1440292116af27b267c7a1fbec1c2336784efe
If we pass CFLAGS params to configure, then the CFLAGS set inside
configure.ac are not applied and the build fails if -Werror is enabled
because then BUILDING_LIBOSMOCORE is not defined.
Seeing the following output at configure time in FreeBSD build:
./configure: CFLAGS+= -fsanitize=address -fsanitize=undefined: not found
./configure: CPPFLAGS+= -fsanitize=address -fsanitize=undefined: not found
./configure: CFLAGS+= -DBUILDING_LIBOSMOCORE -Wall: not found
./configure: CPPFLAGS+= -DBUILDING_LIBOSMOCORE -Wall: not found
Change-Id: I692bbc66e5343998eec360a02a3a1fe73478c5c6
Files in include/osmocom/gprs/ and src/gb/ are not included in any doxygen
generated API docs. Add Doxyfile.gb.in and adjust configure.ac and Makefile.am.
Change-Id: Ieb64f497f55368e396872083237c9ff28da2dd93
A warning was printed even if the deprecation didn't apply to
libosmocore because it is still allowed to use it internally.
This patch fixes this case while still printing a warning if external
projects build using libosmocore headers.
Change-Id: I32212f20756f828af1017482a71e29e4b3adbad4
In tightly embedded builds (--enable-embedded), we want the ability to
replace talloc with a very simple heap allocator to avoid the complexity
of talloc without modifying all our code that assumes talloc.
This will break the hierarchical notion of the allocator, but
libosmo{core,gsm,coding,codec} don't rely on that anyway.
Change-Id: Ie341034076f242a813f081919dd09d845775ad35
Harald Welte
Oliver Smith
Vadim Yanitskiy
Pau Espin
Eric Wild
Vasil Velichkov
Daniel Willmann
Neels Hofmeyr