This reverts commit 7f1fb3ea81 - slightly
amended with the new log_cache_enalbe() function. The cache is hence
disabled by default, and applications can enable it, if they wish to
benefit from it.
Reason for the original revert was: some fallout expected due to log
manipulation in test code
Change-Id: I539872fc9e3c50b407e6bc388f1e091fa2c826c3
This ensures multithreaded logging attempts, in particular ones that do
nothing, do not hold the lock just for checking the level, which
interferes with other logging attempts.
Closes: OS#5818
Change-Id: I35f8dd9127dd6e7feae392094fd6b3ce2d32558d
* make backend configurable for later
* segmentation callback for chunked streams
* logging target for osmo_io
* support partial writes
Change-Id: I50d73cf550d6ce8154bf827bf47408131cf5b0a0
Related: SYS#5094, OS#5751
When the logging framework is not initialized we get an error:
"ERROR: osmo_log_info == NULL! You must call log_init() before
using logging in ..."
There are sometimes situations where some code tries to log before
logging was initialied. This is a problem because the actual log
line with the debug info we need is covered up by the error message
from the logging framework.
Lets introduce a fallback logging function that is called when the
the logging framework is not available. This function can just use
fprintf to output to stderr.
Change-Id: I9b1b0988e02322e3e44fd4ceea3e1bc2d4df3c45
Only support for SMpSDU mode is introduced in this commit.
Not supported explicit list:
- Transparent mode
- ATM/AAL2 based Transport layer
- GTP-U based Transport Layer
- Iu Rate Control procedure
- Time Alignment procedure
APIs are provided to allocate the primitives properly inside the related
msgb. This way primitives can be placed in the headroom, leaving the
data part of the msgb for the IuUP payload, hence allowing re-use of the
msgb and 0 copy of IuUP payload when forwarding data over RNL<->TNL.
Since RNL and TNL primitives relu struct osmo_prim_header, which is not
packed, they cannot be set to packed, and hence proper memory alignment
in the msgb must be done to avoid misaligned accesses (Asan errors about
it otherwise).
Related: SYS#5516
Change-Id: Ibe356fa7b1abaca0091e368db8478e79c09c6cb0
The macro introduced in d02090bba5 was not
enough: the actual logging macros are being used, i.e. by the fsm, so
wrap those as well, and provide a flag to disable this at build time.
Change-Id: Ia4c78abe5f198139f96ffa289998855be2477585
This was previously unconditionally defined, so embedded targets were
unable to get rid of the log macros and functions.
Change-Id: I589f93d98a6bc5cf6221c56e2fe3f27bfdd200e8
So far, we used blocking, buffered fwrite() to write to stderr
and file targets. This causes problems if there are [slow] consumers
causing delays, such as gnome-terminal (when the program is started
interactively) or systemd/journald (where we observe 64..128ms blocks on
stderr).
This patch introduces stderr/file based logging via write_queue
and osmo_select_main(), i.e. switch from glibc-buffered, blocking
to internally buffered, non-blocking writes.
* when osmo_stderr_target is created via application.c, we create it
in blocking stream mode for backwards compatibility, particularly
for [smaller] programs that don't use osmo_select_main()
* when the VTY code encounters 'log stderr' or 'log file FILENAME',
we switch that respective target to non-blocking write-queue mode,
as this means the application is in fact using osmo_select_main()
* The config file can now state 'log stderr blocking-io' or
'log file FILENAME blocking-io' to explicitly enforce using blocking
stream based I/O
* The application can at any time use API functions to switch either way
Closes: OS#4311
Change-Id: Ia58fd78535c41b3da3aeb7733aadc785ace610da
Introduce 2 new logging sub systems for signal and unit data.
Unify log messages so all log messages look similiar.
Log also Rx PDUs. Ensure dropped Tx packets (BLOCK/RESET on SNS)
contain *Tx*.
Change-Id: I34b8fde2955ecc010d1dcd9512e1bba9211e2c0d
Let's flag the API as deprecated so that people start using
log_set_print_filename2() API instead, which has less ackward
behavior implications like changing the print status of category-hex.
Related: OS#5034
Change-Id: If9b6b322989536a12094e6105c3aabc84d8be24a
Historically, BSSGP uses a non-constant, user-configurable integer
varieable for the logging sub-system. Let's replace this with a
statically-allocated library logging constant.
This is required if we want to use the subsystem number in e.g.
static initialized for osmo_fsm.log_subsys.
Change-Id: I506190aae9217c0956e4b5764d1a0c0772268e93
NSVC filtering was only implemented on sending messages, this also adds
log_set_context() calls to ns2_recv_vc()
Filtering by NSE is implemented similar to NSVC.
Change-Id: I63c0e85f82f5d08c5a6f535da94b8648498439d2
Related: SYS#5232
This change implements 'systemd-journal' logging target, that is
similar to the existing 'syslog' target. The key difference is
that 'systemd-journal' allows us to offload rendering of the meta
information, such as location (file name, line number), subsystem,
and logging level, to systemd. Moreover, we can attach arbitrary,
user-specific fields [1] to the logging messages, so they can be
used for advanced log filtering (e.g. by IMSI/TMSI/TLLI):
$ journalctl OSMO_SUBSYS=DMSC -f
Since we don't want to make libsystemd a required dependency, this
feature is optional, and needs to be enabled at build-time:
$ ./configure --enable-systemd-logging
The new logging target can be configured in the same way as any
other one - via the VTY interface, or using the configuration file:
log systemd-journal [raw]
logging level set-all notice
logging filter all 1
Two logging handlers are available: generic and raw. The first one
behaves similarly to both 'syslog' and 'stderr', i.e. all the meta
information is rendered by libosmocore itself, and then passed to
systemd together with the logging message. The later is more like
the 'gsmtap' target, so all available meta information is handed
over to systemd in form of fields [1]:
- CODE_FILE / CODE_LINE - location info,
- PRIORITY - syslog-compatible logging level,
- OSMO_SUBSYS - Osmocom-specific sub-system (e.g. DMSC),
- OSMO_SUBSYS_HEX - same as OSMO_SUBSYS, but encoded in hex,
- MESSAGE - the logging message itself,
and then can be rendered in any supported format (e.g. JSON).
More details about the API can be found in [2].
[1] https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html
[2] https://www.freedesktop.org/software/systemd/man/sd-journal.html
Change-Id: I609f5cf438e6ad9038d8fc95f00add6aac29fb23
It's hard to figure out what color logging categories have with those ANSI
color code strings. Instead, define these OSMO_LOGCOLOR_* constants.
Naming: commonly, the logging.h header has the "LOG" prefix in the name, but it
seems saner to include the OSMO_ prefix: it seems too likely that some
libosmocore user somewhere already has defined "LOGCOLOR_RED" somewhere.
Change-Id: I03b6b1f73ae7ee61d37ff921e071a3d0881d3e9a
log_enable_multithread() enables use of locks inside the
implementation. Lock use is disabled by default, this way only
multi-thread processes need to enable it and suffer related
complexity/performance penalties.
Locks are required around osmo_log_target_list and items inside it,
since targets can be used, modified and deleted by different threads
concurrently (for instance, user writing "logging disable" in VTY while
another thread is willing to write into that target).
Multithread apps and libraries aiming at being used in multithread apps
should update their code to use the locks introduced here when
containing code iterating over osmo_log_target_list explictly or
implicitly by obtaining a log_target (eg. osmo_log_vty2tgt()).
Related: OS#4088
Change-Id: Id7711893b34263baacac6caf4d489467053131bb
This list is really not needed by applications and currently only used
internally in logging.c and logging_vty.c.
Change-Id: I5dca069512bfcd0826194427c5482fad8bfd0232
the DEBUG macro name and ARRAY_SIZE macro function are frequently
used in other projects. If these projects also use libosmocore,
the macros will be redefined. This also generates a warning message
during compilation.
Not redefining the macros removes the warning message and possible
(but unlikely) mis-redefinition.
Change-Id: I0ba91eae8eacc5542d1647601b372e417ed1713c
Completely drop the implementations of log_vty_command_{str,description}().
These functions have been public API once, marked as deprecated since
c65c5b4ea0 (March 2017). I considered to keep
them, or reduce them to useless stubs, but it is quite silly, really. These
functions are completely and utterly useless outside of libosmocore. Any
program linking these deserves to fail.
Re-implement vty logging level command gen, in logging_vty.c. logging.c is
simply the wrong place for that.
Introduce logging_internal.h to share logging definitions to logging_vty.c
without publishing as API.
Introduce static gen_logging_level_cmd_strs() to compose a list of category
arguments with their descriptions for VTY commands. Use osmo_talloc_asprintf()
instead of the previous error prone and chaotic strlen() counting method.
Do not dynamically generate log level arguments, just keep static strings. We
are super unlikely to ever change the log levels we have.
No changes in logging_vty_test.vty: proves that there is no functional change.
All of this, besides introducing basic sanity, is cosmetic preparation to be
able to re-use the generic command generation code for arbitrary commands with
category or level args (for deprecated and new keywords).
Rationale: I want to hide 'all' and 'everything' from the VTY command
documentation, by means of deprecating. I first tried to simply define a
deprecated 'logging level CAT everything' command:
logging level (all|rsl|rr|...) (debug|info|notice|error|fatal)
logging level CAT everything # <- deprecated and hidden
But unfortunately, command matching doesn't work as intended when the CAT
argument reflects a valid category; I want it to invoke the deprecated function
as soon as the 'everything' keyword follows, but it stays stuck to the "valid"
command when the category argument matches an explicit keyword in that list,
and will throw an error on the following 'everything' keyword. I.e.:
logging level rsl everything
% Unknown command # <-- leads to config file parse error
logging level unknown_string everything
% Ignoring deprecated 'everything' # <-- works only for invalid categories
So I need to define 'everything' separately, again with a list of each valid
category instead of a generic CAT arg.
Change-Id: I3b083f27e3d751ccec258880ae7676e9af959a63
Allow printing the source file information *after* the log information.
Add target->print_filename_pos, log_set_print_filename_pos() and enum
log_filename_pos, and the optional 'last' keyword to the 'logging print file'
vty command. (An enum in case we want to add other positions later.)
Rationale: on log lines, the logging context is usually printed first in the
logging text. Without the source file information, this logging context pretty
much aligns in an easily readable way. When adding the source file information,
each line's logging context info is shifted right by a pretty much random
amount and it is hard to spot recurring logging contexts.
One solution is to switch off source file info, of course, but that's not an
option when actively hacking on bugs and new features.
For example, it is unnecessarily hard to spot lchan FSM related log lines in
this osmo-bsc log snippet:
DRSL NOTICE mgw_endpoint_fsm.c:603 mgw-endpoint(conn0){WAIT_MGW_RESPONSE}: Freeing instance
DRSL NOTICE fsm.c:381 mgw-endpoint(conn0){WAIT_MGW_RESPONSE}: Deallocated
DMSC NOTICE mgw_endpoint_fsm.c:603 SUBSCR_CONN(conn0){CLEARING}: Received Event FORGET_MGW_ENDPOINT
DCHAN DEBUG lchan_rtp_fsm.c:741 lchan(0-0-1-TCH_F-0){WAIT_RLL_RTP_ESTABLISH}: Received Event LCHAN_EV_RTP_RELEASED
DCHAN DEBUG lchan_rtp_fsm.c:520 lchan_rtp(0-0-1-TCH_F-0){ROLLBACK}: Freeing instance
DCHAN DEBUG fsm.c:381 lchan_rtp(0-0-1-TCH_F-0){ROLLBACK}: Deallocated
DCHAN DEBUG lchan_rtp_fsm.c:520 lchan(0-0-1-TCH_F-0){WAIT_RLL_RTP_ESTABLISH}: Received Event LCHAN_EV_RTP_RELEASED
DCHAN DEBUG lchan_fsm.c:1232 lchan(0-0-1-TCH_F-0){WAIT_RLL_RTP_ESTABLISH}: state_chg to WAIT_RF_RELEASE_ACK
DRSL NOTICE handover_fsm.c:762 handover(interBSCin_conn0){WAIT_RR_HO_DETECT}: Terminating (cause = OSMO_FSM_TERM_REGULAR)
DRSL NOTICE handover_fsm.c:762 handover(interBSCin_conn0){WAIT_RR_HO_DETECT}: Freeing instance
DRSL NOTICE fsm.c:381 handover(interBSCin_conn0){WAIT_RR_HO_DETECT}: Deallocated
DCHAN DEBUG lchan_fsm.c:1196 lchan(0-0-1-TCH_F-0){WAIT_RF_RELEASE_ACK}: state_chg to BORKEN
DCHAN DEBUG lchan_fsm.c:354 lchan(0-0-1-TCH_F-0){BORKEN}: (type=TCH_F) Clearing lchan state
DLMGCP DEBUG mgcp_client.c:741 Tx MGCP msg to MGCP GW: 'DLCX 3 2@mgw MGCP 1.0'
DLMGCP DEBUG mgcp_client.c:743 Sending msg to MGCP GW size: 66
DRLL NOTICE mgcp_client_fsm.c:422 MGCP_CONN(to-BTS){ST_DLCX_RESP}: Received Event EV_DLCX_RESP
Placing the source file info behind the log text makes it much easier to
follow, while the source file info is still available:
DRSL NOTICE mgw-endpoint(conn0){WAIT_MGW_RESPONSE}: Freeing instance (mgw_endpoint_fsm.c:603)
DRSL NOTICE mgw-endpoint(conn0){WAIT_MGW_RESPONSE}: Deallocated (fsm.c:381)
DMSC NOTICE SUBSCR_CONN(conn0){CLEARING}: Received Event FORGET_MGW_ENDPOINT (mgw_endpoint_fsm.c:603)
DCHAN DEBUG lchan(0-0-1-TCH_F-0){WAIT_RLL_RTP_ESTABLISH}: Received Event LCHAN_EV_RTP_RELEASED (lchan_rtp_fsm.c:741)
DCHAN DEBUG lchan_rtp(0-0-1-TCH_F-0){ROLLBACK}: Freeing instance (lchan_rtp_fsm.c:520)
DCHAN DEBUG lchan_rtp(0-0-1-TCH_F-0){ROLLBACK}: Deallocated (fsm.c:381)
DCHAN DEBUG lchan(0-0-1-TCH_F-0){WAIT_RLL_RTP_ESTABLISH}: Received Event LCHAN_EV_RTP_RELEASED (lchan_rtp_fsm.c:520)
DCHAN DEBUG lchan(0-0-1-TCH_F-0){WAIT_RLL_RTP_ESTABLISH}: state_chg to WAIT_RF_RELEASE_ACK (lchan_fsm.c:1232)
DRSL NOTICE handover(interBSCin_conn0){WAIT_RR_HO_DETECT}: Terminating (cause = OSMO_FSM_TERM_REGULAR) (handover_fsm.c:745)
DRSL NOTICE handover(interBSCin_conn0){WAIT_RR_HO_DETECT}: Freeing instance (handover_fsm.c:745)
DRSL NOTICE handover(interBSCin_conn0){WAIT_RR_HO_DETECT}: Deallocated (fsm.c:381)
DCHAN DEBUG lchan(0-0-1-TCH_F-0){WAIT_RF_RELEASE_ACK}: state_chg to BORKEN (lchan_fsm.c:1196)
DCHAN DEBUG lchan(0-0-1-TCH_F-0){BORKEN}: (type=TCH_F) Clearing lchan state (lchan_fsm.c:354)
DLMGCP DEBUG Tx MGCP msg to MGCP GW: 'DLCX 3 12@mgw MGCP 1.0' (mgcp_client.c:741)
DLMGCP DEBUG Sending msg to MGCP GW size: 67 (mgcp_client.c:743)
DRLL NOTICE MGCP_CONN(to-BTS){ST_DLCX_RESP}: Received Event EV_DLCX_RESP (mgcp_client_fsm.c:422)
Implementation: for 'last', insert source file info only when an '\n' is found
at the end of the log line composition buffer, so that LOGP()...LOGPC()
constructs also print source file info only when a log line actually ends.
Change-Id: I393907b3c9e0cc1145e102328adad0a83ee13a9f
The intention was to use the file's basename, but __BASE_FILE__ means "the root
file that is being parsed and contains #include statements".
If we had a function using __BASE_FILE__ and that was defined in an #included
file, __BASE_FILE__ would indicate the first file where the #include is, and
not the file where the function is defined. __BASE_FILE__ works for us because
we don't ever include function definitions that log something, so __BASE_FILE__
always coincides with __FILE__ for our logging; but still __BASE_FILE__ is
semantically the wrong constant.
Related: OS#2740
Change-Id: Ibc1d3746f1876ac42d6b1faf0e5f83bd2283cdcc
In the C API, add another enum log_file_type value, and when set print only the
basename of the source file path.
Rationale: especially when not building directly in the source dir, the paths
to the source files can become rather long. Usually, just the basename of the
file is sufficient to identify the source line.
Change-Id: If3e4d5fb2066f8bf86e59c82d1752b1a843cf58e
Add a separate flag and API to switch the category-in-hex output:
log_set_print_category_hex().
Add log_set_print_filename2() to modify only the print_filename flag. The old
log_set_print_filename() function still affects both flags. Explain the
rationale in the comment for log_set_print_filename().
There is no need to deprecate log_set_print_filename(); it might cause compiler
warnings and break strict builds unnecessarily.
Add VTY command 'logging print category-hex (0|1)'.
Since there is no VTY command to switch filename output, nothing needs to be
adjusted there (a command will be added in a subsequent patch).
Change-Id: Iba03a2b7915853c6dccaf6c393c31405320538b4
Log the log level string after the category name, if enabled.
The default behavior remains unchanged.
Change-Id: Ie6be365cfa6aeabdf115bff19bac198440c9adf1
For the lua console printing I need to print several values with
continuation but also specify the filename. Add a "C" for continue
and forward arguments.
Change-Id: I1d6dcb2567b9ed2c8767f661737b979bc3d1377e
In the course of splitting up the openbsc.git repository, we will create
libosmo-mgcp and need a library logging category for that purpose.
Change-Id: I09c587e2d59472cbde852d467d457254746d9e67
Add deprecation notice for public API use: it shouldn't be used outside
of libosmocore anyway.
Change-Id: I792e30dc44f027fd94e1f65af19fe08bac52b95c
Related: OS#71
Considering the various styles and implications found in the sources, edit
scores of files to follow the same API doc guidelines around the doxygen
grouping and the \file tag.
Many files now show a short description in the generated API doc that was so
far only available as C comment.
The guidelines and reasoning behind it is documented at
https://osmocom.org/projects/cellular-infrastructure/wiki/Guidelines_for_API_documentation
In some instances, remove file comments and add to the corresponding group
instead, to be shared among several files (e.g. bitvec).
Change-Id: Ifa70e77e90462b5eb2b0457c70fd25275910c72b
Especially for short descriptions, it is annoying to have to type \brief for
every single API doc.
Drop all \brief and enable the AUTOBRIEF feature of doxygen, which always takes
the first sentence of an API doc as the brief description.
Change-Id: I11a8a821b065a128108641a2a63fb5a2b1916e87
A warning was printed even if the deprecation didn't apply to
libosmocore because it is still allowed to use it internally.
This patch fixes this case while still printing a warning if external
projects build using libosmocore headers.
Change-Id: I32212f20756f828af1017482a71e29e4b3adbad4
* remove unused parameter from logging_vty_add_cmds()
* mark log level descriptors static
* change internal static function int check_log_to_target() to more
appropriate bool should_log_to_target()
* deprecate log_vty_command_*() from public API as it should only be
used by logging_vty_add_cmds()
Change-Id: I0e9ddd7ba3ce211302d99a3494eb408907a2916e
Related: OS#71
My recent logging patch was merged to master a bit too soon. Accomodate the
request for naming that matches the general "LOG" prefix instead of "LOGGING".
libosmocore will not be backwards-compatible with the few commits from
change-id I5c343630020f4b108099696fd96c2111614c8067 up to this one. This and
following commits are backwards compatible with those before that short window.
See also:
* openbsc change-id Ib2ec5e4884aa90f48051ee2f832af557aa525991
* osmo-pcu change-id I4db4a668f2be07f3d55f848d38d1b490d8a7a685
Change-Id: I424fe3f12ea620338902b2bb8230544bde3f1a93
Commit 812ba6dc63
"logging: centrally define ctx and filter indexes"
Removed definitions, which causes older e.g. openbsc and osmo-pcu code trees to
fail to build against a newer libosmocore. Re-introduce the legacy definitions
to redirect to the new ones and re-establish backwards compatibility.
The GPRS_* constants used to be defined in gprs_msgb.h, but since that header
also includes logging.h, rather place the legacy shims in logging.h next to the
other ones.
Change-Id: I455bb1bb474d758af0fd5b6397f7e57260ad739d
It is too easy for calling code to use the same filter and context indexes for
different filters and structs. For example, openbsc's IMSI filter and libgb's
GPRS_BVC filter both fall on index 1 even though there are plenty more indexes
to choose from. To alleviate this, have one central definition here, sort of
like ports.h does for VTY and CTRL port numbers.
Add static asserts to make sure the indexes fit in the available array and bit
mask space.
Calling code like openbsc.git and osmo-pcu need adjustments and/or should move
to using these enum values instead of their local definitions.
Taking this opportunity to also prepare for a split of struct gsm_subscriber in
openbsc into bsc_subsciber and vlr_subscriber with appropriate separate filter
index constants for both subscriber types.
Include previous LOG_FILTER_ALL in the LOGGING_FILTER_* enum, and replace its
use by (1 << LOGGING_FILTER_ALL).
Change-Id: I5c343630020f4b108099696fd96c2111614c8067
Eric Wild
Harald Welte
Vadim Yanitskiy
Pau Espin
Philipp Maier
Neels Hofmeyr
Alexander Couzens
Daniel Willmann
Oliver Smith
Kevin Redon