In many cases, a lot of the counters are zero, and we're likely
not interested in those, but only the non-zero counters. Add a version
of the 'show stats' command which dumps only those items with a non-zero
total value.
Change-Id: Ie4df1c139e3c82deca1dd3cdab5d3909e0513684
pthread_getname_np() is a non-portable extension of pthreads. While
it exists in glibc, for example musl didn't have it until rather
recently (April 2021) and there still hasn't yet been a musl release
with this change, resulting even current OpenWRT not yet supporting
pthread_getname_np.
So let's check if pthread_getname_np is supported, and only use it
in that case.
Change-Id: Ibd01485af24e2fe574006f8d049bf37226dda966
Ever since Change-Id If76a4bd2cc7b3c7adf5d84790a944d78be70e10a in 2020
(part of libosmocore >= 1.4.0) we have introduced cpu_sched_vty.c, which
directly uses libpthread. As a result, libosmovty should be using
pthread compiler flags and link against libpthread.
This missing dependency is causing osmocom applications to
fail to link on OpenWRT (at leats for ath79-generic).
Change-Id: I7febbf88cbe61eacd05f46a9316e773b5c148e77
Related: SYS#4986
osmocom applications are deployed in a variety of different situations.
Dependung on the medium that interconnects the network components
unexpected behaviour may occur. To debug problems with the
interconnection between network components it might help to monitor the
health of the related TCP connections.
Change-Id: I1416f95aff2adcf13689646b7574845de169fa3d
Related: SYS#5701
Remove the paragraph about writing to the Free Software Foundation's
mailing address. The FSF has changed addresses in the past, and may do
so again. In 2021 this is not useful, let's rather have a bit less
boilerplate at the start of source files.
Change-Id: I5050285e75cf120407a1d883e99b3c4bcae8ffd7
It's the usual naming for unit test binaries. Without the '_test' endig,
the tdef_vty_test_{config_root,config_subnode,dynamic} binaries do not
match the 'tests/*/*_test' pattern and appear as untracked files in git.
Change-Id: I828fa45132e11a41c527d4b25df850c19871cb75
So far, we used blocking, buffered fwrite() to write to stderr
and file targets. This causes problems if there are [slow] consumers
causing delays, such as gnome-terminal (when the program is started
interactively) or systemd/journald (where we observe 64..128ms blocks on
stderr).
This patch introduces stderr/file based logging via write_queue
and osmo_select_main(), i.e. switch from glibc-buffered, blocking
to internally buffered, non-blocking writes.
* when osmo_stderr_target is created via application.c, we create it
in blocking stream mode for backwards compatibility, particularly
for [smaller] programs that don't use osmo_select_main()
* when the VTY code encounters 'log stderr' or 'log file FILENAME',
we switch that respective target to non-blocking write-queue mode,
as this means the application is in fact using osmo_select_main()
* The config file can now state 'log stderr blocking-io' or
'log file FILENAME blocking-io' to explicitly enforce using blocking
stream based I/O
* The application can at any time use API functions to switch either way
Closes: OS#4311
Change-Id: Ia58fd78535c41b3da3aeb7733aadc785ace610da
Replace some with atoi(), where the VTY has already validated correct
range of the argument.
Replace others with the new osmo_str_to_int() or osmo_str_to_int64()
functions, possibly covering more detection of invalid number strings.
Leave those strtol() callers that depend on endptr to provide the next
string token.
Related: SYS#5542
Change-Id: I0ebb06e751c28f7d1cdf328de29cd227a2449391
Intead of attempting to store all distinct values of a reporting period,
just store min, max, last as well as a sum and N of each reporting
period.
This gets rid of error messages like
DLSTATS ERROR stat_item.c:285 num_bts:oml_connected: 44 stats values skipped
while at the same time more accurately reporting the max value for each
reporting period. (So far stats_item only reports the max value; keep
that part unchanged, as shown in stats_test.c.)
With the other so far unused values (min, sum), we are ready to also
report the minimum value as well as an average value per reporting
period in the future, if/when our stats reporter allows for it.
Store the complete record of the previous reporting period. So far we
only compare the 'max' value, but like this we are ready to also see
changes in min, last and average value between reporting periods.
This patch breaks API by removing:
- struct members osmo_stats_item.stats_next_id, .last_offs and .values[]
- struct osmo_stats_item_value
- osmo_stat_item_get_next()
- osmo_stat_item_discard()
- osmo_stat_item_discard_all()
and by making struct osmo_stats_item opaque.
In libosmocore, we do have a policy of never breaking API. But since the
above should never be accessed by users of the osmo_stats_item API -- or
if they are, would no longer yield useful results, we decided to make an
exception in this case. The alternative would be to introduce a new
osmo_stats_item2 API and maintaining an unused legacy osmo_stats_item
forever, but we decided that the effort is not worth it. There are no
known users of the removed items.
Related: SYS#5542
Change-Id: I137992a5479fc39bbceb6c6c2af9c227bd33b39b
vty_out_uptime() calculates the time difference to a given timespec
and print it in a human readable format (days, hours,
minutes, seconds) to the vty.
Related: OS#5028
Change-Id: I264a3f49096b96646e0a1f5366623ac20d860793
It was so far sufficient to wait for the buffers to drain at some
random point in time, but this is not always the case, sometimes it is
important that the output is flushed immediately.
Change-Id: If984b9ad2eba9f400bc29a7aa8825e241fd1d2a9
Exempt all stat_item statistics from 'stats reset'. Only reset rate_ctr
statistics to zero.
The rate_ctr statistics have an implicit time scale, counting occurences
per time unit. For them it makes sense to reset all ratings and start
from zero, for example in a test suite (e.g. our TTCN3 BSC_Tests).
In contrast, stat_item statistics count number of objects or nr of
specific object stati at any given time, and they do not deteriorate
over time. Many stat items depend on increment/decrement to be sane.
For example, in osmo-bsc, if the nr of connected BTS is 3, that does not
make sense to be reset to zero. There are still 3 BTS connected, only
the stat_item would suddenly reflect zero. From then on, it'd be wrong.
All stat_items are by definition wrong after a 'stats reset'.
- Those that depend on increment/decrement will be wrong until the
program exits, and
- those that are set to absolute values will be wrong up until the next
value is set. That could be seconds or hours later, depending.
Related: SYS#5542
Change-Id: If2134768b1076e7af189276c45f2a09a4944303e
Use ANSI escape characters to clear the screen with ^L, like it works
in typical Linux shells. I always found it slightly inconvenient that
this didn't work in the VTY.
Change-Id: Ie2356cd92f39b4dc28b5c20bbe4557fb0d972747
Kill programs with SIGTERM with the new vty command, as it would happen
with "systemctl stop".
I've considered running the select shutdown logic too in order to ensure
that pending OSMO_FD_WRITE requests are serviced. But as noted in
review, it's better to ensure that the regular SIGTERM code path works
as expected and call that instead of introducing an alternative code
path.
Closes: OS#4609
Change-Id: I3596a35b0f4a24e49484ec3f24a98e4d4f583e1e
Certain control interface commands also may require to verfy a range in
their verify function. cmd_range_match() from the VTY does exactly that
and the range can be specified as string, the same way as we would
specify it in the VTY.
Change-Id: I53fc207677f52b1dc748b01d58424839cdba807c
related: SYS#5369
Use section for all counter headings, the idea is that the generated doc
is included below a chapter describing the implemented counters which
can optionally describe individual items.
If not osmo_counters are present simply omit the whole section to tidy
up the manual. Those are deprecated anyway and not used in the osmo-*
projects that I checked.
Change-Id: I126b39cb9458e0f62efa338967e07151a0eece0f
Related: SYS#5005, OS#4523
This patch adds a new field "name" to the rate_ctr and osmo_stat_item_group
structs, together with an API to set it. This new field allows for easy
identification of specific group instances when several of them exists,
rather than using a sometimes random/increasing index value.
If set, this name (string) is used instead of the index by the stats
reporter.
The name, if set, is also printed during "show stats" VTY commands.
It's up to the user or application to set up unique or meaningful names
to fullfill one's needs.
WARNING: this commit breaks ABI and possibly creates unexpected behavior
when run with non-rebuilt apps which use the modified structs directly
to get the coutners, or if use the static inline API rate_ctr_inc2().
Existing users of these structs should migrate to use new APIs
introduced in follow-up commits instead of accessing the field directly.
Related: SYS#5456
Change-Id: I0dc510783dd9ae8436dae8005a7b3330e80d36f3
This new API doesn't use host_config_set(), and allows passing a FILE*
from any source, not only a filesystem path.
Related: SYS#5369
Change-Id: I720ac04386261628c0798a1bfcaa91e2490a86c3
The library specific sub-systems are kind of special, because their
position in the 'osmo_log_info' may vary depending on the number of
application specific sub-systems. This is why their associated
constant values (like DLGLOBAL) are negative, and this is what
the LOGP() macro expects as the first argument.
Before this change, invoking 'logp' command with any library
specific logging sub-system would result in getting messages
printed with the fall-back DLGLOBAL sub-systems.
Change-Id: If86563e169fe1243adfa7b09c9d65d9f88c8a99e
Back in 2018 in I393907b3c9e0cc1145e102328adad0a83ee13a9f Neels
introduced "last" as an optional flag to log the file/line number at the
end of the line, rather than at the end of the header. It seems nobody
has been usingi this feature, or at least never tried to save a related
config file, as there was no code whatsoever that would ever save this
optional "last" flag.
Change-Id: I7b6245256aecc425722242aaabc154ac58ba27a0
Fixes following ASan runtime errors while running vty tests:
command.c:730:27: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Somehow we didn't catch this one in Ie11ff18d6fd9f6e1e91a51b6156fb6b0b7d3a9a8
Change-Id: I601caf7daa947f3cf391316f1011007ef9188c90
Having the expert mode flag stored in the global 'host' structure
was a bad idea, because this way it applies globally. In other
words, if user Bob activates the expert mode in his dedicated
session (e.g. a telnet connection), then not only him, but all
other users would see the hidden commands in their VTYs.
Moreover, if somebody deactivates the expert mode, it would also
affect the Bob's VTY session. And finally, terminating a VTY
session would not deactivate the expert mode.
Let's move that flag from the global 'struct host' to 'struct vty'
representing an individual VTY session, so then the expert mode
would only affect the session where it was activated.
In functions related to the XML VTY reference generation we don't
have access to 'struct vty' (there may be no VTY session at all).
Add two additional arguments to vty_dump_nodes(), indicating the
global flag mask and a matching mode. This would allow to match
the VTY commands in many different ways, e.g. one can dump hidden
commands only, or all commands except the library specific ones.
Change-Id: Iba13f0949061e3dadf9cf92829d15e97074fe4ad
Related: SYS#4910
Otherwise in the expert mode, hidden commands do not get the
associated attribute printed to the XML VTY reference.
Change-Id: I8ded973031beb6d8c7ad55f06641c9651b1d9e71
Related: OS#4910
This change introduces an enumerated type 'vty_ref_gen_mode' that
(as the name suggests) defines the VTY reference generation mode:
- DEFAULT - all commands except deprecated and hidden,
- EXPERT - all commands including hidden, excluding deprecated;
and a new function vty_dump_xml_ref_mode(), that allows to specify
that mode. The old vty_dump_xml_ref() is now deprecated.
Change-Id: Ie2022a7f9e167e5ceacf15350c037dd43768ff40
Related: SYS#4910
Some VTY commands are intentionally hidden, e.g. because they might
by relatively dangerous if used in production operation. We equip
such commands with a special attribute - CMD_ATTR_HIDDEN.
The problem is that neiter they appear in the XML VTY reference,
nor in the online VTY help, so it's a bit tricky to invoke them.
This change introduces so-called 'expert' mode, in which hidden
(but not deprecated) commands are getting visible.
In the (telnet) VTY session, this mode can be activated by passing
an additional argument to well-known 'enable' command:
OsmoApp> enable ?
[expert-mode] Enable the expert mode (show hidden commands)
OsmoApp> enable expert-mode
OsmoApp#
so then hidden commands will appear together with all the other
commands. They will be marked with a special '^' flag:
OsmoApp# list with-flags
^ ... foo-hidden [expert-mode]
. ... foo-regular-one
! ... foo-immediate
^ u.. app-hidden-unbelievable
For the XML reference generation, additional API needs to be
introduced. This will be implemented in subsequent patches.
Change-Id: Ie69c2a19b22fb31d7bd7f6412f0aeac86ea5048f
Related: SYS#4910
This change implements 'systemd-journal' logging target, that is
similar to the existing 'syslog' target. The key difference is
that 'systemd-journal' allows us to offload rendering of the meta
information, such as location (file name, line number), subsystem,
and logging level, to systemd. Moreover, we can attach arbitrary,
user-specific fields [1] to the logging messages, so they can be
used for advanced log filtering (e.g. by IMSI/TMSI/TLLI):
$ journalctl OSMO_SUBSYS=DMSC -f
Since we don't want to make libsystemd a required dependency, this
feature is optional, and needs to be enabled at build-time:
$ ./configure --enable-systemd-logging
The new logging target can be configured in the same way as any
other one - via the VTY interface, or using the configuration file:
log systemd-journal [raw]
logging level set-all notice
logging filter all 1
Two logging handlers are available: generic and raw. The first one
behaves similarly to both 'syslog' and 'stderr', i.e. all the meta
information is rendered by libosmocore itself, and then passed to
systemd together with the logging message. The later is more like
the 'gsmtap' target, so all available meta information is handed
over to systemd in form of fields [1]:
- CODE_FILE / CODE_LINE - location info,
- PRIORITY - syslog-compatible logging level,
- OSMO_SUBSYS - Osmocom-specific sub-system (e.g. DMSC),
- OSMO_SUBSYS_HEX - same as OSMO_SUBSYS, but encoded in hex,
- MESSAGE - the logging message itself,
and then can be rendered in any supported format (e.g. JSON).
More details about the API can be found in [2].
[1] https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html
[2] https://www.freedesktop.org/software/systemd/man/sd-journal.html
Change-Id: I609f5cf438e6ad9038d8fc95f00add6aac29fb23
Fixes following ASan runtime errors while running vty tests:
src/vty/command.c:3088:27: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
src/vty/command.c:3136:23: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Change-Id: Ie11ff18d6fd9f6e1e91a51b6156fb6b0b7d3a9a8
Harald Welte
Vadim Yanitskiy
Philipp Maier
Oliver Smith
Pau Espin
Neels Hofmeyr
Alexander Couzens
Eric Wild
Daniel Willmann