osmocom applications are deployed in a variety of different situations.
Dependung on the medium that interconnects the network components
unexpected behaviour may occur. To debug problems with the
interconnection between network components it might help to monitor the
health of the related TCP connections.
Change-Id: I1416f95aff2adcf13689646b7574845de169fa3d
Related: SYS#5701
Remove the paragraph about writing to the Free Software Foundation's
mailing address. The FSF has changed addresses in the past, and may do
so again. In 2021 this is not useful, let's rather have a bit less
boilerplate at the start of source files.
Change-Id: I5050285e75cf120407a1d883e99b3c4bcae8ffd7
Change the functionality of skipping unchanged values: instead of
looking up whether new values have been set on a stat item, rather
remember the last reported value and skip reporting identical values.
stats_test.c shows that previously, a stat item reported a value of 10
again, even though the previous report had already sent a value of 10.
That's just because the value 10 was explicitly set again, internally.
From a perspective of preserving all data points, it could make sense to
send consecutive identical values. But since we already collapse all
data points per reporting period into a max, that is pointless.
Related: SYS#5542
Change-Id: I8f4cf34dfed17e0879716fa2cbeee137c158978b
Intead of attempting to store all distinct values of a reporting period,
just store min, max, last as well as a sum and N of each reporting
period.
This gets rid of error messages like
DLSTATS ERROR stat_item.c:285 num_bts:oml_connected: 44 stats values skipped
while at the same time more accurately reporting the max value for each
reporting period. (So far stats_item only reports the max value; keep
that part unchanged, as shown in stats_test.c.)
With the other so far unused values (min, sum), we are ready to also
report the minimum value as well as an average value per reporting
period in the future, if/when our stats reporter allows for it.
Store the complete record of the previous reporting period. So far we
only compare the 'max' value, but like this we are ready to also see
changes in min, last and average value between reporting periods.
This patch breaks API by removing:
- struct members osmo_stats_item.stats_next_id, .last_offs and .values[]
- struct osmo_stats_item_value
- osmo_stat_item_get_next()
- osmo_stat_item_discard()
- osmo_stat_item_discard_all()
and by making struct osmo_stats_item opaque.
In libosmocore, we do have a policy of never breaking API. But since the
above should never be accessed by users of the osmo_stats_item API -- or
if they are, would no longer yield useful results, we decided to make an
exception in this case. The alternative would be to introduce a new
osmo_stats_item2 API and maintaining an unused legacy osmo_stats_item
forever, but we decided that the effort is not worth it. There are no
known users of the removed items.
Related: SYS#5542
Change-Id: I137992a5479fc39bbceb6c6c2af9c227bd33b39b
Background:
* Individual values can be added to osmo_stat_item.values at any time.
* Stats are reported at a fixed interval (see vty 'stats interval'),
e.g. every 10 seconds.
* In order to report a new stat value, we use the maximum of all
osmo_stat_item.values added since the last report.
* By default, we do not send new stat values if they did not change
(see vty 'config-stats' -> 'flush-period' default of 0).
Fix the following bug:
* If 'flush-period' is 0, and no new osmo_stat_item.values are coming
in, the last value that gets reported is not necessarily the last
entry in osmo_stat_item.values.
* For attached reporters (statsd), it could then be that the given stat
stays at the wrong value for a long stretch of time (think of several
hours/days/forever).
Explanation of how the test shows that it is fixed:
* stats get reported (value is irrelevant)
* osmo_stat_item gets a new value: 20
* osmo_stat_item gets a new value: 10
* stats get reported (value: 20, the maximum of both new values)
* osmo_stat_item gets no new values
* stats get reported (value: 10, this is new because of the bug fix,
the real last value in osmo_stat_item, different from the 20 sent
earlier, without the fix it would not send anything here and the last
sent value would be 20)
* osmo_stat_item gets no new values
* stats get reported (nothing gets sent, since the real last value was
already sent and 'flush-period' is 0)
Fixes: OS#5215
Change-Id: Ibeefd0e3d1dbe4be454ff05a21df4848b2abfabe
Fix counting of values missed because of FIFO overflow in
osmo_stat_item_get_next(), by assigning a new item value id effectively
as item->value[n + 1].id = item->value[n].id + 1, instead of increasing
a global_value_id that is shared between all items and groups. With
global_value_id, the count of values missed was wrong for one item, as
soon as a new value was added to another item.
This partially reverts b27b352e ("stats: Use a global index for stat
item values") from 2015, right after stats was added to libosmocore. It
was supposed to make multiple readers (reporters) possible, which could
read independently from stat_item (and later added comments explain it
like that). But this remained unused, stats has implemented multiple
reporters by reading all stat_items once and sending the same data to
all enabled reporters. The patch caused last_value_index in struct
osmo_stat_item to always remain at -1.
Replace this unused last_value_index with stats_next_id, so stats can
store the item-specific next_id in the struct again. It appears that
stats is the only direct user of osmo_stat_item, but if there are
others, they can bring their own item-specific next_id: functions in
stat_item.c still accept a next_id argument.
Related: OS#5088
Change-Id: Ie65dcdf52c8fc3d916e20d7f0455f6223be6b64f
Let osmo_stat_item_get_next, osmo_stat_item_discard,
osmo_stat_item_discard_all consistently refer to their next_id arg as
such (and not idx or next_idx). It refers to an ID (item->values[i].id),
not an index (item->values[i]), and it is always the next one, never the
current one.
Do the same change for _index/_idx variables in stats.c, which are used
as arguments to these functions. Replace rd_ with next_id_ in
stats_test.c, too.
Related: OS#5088
Change-Id: I5dd566b08dff7174d1790f49abd2d6ac020e120e
We should never report multiple values for a metric. It is confusing for
the log reporter and wrong for statsd. Statsd will record only one value,
but will it be the first, last, ...?
This can happen if an osmo_stat_item changes more than once within the
same reporting interval.
With this patch only one aggregate value is sent to the log reporters.
The value reported is the maximum during this interval. Other
aggregations could be possible (min, last), but reporting a (useful)
average is not because the values don't include a timestamp and most
osmo_stat_items change at unregular intervals.
Change-Id: I366ab1c66f4ae6363111ea4e41b66b7d5bcade9c
Related: SYS#4877
We currently only have probes for the logging sub-system.
This patch adds two tracepoints for tracing the performance
impact of statistics reporting: stat_start and stat_done.
They can be used to trace the amount of time a libosmocore-using
application spends in reporting/exporting statistics. This includes
both the CPU time for encoding the statistics, as well as the system
calls for sending them.
Change-Id: I7208c45f6d051505dd2435305c67b4d26c0b1dd2
Related: OS#4311
Related: SYS#4877
* Allow to set the reporting interval to 0 for manual reporting
* stats reset command to reset all statistics
* stats report command to manually trigger statistics reporting
Change-Id: I9febcb65650abfd538caedfbca77a787e66d517a
Related: SYS#4877
Previously the interval between stats flushes would slowly increase
which would lead to reporting time jitter and confuse a timescale
database.
Change-Id: I23d8b5157ef8a9833ba16a81d9b28a126f303c30
Reliable monitoring requires regular flush of all stat values, even
if they have not changed. Otherwise (1) the monitoring app has to
maintain state and (2) can go out of sync if it's restarted while
the app is still running.
Change-Id: I04f1e7bdf0d6f20e4f15571e94191de61c47ddad
Let's fix some erroneous/accidential references to wrong license,
update copyright information where applicable and introduce a
SPDX-License-Identifier to all files.
Change-Id: I39af26c6aaaf5c926966391f6565fc5936be21af
With stat_item, stats.c and stats_statsd.c, it is becoming a bit
difficult to understand file naming. Also, the 'statistics.c' file
actually only contained osmo_counter handling, so let's rename it to
counter.c altogether.
Change-Id: I2cfb2310543902b7da46cb15a76e2da317eaed7d
Considering the various styles and implications found in the sources, edit
scores of files to follow the same API doc guidelines around the doxygen
grouping and the \file tag.
Many files now show a short description in the generated API doc that was so
far only available as C comment.
The guidelines and reasoning behind it is documented at
https://osmocom.org/projects/cellular-infrastructure/wiki/Guidelines_for_API_documentation
In some instances, remove file comments and add to the corresponding group
instead, to be shared among several files (e.g. bitvec).
Change-Id: Ifa70e77e90462b5eb2b0457c70fd25275910c72b
Add a new function timer function to set up the timer, similar to what
we have in the Linux kernel. This patch also converts existing opencoded
timer setup in the libosmocore tree as initial client of this new
function.
This patch implicitly removes function callback passed by reference that
defeat compile time type validation.
Compile-tested only, but I ran make check that reports success when
testing timer infrastructure.
Change-Id: I2fa49972ecaab3748b25168b26d92034e9145666
Fixes the following compiler warning:
stats_statsd.c: In function ‘osmo_stats_reporter_create_statsd’: stats_statsd.c:54:18: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
srep->send_item = osmo_stats_reporter_statsd_send_item;
Change-Id: Id36914906e0982f6ac092a311210727de66b343a
Due to prior refactoring, the functions do not have an sensible order
in the file. This commit tries to improve that a little bit.
Sponsored-by: On-Waves ehf
This commit moves the stats specific code parts into stats_statsd.c
while keeping the generic parts in stats.c. The code in stats.c no
longer contains references to statsd symbols.
Note that the VTY code still needs to know about every stats reporter
backend.
Sponsored-by: On-Waves ehf
Currently the global stat item index is being advanced in
osmo_stat_item_group_handler, so that the second and further groups
in the list will skip new item values.
This commit moves the call to osmo_stat_item_discard_all into
osmo_stats_report to a place where all groups have been processed
already.
Sponsored-by: On-Waves ehf
Currently the reporter loop is aborted, if the event would be ignored
due to an insufficient max_level. Thus the reporters that happen to
be stored afterwards would be skipped even if their level were
sufficient.
This commit replaces the conditional 'return' statement by a
considtional 'continue' statement to ensure the processing of the
list tail.
Sponsored-by: On-Waves ehf
Currently only counter changes and new item values are being
reported. This makes it cumbersome to configure reporting clients,
since there is nothing like a list of all parameters.
This commit changes this behaviour such that all currently existing
counters and items that would be reported eventually, are passed to
the reporter when it has been reconfigured or enabled. If a counter
has not been incremented, 0 is sent. If a stat item value has not
been added, the last item value (or the default value if there is
none) is resent again.
Note that this will not catch transient counters/items that will be
created later on, e.g. triggered by new peers or subscribers.
To just force this kind of dump on a running reporter, it is
sufficient to invoke the 'enable' command in its configuration node.
Sponsored-by: On-Waves ehf
Currently the having an unset prefix leads to an abort() in the
statsd reporter due to an fprintf format string error. In addition
the prefix cannot be reset to its initial state (NULL) by using 'no
prefix', which just sets the prefix to the empty string, causing a
single leading dot to appear in front of the name.
This commit changes the implemenation to consistly use NULL for
the unset name prefix ('no prefix') and to handle this case correctly
in the statsd reporter.
Sponsored-by: On-Waves ehf
we need to put the default value into inaddr and not put a
32bit value into the addr pointer.
Spotted by cppcheck:
[src/stats.c:231]: (error) Uninitialized variable: inaddr
Nothing is being done with these return values. Remove them
for now and update the TODO entry that is present at two of
the three places.
stats.c: In function ‘rate_ctr_handler’:
stats.c:570:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]
int rc;
^
stats.c: In function ‘osmo_stat_item_handler’:
stats.c:617:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]
int rc;
^
stats.c: In function ‘handle_counter’:
stats.c:651:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]
int rc;
This log level is used by the stats subsystem log reporter to report
statistics to level INFO. Note that the default level of DLSTATS is
NOTICE.
Sponsored-by: On-Waves ehf
This commit adds class_id fields to the rate_ctr and stat_item group
descriptions. The stats reporter code is extended to only process
groups whose class_id does not exceed a per reporter max_class level.
If the class_id is not set, the code assumes 'global' for groups with
idx == 0 and 'subscriber' otherwise.
The following vty command is added to config-stats:
level (global|peer|subscriber) Set the maximum group level
Sponsored-by: On-Waves ehf
Currently the unsigned group index value is silently being cast to
(signed) int in the log and statsd reporter code. If the resulting
value is negative (which can happen for instance with MMCTX
counters), the index is assumed to be unset.
This commit changes the affected types to unsigned. The index value 0
is then the only value indicating an unset group.
Sponsored-by: On-Waves ehf
Since the the stat_item and stats functions and data types are meant
to be exported, they get an osmo_ prefix.
Sponsored-by: On-Waves ehf
[hfreyther: Prepended the enum values too. This was requested by
Jacob]
This reporter passes the measurement values to the logging subsystem
as DSTATS (which is currently DLGLOBAL) level INFO messages.
Sponsored-by: On-Waves ehf
Currently case statements are used to select the right reporter
functions. This makes it difficult to add new reporter types,
especially if they are not going to reside in the same file.
This commit introduces per reporter function pointer for
open, close, send_count, and send_item. They are checked for
non-NULL before being called or skipped.
Sponsored-by: On-Waves ehf
To support reporters without network configuration, this commit
introduces the have_net_config flag to provide corresponding error
messages.
Sponsored-by: On-Waves ehf
If the MTU is given, combine several messages into a single UDP
packet until the limit is reached. Flush all reporters after the
values have been scanned.
New vty commands (node config-stats):
mtu <100-65535> Enable multi-metric packets and set the maximum
packet size (in byte)
no mtu Disable multi-metric packets
Note that single messages that are longer than the given MTU (minus
28 octets protocol overhead) will be dropped.
Sponsored-by: On-Waves ehf
This commit changes the reporting code to also show all modified
osmo_counter values. Since there is no grouping of these values, the
name string just consists of the optional prefix and the counter
name.
Sponsored-by: On-Waves ehf
Currently only rate counter are being supported.
This commit adds support for stat items. All groups are polled for
changed values.
Sponsored-by: On-Waves ehf
This calls stats_flush in regular intervals which polls the
statistical values and calls the active reporters when values have
changed.
Sponsored-by: On-Waves ehf
This commit provides the stats reporting framework that can manage
several types of measurement reporters. Initially support for
rate_ctr and the statsd protocol is included.
Sponsored-by: On-Waves ehf
Philipp Maier
Oliver Smith
Vadim Yanitskiy
Neels Hofmeyr
Daniel Willmann
Harald Welte
Alexander Chemeris