207 lines
6.5 KiB
Plaintext
207 lines
6.5 KiB
Plaintext
[[common-counters]]
|
|
== Osmocom Counters
|
|
|
|
The following gives an overview of all the types of counters available:
|
|
|
|
=== Osmo Counters (deprecated)
|
|
|
|
Osmo counters are the oldest type of counters added to Osmocom projects.
|
|
They are not grouped.
|
|
|
|
* Printed as part of VTY show stats
|
|
* Increment, Decrement
|
|
* Accessible through the control interface: counter.<counter_name>
|
|
|
|
=== Rate Counters
|
|
|
|
Rate counters count rates of events.
|
|
|
|
* Printed as part of VTY show stats
|
|
* Intervals: per second, minute, hour, day or absolute value
|
|
* Increment only
|
|
* Accessible through the control interface
|
|
* Rate counters are grouped and different instances per group can exist
|
|
|
|
The control interface command to get a counter (group) is:
|
|
|
|
rate_ctr.per_{sec,min,hour,day,abs}.<group_name>.<idx>.[counter_name]
|
|
|
|
It is possible to get all counters in a group by omitting the counter name
|
|
|
|
=== Stat Item
|
|
|
|
Stat items are a grouped replacement for osmo counters.
|
|
|
|
* Printed as part of VTY show stats
|
|
* Replacement for osmo counters
|
|
* Not yet available through the control interface
|
|
* Grouped and indexed like rate counters
|
|
* Items have a unit
|
|
* Keeps a list of the last values measured, so could return an average, min,
|
|
max, std. deviation. So far this is not implemented in any of the reporting
|
|
options.
|
|
|
|
=== Statistic Levels
|
|
|
|
There are three levels on which a statistic can be aggregated in Osmocom
|
|
projects: globally, per-peer and per-subscriber.
|
|
|
|
==== Global
|
|
|
|
These are global statistics.
|
|
|
|
==== Peer
|
|
|
|
These statistics relate to a peer the program connects to such as the NSVC in
|
|
an SGSN.
|
|
|
|
This level also includes reporting global statistics.
|
|
|
|
==== Subscriber
|
|
|
|
These statistics are related to an individual mobile subscriber. An example
|
|
would be bytes transferred in an SGSN PDP context.
|
|
|
|
This level also includes global and peer-based statistics.
|
|
|
|
=== Stats Reporter
|
|
|
|
The stats reporter periodically collects osmo counter, rate counter and
|
|
stat item values and sends them to a backend. Currently implemented are
|
|
outputting to the configured log targets and a statsd connector.
|
|
|
|
==== Configuring a stats reporter
|
|
|
|
Periodically printing the statistics to the log can be done in the following
|
|
way:
|
|
|
|
.Log statistics
|
|
====
|
|
|
|
----
|
|
OsmoBSC> enable
|
|
OsmoBSC# configure terminal
|
|
OsmoBSC(config)# stats interval 60 <1>
|
|
OsmoBSC(config)# stats reporter log <2>
|
|
OsmoBSC(config-stats)# level global <3>
|
|
OsmoBSC(config-stats)# enable <4>
|
|
----
|
|
====
|
|
|
|
<1> The interval determines how often the statistics are reported.
|
|
<2> Write the statistic information to any configured log target.
|
|
<3> Report only `global` statistics (can be `global`, `peer`, or
|
|
`subscriber`).
|
|
<4> Enable the reporter, `disable` will disable it again.
|
|
|
|
The counter values can also be sent to any aggregation/visualization tool that
|
|
understands the statsd format, for example a statsd server with graphite or
|
|
prometheus using the statsd_exporter together with grafana.
|
|
|
|
The statsd format is specified in https://github.com/b/statsd_spec
|
|
|
|
.Report statistics to statsd
|
|
====
|
|
|
|
----
|
|
OsmoBSC> enable
|
|
OsmoBSC# configure terminal
|
|
OsmoBSC(config)# stats interval 10
|
|
OsmoBSC(config)# stats reporter statsd <1>
|
|
OsmoBSC(config-stats)# prefix BSC1 <2>
|
|
OsmoBSC(config-stats)# level subscriber <3>
|
|
OsmoBSC(config-stats)# remote-ip 1.2.3.4 <4>
|
|
OsmoBSC(config-stats)# remote-port 8125 <5>
|
|
OsmoBSC(config-stats)# enable
|
|
----
|
|
====
|
|
|
|
<1> Configure the statsd reporter.
|
|
<2> Prefix the reported statistics. This is useful to distinguish statistics
|
|
from multiple instances of the same service.
|
|
<3> Report only `global` statistics or include `peer` or `subscriber`
|
|
statistics as well.
|
|
<4> IP address of the statsd server.
|
|
<5> UDP port of the statsd server. Statsd by default listens to port 8125.
|
|
|
|
You can use Netdata (https://learn.netdata.cloud/) as a statsd server which does not require special configuration
|
|
to show rate counters.
|
|
By default all the rate counters will be exposed to the StatsD plugin (listening on 127.0.0.1:8125) and displayed on the
|
|
Netdata dashboard available via: http://localhost:19999
|
|
The list of available charts which includes all the rate counters reported via statsD is available through:
|
|
http://localhost:19999/api/v1/charts
|
|
|
|
=== Socket stats
|
|
|
|
libosmocore provides features to monitor the status of TCP connections. This
|
|
can be a helpful source of information when the links between network
|
|
components are unreliable (e.g. satellite link between BTS and BSC).
|
|
|
|
NOTE: This feature is only available for certain types of TCP connections. At
|
|
the moment only RSL/OML connections between OsmoBSC and the connected BTSs can
|
|
be monitored.
|
|
|
|
==== Configuration
|
|
|
|
The gathering of the TCP connection statistics is done via syscalls. This has
|
|
to be taken into account for the configuration. Since syscalls are rather
|
|
expensive and time consuming the overall performance of the application may
|
|
suffer when many TCP connections are present. This may be the case for BSCs
|
|
with a large number of BTSs connected to it.
|
|
|
|
The statistics are gathered in batches per interval. A batch size of 5 would
|
|
mean that only 5 TCP connections per interval are evaluated and the next 5
|
|
connections in the next interval and so on.
|
|
|
|
It is recommended to choose a large reporting interval and a reasonable small
|
|
batch size to distribute the syscall load as even as possible.
|
|
|
|
.Report statistics to statsd
|
|
====
|
|
----
|
|
OsmoBSC> enable
|
|
OsmoBSC# configure terminal
|
|
stats-tcp interval 10 <1>
|
|
stats-tcp batch-size 5 <2>
|
|
----
|
|
====
|
|
|
|
<1> Set the gathering interval (sec.)
|
|
<2> Set how many TCP sockets statistics to gather per interval.
|
|
|
|
==== Generated stats items
|
|
|
|
[options="header"]
|
|
|===
|
|
| Name | Description
|
|
| tcp:unacked | unacknowledged packets.
|
|
| tcp:lost | unacknowledged packets.
|
|
| tcp:retrans | lost packets.
|
|
| tcp:rtt | retransmitted packets.
|
|
| tcp:rcv_rtt | roundtrip-time (receive).
|
|
| tcp:notsent_bytes | bytes not yet sent.
|
|
| tcp:rwnd_limited | time (usec) limited by receive window.
|
|
| tcp:sndbuf_limited | Time (usec) limited by send buffer.
|
|
| tcp:reord_seen | reordering events seen.
|
|
|===
|
|
|
|
The item group index is the file descriptor number. The item group name
|
|
consists of a static prefix (e.g. "ipa-rsl"), followed by the IP addresses
|
|
and ports of both peers.
|
|
|
|
.VTY output of a stats item group of a TCP connection
|
|
====
|
|
----
|
|
stats tcp (15)('ipa-rsl,r=10.9.1.143:38455<->l=10.9.1.162:3003'):
|
|
unacknowledged packets: 0
|
|
lost packets: 0
|
|
retransmitted packets: 0
|
|
roundtrip-time: 583
|
|
roundtrip-time (receive): 0
|
|
bytes not yet sent: 0
|
|
time (usec) limited by receive window: 0
|
|
Time (usec) limited by send buffer: 0
|
|
reordering events seen: 0
|
|
----
|
|
====
|