Commit Graph

2232 Commits

Author SHA1 Message Date
Max d7ee5759cb Disable doxygen in jenkins for embedded tests
There's no point in regenerating documentation with Doxygen in embedded
environment.

Change-Id: I6504db04445f5e9c1117379c8803e10df5634324
2017-10-20 18:17:42 +00:00
Max 15b05fef76 Cleanup jenkins build scripts
* reorder builds to avoid rm -rf invocation
* avoid useless double autoreconf
* move common parts into shared helper
* move common build steps into separate function

Change-Id: I24e500e132f5c8e8133d35548cb7b4e4552331d0
2017-10-20 18:17:42 +00:00
Max 32f9971a71 GPRS: print remote state info for NS peer
Change-Id: I1714c68909507eae6ef2933661a085859aca3b54
Related: SYS#3610
2017-10-20 12:27:49 +02:00
Neels Hofmeyr b525b9ea44 api doc: fix two doxygen group closing braces
Change-Id: I6c1f47d13c5b7a9cb2281de9f017df4f1ed88ba7
2017-10-17 11:53:45 +00:00
Harald Welte 37b6165513 [doc] Properly define gsm0800 group and move all related files into it
Change-Id: I91920c69c86d6a1932172becacb76faff2d3eb1e
2017-10-17 07:53:02 +02:00
Harald Welte a338983d3b [doc] Properly define 'oap' group and add introductory text
Change-Id: I1e875991ae1dd93862f850f85d40b3dac61ece72
2017-10-17 07:53:02 +02:00
Harald Welte 381a1aa1c1 [doc] Define 'gsup' group with proper name, add intro text
Change-Id: Ieee6213dc5aad082a2d439c7418b51f281b80b1a
2017-10-17 07:53:02 +02:00
Harald Welte 55d724addc [doc] make sure all SMS related code is part of the 'sms' group
Change-Id: I24c56ccb56d5b39cfb887808f91b715da54c0f8b
2017-10-17 07:53:02 +02:00
Harald Welte 8cc2767891 [doc] gea has separate group; A5 is not part of crypto but a5 group
Change-Id: I2fd24c86f9b52244073ec800a3287e3d38e660d9
2017-10-17 07:53:02 +02:00
Harald Welte c368b5420a [doc] Provide proper name for 'auth' group
Change-Id: I5fb1c34475ea9db36c2ec5cddc988074a72a35f7
2017-10-17 07:53:02 +02:00
Harald Welte 45c0ad845d [doc] Ensure include/crypt/* is part of libosmogsm documentation
Change-Id: Ic41f807dabb3b0d2d20b9d8963cad639e98f04dd
2017-10-17 05:52:04 +00:00
Harald Welte b10d214365 [doc] Ensure stats.h is parsed properly for Doxygen
stats.h uses a rather crude '#ifdef unix' hack, which means that Doxygen
will not scan the body of the file.  This patch ensures 'unix' is defined
to parse the stats.h body and thus have all relevant information in it
is parsed.

Change-Id: I65384e5cc01f2ad7783cf6b71ab78b4422902aa1
2017-10-17 05:47:09 +00:00
Harald Welte 9325d86192 [doc] TLV parser has far grown beyond GSM L3
Let's hence not call the group "GSM L3 compatible TLV parser"

Change-Id: Ic53f9c0d5ee787f994f42cf2aff313c6e131fbe6
2017-10-16 15:32:43 +02:00
Harald Welte 84bd33349c [doc] Rename "RSL" to "A-bis RSL" in documentation.
We also have "A-bis OML" and it's a bit odd to have one with prefix
and the other without.

Change-Id: Ida325f8a9c40b40c1aeaa4edfd1123ced7cc92b8
2017-10-16 15:32:06 +02:00
Harald Welte 8c64825249 [doc] Don't add libosmovty bits to libosmocore groups
Let's not put files of libosmovty into Doxygen groups of libosmocore,
as this seems to confuse Doxygen.  Also, some minor updates/fixes
of libosmovty documentation.

Change-Id: I70e612b8d06aabefe634fcd7861641ffb941d974
2017-10-16 15:17:03 +02:00
Harald Welte 7166094a85 [doc] Expand Doxuygen documentation for osmo_prim
Also, make sure prim.c is actually part of the 'prim' module.

Change-Id: I4363e46a5f606eb2114a9cc1d2086007eaa58d31
2017-10-16 14:52:37 +02:00
Harald Welte df8c57f328 [doc] Introductory text on what the logging ring buffer is about
Change-Id: I876cd31e03a53ca90e658e353c0c8a839dece8ad
2017-10-16 14:37:31 +02:00
Harald Welte efee361ce4 [doc] Make sure loggingrb.c file is in correct Doxygen group.
loggingrb.c, like loggingrb.h should be part of the loggingrb group!

Change-Id: I4822a46631b45df1afda202c00038b6ad2c8b5ae
2017-10-16 14:37:05 +02:00
Harald Welte 096a666712 [doc] Doxygen documentation on osmo_hexdump print buffer size
Change-Id: I83ca2a3a555e5b3c1b7f23bf9e463a5063a8006c
2017-10-16 14:33:11 +02:00
Harald Welte 197a4ac06d [doc] Put CRC-16 and generic CRC code in one Doxygen module
.. and add missing API documentation bits

Change-Id: I67119894bcbf8c779426a0272bae4c5ce1fbd1ed
2017-10-16 14:29:26 +02:00
Harald Welte ef7a44e33d [doc] bits.c: Better / more Doxygen documentation
Change-Id: If824a5c8d8ee6e3dc96a3fddeb105786c0c027c1
2017-10-16 14:19:12 +02:00
Harald Welte 53de0d3169 [doc] rate_ctr: Extend Doxygen API documentation with human-readable text on its use
Change-Id: If9abd46e1b0ebb6114522418fd3b45c1d802968a
2017-10-16 14:19:03 +02:00
Harald Welte f01b2383b1 [doc] src/logging.c: Fix Doxygen syntax to make logging.c show up in logging group
Change-Id: I56718e1eca1739a1b0a790f2f3a1b554899a93ab
2017-10-16 14:19:00 +02:00
Harald Welte eb5b6ce444 [doc] Add Doxygen API documentation for stats.c and stats_statsd.c
Change-Id: I8e49505f5c19beac90290fdba8821714e8eecd97
2017-10-16 14:18:56 +02:00
Harald Welte 17bbaa324b [doc] counter.[ch] Add Doxygen API documentation
This adds a more complete set of API documentation for all
osmo_counter relatedd functions and definitions.

Change-Id: I24283c05620ee86a8beb165af98a85d754549efb
2017-10-16 14:18:54 +02:00
Harald Welte 781951bcf9 [doc] stat_item: Complete doxygen documentation for API
The stat_item code base had some incomplete doxygen documentation
so far.  Let's complete it, and at the same time fix some cosmetic
as well as copy+paste issues in the existing documentation bits.

Change-Id: Ib514c137b40bf7b9791bd74be99af0b65575f2b6
2017-10-16 14:18:50 +02:00
Harald Welte 216338c369 Rename 'statistics.c' to 'counter.c'
With stat_item, stats.c and stats_statsd.c, it is becoming a bit
difficult to understand file naming.  Also, the 'statistics.c' file
actually only contained osmo_counter handling, so let's rename it to
counter.c altogether.

Change-Id: I2cfb2310543902b7da46cb15a76e2da317eaed7d
2017-10-15 19:51:35 +02:00
Neels Hofmeyr 4b7c791e15 gsm: make osmo_imsi_str_valid() NULL-safe
No callers that would pass NULL exist, but let's check against NULL from the
start.

Fixup for recent change I1e94f5b0717b947d2a7a7d36bacdf04a75cb3522.

Change-Id: I111fbf29228929f2cd6ffa06bcb1f69da223224e
2017-10-10 14:38:10 +00:00
Max f624546ed1 Fix build on older systems
Make sure GRND_NONBLOCK is always defined, even when using syscall
directly.

Change-Id: I1bcac37ee1847596b49122f9307bd2689ba71b1b
Related: OS#1694
2017-10-10 03:03:58 +00:00
Neels Hofmeyr 4335badd0e utils: add osmo_is_hexstr(), add unit test
Will be used by OsmoHLR to validate VTY and CTRL input.

Change-Id: Idf75946eb0a84e145adad13fc7c78bb7a267aa0a
2017-10-09 16:30:45 +02:00
Neels Hofmeyr 1a02e36c4c auth: add OSMO_MILENAGE_IND_BITLEN_MAX
Will be used by OsmoHLR to validate VTY and CTRL input.

Change-Id: Ic39f3404d1a49ffd06070aa9897b36f219eacf4d
2017-10-09 16:30:45 +02:00
Neels Hofmeyr 26e30b1309 auth: add value_strings for osmo_sub_auth_type, comment on osmo_auth_alg_name()
Add osmo_sub_auth_type_names[] and osmo_sub_auth_type_name().

Also add a hint to enum osmo_auth_algo's API doc that osmo_auth_alg_name()
already exists (it is defined further below).

Change-Id: I652a929bcd11c694d86812fb03d0a1cbd985efda
2017-10-09 16:30:45 +02:00
Max 4b2b0cc15d Add function to generate random identifier
The function is a wrapper on top of getrandom() (if available via glibc) or
corresponding syscall. If neither is available than failure is always
returned.

It's intended to generate small random data good enough for session
identifiers and keys. To generate long-term cryptographic keys it's
better to use special crypto libraries (like GnuTLS for example)
instead.

As an example it's used to replace old insecure random number generator
in osmo-auc-gen utility.

Change-Id: I0241b814ea4c4ce1458f7ad76e31d390383c2048
Related: OS#1694
2017-10-09 10:18:07 +00:00
Harald Welte 81dc67d03f Ensure __always_inline is defined on operating systems that don't have it
For example, Alpine Linux fails to build libosmocore without us locally
defining __always_inline.

Change-Id: I0c22aac410e668f8fb8d5de57281bcaeb343eeb1
2017-10-09 11:00:56 +08:00
Neels Hofmeyr 9cd1e7417e add osmo_imsi_str_valid() and osmo_msisdn_str_valid()
Add GSM23003_IMSI_MIN_DIGITS definition.
Add regression test gsm23003_test.c to test the two new functions.

Will be used by OsmoHLR to validate VTY and CTRL input.

Change-Id: I1e94f5b0717b947d2a7a7d36bacdf04a75cb3522
2017-10-05 19:44:28 +02:00
Harald Welte ee497f245e Fixup severe build performance issues
Make variable substitution calling bumpversion and other commands
introduce by osmo-release.mk has severely slowed down builds.

Inside the makefile we could use $(eval FOO:=$(shell...)) constructs
to have variable substitution only happen once the 'release' target
is executed. However, 'ifeq' and friends don't work with such late
constructs. Let's shift all release action into a helper shell script
that is called from the Makefile instead.

This way we get the best of both worlds: No performance impact during
normal builds, and the convenience of 'make release'.

Modified-by: Max <msuraev@sysmocom.de>

Related: OS#2524

Change-Id: I98b3b5fe3db39953cea969a9dfbb75889df2e1ea
2017-10-04 16:26:37 +02:00
Neels Hofmeyr b697df0c4a doxygen: adjust copyright to include 2017
Change-Id: If7c09a676f67da15454aedcda99d0e9b301c9945
2017-09-28 19:41:57 +02:00
Neels Hofmeyr 505c965e36 CTRL: add unit tests for CTRL command parsing
This uncovers some interesting behavior of the CTRL interface which we may want
to guard against in subsequent patches: trailing whitespace, ignored tokens,
special characters as cmd->id.

Change-Id: If7af06d50ca71fd528b08cd70310774d5a53f0f7
2017-09-27 14:04:23 +00:00
Neels Hofmeyr f4f23bd682 vty: install 'exit', 'end',... commands on *all* nodes
In many callers of the VTY API, we are lacking the vty_install_default() step
at certain node levels. This creates nodes that lack the 'exit' command, and
hence the only way to exit such a node is to restart the telnet session.

Historically, the VTY looked for missing commands on the immediate parent node,
and hence possibly found the parent's 'exit' command when the local node was
missing it. That is why we so far did not notice the missing default commands.

Furthermore, some callers call install_default() instead of
vty_install_default(). Only vty_install_default() also includes the 'exit' and
'end' commands. There is no reason why there are two sets of default commands.

To end this confusion, to catch all missing 'exit' commands and to prevent this
from re-appearing in the future, simply *always* install all default commands
implicitly when calling install_node().

In cmd_init(), there are some top-level nodes that apparently do not want the
default commands installed. Keep those the way they are, by changing the
invocation to new install_node_bare() ({VIEW,AUTH,AUTH_ENABLE}_NODE).

Make both install_default() and vty_install_default() no-ops so that users of
the API may still call them without harm. Do not yet deprecate yet, which
follows in Icf5d83f641e838cebcccc635a043e94ba352abff.

Drop all invocations to these two functions found in libosmocore.

Change-Id: I5021c64a787b63314e0f2f1cba0b8fc7bff4f09b
2017-09-27 14:04:09 +00:00
Neels Hofmeyr 607275c30b gb: drop special vty exit commands, use vty_install_default()
L_NS_NODE and L_BSSGP_NODE had specialized 'exit' and 'end' vty commands, but
all they do is return to the CONFIG and ENABLE_NODEs like the default 'exit'
and 'end' commands. Drop them and use the default 'exit' and 'end' cmds.

Examining BSSGP and NS node behavior in osmo-sgsn exhibited identical list and
exit/end behavior before and after this patch.

Prepares for an upcoming commit incorporating vty_install_default() into
install_node(), see I5021c64a787b63314e0f2f1cba0b8fc7bff4f09b: this patch
changes to the default commands, the upcoming change implies them.

Change-Id: I5b0de066b4249d482c22620d5b1bcb03f381293c
2017-09-27 14:04:08 +00:00
Neels Hofmeyr 6f4176785f vty: fix style: bssgp node vty prompt
Add trailing space and 'config-' prefix to match our common VTY node prompt
style.

Change-Id: I88db128cad9fcc6e53326b4aed5d06ea9102f328
2017-09-27 12:01:17 +00:00
Vadim Yanitskiy 463deef8c2 VTY: implement talloc context introspection command
This change introduces a new command, which could be used to
inspect the application's talloc context directly from VTY.
To enable this feature, an application need to provide it's
context via the 'vty_app_info' struct, and register the VTY
command by calling the osmo_talloc_vty_add_cmds().

The new command is a sub-command of 'show':

  show talloc-context <context> <depth> [filter]

Currently the following contexts may be inspected:

  - application - a context provided by an application;
  - null - all contexts, if NULL-context tracking is enabled.

A report depth is defined by the next parameter, and could be:

  - full - full tree report, as the talloc_report_full() does;
  - brief - brief tree report, as the talloc_report() does;
  - DEPTH - user defined maximal report depth.

Also, there are two optional report filters:

  - regexp - print only contexts, matching a regular expression;
  - tree - print a specific context, pointed by specified address.

The command output is formatted the same way as in case of calling
the talloc_report() or talloc_report_full().

Change-Id: I43fc42880b22294d83c565ae600ac65e4f38b30d
2017-09-27 11:02:19 +00:00
Vadim Yanitskiy 5584a14a31 vty/vty.c: do not bind vty context to application's one
The 'vty_app_info' struct could be used by some applications to
provide its talloc context. In the future, it will facilitate
the implementation of talloc context introspection via VTY.

But the 'vty' talloc context, that contains lots of items
(memory chunks), is being bound to an application's one,
so it becomes hard to read the last.

Let's do not bind the 'vty' context automatically, until some
common talloc context export policy is implemented.

Change-Id: I9cb6ce9f24dbae400029e2d9f9c933fbfb16248f
2017-09-27 11:02:18 +00:00
Max 383c5635e3 Add time conversion helpers
Add macros to break down absolute time in seconds into days/hours/minutes
passed.

Change-Id: I46b9a405c18ed9da8f31b7d5b6dcece5468bafbf
2017-09-25 14:41:41 +00:00
Neels Hofmeyr 657c5b6cad vty: derive node name from prompt, use as XML ids
The 'show online-help' produces XML output with <node id="..."> ids.  We
reference those from the osmo-gsm-manuals.

Instead of numeric IDs coming from internal code, rather use a human-readable
node ID -- referencing id='config-msc' is much easier than referencing id='23'.

Add a char name[] to struct cmd_node, to hold this name. This may be provided
upon struct definition.

Since callers of the VTY API so far don't have a name yet, we would need to add
names everywhere to get meaningful node IDs. There is a way to get node ID
names without touching dependent code:

My first idea was to find out which command entered the node, i.e. command
'msc' enters the MSC_NODE. But it is impossible to derive which command entered
which node from data structs, it's hidden in the vty command definition.

But in fact all (TM) known API callers indeed provide a prompt string that
contains a logical and human readable string name. Thus, if the name is unset
in the struct, parse the prompt string and strip all "weird" characters to
obtain a node name from that. We can still set names later on, but for now will
have meaningful node IDs (e.g. 'config-msc' from '%s(config-msc)# ') without
touching any dependent code.

When VTY nodes get identical node names, which is quite possible, the XML
export de-dups these by appending _2, _3,... suffixes. The first occurence is
called e.g. 'name', the second 'name_2', then 'name_3', and so forth.

If a node has no name (even after parsing the prompt), it will be named merely
by the suffix. The first empty node will become id='_1', then '_2', '_3', and
so forth. This happens for nodes like VIEW_NODE or AUTH_NODE.

If this is merged, we need to adjust the references in osmo-gsm-manuals.git.
This can happen in our own time though, because we manually create the vty
reference xml and copy it to the osmo-gsm-manuals.git and then update the
references from the vty_additions.xml. This anyway has to happen because
currently the references tend to be hopelessly out of sync anyway, placing
comments at wildly unrelated VTY commands.

Change-Id: I8fa555570268b231c5e01727c661da92fad265de
2017-09-23 12:24:58 +00:00
Neels Hofmeyr 0b723f6a6c vty: use internal node id for xml ids
The 'show online-help' produces XML output with <node id="..."> ids.  We
reference those from the osmo-gsm-manuals, but until now, these ids fall out of
sync when the amount of VTY nodes changes.

Change these ids to use the internal node ID constant (as in enum bsc_vty_node)
instead of a simple counter.

If this is merged, we need to adjust the references in osmo-gsm-manuals.git.

Change-Id: Ib07fb9d9106e19f5be6539493e82b5d5991f8bc2
2017-09-23 12:24:56 +00:00
Neels Hofmeyr b022c867e8 vty_test: add artificial node levels for better testing
In vty_test, add three levels of parent nodes (level1, level2, level3) with
each having a leaf child (child1, child2, child3).

Use these to enhance the vty_test cfg files and test more diverse situations.

The current VTY code expects a go_parent_cb() to be present, otherwise it will
bump right back to the CONFIG_NODE, which will not work with more than one node
level below the CONFIG_NODE. Hence provide a minimal go_parent_cb().

Change-Id: Ib9bcf58b655fbd85e196f363fb7d8305d7dfc997
2017-09-20 03:32:24 +02:00
Neels Hofmeyr 00b5ed3d92 comment: describe indent_cmp()
Change-Id: Ia58c16d995f6751bdd69defe8a46665aee163f3d
2017-09-20 00:57:33 +02:00
Neels Hofmeyr 430636328c fix vty regression: empty parent node
The recent exit-by-indent patch breaks a VTY case where a node is entered but
directly followed by a sibling or ancestor without listing any child nodes.
Regression introduced by I24cbb3f6de111f2d31110c3c484c066f1153aac9.

An example is a common usage in osmo-bts, where 'phy N' / 'instance N' is a
parent node that is commonly left empty:

	phy 0
	 instance 0
	bts 0
	 band 1800

Before this patch, this case produces the error:

	There is no such command.
	Error occurred during reading the below line:
	bts 0

Fix indentation parsing logic in command.c to accomodate this case.

Add a unit test for empty parent node.

Change-Id: Ia0880a17ae55accb092ae8585cc3a1bec9986891
2017-09-20 00:57:33 +02:00
Neels Hofmeyr 4a31ffa2f0 VTY: implicit node exit by de-indenting, not parent lookup
Note: This will break users' config files if they do not use consistent
indenting. (see below for a definition of "consistent".)

When reading VTY commands from a file, use indenting as means to implicitly
exit child nodes. Do not look for commands in the parent node implicitly.

The VTY so far implies 'exit' commands if a VTY line cannot be parsed on the
current node, but succeeds on the parent node. That is the mechanism by which
our VTY config files do not need 'exit' at the end of each child node.

We've hit problems with this in the following scenarios, which will show
improved user experience after this patch:

*) When both a parent and its child node have commands with identical names:

  cs7 instace 0
   point-code 1.2.3
   sccp-address osmo-msc
    point-code 0.0.1

If I put the parent's command below the child, it is still interpreted in the
context of the child node:

  cs7 instace 0
   sccp-address osmo-msc
    point-code 0.0.1
   point-code 1.2.3

Though the indenting lets me assume I am setting the cs7 instance's global PC
to 1.2.3, I'm actually overwriting osmo-msc's PC with 1.2.3 and discarding the
0.0.1.

*) When a software change moves a VTY command from a child to a parent. Say
'timezone' moved from 'bts' to 'network' level:

  network
   timezone 1 2

Say a user still has an old config file with 'timezone' on the child level:

  network
   bts 0
    timezone 1 2
    trx 0

The user would expect an error message that 'timezone' is invalid on the 'bts'
level. Instead, the VTY finds the parent node's 'timezone', steps out of 'bts'
to the 'network' level, and instead says that the 'trx' command does not exist.

Format:

Consistent means that two adjacent indenting lines have the exact
same indenting characters for the common length:

Weird mix if you ask me, but correct and consistent:

  ROOT
  <space>PARENT
  <space><tab><space>CHILD
  <space><tab><space><tab><tab>GRANDCHILD
  <space><tab><space><tab><tab>GRANDCHILD2
  <space>SIBLING

Inconsistent:

  ROOT
  <space>PARENT
  <tab><space>CHILD
  <space><space><tab>GRANDCHILD
  <space><tab><tab>GRANDCHILD2
  <tab>SIBLING

Also, when going back to a parent level, the exact same indenting must be used
as before in that node:

Incorrect:

  ROOT
  <tab>PARENT
  <tab><tab><tab>CHILD
  <tab><tab>SIBLING

As not really intended side effect, it is also permitted to indent the entire
file starting from the root level. We could guard against it but there's no
harm:

Correct and consistent:

  <tab>ROOT
  <tab><tab>PARENT
  <tab><tab><tab><tab>CHILD
  <tab><tab>SIBLING

Implementation:

Track parent nodes state: whenever a command enters a child node, push a parent
node onto an llist to remember the exact indentation characters used for that
level.

As soon as the first line on a child node is parsed, remember this new
indentation (which must have a longer strlen() than its parent level) to apply
to all remaining child siblings and grandchildren.

If the amount of spaces that indent a following VTY command are less than this
expected indentation, call vty_go_parent() until it matches up.

At any level, if the common length of indentation characters mismatch, abort
parsing in error.

Transitions to child node are spread across VTY implementations and are hard to
change. But transitions to the parent node are all handled by vty_go_parent().
By popping a parent from the list of parents in vty_go_parent(), we can also
detect that a command has changed the node without changing the parent, hence
it must have stepped into a child node, and we can push a parent frame.

The behavior on the interactive telnet VTY remains unchanged.

Change-Id: I24cbb3f6de111f2d31110c3c484c066f1153aac9
2017-09-19 01:35:30 +00:00