The osmo_escape_str_c() and osmo_quote_str_c() functions return truncated
results when characters need escaping. For example:
osmo_quote_str_c(NULL, "foo"); --> "foo"
osmo_quote_str_c(NULL, "foo\n"); --> "foo\n
osmo_quote_str_c(NULL, "foo\tbar\t\n"); --> "foo\tbar\t
Implement these _c variants using OSMO_NAME_C_IMPL() to always allocate
sufficient memory.
However, current osmo_escape_str_buf2() and osmo_quote_str_buf2() fail to
return the required buffer size (even though that information is readily
avaiable), so these don't qualify for accurate use of OSMO_NAME_C_IMPL().
Hence, move the implementations of osmo_escape_str and osmo_quote_str to an
internal static function that returns the characters needed, so that all
dynamically allocating implementations can return un-truncated results.
Of course, external callers would also benefit from escape/quote API that
accurately returns the amount of characters needed, but I am not changing
public API in this patch, on purpose, ... yet.
Change-Id: I16c08eced41bf1b7acf6e95f658068ace99ca4c8
This way we get rid of extra 128 bytes in memory per thread created.
It makes sense to share the buffer since it's same size and it doesn't
make much sense to be using both osmo_str_tolower and osmo_strtoupper at
the same time (usually you either want to move everything to uppercase
or everything to lowerase). In required scenarios, one can still use the
_buf versions.
Change-Id: I032803faa0e27c2efdff1ff276acabab95a8319a
We have a number of library-internal static global buffers which are
mainly used for various stringification functions. This worked as
all of the related Osmocom programs were strictly single-threaded.
Let's make those buffers at least thread-local. This way every thread
gets their own set of buffers, and it's safe for multiple threads to
execute the same functions once. They're of course still not
re-entrant. If you need re-entrancy, you will need to use the _c()
or _buf() suffix version of those functions and work with your own
(stack or heap) buffers.
Change-Id: I50eb2436a7c1261d79a9d2955584dce92780ca07
Calling sizeof() on a pointer to dynamically allocated memory would
result in getting size of the pointer (usually 4 or 8 bytes) itself,
but not the size of allocated memory.
Change-Id: I8ffda4dea2b7f9b4b76dfeecad1fab6384c5a62c
Fixes: CID#197629, CID#197628, CID#197627
Fixes: CID#197626, CID#197625, CID#197624
We often compose FSM instance IDs from context information, for example placing
an MSISDN string or IP:port information in the FSM instance id, using
osmo_fsm_inst_update_id_f(). This fails if any characters are contained that
don't pass osmo_identifier_valid(). Hence it is the task of the caller to make
sure only characters allowed in an FSM id are applied.
Provide API to trivially allow this by replacing illegal chars:
- osmo_identifier_sanitize_buf(), with access to the same set of illegal
characters defined in utils.c,
- osmo_fsm_inst_update_id_f_sanitize() implicitly replaces non-identifier
chars.
This makes it easy to add strings like '192.168.0.1:2342' or '+4987654321' to
an FSM instance id, without adding string mangling to each place that sets an
id; e.g. replacing with '-' to yield '192-168-0-1:2342' or '-4987654321'.
Change-Id: Ia40a6f3b2243c95fe428a080b938e11d8ab771a7
To be able to append an escaped or quoted string using
OSMO_STRBUF_APPEND_NOLEN(), the function signature must have the buf and len as
first args, like most other *_buf() functions.
Add osmo_escape_str_buf2() and osmo_quote_str_buf2() to match this signature.
A recent patch [1] has changed the return value of osmo_escape_str_buf() to
char*, removing the const. However, the functions may return const strings,
hence re-add the const. The new signatures always return the non-const buffer.
To avoid code duplication, implement osmo_quote_str_buf() and
osmo_escape_str_buf() by calling the new functions.
I decided to allow slight changes to the behavior for current osmo_escape_str()
and osmo_escape_str_buf(), because impact on callers is minimal:
(1) The new implementation uses OSMO_STRBUF_*, and in consequence
osmo_quote_str() no longer prints an ending double quote after truncated
strings; Before, a truncated output was, sic:
"this string is trunca"
and now this becomes, sic:
"this string is truncat
I decided to not keep the old behavior because it is questionable to begin
with. It looks like the string actually ended at the truncation boundary
instead of the reason being not enough space in the output buffer.
(2) The new osmo_escape_str_buf2() function obviously cannot pass-thru an
unchanged char* if no escaping was needed. Sacrifice this tiny optimization
feature to avoid code duplication:
- it is an unnoticeable optimization,
- the caller anyway always passes a string buffer,
- the feature caused handling strings and buffers differently depending on
their content (i.e. code that usually writes out strings in full length
"suddenly" truncates because a non-printable character is contained, etc.)
I considered adding a skip_if_unescaped flag to the osmo_quote_str_buf2()
function signature, but in the end decided that the API clutter is not worth
having for all the above reasons.
Adjust tests to accomodate above changes.
[1] 4a62eda225
Ibf85f79e93244f53b2684ff6f1095c5b41203e05
Change-Id: Id748b906b0083b1f1887f2be7a53cae705a8a9ae
Move from a static implementation in tdef_vty.c to utils.c, I also want to use
this in osmo-msc.
The point is that the telnet VTY allows unambiguous partly matches of keyword
args. For example, if I have a command definition of:
compare (apples|oranges)
then it is perfectly legal as for the vty parser to write only
compare app
One could expect the VTY to then pass the unambiguous match of "apples" to the
parsing function, but that is not the case.
Hence a VTY function implementation is faced with parsing a keyword of "app"
instead of the expected "apples".
This is actually a very widespread bug in our VTY implementations, which assume
that exactly one full keyword will always be found. I am now writing new
commands in a way that are able to manage only the starts of keywords.
Arguably, strstr(a, b) == a does the same thing, but it searches the entire
string unnecessarily.
Change-Id: Ib2ffb0e9a870dd52e081c7e66d8818057d159513
We have a habit of returning static buffers from some functions,
particularly when generating some kind of string values. This is
convenient in terms of memory management, but it comes at the expense
of not being thread-safe, and not allowing for two calls of the
related function within one printf() statement.
Let's introduce _c suffix versions of those functions where the
caller passes in a talloc context from which the output buffer shall
be allocated.
Change-Id: I8481c19b68ff67cfa22abb93c405ebcfcb0ab19b
We have a number of static buffers in use in libosmo*. This means
the related functions are not usable in a thread-safe way. While
we so far don't have many multi-threaded programs in the osmocom
universe, the static buffers also prevent us from calling the same
e.g. string-ify function twice within a single printf() call.
Let's make sure there's an alternative function in all those cases,
where the user can pass in a caller-allocated buffer + size, and make
the 'classic' function with the static buffer a wrapper around that
_buf() variant.
Change-Id: Ibf85f79e93244f53b2684ff6f1095c5b41203e05
osmo_escape_str_buf() used to have the somewhat odd semantics that
if no escaping was needed, it would return the original pointer without
making any copy to the output buffer. While this seems like an elegant
optimization, it is a very strange behavior and it works differently
than all of our other *_buf() functions. Let's unify the API and
turn osmo_escape_str_buf() into a strlcpy() if no escaping is needed.
Change-Id: I3a02bdb27008a73101c2db41ac04248960ed4064
get_value_string() conveniently prints the value number to a static buffer if
it is unknown in a value_string array. Do the same if the value_string array
pointer itself is NULL.
If a value string array is user supplied and might be NULL, one could add a
separate NULL check around it; but by making get_value_string() itself guard
against NULL, another static char buffer to print the value number is avoided.
Change-Id: Ie640e9258a959da8f4f9089478de993509853997
Add osmo_hexdump_buf() as an all-purpose hexdump function, which all other
osmo_hexdump_*() implementations now call. It absorbs the static
_osmo_hexdump(). Add tests for osmo_hexdump_buf().
Rationale: recently during patch review, a situation came up where two hexdumps
in a single printf would have been useful. Now I've faced a similar situation
again, in ongoing development. So I decided it is time to provide this API.
The traditional osmo_hexdump() API returns a non-const char*, which should
probably have been a const instead. Particularly this new function may return a
string constant "" if the buf is NULL or empty, so return const char*. That is
why the older implementations calling osmo_hexdump_buf() separately return the
buffer instead of the const return value directly.
Change-Id: I590595567b218b24e53c9eb1fd8736c0324d371d
Verify 14 digit and 15 digit IMEI strings. OsmoHLR will use the 14
digit version to check IMEIs before writing them to the DB.
Place the Luhn checksum code in a dedicated osmo_luhn() function, so
it can be used elsewhere.
Related: OS#2541
Change-Id: Id2d2a3a93b033bafc74c62e15297034bf4aafe61
Add a standalone bcd-to-string conversion function with generic parameters.
Add a regression test in utils_test.c.
So far there is no single universal implementation that converts a BCD to a
string. I could only find gsm48_mi_to_string(), which also interprets
surrounding bytes, MI type and TMSI as non-BCD value.
The idea is to use this function from gsm48_mi_to_string() and similar
implementations in subsequent commits.
Root cause: in osmo-msc, I want to have an alternative MI-to-string function
for composing an FSM name, which needs the BCD part of gsm48_mi_to_string() but
not the TMSI part.
Change-Id: I86b09d37ceef33331c1a56046a5443127d6c6be0
We already have osmo_str2lower() and osmo_str2upper(), but these lack:
* proper destination buffer bounds checking,
* ability to call directly as printf() argument.
Deprecate osmo_str2upper() and osmo_str2lower() because of missing bounds
checking.
Introduce osmo_str_tolower_buf(), osmo_str_toupper_buf() to provide
bounds-safe conversion, also able to safely convert a buffer in-place.
Introduce osmo_str_tolower(), osmo_str_toupper() that call the above _buf()
equivalents using a static buffer[128] and returning the resulting string
directly, convenient for direct printing. Possibly truncated but always safe.
Add unit tests to utils_test.c.
Replace all libosmocore uses of now deprecated osmo_str2lower().
Naming: the ctype.h API is called tolower() and toupper(), so just prepend
'osmo_str_' and don't separate 'to_lower'.
Change-Id: Ib0ee1206b9f31d7ba25c31f8008119ac55440797
Rationale: with osmo_escape_str(), you get the escaped contents of the string,
but not so graceful handling of NULL strings. The caller needs to quote it, and
for NULL strings not quote it.
osmo_quote_str() is like osmo_escape_str() but always quotes a non-NULL string,
and for a NULL string returns a literal NULL, i.e. it should (tm) give the
exact C representation of a string.
That's useful in testing, to show exactly what char* situation we have, without
jumping through hoops like
if (str)
printf("\"%s\"", osmo_escape_str(str, -1));
else
printf("NULL");
Copy the unit test for osmo_escape_str() and adjust. To indicate that the
double quotes are returned by osmo_quote_str(), use single quotes in the test
printf()s.
I considered allowing to pick the quoting characters by further arguments, but
that complicates things: we'd need to escape the quoting characters. Just
hardcode double quotes like C.
Change-Id: I6f1b3709b32c23fc52f70ad9ecc9439c62b02a12
To report invalid characters in identifiers, it is desirable to escape any
weird characters. Otherwise we might print stray newlines or control characters
in the log output.
ctrl_test.c already uses a print_escaped() function, which will be replaced by
osmo_escape_str() in a subsequent patch.
control_cmd.c will use osmo_escape_str() to log invalid identifiers.
Change-Id: Ic685eb63dead3967d01aaa4f1e9899e5461ca49a
For validating CTRL input, we want to verify that an input variable is a series
of valid osmo_identifier_valid() separated by dots. Allow validating any
additional chars with identifiers, for CTRL vars will be just ".".
Change-Id: I13dfd02c8c870620f937d789873ad84c6b1c45de
Some callers pass NULL and len == 0. The semantics are that we then
nul-terminate an emtpy string. Avoid a sanitizer warning by not calling
memcpy() for the NULL case.
Change-Id: I883048cf2807e606c6481634dbd569fc12aed889
Let's fix some erroneous/accidential references to wrong license,
update copyright information where applicable and introduce a
SPDX-License-Identifier to all files.
Change-Id: I39af26c6aaaf5c926966391f6565fc5936be21af
osmo_bcd2char() has always supported both decimal and hex.
However, osmo_char2bcd() use to only implement decimal digits.
With this patch, it also suppots conversion of hex characters from ASCII
to BCD.
This would be relevant in cases where somebdoy would want to use 'code
11', 'code 12' or 'ST' signals in any addresses (SCCP GT e.g.)
Change-Id: I7bbcc6de08024567ab64765c12d7de71df787a7a
We define the notion of an 'osmocom identifier' which is basically a
7-bit US-ASCII without any special characters beyond "-_:@". We
introduce a function to verify if an identifier consists only of the
permitted characters.
Change-Id: I96a8d345c5a69238a12d040f39b70c485a5c421c
utils.c: In function 'osmo_str2lower':
utils.c:277:3: warning: array subscript has type 'char' [-Wchar-subscripts]
out[i] = tolower(in[i]);
And according to man:
If c is neither an unsigned char value nor EOF, the behavior of these func‐
tions is undefined.
Change-Id: I3fed2ab6a4efba9f8a21fcf84a5b3a91e8df084f
Considering the various styles and implications found in the sources, edit
scores of files to follow the same API doc guidelines around the doxygen
grouping and the \file tag.
Many files now show a short description in the generated API doc that was so
far only available as C comment.
The guidelines and reasoning behind it is documented at
https://osmocom.org/projects/cellular-infrastructure/wiki/Guidelines_for_API_documentation
In some instances, remove file comments and add to the corresponding group
instead, to be shared among several files (e.g. bitvec).
Change-Id: Ifa70e77e90462b5eb2b0457c70fd25275910c72b
Especially for short descriptions, it is annoying to have to type \brief for
every single API doc.
Drop all \brief and enable the AUTOBRIEF feature of doxygen, which always takes
the first sentence of an API doc as the brief description.
Change-Id: I11a8a821b065a128108641a2a63fb5a2b1916e87
It's a pity that even with this patch we still are fare away from having
the whole API documented. However, at least we have a more solid
foundation. Updates not only extend the documentation, but also make
sure it is rendered properly in the doxygen HTML.
Change-Id: I1344bd1a6869fb00de7c1899a8db93bba9bafce3
This is particularly useful for hex dumps containing spaces found in a log
(e.g. osmo-nitb authentication rand token), which can now be passed in quotes
to osmo-auc-gen without having to edit the spaces away.
Change-Id: Ib7af07f674a2d26c8569acdee98835fb3e626c45
Stating that it 'truncates src' is misleading. Also clarify whether siz
includes the space needed for the terminating NUL.
Change-Id: I01c1a94408b471f7f54576178a60938bf9ee3261
I'm aware of the existing criticism on stlrcpy(), but I think it is
still better than what we have now: stnrcpy(), sometimes with Coverity
warnings and sometimes with a manual setting of the termination byte.
The implementation follows the linux kernel strlcpy() which is claimed
to be BSD compatible.
We could of course link against libbsd on Linux instead, but I think
it's reasonably small and simple to provide our own implementation.
Future versions of libosmocore could use some autoconf magic and
preprocessor macros to use the system-provided strlcpy() if it exists.
Change-Id: Ifdc99b0e3b8631f1e771e58acaf9efb00a9cd493
Add get_value_string_or_null() to return NULL in case the given value is not
found in the list of strings, to be able to cleanly fall back to another list
of strings. Absorb the lookup loop from get_value_string().
Context: in osmo-bts, I want to look up an RSL message name in rsl_msgt_names
and fall back to rsl_ipac_msgt_names if not found, because the IPAC PDCH ACT
and DEACT messages are sent in a standard ABIS_RSL_MDISC_DED_CHAN.
In a subsequent commit, get_value_string_or_null() will be used by new
rsl_or_ipac_msg_name().
Change-Id: I1fa3907e28d528d2758bc3eae9d19e6c1168f5e5
Reviewed-on: https://gerrit.osmocom.org/230
Reviewed-by: Harald Welte <laforge@gnumonks.org>
Tested-by: Jenkins Builder
We should consider simply removing the alias as we have
had several ABI changes since introducint the alias
utils.c:223:23: error: only weak aliases are supported on darwin
__attribute__((weak, alias("osmo_hexdump_nospc")));
In the osmo-bts and libosmo-abis code the hexdump routine is used
for every incoming/outgoing packet (including voice frames) and the
usage of snprintf showed up inside profiles.
There is a semantic change when more than 4096 characters are used.
The code will now truncate at byte boundaries (and not nibbles).
Code:
static const int lengths[] = { 23, 1000, 52 };
char buf[4096];
int i;
for (i = 0; i < 30000; ++i)
char *res = osmo_hexdump(buf, lengths[i & 3]);
Results:
before: after:
real 0m3.233s real 0m0.085s
user 0m3.212s user 0m0.084s
sys 0m0.000s sys 0m0.000s
Neels Hofmeyr
Pau Espin
Kevin Redon
Harald Welte
Oliver Smith
Neels Hofmeyr