Remove the paragraph about writing to the Free Software Foundation's
mailing address. The FSF has changed addresses in the past, and may do
so again. In 2021 this is not useful, let's rather have a bit less
boilerplate at the start of source files.
Change-Id: Ifd7b7e548c9c6db140fa8cbc8a3c8dfaf4603d18
The existing code assumed that tmpdir only contained regular files,
not an entire hierarchy of files with sub-directories or the like.
In case sub-directories exist, the current code fails as follows:
line 176, in test_all_apps
remove_tmpdir(tmpdir)
File "/usr/local/lib/python3.5/dist-packages/osmopython-0.2.0-py3.5.egg/EGG-INFO/scripts/osmotestconfig.py", line 142, in remove_tmpdir
os.unlink(os.path.join(tmpdir, f))
IsADirectoryError: [Errno 21] Is a directory: 'writtenconfig/ericsson'
Change-Id: I896b99032d94ba0cdd340a8eed7c7b625661ad69
Re-apply reverted commit Iabda95073faa2191fd117e9637e0858c589e9d9e
("Drop python2 support"), but with additional changes to make the
scripts actually work with python3 and to make it build without python2.
I have verified, that the contrib/jenkins.sh scripts of all Osmocom
repositories (with their python3 patches on top) are working with this
patch and that all Osmocom repositories with the python3 patches build
in OBS (tested in own namespace).
All related patches for changing from python2 to 3 in other repositories
must be merged shortly after this one, as soon as the build slaves were
(automatically) updated to have the new osmo-python-tests installed:
https://gerrit.osmocom.org/q/topic:drop-py2
New fixes:
* osmopy/obscvty.py: verify: fix compare
Comparing maps in python3 does not work the same as in python2. Convert
them to lists first, so the compare works as intended again.
Fix error:
File "/home/user/code/osmo-dev/src/osmo-python-tests/scripts/osmotestvty.py", line 57, in test_history
assert(self.vty.w_verify(test_str, [t1]))
AssertionError
* osmopy/obscvty.py: use enc/dec with send/recv
Fix error:
self.socket.send("%s\r" % request)
TypeError: a bytes-like object is required, not 'str'
* scripts/osmotestconfig.py: use encode() before writing to file
Fix error:
File "/home/user/code/osmo-dev/src/osmo-python-tests/scripts/osmotestconfig.py", line 91, in copy_config
tmpfile.write(open(config).read())
File "/usr/lib/python3.5/tempfile.py", line 622, in func_wrapper
return func(*args, **kwargs)
TypeError: a bytes-like object is required, not 'str'
* debian/control: add --buildsystem=pybuild. Otherwise "--with python3"
is ignored and the build fails if python2 is not installed, with:
Can't exec "pyversions": No such file or directory at /usr/[...]/python_distutils.pm line 120.
Related: OS#2819
Change-Id: I3ffc3519bf6c22536a49dad7a966188ddad351a7
This reverts commit fb1dc7c405.
I was under the impression, that all previous scripts were already
working with python 3. But as it turns out, this isn't true. Reverting,
so I can properly post follow-up patches, that fix the issues before we
apply this "drop python2" patch again.
Related: OS#2819
Change-Id: Ic1559d1a9f7839fa86a841d62a04b22e1caed466
Remove all compatibility code for python2.
All scripts are already python 3 compatible since
I80e5850a8978d78cda793e2192ef4bd3fd54a121 and
I1b4a629f12863c498a8681b555f57b4e255cebfb.
dpkg-buildpackage shows that it is still invoking setup.py with python
in addition to python3, on debian stretch. But after spending quite some
time on trying to convince it to not care about python2 without success
(trying different variables, overrides, --without python2 flags etc.),
I'm leaving it as is. The resulting package is the python3 package, which
is what we want.
Related: OS#2819
Change-Id: Iabda95073faa2191fd117e9637e0858c589e9d9e
Use a loop instead. Without it the script will eventually crash with a
RecursionError.
File "/usr/bin/osmo_trap2cgi.py", line 211, in conn_client
await ctrl_client(proxy, reader, writer)
File "/usr/bin/osmo_trap2cgi.py", line 202, in ctrl_client
proxy.dispatch(wr, data)
[...]
File "/usr/bin/osmo_trap2cgi.py", line 202, in ctrl_client
proxy.dispatch(wr, data)
File "/usr/bin/osmo_trap2cgi.py", line 201, in ctrl_client
[...]
RecursionError: maximum recursion depth exceeded in comparison
Change-Id: Ic909e371771f3056cb87e18793fd4225ffb90a2c
Related: SYS#4399
* split hash generation into simpler functions
* move them into helper file
* split log init into separate function
This facilitates code sharing with new version of ctrl2cgi proxy in
folow-up patches.
Change-Id: Ic5c349349d2c246f934d6701435b1a2d9de77de1
Related: SYS#4399
Previously we've tried to determine the type of TRAP and handle as many
as possible via generic method dispatcher similar to parent class. Let's
make code simpler by explicitly ignoring all but 'location-state'
TRAPs. This should also reduce debug log output because we'll only log
the TRAP variable for ignored messages instead of entire content.
Change-Id: I42b715cb82eb309950ff387649726504b6c7cf61
Related: SYS#4399
Save and propagate separate timestamp for HTTP request duration to
distinguish between time spent waiting on connection limit availability
from actual http request time.
Change-Id: I14d1db5206864ac9ab7c64f96f67139113ab3fcf
Related: SYS#4399
osmo-mgw's configure.ac won't find osmotestvty.py, unless it is
executable. This allows passing its ./configure with placing the scripts
dir of osmo-python-tests in PATH, instead of installing it.
Change-Id: If6cefd32d0dc544093db08efc114db86b9bb18b0
Log repr() of error because it's more conscience which makes it easier to
read the logs: in case of "e" there's additional
"Failure instance: Traceback (failure with no frames)" prefix which is
pretty useless because we won't have proper tracebacks for deferred anyway.
Change-Id: I39870b83cacee9a589636974d04e356ef0d32345
Log http request timestamp and duration on debug loglevel. This is
especially helpful while troubleshooting issues with multiple concurrent
requests under significant load while network issues are present.
Change-Id: I11c8ac67a2730a9c6912694e5b83bbdf08fe357d
Related: SYS#4399
Log the basic location-state trap triggering request and command reply
as info level - that way we can receive useful information without
turning on full debug output.
Change-Id: If8eae9f36292930571954becf6613fada43d301c
When receiving commands for particular BSC, log BSC id known at the time
when request was made, not the one which is part of the reply.
Change-Id: I6acdfddb9a1132f978f2b55c769559b0c29eb3e8
Manual acquire()/release() of semaphore does not limit number of
concurrent requests when combined with explicit yield. Fix this by using
semaphore.run() and removing inilineCallbacks decorator.
Change-Id: I47b8b9f5b726ca0905bb7c023d63b325c7f7d85f
Related: SYS#4399
Previously command-line arguments without defaults took precedence over
config file variables while values from config file which had
command-line counterparts with default value were silently ignored.
Let's fix this by making config file option mandatory values and
removing overlap between command-line and config file parameters.
Change-Id: I471b5a6497eadce6456e835233fdaba88a593324
Related: SYS#4399
Since the scripts are intended to be used as systemd services, there's
no need in separate logging via syslog: systemd will take care of
properly collecting and storing script output. Hence we can drop extra
options and function parameters.
Change-Id: Ifcad1877d45d43b3a2e617775a1c9b256e190591
Related: SYS#4399
* reorder imports to make pylint3 happy
* drop unused imports
* use proper spacing for list constants
* don't use reserved names for internal variables
The check was run as follows:
pylint3 -d C0103,C0301,C0410,C0326,R0913,R0901 ...
to disable useless warnings.
Change-Id: I5b90ee790f73dc509081401776911f25e43f1801
Previously handle_reply() was marked as deferred callback unlike soap.py
where it's synchronous function. This seems to be causing issues where
some of the callbacks are not yield properly. Let's move to the
known-to-work semantics of soap.py where async functions are limited to
Trap() class.
Change-Id: Ib2c28dd7f79cbd28d475de93750703659ddd18f1
Related: SYS#4399
Sometimes when we set variable we might receive arbitrary number of TRAP
messages before we get SET_REPLY. Those could be either separate
messages or combined together with SET_REPLY depending on tcp buffering
at server side. Let's handle this gracefully by skipping over all TRAP
messages.
An example command which often triggers this behavior:
./osmo_ctrl.py -s -d 127.0.0.1 -p 4249 bts.0.location (date +%s)",fix2d,1,2,3"
Change-Id: Ia6de02c2f13a56f0381c97a9ab02c6c7a31cc32f
Related: SYS#4399
After further testing it turns out that Trap() have too many
implementation details which makes it cumbersome to be shared. Instead,
it's easier to make it into wrapper over shared functions.
Change-Id: I8a3c62bcdf4286f8127c5b6d8dee6d740aca23b9
The ctrl2cgi.py is heavily based upon soap.py - let's move all the
shared code into separate file to make further modifications easier.
Change-Id: I7b59f2dbded9074d15f2d2f40bf5a92ed02601e2
Related: SYS#4399
This scripts implements a similar proxy to the one already available in
soapy.py, but aiming at converting CTRl into a regular HTTP POST Request
to be handled by a CGI script in the HTTP server.
Furthermore, it includes several improvements over the soapy.py script,
such as using a token to authenticate users and capability to read some
parameters from a cfg file.
Change-Id: I7a83ecb379835e295c8cb4c27c39883783746544
twisted_ipa is a module expected to be used by other python scripts, so
move it to osmopy dir to be installed as a lib (in python path) so it
can be later used.
twisted_ipa has some features to use it standalone for testing purposes.
This feature can still be used from new installation path.
Change-Id: I33a0c4263297abd9e111bf43e6e9967c06474318
This simple tool dump all the rate counters available via ctrl interface
to cvs file (or stdout).
Change-Id: I12b369434e4ee9b38f92872f297e1f3d4f0698c2
Fixes: OS#2550
Use proper print() function to make scripts compatible with both python
2 and 3. This paves the way to deprecating python 2 support altogether.
Change-Id: I80e5850a8978d78cda793e2192ef4bd3fd54a121
* make parse() return command id in addition to variable name and value
* introduce parse_kv() wrapper which ignores that id and use it instead
of old parse()
* make parse() compatible with python3 where we got bytes, not string
from the socket so we have to decode it properly before using split()
* expand test_py3.py with simply asyn server which verifies that
osmo_ctrl.py works properly
Change-Id: I599f9f5a18109929f59386ab4416b8bfd75c74d1
It's a standalone script illustrating the use of ctrl protocol from
python. Since it's not used as a library and nothing depends on it, we
can safely switch to python3.
Change-Id: I2461dd9af67771beed5306116e8a1b0ee2285aa8
To allow running the scripts from within the scripts/ dir without having to
install osmopy on the system first, place a symlink to the osmopy module dir.
This allows testing modifications without having to run setup.py all the time.
Change-Id: I880c8afa8d02ef112a80cb54f649ec71902bbd4c
After I30cdf0f85b2a60a235960911c9827f4129da40db,
* the osmo_interact_{vty,ctrl}.py can no longer import osmo_interact_common,
since it was moved to scripts/ in error.
* the osmo_verify_{vty,ctrl} scripts can no longer import osmo_interact_{vty,ctrl},
since it is also in scripts/. Notably, the osmo_interact_{vty,ctrl}.py also
served as scripts while being modules at the same time, which is not good.
Fix these issues by adding a new osmopy/osmo_interact/ submodule with
osmopy/osmo_interact/common.py, /vty.py and /ctrl.py as modules, and add in
scripts thin wrappers that invoke the modules' main().
Change-Id: I40a37b212274cb70ebb1e1d9d1b3743eb2d64d05
Having the scripts in the same directory with library code means that
it'll be installed unconditionally regardless of version check in
setup.py which makes it impossible to write separate tests for python 2
and 3. Fix this by moving the scripts into separate directory and
adjusting init and setup files accordingly.
This is necessary for implementing proper CI tests in follow-up patches.
Change-Id: I30cdf0f85b2a60a235960911c9827f4129da40db
Oliver Smith
Vadim Yanitskiy
Harald Welte
Daniel Willmann
Max
Pau Espin