The tools are so far badly under-documented. Alleviate that by comprehensive
description and examples shown by -h option output. Hint at that from the
README.
Change-Id: I94dcad257558b917cb54fc877122594cd164f496
To allow running the scripts from within the scripts/ dir without having to
install osmopy on the system first, place a symlink to the osmopy module dir.
This allows testing modifications without having to run setup.py all the time.
Change-Id: I880c8afa8d02ef112a80cb54f649ec71902bbd4c
After I30cdf0f85b2a60a235960911c9827f4129da40db,
* the osmo_interact_{vty,ctrl}.py can no longer import osmo_interact_common,
since it was moved to scripts/ in error.
* the osmo_verify_{vty,ctrl} scripts can no longer import osmo_interact_{vty,ctrl},
since it is also in scripts/. Notably, the osmo_interact_{vty,ctrl}.py also
served as scripts while being modules at the same time, which is not good.
Fix these issues by adding a new osmopy/osmo_interact/ submodule with
osmopy/osmo_interact/common.py, /vty.py and /ctrl.py as modules, and add in
scripts thin wrappers that invoke the modules' main().
Change-Id: I40a37b212274cb70ebb1e1d9d1b3743eb2d64d05
By default setup.py ignores compilation errorsand proceed with
installation anyway. Make sure that it fails if any errorsis encountered
bymanually call compilation step.
Change-Id: I2235e3fff21b7c09d3fba00ecdccca7f4515e071
Related: OS#2770
Having the scripts in the same directory with library code means that
it'll be installed unconditionally regardless of version check in
setup.py which makes it impossible to write separate tests for python 2
and 3. Fix this by moving the scripts into separate directory and
adjusting init and setup files accordingly.
This is necessary for implementing proper CI tests in follow-up patches.
Change-Id: I30cdf0f85b2a60a235960911c9827f4129da40db
* make sure jenkins.sh fails on any errors similar to other jenkins jobs
* always explicitly use python2 instead of generic python
* add basic module import tests for python 2 and 3
* add comments
Change-Id: I0f4639537d227c513859d4552533ce1e41df9deb
According to https://docs.python.org/3/library/distutils.html the
setuptools are used in place of distutils anyway. Using it directly
allows us to make packaging more flexible: specify dependencies,
automatically find package name etc.
Change-Id: I39ee53f352001e47c6df055cbec52d638480253d
Nowadays bin/python usually points to python3, and this script is written
in python2, which means if run directly from terminal it will fail with
some print syntax errors.
Change-Id: I24a568fc2c3e69e7c9f7b9729bd023fe1f0691c7
Sometimes it is desirable to match any lines, just make sure that a given regex
does *not* exist in the command reply.
For example, during handover config tests, to verify that no handover config
exists, one would have to match an entire 'show running-config' output, which
is subject to change whenever the VTY changes.
Add a wildcard '... !' which allows passing a regex that must not be
encountered, e.g.:
OsmoBSC(config-net)# show running-config
...
network
... !handover
bts 0
...
handover 1
...
This checks that there is no handover config on the network level while bts 0
does have handover config.
Change-Id: I7c1ebb2e7f059047903a53de26a0ec1ce7fa9b98
When running jenkins job, osmo_ipa is not available despite latest
osmo-python-tests being installed on the build slave. Let's try to fix
it by explicitly adding it to setup.py
Change-Id: Ic3e160c220ca6d2b55df0dfcdcdcb56e4e240bb3
Those are generic enough and can be used as advanced example of uzing
CTRL interface from python on top of osmo_ipa.
Change-Id: Ic4e1155d3bd546feaabab34a46e354c69058056e
Have common implementations in one place, and expose distinct command line
argument signatures to obtain the separate tasks as separate scripts.
osmo_interact_common.py implements the parts common to all VTY and CTRL
interactions.
osmo_interact_vty.py and osmo_interact_ctrl.py implement commands I/O but only
expose command line args to directly pipe commands and responses.
osmo_verify_transcript_vty.py and osmo_verify_transcript_ctrl.py act as before,
now implemented by importing osmo_interact_{vty,ctrl}.py, only exposing the
verifification command line arguments.
Change-Id: Ie0cbd5db85ebebc893df638a07f5568632563dc9
To retrieve the VTY reference XML from 'show online-help', which we need for
the osmo-gsm-manuals, it is useful to directly print the output of a single VTY
command.
I have also always wanted to be able to just type CTRL commands and get their
results.
Add --command and --output arguments to run specific commands and pipe to
stdout instead of verifying a transcript file, in a way that works generically
for both VTY and CTRL interface.
The scripts' names are now a bit of a misnomer, to be tweaked separately.
Change-Id: I2c31a87fa255092bcd06da239651cc50174c8faa
If one calls 'python2 setup.py install', python replaces the script shebangs
with python2 ones, regardless of the code being mint python3, and vice versa.
In setup.py, name only the scripts that match the current python version.
Change-Id: I84a5c8ec108a99f527b0d2fb5bf4e312df5f7403
While adding VTY and CTRL tests to new programs like OsmoHLR, I wanted to have
a simple way to translate a VTY interaction transcript to a VTY python test. It
is fairly trivial to simply read in a transcript, extract both the commands to
send as well as the expected results, and to verify these without having to
write one line of application-specific code. From there it was just a little
step to allow the same for CTRL interaction.
With osmo_verify_transcript_vty.py and osmo_verify_transcript_ctrl.py, it is
possible to have a simple text file of a telnet VTY or CTRL interface
interaction and run it against a given application. With the --update option,
the scripts run the given command and rewrite the transcript file to whatever
the application currently produces as response. Backed by version control, it
is super easy to tweak commands, --update the test results and verify that only
the desired bits changed. A '...' wildcard can skip any number of lines in the
expected result and is usually preserved during --update.
This python3 implementation is independent from the previous obscvty
implementations.
Take the opportunity to clarify/fix a few aspects: for example, it is now
possible to verify the hints that the interactive VTY displays when the user
enters '?' in various places, and to evaluate the prompt character '>'/'#'.
Unitl now, code is duplicated/scattered across various vty_test_runner.py
scripts in different git repositories. Now, a VTY or CTRL transcript is enough
to put a complete test in place.
The simplest invocation is directly from the Makefile, feeding an application
commandline, the proper port number to contact it and e.g. a VTY prompt name.
This new code is also usable as python modules, to be able to build more
complex tests that require specialized intermediate actions, possibly
coordinating launch of applications or data manipulation.
The first repository to employ this is osmo-hlr.git. See change-ids
I42b3b70a0439a8f2e4964d7cc31e593c1f0d7537 for VTY and
Iff93abe370b8f3ecf42082d1d0eaa1fbeca5b122 for CTRL.
Change-Id: Id47331009910e651372b9c9c76e12f2e8964cc2c
At least now that openbsc.git has been split, the IPA and Ctrl API offered by
this file are used across multiple git repositories for python tests. Instead
of having a separate copy of ipa.py in each, have one here.
Rename to osmo_ipa.py to avoid any confusion between old and new files that
might be lying around in the python path.
Change-Id: Iaef955f04b97e8f266339b37b46d2523d1247cb9
When the binary has not been built in the source tree, it was impossible to
call osmodumpdoc.py despite the -p <config-path> option. Look for config files
in that config-path.
Make sure a relative config-path is adjusted when changing to the workdir.
Change-Id: I5427d354c289ec4602411c7059c8d80e2b451f7c
/proc/self/fd refers to the process opening the file. In this case
self will be "ls". Maybe it is better to look at the FDs open in
the test runner itself.
Change-Id: I638e94f16f4019a7b6026d1fae69e677870fb253
Each test run returns 1 on error, so instead of |=, why not count the number of
errors with +=. Also print the final error count.
Change-Id: I690dde3711555a3447e5ad4cc0a04a7a869a8296
When a test run encounters an Exception, the test config file name is printed,
but that may already be gone when done, so also print the complete actual test
config file contents, making it easy to reproduce the failure manually.
Change-Id: I9b00f170fb0cdceb35b9231eb1f2c545ba079d1d
Recent commit b59b677c9b called proc.terminate()
instead of killing right away, with a .1 second sleep. Reduce this sleep to
a minuscule first wait_time, remaining tolerant for processes that take longer.
Actually all of our current processes are very fast to terminate. This patch
was created while looking for a different problem, now that it's there we might
as well keep it.
Change-Id: I98849e4550116c5666fdf6f5d4cbb576ffa3e14a
The VTYInteract tests gave a constant sleep(1) grace period for the process to
startup. This caused the test to take minutes for no reason at all.
Add code to VTYInteract._connect_socket() to try and connect right away,
retrying up to three seconds in .1 second intervals. This flies through most
tests without any sleep() at all.
When TCP socket debugging is switched on, also print how many connection tries
it took to connect the VTY socket.
Note that the openbsc python tests also add some sleep()s that also need to be
removed to benefit from this.
Change-Id: Icc337f52a93d5fe31fc4ff235ccaf4e0fe75fa39
It appears that during some error conditions, the socket will not be in the tcp
socket debug tracking list, and on top of an exception this barfs as well.
Let's not care about the tcp debug list and avoid confusing error messages.
Change-Id: I8daa317fed8fc7e720dccb70fd5f7fc74fde423f
It appears that during some error conditions, the socket will not be in the tcp
socket debug tracking list, and on top of an exception this barfs as well.
Let's not care about the tcp debug list and avoid confusing error messages.
Change-Id: Ib7b0e45fa1f5551da2fc81b71dcc227eee533f44
Print monitoring info about TCP sockets when the env var
OSMOPY_DEBUG_TCP_SOCKETS is present and not "0".
Makes it easy to enable it on jenkins without blowing up output of normal runs
"at home".
Change-Id: I46212d07a2c1be05672a37766457e6c98b68f90d
Each running test would open up another socket without ever closing unused
ones. Close the sockets after each test is done.
Change-Id: If97944a079e901439139161eae39181f5c800200
Log Popen in osmoutil.py instead of in each python test. Launch message is
being removed in openbsc's python tests and added here. (change-id
Id8eb70ddfdc1d0d9f90aa5343a4ea522042c34ee)
Change-Id: I7e539abc3aeee91ae3e7635f9e28b9e557eaf969
Add global flag debug_tcp_sockets; if true, monitors all open TCP sockets of
the system and running processes by regular 'ss' and 'ps' output, and logs
which VTY sockets are opened and closed.
Change-Id: I0cc7dc049c66f29fe58fee204e74aa9e82c21989
In the OpenBSC VTY tests it seems no one is calling
the command with close=True. So make sure the socket
is released. Modify _close_socket to be idempotent.
Change-Id: I717537772e89113ef352d32ffba1b7a840f2c6e6
When running on top of vboxsf the write option doesn't work and we
have a success that should be a failure. Check the success string
we want to see on write.
Change-Id: I460e2da10f049807e4729f53fc69448c9c30f5f9
After copytree the destination directory might end up not being writable
- this causes failure for subsequent writeconfig tests. Force rwx
permissions to fix it.
In addition to binary name it's now possible to provide extra
command-line arguments in osmoappdesc.py for application under
test. This change is fully optional and backward compatible.
This attribute in osmoappdesc.py allows to ignore certain configs while
running vty tests. It's handy for hardware-specific or otherwise special
configuration examples.