Commit Graph

114 Commits

Author SHA1 Message Date
Pau Espin 1687d3dea3 fake_trx: Implement RFMUTE TRXC cmd
Change-Id: I67d16858cd70cb0527c1da77bd3787d5e53100b4
2020-08-26 18:04:57 +02:00
Vadim Yanitskiy 4e4babc7db trx_toolkit/data_if.py: fix: handle encoding exceptions
Change-Id: I78163d41be3a912da1dd8c0543b1c3af3a0649fa
Related: OS#4681
2020-07-27 02:58:25 +07:00
Vadim Yanitskiy 129c976c3a trx_toolkit/data_if.py: do not validate TRXD message twice
DATAMSG.gen_msg() does validete the message before encoding.

Change-Id: Ia3691b3c18778cf7a1f16c71bef5c0b2e6241190
Related: OS#4681
2020-07-27 02:25:20 +07:00
Vadim Yanitskiy d4ed09df57 Revert "trx_toolkit/transceiver.py: implement the transmit burst queue"
This reverts commit 6e1c82d298.
Unfortunately, solving one problem it introduced even more regressions.

Change-Id: If29b4f6718cbc8af18fe18a5e3eca3912e8af01e
Related: OS#4658
2020-07-22 19:59:06 +07:00
Vadim Yanitskiy 7eb355ab3e trx_toolkit/clck_gen.py: remove unused import of 'time' module
Change-Id: I40628d32409543c9f4b40b7268a4538b4671102d
2020-07-16 14:16:34 +07:00
Vadim Yanitskiy 1e20f8e042 trx_toolkit: get rid of Python2 specific workarounds
Change-Id: I16c63205c9133d964048588c25867ac7c310f951
2020-07-16 13:58:34 +07:00
Vadim Yanitskiy 49673e4073 trx_toolkit: use python3 in shebang of executable scripts
TRX Toolkit is still backwards compatible with Python2, but Python3
does much better in terms of performance.  Also, on Debian Stretch
that is used as a base for our Docker images, Python 2.7 is still
the default.  Let's require Python3 in shebang.

Change-Id: I8a1d7c59d3b5d49ec2ed94a7c77905e02134f216
2020-07-16 13:46:19 +07:00
Vadim Yanitskiy 270af48118 trx_toolkit: remove shebang from non-executable scripts
Change-Id: I5ddc531a4e98d4d6f8672d6ef14034fce605ba3d
2020-07-16 13:42:56 +07:00
Vadim Yanitskiy 6e1c82d298 trx_toolkit/transceiver.py: implement the transmit burst queue
In order to reflect the UL/DL delay caused by the premature burst
scheduling (a.k.a. 'fn-advance') in a virtual environment, the
Transceiver implementation now queues all to be transmitted bursts,
so they remain in the queue until the appropriate time of transmission.

The API user is supposed to call recv_data_msg() in order to obtain
a L12TRX message on the TRXD (data) inteface, so it gets queued by
this function.  Then, to ensure the timeous transmission, the user
of this implementation needs to call clck_tick() on each TDMA
frame.  Both functions are thread-safe (queue mutex).

In a multi-trx configuration, the use of queue additionally ensures
proper burst aggregation on multiple TRXD connections, so all L12TRX
messages are guaranteed to be sent in the right order, i.e. with
monolithically-increasing TDMA frame numbers.

Of course, this change increases the overall CPU usage, given that
each transceiver gets its own queue, and we need to serve them all
on every TDMA frame.  According to my measurements, when running
test cases from ttcn3-bts-test, the average load is ~50% higher
than what it used to be.  Still not significantly high, though.

Change-Id: Ie66ef9667dc8d156ad578ce324941a816c07c105
Related: OS#4658, OS#4546
2020-07-14 17:21:14 +07:00
Vadim Yanitskiy f262caca79 trx_toolkit/clck_gen.py: support optional clock handler
Change-Id: I85b2182d9835ed035cf370e45ea039ac6a7e8405
2020-07-14 17:21:10 +07:00
Vadim Yanitskiy 8d19fbef57 trx_toolkit/clck_gen.py: fix TDMA clock counter wrapping
Change-Id: I157447c7610402f6d62d2b74c9f04fcaa0bc1724
2020-07-14 17:20:14 +07:00
Vadim Yanitskiy 93beb3f5c5 trx_toolkit/clck_gen.py: call send_clck_ind() on every TDMA frame
Change-Id: I6d53e5266fa3b1f2eb55822d1c14975789b202ed
2020-07-14 17:17:38 +07:00
Vadim Yanitskiy e2aaeb59b3 trx_toolkit/fake_trx.py: move Rx burst handling to Transceiver
Change-Id: Ic1f44bfb21ac3173e9530a0a9966cd5e64b8bd48
2020-07-13 08:51:19 +00:00
Vadim Yanitskiy 37c81fea95 trx_toolkit/fake_trx.py: avoid using TRXList.__getitem__()
Running with cProfile shows that there are quite a lot calls:

  469896    0.254    0.000    0.254    0.000 trx_list.py:37(__getitem__)

Let's better avoid using it in performance critical parts.

Change-Id: I2bbc0a2af8218af0b9a02d8e16d4216cf602892a
2020-07-13 08:51:19 +00:00
Vadim Yanitskiy bb0155d0e7 trx_toolkit/burst_fwd.py: inherit trx list API from TRXList
Change-Id: I1c589888991add435d88517094c7b4a7db93cbae
2020-07-13 08:51:19 +00:00
Pau Espin 3ca971eb2e fake_trx: Support SETPOWER and NOMTXPOWER TRXC cmds
By default RSSI on the Rx side is computed based on transmitter's
tx power and then substracting the the Rx path loss.
If FAKE_RSSI is used, then the values in there are used instead.

A default hardcoded value of tx nominal power = 50 dBm is set to keep
old behavior of RSSI=-60dB after calculations.

Change-Id: I3ee1a32ca22c3272e66b3ca78e4f67d283844c80
2020-06-03 20:22:59 +02:00
Vadim Yanitskiy cab00398c2 trx_toolkit: cosmetic: get rid of 'i' where it is not used
Change-Id: I00126a90446e5f3fb77a46be9d7d5dbff89fa221
2020-05-22 18:48:23 +07:00
Vadim Yanitskiy 0a6e083e8a trx_toolkit/data_dump.py: fix return value of parse_msg()
Jenkins build #2516 has uncovered a problem in DATADumpFile.parse_msg():

  ======================================================================
  FAIL: test_parse_empty (test_data_dump.DATADump_Test)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/build/src/target/trx_toolkit/test_data_dump.py",
         line 138, in test_parse_empty
      self.assertEqual(msg, False)
  AssertionError: None != False

I did a quick investigation, and figured out that this failure
happens when trying to call parse_msg() with idx == 0, because
DATADumpFile._seek2msg() basically does nothing in this case
and thus always returns True. The None itself comes from
DATADumpFile._parse_msg().

Let's ensure that DATADumpFile.parse_msg() always returns None,
even if DATADumpFile._seek2msg() fails. Also, update the unit
test, so we always test a wide range of 'idx' values.

Change-Id: Ifcfa9c5208636a0f9309f5ba8e47d282dc6a03f4
2020-05-22 18:22:32 +07:00
Vadim Yanitskiy 7ec1c1ccc8 trx_toolkit/transceiver.py: add frequency hopping support
There are two ways to implement frequency hopping:

  a) The Transceiver is configured with the hopping parameters, in
     particular HSN, MAIO, and the list of ARFCNs (channels), so the
     actual Rx/Tx frequencies are changed by the Transceiver itself
     depending on the current TDMA frame number.

  b) The L1 maintains several Transceivers (two or more), so each
     instance is assigned one dedicated RF carrier frequency, and
     hence the number of available hopping frequencies is equal to
     the number of Transceivers. In this case, it's the task of
     the L1 to commutate bursts between Transceivers (frequencies).

Variant a) is commonly known as "synthesizer frequency hopping"
whereas b) is known as "baseband frequency hopping".

For the MS side, a) is preferred, because a phone usually has only
one Transceiver (per RAT). On the other hand, b) is more suitable
for the BTS side, because it's relatively easy to implement and
there is no technical limitation on the amount of Transceivers.

FakeTRX obviously does support b) since multi-TRX feature has been
implemented, as well as a) by resolving UL/DL frequencies using a
preconfigured (by the L1) set of the hopping parameters. The later
can be enabled using the SETFH control command:

  CMD SETFH <HSN> <MAIO> <RXF1> <TXF1> [... <RXFN> <TXFN>]

where <RXFN> and <TXFN> is a pair of Rx/Tx frequencies (in kHz)
corresponding to one ARFCN the Mobile Allocation. Note that the
channel list is expected to be sorted in ascending order.

NOTE: in the current implementation, mode a) applies to the whole
Transceiver and all its timeslots, so using in for the BTS side
does not make any sense (imagine BCCH hopping together with DCCH).

Change-Id: I587e4f5da67c7b7f28e010ed46b24622c31a3fdd
2020-05-17 14:36:12 +07:00
Vadim Yanitskiy 220e60184f trx_toolkit/gsm_shared.py: implement hopping sequence generation
Based on firmware/layer1/rfch.c:rfch_hop_seq_gen() by Sylvain Munaut.

Change-Id: I9ecabfef6f5a4e4180956c6a019c386ccb1c9acd
2020-05-17 07:26:07 +00:00
Vadim Yanitskiy 2db781a5d7 trx_toolkit/rand_burst_gen.py: use list comprehension
See previous commit, TL;DR this approach is significantly faster.

Change-Id: I5dc0dda89443d2763bfae50cc402724935cc91b3
2020-05-16 20:17:18 +00:00
Vadim Yanitskiy 86b621b36b trx_toolkit/data_msg.py: use list comprehension for bit conversion
This approach is much better than buf.append() in terms of performance.
Consider the following bit conversion benchmark code:

  usbits = [random.randint(0, 254) for i in range(GSM_BURST_LEN)]
  ubits = [int(b > 128) for b in usbits]

  for i in range(100000):
      sbits = DATAMSG.usbit2sbit(usbits)
      assert(DATAMSG.sbit2usbit(sbits) == usbits)

      sbits = DATAMSG.ubit2sbit(ubits)
      assert(DATAMSG.sbit2ubit(sbits) == ubits)

=== Before this patch:

 59603795 function calls (59603761 primitive calls) in 11.357 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 59200093    3.389    0.000    3.389    0.000 {method 'append' of 'list' objects}
   100000    2.212    0.000    3.062    0.000 data_msg.py:191(usbit2sbit)
   100000    1.920    0.000    2.762    0.000 data_msg.py:214(sbit2ubit)
   100000    1.835    0.000    2.677    0.000 data_msg.py:204(sbit2usbit)
   100000    1.760    0.000    2.613    0.000 data_msg.py:224(ubit2sbit)

=== After this patch:

  803794 function calls (803760 primitive calls) in 3.547 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   100000    1.284    0.000    1.284    0.000 data_msg.py:203(<listcomp>)
   100000    0.864    0.000    0.864    0.000 data_msg.py:193(<listcomp>)
   100000    0.523    0.000    0.523    0.000 data_msg.py:198(<listcomp>)
   100000    0.500    0.000    0.500    0.000 data_msg.py:208(<listcomp>)
        1    0.237    0.237    3.547    3.547 data_msg.py:25(<module>)
   100000    0.035    0.000    0.899    0.000 data_msg.py:191(usbit2sbit)
   100000    0.035    0.000    0.558    0.000 data_msg.py:196(sbit2usbit)
   100000    0.033    0.000    0.533    0.000 data_msg.py:206(ubit2sbit)
   100000    0.033    0.000    1.317    0.000 data_msg.py:201(sbit2ubit)

So the new implementation is ~70% faster in this case, and takes
significantly less function calls according to cProfile [1].

[1] https://docs.python.org/3.8/library/profile.html

Change-Id: I01c07160064c8107e5db7d913ac6dec6fc419945
2020-05-16 20:17:18 +00:00
Vadim Yanitskiy 75ae23674c trx_toolkit/app_common: add options to enable time printing
Change-Id: Ie5d14a261e17af554f7132b03d58549a4831dcdb
2020-04-09 04:44:49 +07:00
Vadim Yanitskiy ec71203e79 trx_toolkit/app_common: introduce auxiliary add_log_handler()
Change-Id: Ied32764cf1c34dc7e0f746f4f085ea20168775cb
2020-04-09 04:42:45 +07:00
Vadim Yanitskiy 930d7240bf trx_toolkit/trx_sniff.py: add options to filter bursts by RSSI
Change-Id: I16dd29d2f1e14e634029195599fa49a9be9219ab
2020-03-30 19:22:47 +07:00
Vadim Yanitskiy b1f0772002 trx_toolkit/trx_sniff.py: add option to ignore NOPE / IDLE indications
Change-Id: If51052af04289f10bfaefd5374049908de05319a
2020-03-30 19:20:32 +07:00
Vadim Yanitskiy bbd1edccae trx_toolkit/trx_sniff.py: pass the whole msg to burst_pass_filter()
Change-Id: I3da62249a4d62078b79ce5e79c86923e59c1e457
2020-03-30 19:00:45 +07:00
Vadim Yanitskiy 7a49fc31c1 trx_toolkit/data_dump.py: rewrite unit tests to use unittest framework
Change-Id: I8b934c15ba96d856aa79d10bf296d1446f043dd1
2019-12-31 17:03:12 +00:00
Vadim Yanitskiy 5cee398277 trx_toolkit/data_msg.py: rewrite unit tests to use unittest framework
Change-Id: Ia0cc7447b193a705e994078d16f3902339219916
2019-12-31 17:03:12 +00:00
Vadim Yanitskiy a15fbcbdf3 trx_toolkit/clck_gen.py: drop unneeded debug print()
Change-Id: I372af77d5b0b24fa38e304b782ca5b3d0888211b
2019-11-24 02:24:42 +07:00
Vadim Yanitskiy 6b0946ee69 trx_toolkit/clck_gen.py: turn CLCKGen's thread into a daemon
If the main thread crashes, the CLCKGen's thread would never stop.
It would also happen if the main thread terminates without calling
CLCKGen.stop().  Let's prevent this by creating a daemon thread.

Change-Id: I9d41c5baa25fa0a263758414a164c1bded25e04e
2019-11-24 02:24:42 +07:00
Vadim Yanitskiy baf07c4be2 trx_toolkit/clck_gen.py: refactor CLCKGen to use a single thread
The previous approach was based on threading.Timer, so on each clock
iteration one thread spawned another new thread.  So far it worked
well, but such frequent spawning involves an additional overhead.

After this change, CLCKGen.start() allocates and starts a new thread,
that periodically sends clock indications and sleep()s during the
indication intervals.  The CLCKGen.stop() in its turn terminates
that thread and frees the memory.

Change-Id: Ibe477eb0a1ee2193c1ff16452a407be7e858b2ef
2019-11-24 02:24:34 +07:00
Vadim Yanitskiy ae8e5ad648 trx_toolkit/fake_trx.py: send NOPE.ind in case of path loss simulation
Since TRXD header version 1, we should send NOPE indications to the
L1 side in absence of TRX2L1 bursts, and IDLE indications during
IDLE TDMA frames (basically noise measurements).

This change is the first step towards the goal: if a given burst
is to be dropped due to the path loss simulation (see FAKE_DROP),
mark the carrier TRX2L1 message as NOPE.ind and send anyway.

Change-Id: Iabd0af665e3108d23a908638f943a5b689986e2c
Related: OS#3428, OS#2975
2019-11-20 15:05:57 +00:00
Vadim Yanitskiy 7108c28c38 trx_toolkit/data_msg.py: fix: NOPE.ind also contains C/I filed
Change-Id: I281fb7387a83fec7e097ebf8650c95713d3f70e9
2019-11-20 15:05:57 +00:00
Vadim Yanitskiy b1ae186c55 trx_toolkit/fake_trx.py: refactor L12TRX -> TRX2L1 burst transformation
The burst transformation in BurstForwarder.forward_msg() used to be
done only once, so then the resulting message was distributed over
the list of connected (and active) transceivers.

This approach limits the path loss simulation capabilities, because
a reference to the same message is passed to FakeTRX.send_data_msg().
If one transceiver changes (or removes) the burst bits, the other
transceivers would not receive the original message.

Let's do the transformation individually for each transceiver,
so the original message will always remain unchanged.

Change-Id: Ia016a3a9bb6e9f17182a7168aa5a501ae9b9978b
2019-11-20 15:05:57 +00:00
Vadim Yanitskiy 2605d96720 trx_toolkit: fix: do not use 'is' / 'is not' with string and numerical literals
Since version 3.8, Python warnins us that using the "is" and "is not"
operators with string and numerical literals is a bad idea. Let's
avoid this and use the classical '==' and '!=' operators instead.

Change-Id: Iaed86d630ac1e0b9b4f72bbf3c788e325783456d
Bug description: https://bugs.python.org/issue34850
2019-11-17 17:46:48 +07:00
Martin Hauke 1f7a2ab5d3 Fix common misspellings and typos
Change-Id: I962b42871693f33b1054d43d195817e9cd84bb64
2019-10-17 08:07:39 +00:00
Vadim Yanitskiy 5afc0e4cb0 trx_toolkit/data_msg.py: enrich some ValueError messages
Change-Id: Icdf0d136a9b820cfaec534e2604204da9ea42092
2019-08-29 18:02:04 +02:00
Vadim Yanitskiy 171773b3e8 trx_toolkit/data_msg.py: also print burst length in desc_hdr()
Change-Id: Ia0036cbf70a3736a7f9779e030e05221cba8add9
2019-08-29 18:02:00 +02:00
Vadim Yanitskiy 59903772fa trx_toolkit/data_msg.py: fix: make sure header version is known
Before using DATA_MSG.HDR_LEN, we need to make sure that a parsed
header version is known and supported. Otherwise we will get an
IndexError exception.

Change-Id: Ie1887aa8709da1a2a287aa58a7873e72c0b4ed33
2019-08-29 15:09:40 +02:00
Vadim Yanitskiy df86074abe trx_toolkit/data_msg.py: fix message length check in parse_msg()
Unlike DATA_MSG.HDR_LEN, the CHDR_LEN is a constant that defines
length of the common header, which is mandatory for every version.
DATA_MSG.HDR_LEN in its turn defines length of the whole header,
including the version specific fields. Thus we need to know the
header version before using it.

In DATA_MSG.parse_msg() we need to parse the common header first,
so then we know the version and length of the whole header. After
that we can safely use DATA_MSG.HDR_LEN.

Change-Id: I2809f5f96209eed64bdabf7a15575144313f7cc9
2019-08-29 15:04:28 +02:00
Vadim Yanitskiy 5fed799df3 trx_toolkit/trx_sniff.py: also print header for incorrect messages
Change-Id: I376f1e7af46750e53305109cf4a9f64427b9960c
2019-08-27 21:13:31 +02:00
Vadim Yanitskiy 83a41a63b2 trx_toolkit/trx_sniff.py: also validate() sniffed messages
Change-Id: Ica3e4702fdabfdae0c1025b650ff32c8e85a6edf
2019-08-27 21:06:58 +02:00
Vadim Yanitskiy 78199f4f7b trx_toolkit/trx_sniff.py: print exact message parsing error
For sure, the following message is much more informative:

  Ignoring an incorrect message: Unhandled version 12

than:

  Failed to parse message, dropping...

NOTE: since the way of printing exceptions is different in both
Python versions, I had to drop Python 2 support.

Change-Id: I5fb02ce508c58ff94e47accc0ed655939eb53062
2019-08-27 21:04:47 +02:00
Vadim Yanitskiy 6af5d8da2c trx_toolkit/data_msg.py: raise exceptions from validate() methods
Raising exceptions is a Pythonic way to handle errors, which in this
particular case will help us to know *why* exactly a given message
is incorrect or incomplete.

Change-Id: Ia961f83c717066af61699c80536468392b8ce064
2019-08-27 20:30:23 +02:00
Pau Espin 2cc3392706 trx_toolkit: Fixes in TRXD documentation
Change-Id: I126a8aed6b2bac7a620e95f06ecb98642a63b5f0
2019-08-05 16:24:40 +02:00
Vadim Yanitskiy fe981d65da trx_toolkit/trx_sniff.py: support additional capture filter
Change-Id: I761debb2e1c411f2c6d489eac0adf32060966a4c
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy 710503bc4c trx_toolkit/trx_sniff.py: support sniffing on multiple ports
Change-Id: I76b818c673b98c427b5621ddb852f947c74557d6
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy e92c9cd923 trx_toolkit/trx_sniff.py: add support for reading from PCAP file
Change-Id: I8a3481fbea5f6c917ae5684d0b5b806f4a76ff78
2019-07-24 05:44:19 +07:00
Vadim Yanitskiy 7cde195e30 trx_toolkit/burst_gen.py: basic TRXD header version 1 support
Change-Id: Icd4505d211816dc80e91d65094be92f2aed856bd
2019-07-24 05:44:19 +07:00