Introduce Android UEs as new modems

To expand the test capacities we would like to introduce
Android UEs as new modems. Currently the following tests
are supported:
- Ping
- iPerf3 DL/UL
- RRC Mobile MT Ping

In the following is a small description.

Prerequisites:
    - Android UE
        - Rooted (Ping, iPerf, RRC Idle MT Ping)
        - Qualcomm baseband with working diag_mdlog (RRC Idle MT Ping)
        - iPerf3
        - Dropbear
    - OGT Slave Unit
        - Android SDK Platform-Tools
	  (https://developer.android.com/studio/releases/platform-tools#downloads)
        - Pycrate (https://github.com/P1sec/pycrate)
        - SCAT
            clone https://github.com/bedrankara/scat/ & install dependencies
            checkout branch ogt
            symlink scat (ln -s ~/scat/scat.py /usr/local/bin/scat)

Infrastructure explaination:
The Android UEs are connected to the OGT Units via USB. We
activate tethering and set up a SSH server (with Dropbear).
We chose tethering over WiFi to have a more stable route
for the ssh connection. We forward incoming connections to
the OGT unit hosting the Android UE(s) on specific ports
to the UEs via iptables. This enables OGT to issue commands
directly to the UEs. In case of local execution we use ADB
to issue commands to the AndroidUE. The set up was tested
with 5 Android UEs connected in parallel but it should be
scalable to the number of available IPs in the respective
subnet. Furthermore, we need to cross compile Dropbear
and iPerf3 to use them on the UEs. These tools have to be
added to the $PATH variable of the UEs.

Examplary set up:
In this example we have two separate OGT units (master
and slave) and two Android UEs that are connected to the
slave unit. An illustration may be found here: https://ibb.co/6BXSP2C

On UE 1:
ip address add 192.168.42.130/24 dev rndis0
ip route add 192.168.42.0/24 dev rndis0 table local_network
dropbearmulti dropbear -F -E -p 130 -R -T /data/local/tmp/authorized_keys  -U 0 -G 0 -N root -A

On UE 2:
ip address add 192.168.42.131/24 dev rndis0
ip route add 192.168.42.0/24 dev rndis0 table local_network
dropbearmulti dropbear -F -E -p 131 -R -T /data/local/tmp/authorized_keys  -U 0 -G 0 -N root -A

On OGT slave unit:
sudo ip link add name ogt type bridge
sudo ip l set eth0 master ogt
sudo ip l set enp0s20f0u1 master ogt
sudo ip l set enp0s20f0u2 master ogt
sudo ip a a 192.168.42.1/24 dev ogt
sudo ip link set ogt up

Now we have to manually connect to every UE from OGT Master
to set up SSH keys and verify that the setup works.
Therefore, use:
ssh -p [UE-PORT] root@[OGT SLAVE UNIT's IP]

Finally, to finish the setup procedure create the
remote_run_dir for Android UEs on the slave unit like
following:
mkdir /osmo-gsm-tester-androidue
chown jenkins /osmo-gsm-tester-androidue

Example for modem in resource.conf:
- label: mi5g
  type: androidue
  imsi: '901700000034757'
  ki: '85E9E9A947B9ACBB966ED7113C7E1B8A'
  opc: '3E1C73A29B9C293DC5A763E42C061F15'
  apn:
    apn: 'srsapn'
    mcc: '901'
    mnc: '70'
    select: 'True'
    auth_algo: 'milenage'
  features: ['4g', 'dl_qam256', 'qc_diag']
  run_node:
    run_type: ssh
    run_addr: 100.113.1.170
    ssh_user: jenkins
    ssh_addr: 100.113.1.170
    ue_ssh_port: 130
    adb_serial_id: '8d3c79a7'
  scat_parser:
    run_type: local
    run_addr: 127.0.0.1
    adb_serial_id: '8d3c79a7'

Example for default-suites.conf:
- 4g:ms-label@mi5g+srsenb-rftype@uhd+mod-enb-nprb@25+mod-enb-txmode@1

Change-Id: I79a5d803e869a868d4dac5e0d4c2feb38038dc5c

src/osmo_gsm_tester/core/process.py

@@ -493,6 +493,14 @@ class RemoteNetNSProcess(RemoteProcess):
         args = ['sudo', self.NETNS_EXEC_BIN, self.netns] + list(popen_args)
         super().__init__(name, run_dir, remote_user, remote_host, remote_cwd, args, **popen_kwargs)
 
+class AdbProcess(Process):
+    def __init__(self, name, run_dir, adb_serial, popen_args, **popen_kwargs):
+        super().__init__(name, run_dir, popen_args, **popen_kwargs)
+        self.adb_serial = adb_serial
+
+        self.popen_args = ['adb', '-s', self.adb_serial, 'exec-out', 'su', '-c'] + list(popen_args)
+        self.dbg(self.popen_args, dir=self.run_dir, conf=self.popen_kwargs)
+
 def run_local_sync(run_dir, name, popen_args):
     run_dir =run_dir.new_dir(name)
     proc = Process(name, run_dir, popen_args)

src/osmo_gsm_tester/core/schema.py

@@ -111,7 +111,7 @@ def cipher(val):
     raise ValueError('Unknown Cipher value: %r' % val)
 
 def modem_feature(val):
-    if val in ('sms', 'gprs', 'voice', 'ussd', 'sim', '2g', '3g', '4g', 'dl_qam256', 'ul_qam64'):
+    if val in ('sms', 'gprs', 'voice', 'ussd', 'sim', '2g', '3g', '4g', 'dl_qam256', 'ul_qam64', 'qc_diag'):
         return True
     raise ValueError('Unknown Modem Feature: %r' % val)
 

src/osmo_gsm_tester/obj/android_apn.py

@@ -0,0 +1,210 @@
+# osmo_gsm_tester: specifics for setting an APN on an AndroidUE modem
+#
+# Copyright (C) 2020 by Software Radio Systems Limited
+#
+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com>
+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import re
+
+from ..core import log
+from ..core import schema
+from .android_host import AndroidHost
+
+
+class AndroidApn(AndroidHost):
+##############
+# PROTECTED
+##############
+    def __init__(self, apn, mcc, mnc, select=None):
+        self.logger_name = 'apn_worker_'
+        super().__init__(self.logger_name)
+        self._apn_name = apn
+        self._apn = apn
+        self._mcc = mcc
+        self._mnc = mnc
+        self._select = select
+        if not self._apn:
+            raise log.Error('APN name not set')
+        if not self._mcc:
+            raise log.Error('MCC not set')
+        if not self._mnc:
+            raise log.Error('MNC not set')
+
+        # optional parameters, set with set_additional_params()
+        self.proxy = None
+        self.port = None
+        self.user = None
+        self.password = None
+        self.server = None
+        self.mmsc = None
+        self.mmsport = None
+        self.mmsproxy = None
+        self.auth = None
+        self.type = None
+        self.protocol = None
+        self.mvnoval = None
+        self.mvnotype = None
+
+    def get_carrier_id(self, carrier_name):
+        qry_carrier_cmd = "content query --uri \"content://telephony/carriers\""
+        proc = self.run_androidue_cmd('get-carrier-id', [qry_carrier_cmd])
+        proc.launch_sync()
+        available_carriers = proc.get_stdout().split('\n')
+        carr_id = -1
+        for carr in available_carriers:
+            if 'name=' + carrier_name in carr:  # found carrier
+                carr_id = re.findall(r'_id=(\S+),', carr)[0]
+                break
+        return carr_id
+
+    def set_new_carrier(self, apn_parameter, carr_id):
+        # check if carrier was found, delete it if exists
+        if carr_id != -1:
+            self.delete_apn(apn_parameter['carrier'])
+
+        set_carrier_cmd = "content insert --uri content://telephony/carriers" \
+                          + " --bind name:s:\"" + apn_parameter["carrier"] + "\"" \
+                          + " --bind numeric:s:\"" + apn_parameter["mcc"] + apn_parameter["mnc"] + "\"" \
+                          + " --bind mcc:s:\"" + apn_parameter["mcc"] + "\"" \
+                          + " --bind mnc:s:\"" + apn_parameter["mnc"] + "\""\
+                          + " --bind apn:s:\"" + apn_parameter["apn"] + "\"" \
+                          + " --bind user:s:\"" + apn_parameter["user"] + "\"" \
+                          + " --bind password:s:\"" + apn_parameter["password"] + "\"" \
+                          + " --bind mmsc:s:\"" + apn_parameter["mmsc"] + "\"" \
+                          + " --bind mmsport:s:\"" + apn_parameter["mmsport"] + "\"" \
+                          + " --bind mmsproxy:s:\"" + apn_parameter["mmsproxy"] + "\"" \
+                          + " --bind authtype:s:\"" + apn_parameter["auth"] + "\"" \
+                          + " --bind type:s:\"" + apn_parameter["type"] + "\"" \
+                          + " --bind protocol:s:\"" + apn_parameter["protocol"] + "\"" \
+                          + " --bind mvno_type:s:\"" + apn_parameter["mvnotype"] + "\"" \
+                          + " --bind mvno_match_data:s:\"" + apn_parameter["mvnoval"] + "\""
+        proc = self.run_androidue_cmd("set-new-carrier", [set_carrier_cmd])
+        proc.launch_sync()
+        return self.get_carrier_id(apn_parameter['carrier'])
+
+    def set_preferred_apn(self, carr_id):
+        if carr_id != -1:
+            set_apn_cmd = "content insert --uri content://telephony/carriers/preferapn --bind apn_id:s:\"" + str(carr_id) + "\""
+            proc = self.run_androidue_cmd('set-preferred-apn', [set_apn_cmd])
+            proc.launch_sync()
+
+    def select_apn(self, carr_name):
+        carr_id = self.get_carrier_id(carr_name)
+        if carr_id == 0:
+            return False
+
+        # select carrier by ID
+        sel_apn_cmd = "content update --uri content://telephony/carriers/preferapn --bind apn_id:s:\"" + str(carr_id) + "\""
+        proc = self.run_androidue_cmd('select-apn', [sel_apn_cmd])
+        proc.launch_sync()
+        return True
+
+    def delete_apn(self, carr_name):
+        set_apn_cmd = "content delete --uri content://telephony/carriers --where \'name=\"" + str(carr_name) + "\" \'"
+        proc = self.run_androidue_cmd('delete-apn', [set_apn_cmd])
+        proc.launch_sync()
+
+########################
+# PUBLIC - INTERNAL API
+########################
+    @classmethod
+    def from_conf(cls, conf):
+        return cls(conf.get('apn', None), conf.get('mcc', None),
+                   conf.get('mnc', None), conf.get('select', None))
+
+    @classmethod
+    def schema(cls):
+        resource_schema = {
+            'apn': schema.STR,
+            'mcc': schema.STR,
+            'mnc': schema.STR,
+            'select': schema.BOOL_STR,
+            }
+        return resource_schema
+
+    def configure(self, testenv, run_dir, run_node, rem_host):
+        self.testenv = testenv
+        self.rem_host = rem_host
+        self._run_node = run_node
+        self.run_dir = run_dir
+        self.logger_name += self._run_node.run_addr()
+        self.set_name(self.logger_name)
+
+    def set_additional_params(self, proxy=None, port=None, user=None, password=None, server=None, auth=None, apn_type=None,
+                              mmsc=None, mmsport=None, mmsproxy=None,  protocol=None, mvnoval=None, mvnotype=None):
+        self.proxy = proxy
+        self.port = port
+        self.user = user
+        self.password = password
+        self.server = server
+        self.auth = auth
+        self.type = apn_type
+        self.mmsc = mmsc
+        self.mmsport = mmsport
+        self.mmsproxy = mmsproxy
+        self.protocol = protocol
+        self.mvnoval = mvnoval
+        self.mvnotype = mvnotype
+
+    def set_apn(self):
+        apn_params = {
+            'carrier': self._apn_name,
+            'apn': self._apn,
+            'proxy': self.proxy or '',
+            'port': self.port or '',
+            'user': self.user or '',
+            'password': self.password or '',
+            'server': self.server or '',
+            'mmsc': self.mmsc or '',
+            'mmsport': self.mmsport or '',
+            'mmsproxy': self.mmsproxy or '',
+            'mcc': self._mcc,
+            'mnc': self._mnc,
+            'auth': self.auth or '-1',
+            'type': self.type or 'default',
+            'protocol': self.protocol or '',
+            'mvnotype': self.mvnotype or '',
+            'mvnoval': self.mvnoval or '',
+        }
+        self.dbg('APN parameters: ' + str(apn_params))
+
+        # search for carrier in database
+        carrier_id = self.get_carrier_id(apn_params['carrier'])
+
+        # add/update carrier
+        carrier_id = self.set_new_carrier(apn_params, carrier_id)
+
+        # select as preferred APN
+        if self.select:
+            self.set_preferred_apn(carrier_id)
+
+    def __str__(self):
+        return self.name()
+
+    def apn(self):
+        return self._apn
+
+    def mcc(self):
+        return self._mcc
+
+    def mnc(self):
+        return self._mnc
+
+    def select(self):
+        return self._select
+
+# vim: expandtab tabstop=4 shiftwidth=4

src/osmo_gsm_tester/obj/android_bitrate_monitor.py

@@ -0,0 +1,118 @@
+# osmo_gsm_tester: specifics for monitoring the bit rate of an AndroidUE modem
+#
+# Copyright (C) 2020 by Software Radio Systems Limited
+#
+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com>
+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from ..core import log
+from .android_host import AndroidHost
+
+
+class BitRateMonitor(AndroidHost):
+
+##############
+# PROTECTED
+##############
+    def __init__(self, testenv, run_dir, run_node, rem_host, data_interface):
+        super().__init__('brate_monitor_%s' % run_node.run_addr())
+        self.testenv = testenv
+        self.rem_host = rem_host
+        self._run_node = run_node
+        self.run_dir = run_dir
+        self.data_interface = data_interface
+        self.rx_monitor_proc = None
+        self.tx_monitor_proc = None
+
+########################
+# PUBLIC - INTERNAL API
+########################
+    def start(self):
+        # start bit rate monitoring on Android UE
+        popen_args_rx_mon = ['while true; do cat /sys/class/net/' + self.data_interface + '/statistics/rx_bytes;',
+                             'sleep 1;', 'done']
+        popen_args_tx_mon = ['while true; do cat /sys/class/net/' + self.data_interface + '/statistics/tx_bytes;',
+                             'sleep 1;', 'done']
+        self.rx_monitor_proc = self.run_androidue_cmd('start-rx-monitor', popen_args_rx_mon)
+        self.testenv.remember_to_stop(self.rx_monitor_proc)
+        self.rx_monitor_proc.launch()
+        self.tx_monitor_proc = self.run_androidue_cmd('start-tx-monitor', popen_args_tx_mon)
+        self.testenv.remember_to_stop(self.tx_monitor_proc)
+        self.tx_monitor_proc.launch()
+
+    def stop(self):
+        self.testenv.stop_process(self.rx_monitor_proc)
+        self.testenv.stop_process(self.tx_monitor_proc)
+
+    def save_metrics(self, metrics_file):
+        brate_rx_raw = self.rx_monitor_proc.get_stdout().split('\n')
+        brate_tx_raw = self.tx_monitor_proc.get_stdout().split('\n')
+        brate_rx_raw.remove('')
+        brate_tx_raw.remove('')
+        brate_rx_l = brate_rx_raw[1:]
+        brate_tx_l = brate_tx_raw[1:]
+
+        if len(brate_rx_l) < 2 or len(brate_tx_l) < 2:
+            raise log.Error('Insufficient data available to write metrics file')
+
+        # cut of elements if lists don't have the same length
+        if len(brate_rx_l) > len(brate_tx_l):
+            brate_rx_l = brate_rx_l[:len(brate_tx_l) - len(brate_rx_l)]
+        if len(brate_rx_l) < len(brate_tx_l):
+            brate_tx_l = brate_tx_l[:len(brate_rx_l) - len(brate_tx_l)]
+
+        # get start value
+        brate_rx_last = int(brate_rx_l[0])
+        brate_tx_last = int(brate_tx_l[0])
+
+        with open(metrics_file, 'w') as ue_metrics_fh:
+            ue_metrics_fh.write('time;cc;earfcn;pci;rsrp;pl;cfo;pci_neigh;rsrp_neigh;cfo_neigh;'
+                                + 'dl_mcs;dl_snr;dl_turbo;dl_brate;dl_bler;'
+                                + 'ul_ta;ul_mcs;ul_buff;ul_brate;ul_bler;rf_o;rf_u;rf_l;'
+                                + 'is_attached\n')
+            for i in range(1, len(brate_rx_l)):
+                time = '0'
+                cc = '0'
+                earfcn = '0'
+                pci = '0'
+                rsrp = '0'
+                pl = '0'
+                cfo = '0'
+                pci_neigh = '0'
+                rsrp_neigh = '0'
+                cfo_neigh = '0'
+                dl_mcs = '0'
+                dl_snr = '0'
+                dl_turbo = '0'
+                dl_brate = str((int(brate_rx_l[i]) - brate_rx_last) * 8)
+                brate_rx_last = int(brate_rx_l[i])
+                dl_bler = '0'
+                ul_ta = '0'
+                ul_mcs = '0'
+                ul_buff = '0'
+                ul_brate = str((int(brate_tx_l[i]) - brate_tx_last) * 8)
+                brate_tx_last = int(brate_tx_l[i])
+                ul_bler = '0'
+                rf_o = '0'
+                rf_u = '0'
+                rf_l = '0'
+                is_attached = '0'
+
+                line = time + ';' + cc + ';' + earfcn + ';' + pci + ';' + rsrp + ';' + pl + ';' + cfo + ';' \
+                       + pci_neigh + ';' + rsrp_neigh + ';' + cfo_neigh + ';' + dl_mcs + ';' + dl_snr + ';' \
+                       + dl_turbo + ';' + dl_brate + ';' + dl_bler + ';' + ul_ta + ';' + ul_mcs + ';' + ul_buff + ';' \
+                       + ul_brate + ';' + ul_bler + ';' + rf_o + ';' + rf_u + ';' + rf_l + ';' + is_attached
+                ue_metrics_fh.write(line + '\n')

src/osmo_gsm_tester/obj/android_host.py

@@ -0,0 +1,47 @@
+# osmo_gsm_tester: Base class for AndroidUE modems
+#
+# Copyright (C) 2020 by Software Radio Systems Limited
+#
+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com>
+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from ..core import log, process
+from abc import ABCMeta
+
+
+class AndroidHost(log.Origin, metaclass=ABCMeta):
+    """Base for everything AndroidUE related."""
+
+##############
+# PROTECTED
+##############
+    def __init__(self, name):
+        log.Origin.__init__(self, log.C_TST, name)
+
+########################
+# PUBLIC - INTERNAL API
+########################
+    def run_androidue_cmd(self, name, popen_args):
+        # This function executes the given command directly on the Android UE. Therefore,
+        # ADB is used to execute commands locally and ssh for remote execution. Make sure
+        # Android SDK Platform-Tools >= 23 is installed
+        if self._run_node.is_local():
+            # use adb instead of ssh
+            run_dir = self.run_dir.new_dir(name)
+            proc = process.AdbProcess(name, run_dir, self._run_node.adb_serial_id(), popen_args, env={})
+        else:
+            proc = self.rem_host.RemoteProcess(name, popen_args, remote_env={})
+        return proc

src/osmo_gsm_tester/obj/iperf3.py

@@ -227,6 +227,7 @@ class IPerf3Client(log.Origin):
         self.remote_log_file = None
         self.log_copied = False
         self.logfile_supported = False # some older versions of iperf doesn't support --logfile arg
+        self.is_android_ue = False
 
     def runs_locally(self):
         locally = not self._run_node or self._run_node.is_local()
@@ -281,7 +282,8 @@ class IPerf3Client(log.Origin):
         return proc
 
     def prepare_test_proc_remotely(self, netns, popen_args):
-        self.rem_host = remote.RemoteHost(self.run_dir, self._run_node.ssh_user(), self._run_node.ssh_addr())
+        self.rem_host = remote.RemoteHost(self.run_dir, self._run_node.ssh_user(), self._run_node.ssh_addr(), None,
+                                          self._run_node.ssh_port())
 
         remote_prefix_dir = util.Dir(IPerf3Client.REMOTE_DIR)
         remote_run_dir = util.Dir(remote_prefix_dir.child('cli-' + str(self)))
@@ -307,6 +309,8 @@ class IPerf3Client(log.Origin):
 
         if netns:
             self.process = process.NetNSProcess(self.name(), self.run_dir, netns, popen_args, env={})
+        elif self._run_node.adb_serial_id():
+            self.process = process.AdbProcess(self.name(), self.run_dir, self._run_node.adb_serial_id(), popen_args, env={})
         else:
             self.process = process.Process(self.name(), self.run_dir, popen_args, env={})
         return self.process

src/osmo_gsm_tester/obj/ms.py

@@ -48,7 +48,7 @@ class MS(log.Origin, metaclass=ABCMeta):
 # PROTECTED
 ##############
     def __init__(self, name, testenv, conf):
-        super().__init__(log.C_TST, name)
+        log.Origin.__init__(self, log.C_TST, name)
         self.testenv = testenv
         self._conf = conf
         self._msisdn = None
@@ -77,6 +77,9 @@ class MS(log.Origin, metaclass=ABCMeta):
         elif ms_type == 'srsue':
             from .ms_srs import srsUE
             ms_class = srsUE
+        elif ms_type == 'androidue':
+            from .ms_android import AndroidUE
+            ms_class = AndroidUE
         elif ms_type == 'amarisoftue':
             from .ms_amarisoft import AmarisoftUE
             ms_class = AmarisoftUE

src/osmo_gsm_tester/obj/ms_android.py

@@ -0,0 +1,244 @@
+# osmo_gsm_tester: specifics for running an AndroidUE modem
+#
+# Copyright (C) 2020 by Software Radio Systems Limited
+#
+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com>
+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import pprint
+
+from ..core import log, util, config, remote, schema, process
+from .run_node import RunNode
+from .ms import MS
+from .srslte_common import srslte_common
+from ..core.event_loop import MainLoop
+from .ms_srs import srsUEMetrics
+from .android_bitrate_monitor import BitRateMonitor
+from . import qc_diag
+from .android_apn import AndroidApn
+from .android_host import AndroidHost
+
+
+def on_register_schemas():
+    resource_schema = {
+        'additional_args[]': schema.STR,
+        'enable_pcap': schema.BOOL_STR,
+        }
+    for key, val in RunNode.schema().items():
+        resource_schema['run_node.%s' % key] = val
+    for key, val in AndroidApn.schema().items():
+        resource_schema['apn.%s' % key] = val
+    schema.register_resource_schema('modem', resource_schema)
+
+    config_schema = {
+        'enable_pcap': schema.BOOL_STR,
+        'log_all_level': schema.STR,
+        }
+    schema.register_config_schema('modem', config_schema)
+
+
+class AndroidUE(MS, AndroidHost, srslte_common):
+
+    REMOTEDIR = '/osmo-gsm-tester-androidue'
+    METRICSFILE = 'android_ue_metrics.csv'
+    PCAPFILE = 'android_ue.pcap'
+
+##############
+# PROTECTED
+##############
+    def __init__(self, testenv, conf):
+        self._run_node = RunNode.from_conf(conf.get('run_node', {}))
+        self.apn_worker = AndroidApn.from_conf(conf.get('apn', {})) if conf.get('apn', {}) != {} else None
+        self.qc_diag_mon = qc_diag.QcDiag(testenv, conf)
+        super().__init__('androidue_%s' % self.addr(), testenv, conf)
+        srslte_common.__init__(self)
+        self.rem_host = None
+        self.run_dir = None
+        self.remote_run_dir = None
+        self.emm_connected = False
+        self.rrc_connected = False
+        self.conn_reset_intvl = 20  # sec
+        self.connect_timeout = 300  # sec
+        self.enable_pcap = None
+        self.remote_pcap_file = None
+        self.pcap_file = None
+        self.data_interface = None
+        self.remote_metrics_file = None
+        self.metrics_file = None
+        self.brate_mon = None
+
+    def configure(self):
+        values = dict(ue=config.get_defaults('androidue'))
+        config.overlay(values, dict(ue=self.testenv.suite().config().get('modem', {})))
+        config.overlay(values, dict(ue=self._conf))
+        self.dbg('AndroidUE CONFIG:\n' + pprint.pformat(values))
+
+        if 'qc_diag' in self.features():
+            self.enable_pcap = util.str2bool(values['ue'].get('enable_pcap', 'false'))
+
+        self.metrics_file = self.run_dir.child(AndroidUE.METRICSFILE)
+        self.pcap_file = self.run_dir.child(AndroidUE.PCAPFILE)
+        if not self._run_node.is_local():
+            self.rem_host = remote.RemoteHost(self.run_dir, self._run_node.ssh_user(), self._run_node.ssh_addr(), None,
+                                              self._run_node.ssh_port())
+            self.remote_run_dir = util.Dir(AndroidUE.REMOTEDIR)
+            self.remote_metrics_file = self.remote_run_dir.child(AndroidUE.METRICSFILE)
+            self.remote_pcap_file = self.remote_run_dir.child(AndroidUE.PCAPFILE)
+
+        if self.apn_worker:
+            self.apn_worker.configure(self.testenv, self.run_dir, self._run_node, self.rem_host)
+            # some Android UEs only accept new APNs when airplane mode is turned off
+            self.set_airplane_mode(False)
+            self.apn_worker.set_apn()
+            MainLoop.sleep(1)
+            self.set_airplane_mode(True)
+
+        # clear old diag files
+        self._clear_diag_logs()
+
+    def _clear_diag_logs(self):
+        popen_args_clear_diag_logs = \
+            ['su', '-c', '\"rm -r /data/local/tmp/diag_logs/ || true\"']
+        clear_diag_logs_proc = self.run_androidue_cmd('clear-diag-logs', popen_args_clear_diag_logs)
+        clear_diag_logs_proc.launch_sync()
+
+    def verify_metric(self, value, operation='avg', metric='dl_brate', criterion='gt', window=1):
+        self.brate_mon.save_metrics(self.metrics_file)
+        metrics = srsUEMetrics(self.metrics_file)
+        return metrics.verify(value, operation, metric, criterion, window)
+
+    def set_airplane_mode(self, apm_state):
+        self.log("Setting airplane mode: " + str(apm_state))
+        popen_args = ['settings', 'put', 'global', 'airplane_mode_on', str(int(apm_state)), ';',
+                      'wait $!;',
+                      'su', '-c', '\"am broadcast -a android.intent.action.AIRPLANE_MODE\";']
+        proc = self.run_androidue_cmd('set-airplane-mode', popen_args)
+        proc.launch_sync()
+
+    def get_assigned_addr(self, ipv6=False):
+        ip_prefix = '172.16.0'
+        proc = self.run_androidue_cmd('get-assigned-addr', ['ip', 'addr', 'show'])
+        proc.launch_sync()
+        out_l = proc.get_stdout().split('\n')
+        ip = ''
+        for line in out_l:
+            if ip_prefix in line:
+                ip = line.split(' ')[5][:-3]
+                self.data_interface = line.split(' ')[-1]
+        return ip
+
+########################
+# PUBLIC - INTERNAL API
+########################
+    def cleanup(self):
+        self.set_airplane_mode(True)
+
+    def addr(self):
+        return self._run_node.run_addr()
+
+    def run_node(self):
+        return self._run_node
+
+    def features(self):
+        return self._conf.get('features', [])
+
+###################
+# PUBLIC (test API included)
+###################
+    def run_netns_wait(self, name, popen_args):
+        # This function guarantees the compatibility with the current ping test. Please
+        # note that this function cannot execute commands on the machine the Android UE
+        # is attached to.
+        proc = self.run_androidue_cmd(name, popen_args)
+        proc.launch_sync()
+        return proc
+
+    def connect(self, enb):
+        self.log('Starting AndroidUE')
+        self.run_dir = util.Dir(self.testenv.test().get_run_dir().new_dir(self.name()))
+        self.configure()
+        CONN_CHK = 'osmo-gsm-tester_androidue_conn_chk.sh'
+
+        if 'qc_diag' in self.features():
+            self.qc_diag_mon.start()
+
+        if self._run_node.is_local():
+            popen_args_emm_conn_chk = [CONN_CHK, self._run_node.adb_serial_id(), '0', '0']
+        else:
+            popen_args_emm_conn_chk = [CONN_CHK, '0', self.rem_host.host(), self.rem_host.get_remote_port()]
+
+        # make sure osmo-gsm-tester_androidue_conn_chk.sh is available on the OGT master unit
+        name = 'emm-conn-chk'
+        run_dir = self.run_dir.new_dir(name)
+        emm_conn_chk_proc = process.Process(name, run_dir, popen_args_emm_conn_chk)
+        self.testenv.remember_to_stop(emm_conn_chk_proc)
+        emm_conn_chk_proc.launch()
+
+        # check connection status
+        timer = self.connect_timeout
+        while timer > 0:
+            if timer % self.conn_reset_intvl == 0:
+                self.set_airplane_mode(True)
+                MainLoop.sleep(1)
+                timer -= 1
+                self.set_airplane_mode(False)
+
+            if 'LTE' in emm_conn_chk_proc.get_stdout():
+                if not(self.get_assigned_addr() is ''):
+                    self.emm_connected = True
+                    self.rrc_connected = True
+                    self.testenv.stop_process(emm_conn_chk_proc)
+                    break
+
+            MainLoop.sleep(2)
+            timer -= 2
+
+        if timer == 0:
+            raise log.Error('Connection timer of Android UE %s expired' % self._run_node.adb_serial_id())
+
+        self.brate_mon = BitRateMonitor(self.testenv, self.run_dir, self._run_node, self.rem_host, self.data_interface)
+        self.brate_mon.start()
+
+    def is_rrc_connected(self):
+        if not ('qc_diag' in self.features()):
+            raise log.Error('Monitoring RRC states not supported (missing qc_diag feature?)')
+
+        # if not self.qc_diag_mon.running():
+        #     raise log.Error('Diag monitoring crashed or was not started')
+
+        rrc_state = self.qc_diag_mon.get_rrc_state()
+        if 'RRC_IDLE_CAMPED' in rrc_state:
+            self.rrc_connected = False
+        elif 'RRC_CONNECTED' in rrc_state:
+            self.rrc_connected = True
+        return self.rrc_connected
+
+    def is_registered(self, mcc_mnc=None):
+        if mcc_mnc:
+            raise log.Error('An AndroidUE cannot register to any predefined MCC/MNC')
+        return self.emm_connected
+
+    def get_counter(self, counter_name):
+        if counter_name == 'prach_sent':
+            # not implemented so far, return 2 to pass tests
+            return 2
+        elif counter_name == 'paging_received':
+            return self.qc_diag_mon.get_paging_counter()
+        else:
+            raise log.Error('Counter %s not implemented' % counter_name)
+
+    def netns(self):
+        return None

src/osmo_gsm_tester/obj/qc_diag.py

@@ -0,0 +1,274 @@
+# osmo_gsm_tester: specifics for running Qualcomm diagnostics on an AndroidUE modem
+#
+# Copyright (C) 2020 by Software Radio Systems Limited
+#
+# Author: Nils Fürste <nils.fuerste@softwareradiosystems.com>
+# Author: Bedran Karakoc <bedran.karakoc@softwareradiosystems.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import getpass
+import os
+from ..core import remote, util, process, schema, log
+from ..core.event_loop import MainLoop
+from . import ms_android
+from .android_host import AndroidHost
+from .run_node import RunNode
+
+
+def on_register_schemas():
+    resource_schema = {}
+    for key, val in ScatParser.schema().items():
+        resource_schema['scat_parser.%s' % key] = val
+    schema.register_resource_schema('modem', resource_schema)
+
+
+class QcDiag(AndroidHost):
+
+    DIAG_PARSER = 'osmo-gsm-tester_androidue_diag_parser.sh'
+
+##############
+# PROTECTED
+##############
+    def __init__(self, testenv, conf):
+        self._run_node = RunNode.from_conf(conf.get('run_node', {}))
+        super().__init__('qcdiag_%s' % self._run_node.run_addr())
+        self.testenv = testenv
+        self.run_dir = util.Dir(self.testenv.test().get_run_dir().new_dir(self.name()))
+        if not self._run_node.is_local():
+            self.rem_host = remote.RemoteHost(self.run_dir, self._run_node.ssh_user(), self._run_node.ssh_addr(), None,
+                                              self._run_node.ssh_port())
+            self.remote_run_dir = util.Dir(ms_android.AndroidUE.REMOTEDIR)
+        self.scat_parser = ScatParser(testenv, conf)
+        testenv.register_for_cleanup(self.scat_parser)
+        self.diag_monitor_proc = None
+        self.enable_pcap = util.str2bool(conf.get('enable_pcap', 'false'))
+
+########################
+# PUBLIC - INTERNAL API
+########################
+    def get_rrc_state(self):
+        scat_parser_stdout_l = self.scat_parser.get_stdout().split('\n')
+        # Find the first "Pulling new .qmdl file..." and check the state afterwards. This has to be done to
+        # ensure that no process is reading the ScatParser's stdout while the parser is still writing to it.
+        is_full_block = False
+        for line in reversed(scat_parser_stdout_l):
+            if 'Pulling new .qmdl file...' in line:
+                is_full_block = True
+            if is_full_block and 'LTE_RRC_STATE_CHANGE' in line:
+                rrc_state = line.split(' ')[-1].replace('rrc_state=', '')
+                rrc_state.replace('\'', '')
+                return rrc_state
+        return ''
+
+    def get_paging_counter(self):
+        diag_parser_stdout_l = self.scat_parser.get_stdout().split('\n')
+        return diag_parser_stdout_l.count('Paging received')
+
+    def running(self):
+        return self.diag_monitor_proc.is_running()
+
+    def write_pcap(self, restart=False):
+        self.scat_parser.write_pcap(restart)
+
+    def start(self):
+        popen_args_diag = ['/vendor/bin/diag_mdlog', '-s', '90000', '-f', '/data/local/tmp/ogt_diag.cfg',
+                           '-o', '/data/local/tmp/diag_logs']
+        self.diag_monitor_proc = self.run_androidue_cmd('start-diag-monitor_%s' % self._run_node.adb_serial_id(), popen_args_diag)
+        self.testenv.remember_to_stop(self.diag_monitor_proc)
+        self.diag_monitor_proc.launch()
+
+        self.scat_parser.configure(self._run_node, self.enable_pcap)
+        self.scat_parser.start()
+
+    def scp_back_pcap(self):
+        self.scat_parser.scp_back_pcap()
+
+
+class ScatParser(AndroidHost):
+##############
+# PROTECTED
+##############
+    def __init__(self, testenv, conf):
+        self.testenv = testenv
+        self._run_node = RunNode.from_conf(conf.get('scat_parser', {}))
+        super().__init__('scat_parser_%s' % self._run_node.run_addr())
+        self.run_dir = util.Dir(self.testenv.test().get_run_dir().new_dir(self.name()))
+        self.remote_run_dir = None
+        self.rem_host = None
+        self.pcap_file = None
+        self.remote_pcap_file = None
+        self.parser_proc = None
+        self._parser_proc = None
+        self.popen_args_diag_parser = None
+        self._run_node_ue = None
+        self.enable_pcap = False
+
+    def _clear_diag_files(self):
+        name_chown = 'chown-diag-files'
+        diag_dir_local = str(self.run_dir) + '/diag_logs/'
+        diag_dir_remote = str(self.remote_run_dir) + '/diag_logs/'
+        popen_args_change_owner = ['sudo', 'chown', '-R', '', '']
+        run_dir_chown = self.run_dir.new_dir(name_chown)
+        if self._run_node.is_local():
+            if os.path.exists(diag_dir_local):
+                # Due to errors the diag_logs dir can be non-existing. To avoid errors the path
+                # is checked for existence first.
+                popen_args_change_owner[3] = getpass.getuser()
+                popen_args_change_owner[4] = diag_dir_local
+                change_owner_proc = process.Process(name_chown, run_dir_chown, popen_args_change_owner)
+                change_owner_proc.launch_sync()
+        else:
+            popen_args_change_owner = ['sudo', 'chown', '-R', self.rem_host.user(), diag_dir_remote]
+            change_owner_proc = self.rem_host.RemoteProcess(name_chown, popen_args_change_owner, remote_env={})
+            change_owner_proc.launch_sync()
+
+        name_clear = 'clear-diag-files'
+        run_dir_clear = self.run_dir.new_dir(name_clear)
+        popen_args_clear_diag_files = ['rm', '-r', '']
+        if self._run_node.is_local():
+            popen_args_clear_diag_files[2] = diag_dir_local
+            clear_run_dir_proc = process.Process(name_clear, run_dir_clear, popen_args_clear_diag_files)
+        else:
+            popen_args_clear_diag_files[2] = diag_dir_remote
+            clear_run_dir_proc = self.rem_host.RemoteProcess(name_clear, popen_args_clear_diag_files, remote_env={})
+        clear_run_dir_proc.launch_sync()
+
+########################
+# PUBLIC - INTERNAL API
+########################
+    @classmethod
+    def schema(cls):
+        resource_schema = {
+            'run_type': schema.STR,
+            'run_addr': schema.IPV4,
+            'ssh_user': schema.STR,
+            'ssh_addr': schema.IPV4,
+            'run_label': schema.STR,
+            'ssh_port': schema.STR,
+            'adb_serial_id': schema.STR,
+            }
+        return resource_schema
+
+    def configure(self, run_node, enable_pcap):
+        self.enable_pcap = enable_pcap
+        self._run_node_ue = run_node
+
+        if not self._run_node.is_local():
+            self.rem_host = remote.RemoteHost(self.run_dir, self._run_node.ssh_user(), self._run_node.ssh_addr())
+            self.remote_run_dir = util.Dir(ms_android.AndroidUE.REMOTEDIR)
+            self.remote_pcap_file = self.remote_run_dir.child(ms_android.AndroidUE.PCAPFILE)
+        self.pcap_file = self.run_dir.child(ms_android.AndroidUE.PCAPFILE)
+
+    def start(self):
+        # format: osmo-gsm-tester_androidue_diag_parser.sh $serial $run_dir $pcap_path $remote_ip $remote_port
+        self.popen_args_diag_parser = [QcDiag.DIAG_PARSER, '', '', '', '', '']
+        if self._run_node_ue.is_local():
+            if not self._run_node.is_local():
+                # AndroidUE is attached to Master but ScatParser is running remote
+                raise log.Error('Running the network locally and the ScatParser remotely is currently not supported')
+            else:
+                # Master, ScatParser, and AndroidUE are attached to/running on the same host
+                self.popen_args_diag_parser[1] = str(self._run_node.adb_serial_id())    # adb serial
+                self.popen_args_diag_parser[2] = str(self.run_dir)                      # run dir path
+                self.popen_args_diag_parser[3] = str(self.pcap_file)                    # pcap file path
+                self.popen_args_diag_parser[4] = '0'                                    # remote ip
+                self.popen_args_diag_parser[5] = '0'                                    # remote port
+        else:
+            if self._run_node.is_local():
+                # Master and ScatParser running on the same machine, the AndroidUE runs remote
+                self.popen_args_diag_parser[1] = '0'                                    # adb serial
+                self.popen_args_diag_parser[2] = str(self.run_dir)                      # run dir path
+                self.popen_args_diag_parser[3] = str(self.pcap_file)                    # pcap file path
+                self.popen_args_diag_parser[4] = str(self._run_node_ue.ssh_addr())      # remote ip AndroidUE
+                self.popen_args_diag_parser[5] = str(self._run_node_ue.ssh_port())      # remote port AndroidUE
+            elif self._run_node.ssh_addr() == self._run_node_ue.ssh_addr():
+                # ScatParser and AndroidUE are remote but on the same machine
+                self.popen_args_diag_parser[1] = str(self._run_node.adb_serial_id())    # adb serial
+                self.popen_args_diag_parser[2] = str(self.remote_run_dir)               # run dir path
+                self.popen_args_diag_parser[3] = str(self.remote_pcap_file)             # pcap file path
+                self.popen_args_diag_parser[4] = '0'                                    # remote ip
+                self.popen_args_diag_parser[5] = '0'                                    # remote port
+            else:
+                # Master, ScatParser and AndroidUE are running on/attached to different machines
+                self.popen_args_diag_parser[1] = '0'                                    # adb serial
+                self.popen_args_diag_parser[2] = str(self.remote_run_dir)               # run dir path
+                self.popen_args_diag_parser[3] = str(self.remote_pcap_file)             # pcap file path
+                self.popen_args_diag_parser[4] = str(self._run_node_ue.ssh_addr())      # remote ip AndroidUE
+                self.popen_args_diag_parser[5] = str(self._run_node_ue.ssh_port())      # remote port AndroidUE
+
+        if not self._run_node.is_local():
+            # The diag_logs directory only exists here if the ScatParser entity is running remote
+            self._clear_diag_files()
+
+        name = 'scat_parser_%s' % self._run_node.run_addr()
+        if self._run_node.is_local():
+            run_dir = self.run_dir.new_dir(name)
+            self.parser_proc = process.Process(name, run_dir, self.popen_args_diag_parser)
+        else:
+            self.parser_proc = self.rem_host.RemoteProcess(name, self.popen_args_diag_parser, remote_env={})
+        self.testenv.remember_to_stop(self.parser_proc)
+        self.parser_proc.launch()
+
+    def stop(self):
+        self.testenv.stop_process(self.parser_proc)
+
+    def write_pcap(self, restart=False):
+        # We need to stop the diag_parser to avoid pulling a new .qmdl during
+        # the parsing process. The process can be restarted afterwards but keep in
+        # mind that this will overwrite the pcap after some time. The diag_monitor
+        # process can continue, as it does not hinder this process.
+        if self.parser_proc and self.parser_proc.is_running():
+            self.testenv.stop_process(self.parser_proc)
+        self._clear_diag_files()
+
+        name = 'write-pcap_%s' % self._run_node.run_addr()
+        if self._run_node.is_local():
+            run_dir = self.run_dir.new_dir(name)
+            self._parser_proc = process.Process(name, run_dir, self.popen_args_diag_parser)
+        else:
+            self._parser_proc = self.rem_host.RemoteProcess(name, self.popen_args_diag_parser, remote_env={})
+        self.testenv.remember_to_stop(self._parser_proc)
+        self._parser_proc.launch()
+
+        MainLoop.wait(self.finished_parsing, timestep=0.1, timeout=300)
+
+        if restart:
+            self.parser_proc = self._parser_proc
+        else:
+            self.testenv.stop_process(self._parser_proc)
+
+    def finished_parsing(self):
+        scat_parser_stdout = self._parser_proc.get_stdout()
+        # If the parsers pulls the .qmdl file for the second time we know that
+        # the parsing of the first one is done
+        return scat_parser_stdout.count('Pulling new .qmdl file...') > 1
+
+    def get_stdout(self):
+        return self.parser_proc.get_stdout()
+
+    def is_running(self):
+        return self.parser_proc.is_running()
+
+    def scp_back_pcap(self):
+        try:
+            self.rem_host.scpfrom('scp-back-pcap', self.remote_pcap_file, self.pcap_file)
+        except Exception as e:
+            self.log(repr(e))
+
+    def cleanup(self):
+        if self.enable_pcap:
+            self.write_pcap(restart=False)
+            if not self._run_node.is_local():
+                self.scp_back_pcap()

src/osmo_gsm_tester/obj/run_node.py

@@ -30,7 +30,7 @@ class RunNode(log.Origin):
     T_LOCAL = 'local'
     T_REM_SSH = 'ssh'
 
-    def __init__(self, type=None, run_addr=None, ssh_user=None, ssh_addr=None, run_label=None, ssh_port=None):
+    def __init__(self, type=None, run_addr=None, ssh_user=None, ssh_addr=None, run_label=None, ssh_port=None, adb_serial_id=None):
         super().__init__(log.C_RUN, 'runnode')
         self._type = type
         self._run_addr = run_addr
@@ -38,6 +38,7 @@ class RunNode(log.Origin):
         self._ssh_addr = ssh_addr
         self._run_label = run_label
         self._ssh_port = ssh_port
+        self._adb_serial_id = adb_serial_id
         if not self._type:
             raise log.Error('run_type not set')
         if not self._run_addr:
@@ -56,7 +57,8 @@ class RunNode(log.Origin):
     def from_conf(cls, conf):
         return cls(conf.get('run_type', None), conf.get('run_addr', None),
                    conf.get('ssh_user', None), conf.get('ssh_addr', None),
-                   conf.get('run_label', None), conf.get('ssh_port', None))
+                   conf.get('run_label', None), conf.get('ssh_port', None),
+                   conf.get('adb_serial_id', None))
 
     @classmethod
     def schema(cls):
@@ -67,6 +69,7 @@ class RunNode(log.Origin):
             'ssh_addr': schema.IPV4,
             'run_label': schema.STR,
             'ssh_port': schema.STR,
+            'adb_serial_id': schema.STR,
             }
         return resource_schema
 
@@ -94,4 +97,7 @@ class RunNode(log.Origin):
     def ssh_port(self):
         return self._ssh_port
 
+    def adb_serial_id(self):
+        return self._adb_serial_id
+
 # vim: expandtab tabstop=4 shiftwidth=4

sysmocom/scenarios/ms-label.conf

@@ -0,0 +1,3 @@
+resources:
+  modem:
+  - label: ${param1}

utils/bin/osmo-gsm-tester_androidue_conn_chk.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+# This script reads the network type of an Android phone via ADB
+# usage: osmo-gsm-tester_androidue_conn_chk.sh $serial $remote_ip $remote_port
+serial=$1
+remote_ip=$2
+remote_port=$3
+while true; do
+  if [ "${serial}" == "0" ]; then
+    # run_type == ssh
+    ssh -p "${remote_port}" root@"${remote_ip}" getprop "gsm.network.type"
+  else
+    # run_type = local
+    adb -s "${serial}" shell getprop "gsm.network.type"
+  fi
+  sleep 1
+done

utils/bin/osmo-gsm-tester_androidue_diag_parser.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+# This script pulls the diag folder created by diag_mdlog and parses the
+# .qmdl file. Further, it writes all packets to a pcap file.
+# usage: osmo-gsm-tester_androidue_diag_parser.sh $serial $run_dir $pcap_path $remote_ip $remote_port
+serial=$1
+run_dir=$2
+pcap_path=$3
+remote_ip=$4
+remote_port=$5
+while true; do
+  echo "Pulling new .qmdl file..."
+  if [ "${remote_ip}" == "0" ]; then
+    # ScatParser and AndroidUe are attached to/running on the same machine
+    sudo adb -s "${serial}" pull /data/local/tmp/diag_logs "${run_dir}" >/dev/null
+    wait $!
+  else
+    # ScatParser and AndroidUe are attached to/running on different machines
+    scp -r -P "${remote_port}" root@"${remote_ip}":/data/local/tmp/diag_logs/ "${run_dir}"
+    wait $!
+  fi
+  qmdl_fn=$(find "${run_dir}" -maxdepth 2 -type f -name "*.qmdl")
+  wait $!
+  sudo scat -t qc --event -d "${qmdl_fn}" -F "${pcap_path}"
+  wait $!
+done