Commit Graph

533 Commits (865eea68c3a0e6b3292abc2b07790390b8a63587)

Author SHA1 Message Date
Harald Welte 865eea68c3 filesystem: add unit tests for encoder/decoder methods
Lets add test vectors for the per-record/per-file encode/decode of
our various classes for the Elementary Files.

We keep the test vectors as class variables of the respective EF-classes
to ensure implementation and test vectors are next to each other.

The test classes then iterate over all EF subclasses and execute the
decode/encode functions using the test vectors from the class variables.

Change-Id: I02d884547f4982e0b8ed7ef21b8cda75237942e2
Related: OS#4963
2023-02-01 10:52:23 +01:00
Harald Welte d2edd414a8 ts_51_011: Fix decoding/encoding of EF_LOCIGPRS
The P-TMSI signature is a 3-byte value, not a 1-byte value.

Change-Id: I06e8d3efe0b3cf3970159c913acfd2f72280302d
2023-01-31 17:26:09 +01:00
Harald Welte caa94b5a81 Assume first record number if caller specifies none
This fixes a regression introduced in Change-Id
I02d6942016dd0631b21d1fd301711c13cb27962b which added support for
different encoding/decoding of records by their record number.

Change-Id: I0c5fd21a96d2344bfd9551f31030eba0769636bf
2023-01-31 17:26:09 +01:00
Harald Welte 9b9efb6a7a ts_31_102: Fix several bugs in EF_ECC encoder
The encoder function apparently was never tested, it didn't match at all
the output of the decoder, not even in terms of the string keys of the

Change-Id: Id67bc39d52c4dfb39dc7756d8041cbd552ccbbc4
2023-01-31 17:26:09 +01:00
Harald Welte 136bdb065b ts_51_011: EF_SMSP: Use integer division in ValidityPeriodAdapter
ValidityPeriodAdapter() must return integer values when encoding a
value, as only integer values can be expressed in the binary format.

Change-Id: I0b431a591ac1761d875b5697a71b6d59241db87d
2023-01-31 17:26:09 +01:00
Harald Welte 9181a69a55 gsm_r: EF_IC: Network String Table Index is 16bit, not 8bit
As per EIRENE GSM-R SIM-Card FFFIS, EF_IC conatains records of 1+2+2+2
bytes, the network string table index is 16bit and not 8bit as we
implemented so far.

Change-Id: I9e3d4a48b3cb6fb0ecf887b04c308e903a99f547
2023-01-31 16:00:20 +00:00
Harald Welte 5924ec4d97 ts_51_011: Improve decoding of SELECT response for classic SIM
When decoding the SELECT response of a clasic GSM SIM without
UICC functionality, we
* did not decode the record length or number of records
* accidentially reported the EF file_size as available_memory (like DF)

Let's fix those two, and also add a comment on how the output dict
of decode_select_response() should look like.

As a result, code like 'read_records' now knows the number of records
and can iterate over them rather than raising exceptions.

Change-Id: Ia8e890bda74e3b4dacca0673d6e5ed8692dabd87
Closes: OS#5874
2023-01-27 20:46:08 +01:00
Harald Welte a1bb3f7147 ts_51_011: Support EF.LND
This file is a optional file specified by TS 51.011, storing the last
numbers dialled.  As the EIRENE FFFIS for GSM-R SIM refers to this,
we must implement it to have full GSM-R support in pySim.

Change-Id: I3b7d6c7e7504b7cc8a1b62f13e8c0ae83a91d0f0
Related: OS#5784
2023-01-27 20:46:08 +01:00
Harald Welte 0dc6c201e5 ts_51_011, ts_31_102: point to proper EF_EXTn file
We're using a shared class to implement the identical file encoding
for EF.{ADN,SDN,MBDN,BDN,FDN,CFIS}.  However, they all point to
different extension files.

Previosly for EF.SDN:
    "ext1_record_id": 255

Now for EF.SDN:
    "ext3_record_id": 255

Change-Id: I5301d41225266d35c05e41588811502e5595520d
Related: OS#5784
2023-01-27 20:46:08 +01:00
Harald Welte f11f1308b1 ts_51_011: Implement Extended BCD Coding
TS 51.011 specifies an "Extended BCD Coding" in Table 12 of Section
10.5.1. It allows to express the '*' and '#' symbols used in GSM
SS and/or USSD codes.

This improves decoding from
    "dialing_nr": "a753b1200f",
    "dialing_nr": "*753#1200f",

Change-Id: Ifcec13e9b296dba7bec34b7872192b7ce185c23c
Related: OS#5784
2023-01-27 20:46:08 +01:00
Harald Welte 9ba68df3cc ts_51_011: Support EF.SDN
DF.TELECOM/EF.SDN (Service Dialling Numbers) is specified in section
10.5.9 of TS 51.011 and required by EIRENE for GSM-R.

Let's use the pre-existing EF.ADN decoder to decode this file.

Change-Id: If91332b10138096d465a9dccf90744de2c14b2be
Related: OS#5784
2023-01-27 20:46:08 +01:00
Harald Welte 5b9472db7a ts_51_011: Fix bit-order in EF.VGCSS and EF.VBSS
Those files contain a bit-mask of active group IDs stored at the
respective positions in EV.VGCS and EF.VBS.  However, the bit-order
of each byte is reversed.

Change-Id: I77674c23823aae71c9504b1a85cd75266edadc6f
Related: OS#5784
2023-01-27 20:46:08 +01:00
Harald Welte 73a7fea357 gsm_r: Fix byte/nibble ordering of predefined_value1
Change-Id: Ia0dd8994556548a17a0a3101225c23e804511717
Related: OS#5784
2023-01-27 20:46:08 +01:00
Harald Welte 6bf2d5f216 gsm_r: EF_Predefined: Decode first record different from others
In their infinite wisdom, the authors of the EIRENE FFFIS for GSM-R SIM
cards invented yet a new way of encoding data in SIM card files: The
first record of a file may be encoded differently than further records
of files.

This patch implements the feature based on the newly-introduced way by
which we pass the record number to the encoder and decoder methods.

Change-Id: Ib526f6c3c2ac9a945b8242e2e54536628376efc0
Related: OS#5784
2023-01-27 20:46:08 +01:00
Harald Welte f6b37af721 Prepare for decoding/encoding records differently based on record number
In their infinite wisdom, the authors of the EIRENE FFFIS for GSM-R SIM
cards invented yet a new way of encoding data in SIM card files: The
first record of a file may be encoded differently than further records
of files.

Let's add the required infrastructure to pySim so that the encode and
decode methods for record-oriented files get passed in the current
record number.

Change-Id: I02d6942016dd0631b21d1fd301711c13cb27962b
Related: OS#5784
2023-01-24 20:03:02 +01:00
Harald Welte 8dbf714e96 gsm_r: Fix decoding of EF.FN
This fixes the below exception when trying to decode records of EF.FN:

EXCEPTION of type 'TypeError' occurred with message: 'unsupported operand type(s) for &: 'str' and 'int''

Change-Id: I3723a0d59f862fa818bea1622fe43a7b56c92847
Related: OS#5784
2023-01-24 14:37:18 +01:00
Harald Welte e6d7b14f43 gsm_r: Fix typo (it's EF.FN, not EF.EN)
Related: OS#5784
Change-Id: I2c97a02973d2a1eda2cea5412391144726bb0525
2023-01-24 14:37:13 +01:00
Harald Welte bc7437d3b6 pySim-trace: Also consider SW 91xx as successful
Change-Id: I9e4170721be30342bdce7fb4beeefd1927263ca6
2023-01-24 13:50:51 +01:00
Harald Welte 7489947046 pySim-trace: Fix missing MANAGE CHANNEL decode
old output:

00 MANAGE CHANNEL 01       9110

new output:

00 MANAGE CHANNEL 01       9110 {'mode': 'open_channel', 'created_channel': 1}

Change-Id: Iac5b24c14d2b68d526ab347462b72548b8731b30
2023-01-24 13:50:51 +01:00
Harald Welte c95f6e2124 pySim-trace: Add support for reading GSMTAP from pcap files
So far we supported
* GSMTAP live traces via a UDP socket
* RSPRO traces from pcap files (or live)

We were lacking support for reading GSMTAP stored in pcap, which
is what this patch implements.

Change-Id: I46d42774b39a2735500ff5804206ddcfa545568c
2023-01-24 13:50:51 +01:00
Philipp Maier 284ac104af cards: also program EF.AD under ADF.USIM
DF.GSM and ADF.USIM have an EF.AD with nearly the same contents. Usually
there is one file physically present and the other is just a link.
Apparantly this is not always the case for sysmo-ismi-sja2 cards, so
lets program EF.AD in both locations.

Change-Id: Ic9dd4acc8d9a72acbb7376ddf3e2128125d4a8f5
Related: OS#5830
2023-01-19 10:32:13 +01:00
Philipp Maier de0cf1648c cards: fix typo
Change-Id: I81a6074776bdf67b7bea359fe7a24f906936f46d
2023-01-03 13:30:02 +01:00
Philipp Maier 0a8d9f05b8 cards: check length of mnc more restrictively
Since we now ensure that mnc always has a valid length lets make the
check in more strict.

Related: OS#5830
Change-Id: Iee8f25416e0cc3be96dff025affb1dc11d919fcd
2022-12-20 11:33:03 +01:00
Philipp Maier 4f888a0414 sysmocom_sja2: simplify and fix op/opc decoder/encoder
The decoder/encoder of that decodes the EF.xSIM_AUTH_KEY files has an
overcomplicated handling for op/and opc. There is a condition that
checks if milenage is configured and another one that checks if the
string is recognized as OP or OPc. Both is not correct and seems not to
work (op and opc is always displayed as "null")

The encoder/decoder should focus on the physical file layout and
regardless of any other conriguration the OP/OPc field is physically
present and should be displayd and presented for editing.

Change-Id: I6fa3a07e5e473273498d3f13d4cfa33743b787e1
2022-12-02 12:38:08 +01:00
Christian Amsüss 5d26311efc OTA: Adjust IV length for AES
Change-Id: I854c844418244c100c328f9e76c0f37850d3db00
2022-11-25 04:00:55 +01:00
Harald Welte 75a58d1a87 Add new pySim.ota library, implement SIM OTA crypto
This introduces a hierarchy of classes implementing

* ETS TS 102 225 (general command structure)
* 3GPP TS 31.115 (dialects for SMS-PP)

In this initial patch only the SMS "dialect" is supported,
but it is foreseen that USSD/SMSCB/HTTPS dialects can be
added at a later point.

Change-Id: I193ff4712c8503279c017b4b1324f0c3d38b9f84
2022-09-08 15:45:55 +02:00
Vadim Yanitskiy 05d30eb666 construct: use Python's API for int<->bytes conversion
Argument 'signed' was added in [1] and become available since v2.10.63.
Therefore using bytes2integer() and integer2bytes() from construct.core
bumps the minimum required version of construct to v2.10.63.  For
instance, debian:bullseye currently ships v2.10.58.

There is no strict requirement to use construct's API, so let's use
Python's API instead.  This allows using older construct versions
from the v2.9.xx family.

Change-Id: I613dbfebe993f9c19003635371941710fc1b1236
Related: [1] 660ddbe2d9a351731ad7976351adbf413809a715 construct.git
Related: OS#5666
2022-09-05 23:15:10 +07:00
Christian Amsüss 59f3b1154f proactive: Send a Terminal Response automatically after a Fetch
Change-Id: I43bc994e7517b5907fb40a98d84797c54056c47d
2022-08-21 11:54:33 +00:00
Christian Amsüss 98552ef1bd proactive: Avoid clobbering the output of the command that triggered the FETCH
Change-Id: I2b794a5c5bc808b9703b4bc679c119341a0ed41c
2022-08-21 11:54:00 +00:00
Harald Welte fd476b4d62 pySim.transport: Add mechanism for handling for CAT/USAT proactive cmds
This introduces an optional argument to the LinkBase class constructor,
where the application can pass an instance of a ProactiveHandler derived
class in order to handle the proactive commands that the LinkBase is
automatically fetching whenever the card indicates so.

Change-Id: I844504e2fc1b27ce4fc7ede20b2307e698baa0f6
2022-08-06 18:56:42 +02:00
Harald Welte 5a4891a5b7 Add TLV definitions for *a lot more* CAT / USAT data objects
This adds deciding for the bulk of the TLV objects used in the
ETSI CAT (Card Application Toolkit) and 3GPP USAT (USIM Application
Toolkit) systems.

This patch just adds the definitions, but doesn't use them anywhere yet.

Change-Id: I0c66912dbc10164e040e2fec358cef13c45a66ec
2022-08-06 18:56:42 +02:00
Harald Welte 7d8029eb23 tlv: Use self._compute_tag() method rather than direct self.tag
The TLV_IE.from_tlv() method is part of a base class that is inherited
by more specific classes.  The official way to obtain the tag is the
inherited-class-provided self._compute_tag() method, and *not* a direct
reference to the self.tag member.

This allows for some more obscure TLV parsers, such as the upcoming one
for Proactive Commands in the CAT/OTA context.

Change-Id: I0cd70e31567edc5a0584336efcb5e4282734f6dd
2022-08-06 13:19:16 +02:00
Harald Welte f56b6b2a1c ts_31_102: Add missing imports for envelope_sms command
The envelope_sms command fails due to some missing imports prior to
this patch.

Change-Id: I98e692745e7e1cfbc64b88b248700b1e54915b96
2022-07-30 16:37:01 +02:00
Harald Welte 51b3abb000 ts_31_102: Fix terminal_profile, envelope and envelope_sms commands
In commit Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5 I accidentially
broke those commands by adding argparse definitions for better
documentation.  When adding the  @cmd2.with_argparser decorator,
the method argument changes from the raw string to an argparse.Namespace

This patch fixes the below exception:

pySIM-shell (MF/ADF.USIM)> terminal_profile ffffffff
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/cmd2/", line 2129, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/usr/local/lib/python3.10/dist-packages/cmd2/", line 2559, in onecmd
    stop = func(statement)
  File "/usr/local/lib/python3.10/dist-packages/cmd2/", line 336, in cmd_wrapper
    return func(*args_list, **kwargs)
  File "/space/home/laforge/projects/git/pysim/pySim/", line 1274, in do_terminal_profile
    (data, sw) = self._cmd.card._scc.terminal_profile(arg)
  File "/space/home/laforge/projects/git/pysim/pySim/", line 583, in terminal_profile
    data_length = len(payload) // 2
TypeError: object of type 'Namespace' has no len()

Change-Id: Ia861eeb2970627d3ecfd0ca73f75ca571c6885b2
Fixes: Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5
2022-07-30 16:37:01 +02:00
Harald Welte 7416d463a4 Fix printing of SwMatchError after introduction of logical channels
the interpret_sw() method was moved from RuntimeState to RuntimeLchan
in Change-Id I7aa994b625467d4e46a2edd8123240b930305360 - but the code
in pySim/ was not adjusted accordingly.

Change-Id: I0614436c99c6a6ebc22c4dc14fb361c5f5f16686
2022-07-30 16:37:01 +02:00
Harald Welte 93c34aac89 apdu/ts_102_221: SELECT: allow select of SELF
While in the pySim-shell, it's useful to filter the currently selected
file from the choice of available files for select, this doesn't apply
for the tracing case: It's perfectly valid for the UE to SELECT the
file that's already selected right now.  The operation basically
becomes equivalent to a STATUS.

Change-Id: I1a20fb3ba70426333ac34448c6cb782c51363965
2022-07-25 14:25:11 +02:00
Harald Welte dcc689d9c4 apdu/ts_102_221: SELECT: allow select of parent/ancestor DFs
We need to pass the 'PARENT' flag to get_selectables() to be able
to track SELECT on any of the parent/ancestor DF FID.

Change-Id: Ia7ac627d5edccb97160c90688d720d887fad6ec7
2022-07-25 14:25:11 +02:00
Harald Welte f5ff1b896e filesystem: We can select not just immediate parent DF but all ancestors
I didn't check the specs, but at least experience with real-world cards
(and modems) shows that it's not just permitted to select the immediate
parent DF, but all ancestors of the currently selected file.

So adjust the get_selectables() method to not just return the immediate
parent, but to recurse all the way up and report the FID of any ancestor

Change-Id: Ic9037aa9a13af6fb0c2c22b673aa4afa78575b49
2022-07-25 14:25:11 +02:00
Harald Welte 8e9c844130 apdu/ts_102_221: Fix SELECT of 3f00
In order to be able to explicitly select the MF via 3f00,
we need to pass the 'MF' to get_selectables(), so the record
is included in the list of selectable files from the current
working directory.

Change-Id: I27085896142fe547a6e93e01e63e59bbc65c8b8a
2022-07-24 11:56:35 +02:00
Harald Welte 498361f3b5 apdu/ts_102_221: Implement SELECT case "df_ef_or_mf_by_file_id"
This was [sadly] simply missing from the implementation so far.

Change-Id: I7bbd13ce29f5adc1ca3ca01bffabbe02dd17db20
2022-07-24 11:56:35 +02:00
Harald Welte d2c177b396 Make CardDF.get_selectables() respect the flags
All other get_selectables() understand a flag like 'FIDS' to request
only the hexadecimal FIDs and not the file names.  However, the
CardEF.get_selectables() ignored those flags and unconditionally
returned the names.

Change-Id: Icdc37cae3eecd36d167da76c30224b9d48c844fd
2022-07-24 11:56:35 +02:00
Harald Welte c61fbf4daa pySim-trace: Support SELECT with empty response body
If the modem/UE doesn't ask for the FCP to be returned, a SELECT
can exit with 9000 and no response body.  Don't crash in that case.

Change-Id: I66788717bec921bc54575e60f3f81adc80584dbc
2022-07-24 09:46:11 +02:00
Harald Welte 21caf32e3d Introduce APDU/TPDU trace decoder
This introduces a new pySim.apdu module hierarchy, which contains
classes that represent TPDU/APDUs as exchanged between

It contains instruction level decoders for SELECT, READ BINARY and
friends, and then uses the pySim.filesystem.Runtime{Lchan,State} classes
to keep track of the currently selected EF/DF/ADF for each logical
channel, and uses the file-specific decoder classes of pySim to decode
the actual file content that is being read or written.

This provides a much more meaningful decode of protocol traces than
wireshark will ever be able to give us.

Furthermore, there's the new pySim.apdu_source set of classes which
provides "input plugins" for obtaining APDU traces in a variety of
formats.  So far, GSMTAP UDP live capture and pyshark based RSPRO
live and pcap file reading are imlpemented.

Change-Id: I862d93163d495a294364168f7818641e47b18c0a
Closes: OS#5126
2022-07-23 12:18:57 +02:00
Harald Welte cfa3015bcf sysmocom_sja2: Prevent KeyError/None exception on encode
Fix a bug in the pySim.sysmocom_sja2 module, where we defined unnamed
bits in BitStruct without a default value causing exceptions like this:

	EXCEPTION of type 'KeyError' occurred with message: 'None'

Change-Id: Ib2da5adda4fae374ab14bb8100f338691aef719a
Closes: OS#5575
2022-07-23 12:17:21 +02:00
Harald Welte 1272129ea7 ts_31_102: Fix EF_EPSLOCI argument ordering
We were invoking the constructor with the description as 4th positional
argument, but that was actually the 'size' argument in this case.

Let's swap the order to be aligned with other file constructors.

Change-Id: I9acee757f096fef0d8bacbec3b52f56267cd52f6
2022-07-21 22:48:59 +02:00
Harald Welte 99e4cc02e5 filesystem: Use Tuple for record length
The size should be a *tuple*.  In reality we so far passed a set.  The
problem with the set is that ordering is not guaranteed, and hence we
cannot assume the first and second item have meaning (minimum vs.
default record length).

Change-Id: I470f4e69c83cb2761861b3350bf8d49e31f4d957
2022-07-21 22:48:59 +02:00
Harald Welte 13edf30d6c filesystem: Use Tuple for transparent file size
As the documentation strings say: The size should be a *tuple*.  In
reality we so far passed a set.  The problem with the set is that
ordering is not guaranteed, and hence we cannot assume the first and
second item have meaning (minimum vs. default size).

While at it, use a type annotation to catch such bugs easily.

Change-Id: I553616f8c6c4aaa8f635b3d7d94e8e8f49ed5a56
2022-07-21 22:48:59 +02:00
Harald Welte b2e4b4a300 introduce fully_qualified_path_str() method
Reduce all the copy+pasted '/'.join(path_list) constructs with
a method returning the formatted path string.

Change-Id: I5e9bfb425c3a3fade13ca4ccd2b891a0c21ed56d
2022-07-20 19:35:58 +02:00
Harald Welte 3c98d5e91d Never use Bytes without any 'Adapter'
Otherwise we have binary/bytes as values inside the dict, rather than a
hexadecimal string.  That's ugly when printing without json formatting.

Change-Id: Ia3e7c4791d11bd4e3719a43d58e11e05ec986d1f
2022-07-20 19:35:58 +02:00
Harald Welte 857f110492 EF.AD: Avoid NotImplementedErrror regarding network names
Even while we don't yet have a proper decoder, let's at least represent
the network name as hex-string

Change-Id: I4ed626699d1e4e484d4ffd04349676dadff626a0
2022-07-20 19:35:58 +02:00