Commit Graph

63 Commits

Author SHA1 Message Date
Harald Welte 522555710b utils.py: Add more type annotations
Change-Id: I50a0a07132890af0817f4ff0ce9fec53b7512522
2021-04-04 10:53:36 +02:00
Harald Welte 6e0458dda6 Move init_reader() from utils.py to transport/__init__.py
This avoids a circular dependency when introducing type annotations.

Change-Id: I168597ac14497fb188a15cb632f32452128bc1c6
2021-04-04 10:53:36 +02:00
Harald Welte 236a65f02f cosmetic: fix typo in comment
Change-Id: Iac8b310a470b3ad8dee5f61342fd5acedbbd6e5d
2021-04-03 20:33:31 +00:00
Harald Welte 1e45657e0f filesystem: fix various issues found by mypy
Change-Id: Ib4de80451614712bdf5377a3a5b86156008e2c42
2021-04-02 21:09:40 +02:00
Harald Welte ee3501fc62 Add more documentation to the classes/methods
* add type annotations in-line with PEP484
* convert existing documentation to follow the
  "Google Python Style Guide" format understood by
  the sphinx.ext.napoleon' extension
* add much more documentation all over the code base

Change-Id: I6ac88e0662cf3c56ae32d86d50b18a8b4150571a
2021-04-02 21:08:35 +02:00
Philipp Maier 78e32f2b36 utils: fix sw_match()
The SW_match function takes a given status word and compares it against
a pattern that may contain wildcards (x or ?). This works by creating a
masked version of the SW using a pattern first (each hex digit is
replaced by a wildcard charafter if the pattern has a wildcard in the
same position). Once this is done, the resulting masked version is
compared at the pattern. However, the current implementation can not
work, since it compares the input SW against the pattern to decide
wihich chrafters should be masked. The input SW never contains wildcard
charafters.

Change-Id: I805ad32160fcfcb8628bf919b64f7eee0fe03c7e
Related: OS#4963
2021-03-23 12:17:23 +00:00
Philipp Maier 5d3e2592f7 pySim-shell: add "dir" command.
pysim-shell does not have a convinient way to list the files available
in one directory. Tab completion currently is the only way to obtain a
list of the available files. Lets add a dir command to print a file
list.

Change-Id: Ic06a60e0a0ec54d9bb26e151453ceb49d69e3df7
Related: OS#4963
2021-03-18 17:19:17 +01:00
Philipp Maier 47236500fe utils: add is_hex function to check hex strings
since we have added pySim-shell.py that has a lot of locations where the
user can enter hexadecimal data there is an increased need for input
validation. Lets add a central is_hex function that verifies hex
strings.

Change-Id: Ia29a13c9215357dd2adf141f2ef222c823f8456d
Related: OS#4963
2021-03-12 07:35:37 +00:00
Vadim Yanitskiy c8458e2477 pySim/utils.py: fix 3-digit MNC encoding in enc_plmn()
The bug that was attempted to be fixed in [1] actually was in the
encoding API - pySim.utils.enc_plmn().  According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit (E)HPLMN shall be encoded
as shown below (ASCII-art interpretation):

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 2  |  MNC Digit 1  |
  +---+---+---+---+---+---+---+---+

while pySim.utils.enc_plmn() would produce the following:

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 1  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MNC Digit 2  |
  +---+---+---+---+---+---+---+---+

Initially the _decoding_ API was correct, but then got changed in
[1] to follow buggy pySim's encoding API.  As a result, a (E)HPLMN
programmed with pySim-prog.py would look correct if verified by
pySim-read.py, but the actual file content would be wrong.

This situation shows that our 'program-read-match' build verification
approach alone is insignificant.  The lack of unit test coverage,
at least for the core parts of the project, makes it possible to have
symmetrical bugs in both encoding and decoding API parts unnoticed.

This problem was found while trying to enable dead unit tests in [3].
Change [1] that introduced a symmetrical bug is reverted in [2].

Change-Id: Ic7612502e1bb0d280133dabbcb5cb146fc6997e5
Related: [1] I799469206f87e930d8888367890babcb8ebe23a9
Related: [2] If6bf5383988ad442e275efc7c5a159327d104879
Related: [3] I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-12 01:00:32 +01:00
Vadim Yanitskiy b271be3dc0 Revert "utils.py: Fix for parsing MNC"
This reverts commit bdf3d3597b, which
broke pySim.utils.dec_mnc_from_plmn().  According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit EHPLMN shall be encoded
as shown below (ASCII-art interpretation):

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 2  |  MNC Digit 1  |
  +---+---+---+---+---+---+---+---+

So the original implementation was correct, and we even had a unit
test for it.  Most likely, the SIM card itself was programmed
incorrectly?

Makes 'testDecMNCfromPLMN_threeDigitMNC' pass again.

Change-Id: If6bf5383988ad442e275efc7c5a159327d104879
2021-03-12 01:00:26 +01:00
Denis 'GNUtoo' Carikli 79f5b6080b Python 2 is deprecated, remove backwards compatibility chunks
pySim has already been migrated to Python 3 in another change [1],
and the build verification has been migrated to Debian 10 with
Python 3.7.  However, there is still some backwards compatibility
code left.  Let's get rid of it.

[1] Ic78da9c03e99f59d142c93394051bbc2751f0205

Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Tweaked-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Change-Id: I430d173535e0cd5bb895b9dfc9070cbc40cfc8ff
2021-03-07 20:33:46 +01:00
Harald Welte ab34fa895e pySim/utils.py: Attempt to support pycryptodpme
This should resolve the following error when using with pycryptodome
instead of pycrypto:

TypeError: new() missing 1 required positional argument: 'mode'

Change-Id: Ibd3ca00d62b864909f5e89e0feb350268157a4ca
Related: OS#5060
2021-03-05 20:39:18 +00:00
Harald Welte eab8d2adf7 fix TypeError in derive_milenage_opc()
In 4f6ca43e1f we started to use
the bytearray type as 'b' type, but PyCrypto insists on getting
a bytes type.

This fixes the following Exception:
TypeError: argument 1 must be read-only bytes-like object, not bytearray

Change-Id: If2a727ed417ffd56c0f7d7b4e9f633d67fde5ced
Closes: OS#5060
2021-03-05 20:38:50 +00:00
Harald Welte 4f6ca43e1f start using python3 bytearray for our b2h/h2b types
The code was written long ago, when the python3 bytearray type
probably didn't exist yet, or was at least not known.  Let's stop
using string types with binary bytes inside, and instead standardize
on two types:
 * bytearray for binary data
 * string for hexadecimal nibbles representing that binary data

Change-Id: I8aca84b6280f9702b0e2aba2c9759b4f312ab6a9
2021-03-03 08:37:50 +01:00
Harald Welte 67d551a443 move SW matching to a generic utility function
This will allow using it outside the transport/__init__.py

Change-Id: Id26dfefa85d91e3b3a23e0049f3b833e29cb1cef
2021-03-02 14:26:32 +01:00
Harald Welte 79b5ba4bdf utils.py: de-couple sanitize_pin_adm from argparse 'opts'
This allows the function to be re-used in other contexts

Change-Id: I116e85acca3aeb0a0c24f74653c500ac2dc1d844
2021-03-02 07:50:23 +01:00
Vadim Yanitskiy 6d5e0c9272 Remove Python shebang from files where it's not needed
Change-Id: I1d08544c37f50416acf8dc30139c572c029790d0
2021-03-01 17:33:46 +01:00
Philipp Maier c8caec2933 utils: catch exceptions during reader initalitation
Failed reader initializations happen frome time to time, mostly because
of messed up commandline arguments. This results in ugly crashes very
often. Lets control this a bit by catching the exception string and
print it.

Change-Id: I313017355da01bbef7c3d3f1899555aadb509319
2021-02-25 15:16:07 +01:00
herlesupreeth bdf3d3597b utils.py: Fix for parsing MNC
This commit fixes the incorrect parsing of MNC from PLMN.
Previously its was parsing PLMN string 130062 as MCC 310 MNC 260,
whereas it should be MCC 310 MNC 026.

(The SIM was programmed with MCC 310 and MNC 026)

Change-Id: I799469206f87e930d8888367890babcb8ebe23a9
2021-02-11 07:02:50 +01:00
Supreeth Herle 43fd03b627 utils.py: Support IPv4 decoding for Address TLV object present in EF.ePDGId and EF.ePDGIdEm
Change-Id: I96c30c1fcc03e50c55e9db7e6a18297a3b1d889d
2021-01-05 11:46:41 +01:00
Supreeth Herle 654eca72c9 utils.py: Support IPv4 encoding for Address TLV object present in EF.ePDGId and EF.ePDGIdEm
Change-Id: Id46a44257d09c98ad5e0b7558e25e2bc52b23978
2021-01-05 11:46:41 +01:00
Supreeth Herle 556b0fe262 utils.py: Add helper method to get type of address (FQDN, IPv4, IPv6)
The function takes address string as input, then validates it and returns the type.
Return: 0x00 (FQDN), 0x01 (IPv4), 0x02 (IPv6), None (Bad address format)

Change-Id: I0fabd4f17bbb11f6bb191c1a9e6276427f9d001f
2021-01-05 11:46:41 +01:00
Supreeth Herle 3b342c2f14 Re-purpose helper method to be used for parsing Address TLV Object in general
The Address TLV object is used in EF.P-CSCF Address, EF.ePDGId and EF.ePDGIdEm.
See 3GPP TS 31.102 version 13.4.0 Release 13, section 4.2.8, 4.2.102 and 4.2.104.

Address TLV Object format
Tag (1 Byte) - '80'
Length (1 Byte)
Address Type (1 Byte) - '00' (FQDN), '01' (IPv4), '02' (IPv6)
Address (Address Length Bytes)

Change-Id: Ifd8a240f6b5c7736e58a8151295c30ec5b32ed5f
2021-01-05 10:06:19 +01:00
herlesupreeth 3a261d31d5 utils.py: Bugfix for parsing non-programmed EFePDGSelection
Change-Id: I3a16af785d8ae9ea8730771367bba2d50690b414
2021-01-05 10:06:19 +01:00
Supreeth Herle 95ec772b61 utils.py: Add helper method to encode ePDGSelection info TLV
Encodes ePDGSelection info TLV so it can be stored at EF.ePDGSelection or EF.ePDGSelectionEm.
See 3GPP TS 31.102 version 15.2.0 Release 15, section 4.2.104 and 4.2.106.

Take original hex string of EF.ePDGSelection or EF.ePDGSelectionEm, MCC,
MNC, ePDG priority for PLMN and ePDG FQDN Format to use for PLMNas input
and outputs the encoded hex string.

Change-Id: Ia7292d33783c770a3bb91b081c671af36bbb907f
2020-10-21 08:32:57 +02:00
Supreeth Herle 95b4e8d4fa utils.py: Add helper method to parse ePDG Selection info TLV
ePDG selection information TLV data object is made of following elements:

ePDG Selection Information Tag '80' (1 Byte)
Length 5n Note
PLMN 1 (3 Bytes)
ePDG Priority (2 Bytes)
ePDG FQDN format '00' or '01' (1 Byte)
...

PLMN n
...

Note: The length is coded according to ISO/IEC 8825-1 [35]

Note 2: Inconsistency in spec: 3GPP TS 31.102 version 15.2.0 Release 15, 4.2.104

As per spec, Length field value is 5n, where n is number of PLMNs
But, each PLMN entry is made of PLMN (3 Bytes) + ePDG Priority (2 Bytes) + ePDG FQDN format (1 Byte)
Totalling to 6 Bytes, maybe Length should be 6n and not 5n

Change-Id: I0f9f38961a589e3f9a53d2288a3dc6fa71a4b1b0
2020-10-21 08:25:41 +02:00
Daniel Willmann dd014ea306 Lint fixes: false -> False, missing imports, Index list, not map iter
Change-Id: Iff4123a49c8dbcfc405612c0663d5c7d0f549748
2020-10-19 10:35:11 +02:00
herlesupreeth 3409ae7eea Remove redundant hexstr_to_fivebytearr() and hexstr_to_threebytearr() functions
These functions are replaced by a more generic function hexstr_to_Nbytearr().
And, all occurances of redundant functions are replaced by generic functions
so its safe to remove them.

Change-Id: I7848451b90b35dca29d29f630cdc5405b5e9c19b
2020-09-20 09:49:22 +00:00
herlesupreeth 45fa604834 Use generic function hexstr_to_Nbytearr to convert hex string to 3/5 Bytes array
Change-Id: I1165e4928d063667f0b4dfc3437c9278e7ec7450
2020-09-20 09:49:22 +00:00
Supreeth Herle f394853533 utils.py: Add helper method to convert a hex string into array of N bytes string elements
Change-Id: I3af080726079729eae92af183de40dfc70c9390e
2020-09-20 09:49:22 +00:00
Supreeth Herle d84daa12c2 utils.py: Add helper method to encode Service Table
This method helps in encoding of Service Tables in EF.SST, EF.UST, EF.EST, EF.IST.
Takes original hex string of EF.SST, EF.UST, EF.EST, EF.IST, Service number,
Service to be enabled or disable flag as input and outputs the modified Service Table.

Change-Id: I0c97317d5a17aa0df720659d021b5cbf7d30edcc
2020-09-20 09:49:22 +00:00
Harald Welte ca6739458e Add support for ADF_USIM/EF_EHPLMN
If the EF.EHPLMN exists, it contains the "Equivalent Home PLMN List".
The odd part of that list is that it is not just a list of additional
PLMN identities, but if the first digits of the IMSI are *not* listed
in EF.EHPLMN, then the MCC/MNC of the IMSI prefix is suddently no
longer considered the home network, but the subscriber is roaming.

See TS 23.122: "If the HPLMN code derived from the IMSI is not present
in the EHPLMN list, then it shall be treated as a Visited PLMN for PLMN
selection purposes."

Change-Id: I22d96ab4a424ec5bc1fb02f5e80165c646a748d3
2020-08-28 08:32:55 +00:00
Supreeth Herle 3c0bd7a41e utils.py: Add helper method to encode ePDG Identifier
This method encodes ePDG Id so it can be stored to EF.ePDGId or EF.ePDGIdEm.
See 3GPP TS 31.102 version 13.4.0 Release 13, section 4.2.102 and 4.2.104.

Resulting hex string is made of tag value + length + address type + address.
tag value for home ePDG identifier is 80
address type: 0x00 (FQDN), 0x01 (IPv4), 0x02 (IPv6), other (Reserved)

ePDG FQDN example: epdg.mnc001.mcc001.3gppnetwork.org

Note: Only FQDN format is supported for now

Change-Id: I864bda5505e9061391a727add294a6e90c50f9ef
2020-06-23 13:54:21 +02:00
Supreeth Herle d6a5ec51a8 utils.py: Handle parsing of ePDGId TLV with zero length
Change-Id: Ie05ae010948884765363bd9dbc2b0d764ff8d42a
2020-06-23 13:07:01 +02:00
Sebastian Viviani e61170c0eb utils.py add LOCI EFs decode functions
The LOCI, PSLOCI and EPSLOCI contain some info,
 including the PLMN, added helper functions to
decode it

Change-Id: Ibb513ff7d1dc6d33b354ae26cbd9c390ea3c8efc
2020-06-09 10:10:28 +01:00
Vadim Yanitskiy 29ca8049d6 Implement Generic SIM Access interface as per 3GPP TS 27.007
According to 3GPP TS 27.007, sections 8.17 and 8.18, the modem
may *optionally* provide Generic and/or Restricted SIM Access
to the TE (Terminal Equipment) by means of the AT commands.
This basically means that a modem can act as a card reader.

Generic SIM Access allows the TE to send raw PDUs in the format
as described in 3GPP TS 51.011 directly to the SIM card, while
Restricted SIM Access is more limited, and thus is not really
interesting to us.

This change implements a new transport called ModemATCommandLink,
so using it a SIM card can be read and/or programmed without the
need to remove it from the modem's socket. A downside of this
approach is relatively slow I/O speed compared to PC/SC readers.

Tested with Quectel EC20:

  $ ./pySim-read.py --modem-dev /dev/ttyUSB2

Change-Id: I20bc00315e2c7c298f46283852865c1416047bc6
Signed-off-by: Vadim Yanitskiy <axilirator@gmail.com>
2020-06-02 21:51:07 +07:00
Vadim Yanitskiy eb06b45d0e utils: fix list comprehension in h2s(): ignore upper case padding
By definition, h2s() is supposed to skip padding in the given
hexstring, so it was working fine for 'ff', but not for 'FF'.

Change-Id: I2c5d72a0f7f2796115116737f9f7b5299021f6a3
Signed-off-by: Vadim Yanitskiy <axilirator@gmail.com>
2020-05-28 09:59:51 +00:00
Philipp Maier cd3d6268a6 utils: do not crash when all bytes of EF.IMSI are 0xFF
In case try to decode the contents of an uninitalized EF.IMSI, the
function dec_imsi() would crash because it truncates all 0xFF from the
swapped version of the EF.IMSI contents and then accesses the first
element of the buffer. This always works for EF.IMSI contents that
contain valid IMSI data, but if all bytes are set to 0xFF, then no data
is left in the buffer after truncating, so lets check if we even have
bytes left before we move on with the decoding.

Change-Id: I93874a1d7e0b87d39e4b06a5c504643cfabb451c
2020-05-22 13:18:15 +02:00
Philipp Maier e8536c04bc pysim-prog: move ADM sanitation to utils.py
The lower part of gen_parameters() in pySim-prog.py contains some code
that checks whether the ADM pin supplied in its hexadecimal or in its
string form and generates a sanitised pin_adm from that. Lets separate
this part as it may become useful elsewhere too.

Change-Id: Ifead29724cc13a91de9e2e89314d7fb23c063d50
2020-05-17 09:37:40 +02:00
Harald Welte 7f1d3c496f Treat MCC and MNC as strings, not integers
A MNC of 02 and 002 are *not* equal.  The former is a two-digit MNC
and the latter is a three-digit MNC.  Hence, we shouldn't treat
MNCs as integer values as we have no clue how many leading zeroes
(if any) the user entered.

Change-Id: I9d1d07a64888c76703c3e430bbdd822080c05819
Closes: OS#4523
2020-05-13 11:44:13 +02:00
Philipp Maier ff84c23839 pySim-prog, pySim-read, do not echo reader id
pySim-prog and pySim-read currently echo back the pcsc reader id (or
baudrate/socket, depending on the interface used). This makes the output
unecessarly undeterministic, which becomes a problem when verifying the
putput in tests. Lets not echo those variable, user supplied parameters
back. Also lets move the code that does the initalization to utils, so
that it can be used from pySim-prog and from pySim-read (code dup).

Change-Id: I243cc332f075d007b1c111292effcc610e874eb3
Related: OS#4503
2020-05-12 18:11:38 +00:00
Supreeth Herle d572edef1e utils.py: Add helper method to parse ePDG Identifier from hex string
The hex string consists of contains zero or more ePDG identifier data objects.
Each ePDG Identifier TLV data object consists of tag value of '80', length, address type, identifier.

TS 31.102 version 13.4.0 Release 13. The same parsing method applies for both EF.ePDGId and EF.ePDGIdEm

Change-Id: I96fb129d178cfd7ec037989526da77899ae8d344
2020-05-11 10:53:18 +02:00
Supreeth Herle 7d77d2d5d0 Introduce function for converting bytes list in (hex or int) to string
This function is needed for conversion of bytes list output of TLV parser to representable string

Change-Id: I8c1e39ccf9fb517d465e73f69c720e7fdde02ded
2020-05-11 10:39:50 +02:00
Supreeth Herle 9837055f4f Import TLV parsing related function from https://github.com/mitshell/card
The functions are imported from the git commit 2a81963790e27eb6b188359af169c45afb6d3aaf from master branch

Change-Id: I5c7fdbd122e696d272f7480785d0c17ad2af138c
2020-05-11 09:04:41 +02:00
Supreeth Herle df33037e42 Extend parsing and printing of Service table to ISIM
Change-Id: I7657432df76e9de5e4e5fdecfd717d8363de8270
2020-04-27 12:29:00 +02:00
Supreeth Herle 69e5d27e3c Remove redundant functions
Change-Id: I2dfb5b662b8a4f351e4a629040b405c182e58e8d
2020-04-27 12:29:00 +02:00
Supreeth Herle 0c4d82d84a utils.py: Add helper method to parse and print Service Table
This method helps in printing Service Tables in EF.SST, EF.UST, EF.IST.
Takes hex string of Service table, parses it and prints available service along with its description.

Change-Id: Ie1e82e07ead2e28314a5794661e6b2ced0acd72a
2020-04-27 12:29:00 +02:00
Supreeth Herle 441c4a768f utils.py: Add helper method to parse Service Table
This method helps in parsing Service Tables in EF.SST, EF.UST, EF.EST, EF.IST.
Takes hex string as input and output a list of available/enabled services.

Change-Id: I9b72efdb84ba7be4a40928a008a59c67f6fb71d4
2020-04-01 11:06:59 +02:00
Vadim Yanitskiy 7ba2428de5 utils: fix dec_msisdn(): properly detect international numbers
We should match the whole value of ToN, not just one LSB bit.

Change-Id: Idc51f09b3420d827a75a1161372e4e97c3ddfbc1
2020-02-27 01:54:10 +07:00
Supreeth Herle 5a541016fb pySim-prog.py: add support for MSISDN programming
This change implements programming of EF.MSISDN as per 3GPP TS 31.102,
sections 4.2.26 and 4.4.2.3, excluding the following fields:

  - Alpha Identifier (currently 'FF'O * 20),
  - Capability/Configuration1 Record Identifier ('FF'O),
  - Extension1 Record Identifier ('FF'O).

This feature is introduced exclusively for sysmoUSIM-SJS1.
Othere SIM card types need to be tested.

Change-Id: Ie033a0ffc3697ae562eaa7a241a0f6af6c2b0594
2020-02-15 04:57:20 +07:00