Commit Graph

299 Commits

Author SHA1 Message Date
Vadim Yanitskiy 080cc9f794 README.md: add notes about the new 'cmd2' dependency
Change-Id: I314317ab547bc32497839fe70e7a6f6b66bcc8ef
2021-03-12 15:41:01 +01:00
Philipp Maier 7744b6e9d1 filesystem: be case insensitive when selecting files by fid (HEX)
The file identifier (and allso application ids for ADFs), are
hexadecimal. We should be case insensitive when accepting hex
identifiers but file names should still be full matched.

Change-Id: Ibe283a108ddc9058af77c823b7222db555e1e0f6
Related: OS#4963
2021-03-12 07:35:37 +00:00
Philipp Maier 47236500fe utils: add is_hex function to check hex strings
since we have added pySim-shell.py that has a lot of locations where the
user can enter hexadecimal data there is an increased need for input
validation. Lets add a central is_hex function that verifies hex
strings.

Change-Id: Ia29a13c9215357dd2adf141f2ef222c823f8456d
Related: OS#4963
2021-03-12 07:35:37 +00:00
Philipp Maier 786f781a5f filesystem: add flags to filter selectables
When requesting what DF/EF/ADF are selectable it is useful to have some
control of what we do not want in the resulting list.

Change-Id: Idb50a512bfdbfdf2e98f2ce0e89928cb0ff19f5e
Related: OS#4963
2021-03-12 07:35:37 +00:00
Vadim Yanitskiy 3b51f436a4 pySim/exceptions.py: fix referencing an instance member
Change-Id: I6debfc03e9847b907f959e681234daf21df41656
2021-03-12 07:32:17 +00:00
Vadim Yanitskiy d61da8a64c contrib/jenkins.sh: enable automatic execution of unit tests
Change-Id: I7b4bb49efd5e6ae284da063b7899e368ea4f1e22
Related: I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-12 01:13:15 +01:00
Vadim Yanitskiy 46c49d5256 tests/test_utils.py: update expectations for format_xplmn_w_act()
Change-Id: I520328e3490cc3a333d2daad84e745d115196626
2021-03-12 01:12:11 +01:00
Vadim Yanitskiy c8458e2477 pySim/utils.py: fix 3-digit MNC encoding in enc_plmn()
The bug that was attempted to be fixed in [1] actually was in the
encoding API - pySim.utils.enc_plmn().  According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit (E)HPLMN shall be encoded
as shown below (ASCII-art interpretation):

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 2  |  MNC Digit 1  |
  +---+---+---+---+---+---+---+---+

while pySim.utils.enc_plmn() would produce the following:

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 1  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MNC Digit 2  |
  +---+---+---+---+---+---+---+---+

Initially the _decoding_ API was correct, but then got changed in
[1] to follow buggy pySim's encoding API.  As a result, a (E)HPLMN
programmed with pySim-prog.py would look correct if verified by
pySim-read.py, but the actual file content would be wrong.

This situation shows that our 'program-read-match' build verification
approach alone is insignificant.  The lack of unit test coverage,
at least for the core parts of the project, makes it possible to have
symmetrical bugs in both encoding and decoding API parts unnoticed.

This problem was found while trying to enable dead unit tests in [3].
Change [1] that introduced a symmetrical bug is reverted in [2].

Change-Id: Ic7612502e1bb0d280133dabbcb5cb146fc6997e5
Related: [1] I799469206f87e930d8888367890babcb8ebe23a9
Related: [2] If6bf5383988ad442e275efc7c5a159327d104879
Related: [3] I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-12 01:00:32 +01:00
Vadim Yanitskiy b271be3dc0 Revert "utils.py: Fix for parsing MNC"
This reverts commit bdf3d3597b, which
broke pySim.utils.dec_mnc_from_plmn().  According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit EHPLMN shall be encoded
as shown below (ASCII-art interpretation):

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 2  |  MNC Digit 1  |
  +---+---+---+---+---+---+---+---+

So the original implementation was correct, and we even had a unit
test for it.  Most likely, the SIM card itself was programmed
incorrectly?

Makes 'testDecMNCfromPLMN_threeDigitMNC' pass again.

Change-Id: If6bf5383988ad442e275efc7c5a159327d104879
2021-03-12 01:00:26 +01:00
Vadim Yanitskiy 4ae7c49076 pySim/utils_test.py: prepare this to be executed on Jenkins
As it turns out, we had this set of unit tests since 2018, but
so far they were not executed during the build verification.

Let's fix this:

  * run unittest in discovery mode for all files in 'tests/' (commented out);
  * rename this file, so it can be automatically detected and executed;
  * properly import the API to be tested.

Currently 2 out of 16 unit tests are failing, so we need to get
them passing first and then uncomment the unittest execution.

Change-Id: I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-11 23:54:15 +01:00
Vadim Yanitskiy 7d57edfe2d pySim/utils_test.py: use proper shebang for this executable
Change-Id: I8ad843643b5a97d41a12f74e2ada49088a54974d
2021-03-11 23:01:55 +01:00
Vadim Yanitskiy 3e58d38bdf Get rid of Python 2 specific compatibility leftovers
Change-Id: I0068caa775d89349db2ad378fad22e89832b8d20
2021-03-11 22:59:21 +01:00
Vadim Yanitskiy 5452d64120 ts_51_011: fix bitmask compositing in EF_xPLMNwAcT.enc_act()
This commit fixes two problems (found by semgrep):

  * "'foo' and 'bar' in list" is incorrect, because it's interpreted
    as "'foo' and ('bar' in list)".  Strings with a non-zero length
    evaluate to True, thus it's True if at least 'bar' is present.

  * Copy-pasted 'E-UTRAN NB-S1' checked two times.

The first condition is redundant, and the whole block can be
re-implemented using two independent 'if' statements.

Change-Id: Iceb66160cfb571db8879d3810c55d252c763d320
2021-03-07 21:52:13 +01:00
Denis 'GNUtoo' Carikli 79f5b6080b Python 2 is deprecated, remove backwards compatibility chunks
pySim has already been migrated to Python 3 in another change [1],
and the build verification has been migrated to Debian 10 with
Python 3.7.  However, there is still some backwards compatibility
code left.  Let's get rid of it.

[1] Ic78da9c03e99f59d142c93394051bbc2751f0205

Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Tweaked-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Change-Id: I430d173535e0cd5bb895b9dfc9070cbc40cfc8ff
2021-03-07 20:33:46 +01:00
Vadim Yanitskiy 1f8acd9884 transport/pcsc: work around Python 3.5 bug: guard disconnect()
Unfortunately, Debian ships old Python (3.5 vs 3.8) and old pyscard
(1.9.4 vs 1.9.9). Calling PCSCCardConnection.disconnect() from a
destructor causes warnings about ignored exceptions:

  AttributeError: 'NoneType' object has no attribute 'disconnect'
  AttributeError: 'NoneType' object has no attribute 'setChanged'
  AttributeError: 'NoneType' object has no attribute 'SCardDisconnect'
  TypeError: 'NoneType' object is not callable

All these exceptions happen in pyscard's own destructors.

Change-Id: I9c644bc5fe9791b141a30bfc13647d77937a82ee
2021-03-07 19:26:08 +00:00
Harald Welte ab34fa895e pySim/utils.py: Attempt to support pycryptodpme
This should resolve the following error when using with pycryptodome
instead of pycrypto:

TypeError: new() missing 1 required positional argument: 'mode'

Change-Id: Ibd3ca00d62b864909f5e89e0feb350268157a4ca
Related: OS#5060
2021-03-05 20:39:18 +00:00
Harald Welte eab8d2adf7 fix TypeError in derive_milenage_opc()
In 4f6ca43e1f we started to use
the bytearray type as 'b' type, but PyCrypto insists on getting
a bytes type.

This fixes the following Exception:
TypeError: argument 1 must be read-only bytes-like object, not bytearray

Change-Id: If2a727ed417ffd56c0f7d7b4e9f633d67fde5ced
Closes: OS#5060
2021-03-05 20:38:50 +00:00
Harald Welte b2edd14475 Add a new pySim-shell program
pySim-prog was nice when there were only 5 parameters on a SIM that we
could program, and where the use case was pretty limited.  Today, we
have SIM/USIM/ISIM cards with hundreds of files and even more parameters
to program.  We cannot add a command line argument for each file to
pySim-prog.

Instead, this introduces an interactive command-line shell / REPL,
in which one can navigate the file system of the card, read and update
files both in raw format and in decoded/parsed format.

The idea is primarily inspired by Henryk Ploatz' venerable
cyberflex-shell, but implemented on a more modern basis using
the cmd2 python module.

See https://lists.osmocom.org/pipermail/simtrace/2021-January/000860.html
and https://lists.osmocom.org/pipermail/simtrace/2021-February/000864.html
for some related background.

Most code by Harald Welte. Some bug fixes by Philipp Maier
have been squashed.

Change-Id: Iad117596e922223bdc1e5b956f84844b7c577e02
Related: OS#4963
2021-03-03 08:43:38 +01:00
Harald Welte 4f6ca43e1f start using python3 bytearray for our b2h/h2b types
The code was written long ago, when the python3 bytearray type
probably didn't exist yet, or was at least not known.  Let's stop
using string types with binary bytes inside, and instead standardize
on two types:
 * bytearray for binary data
 * string for hexadecimal nibbles representing that binary data

Change-Id: I8aca84b6280f9702b0e2aba2c9759b4f312ab6a9
2021-03-03 08:37:50 +01:00
Harald Welte 85484a977d commands.py: Introduce a real select_file() method
This method, like select_adf(), only selects a single file ID
and unlike select_path() returns the actual status words returned by the
card.

Change-Id: I8bc86654c6d79f2428e196cc8a401e12d93a676b
2021-03-02 14:26:32 +01:00
Harald Welte c0499c8330 commands.py: rename select_file() to select_path()
In reality, the function is not a simple avstraction around the SELECT
command, but it iterates over a list/path and selects at each element.

Change-Id: I63e01155de4ae47aeed8500708c0eb6580c7b8d1
2021-03-02 14:26:32 +01:00
Harald Welte 67d551a443 move SW matching to a generic utility function
This will allow using it outside the transport/__init__.py

Change-Id: Id26dfefa85d91e3b3a23e0049f3b833e29cb1cef
2021-03-02 14:26:32 +01:00
Harald Welte e79cc8069a introduce SwMatchError exception
This allows callers further up the stack to catch the exception and
interpret it in some way (like decoding the number of remaining tries
in case of authentication errors)

Change-Id: Ia59962978745aef7038f750fa23f8dfc820645f4
2021-03-02 08:06:13 +01:00
Harald Welte 79b5ba4bdf utils.py: de-couple sanitize_pin_adm from argparse 'opts'
This allows the function to be re-used in other contexts

Change-Id: I116e85acca3aeb0a0c24f74653c500ac2dc1d844
2021-03-02 07:50:23 +01:00
Harald Welte a670425088 cards.py: SJS1 + SJA2: Implement + Expose verify_adm() method
SJS1 and SJA2 card types don't use the generic verify_adm()
method of the Card base class, so they must override it with their
own methods.  Only this way application code can call card.verify_adm()
irrespective of the card type.

Change-Id: I05f7f3280873f006310266867f04a9ce1b0a63af
2021-03-02 07:48:22 +01:00
Vadim Yanitskiy 6d5e0c9272 Remove Python shebang from files where it's not needed
Change-Id: I1d08544c37f50416acf8dc30139c572c029790d0
2021-03-01 17:33:46 +01:00
Philipp Maier 289fd28091 serial: do not try to close non existing serial connection
The SerialSimLink only has an _sl member if serial initalization was
successfull. If we close a serial connection, check if we even have the
_sl member. Otherwise move on silently.

Change-Id: Ic3f3f5e50d780f424da7d0be5733d7167bb7159c
2021-02-25 16:27:08 +01:00
Philipp Maier 92bdd5e901 serial: don't crash if the device does not exist
The most common reason for pySim to crash is when it is executed without
commandline parameters. Then pySim will expect a serial reader on
/dev/ttyUSB0 since this is the default. Lets check if /dev/ttyUSB0 even
exists before trying to open it.

Change-Id: I7545c728b531e9a796eee8f80f0b08d4097f8399
2021-02-25 16:27:08 +01:00
Philipp Maier c8caec2933 utils: catch exceptions during reader initalitation
Failed reader initializations happen frome time to time, mostly because
of messed up commandline arguments. This results in ugly crashes very
often. Lets control this a bit by catching the exception string and
print it.

Change-Id: I313017355da01bbef7c3d3f1899555aadb509319
2021-02-25 15:16:07 +01:00
herlesupreeth bdf3d3597b utils.py: Fix for parsing MNC
This commit fixes the incorrect parsing of MNC from PLMN.
Previously its was parsing PLMN string 130062 as MCC 310 MNC 260,
whereas it should be MCC 310 MNC 026.

(The SIM was programmed with MCC 310 and MNC 026)

Change-Id: I799469206f87e930d8888367890babcb8ebe23a9
2021-02-11 07:02:50 +01:00
herlesupreeth cebf8b198b pySim-read.py: Add support for reading USIM+ISIM data from third-party UICC
Change-Id: Id8b95630e90cb5833482da2690e423e7adefb95b
2021-01-21 06:13:03 +01:00
Joachim Steiger 06a1256b67 Readme.md: update debian deps, add python3-yaml
Change-Id: Ic0d53ec3f312afee9d28f2f63d35e5c56dfd4686
2021-01-20 19:04:41 +01:00
Supreeth Herle be3b64167a pySim-read.py: Enable reading of UICC IARI from ISIM
Change-Id: Iba222421f2fcc2b9b12605608bf640f7627904d0
2021-01-05 15:59:38 +01:00
Supreeth Herle a97944b6ca sysmoISIM-SJA2: Enable programming of EF.ACC
Change-Id: If318117a2339399191dfc3fcec856d9247a034fb
2021-01-05 15:59:38 +01:00
Supreeth Herle c60192375e sysmoISIM-SJA2: Enable programming of EF.MSISDN
Change-Id: I8ed27142009a50a1cc31a9f2e0e854a53add6e89
2021-01-05 15:59:37 +01:00
Supreeth Herle be7007e1d9 sysmoISIM-SJA2: Add support for programming IMS public user identity
This EF contains one or more records, with each record able
to hold a public SIP Identity (SIP URI) of the user. EF.IMPU consist of URI TLV data
object values see IETF RFC 3261. The URI shall be encoded to an octet string according
to UTF-8 encoding rules as specified in IETF RFC 3629

./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org --pcscf pcscf.ims.testop.org --ims-hdomain ims.testop.org --impi 1234567895@ims.testop.org --impu sip:5987654321@ims.testop.org

Change-Id: If10bc2e50eca390b81755b5cc7211e695233612d
2021-01-05 11:46:41 +01:00
Supreeth Herle a5bd9684d3 sysmoISIM-SJA2: Add support for programming IMS private user identity
This EF contains the private user identity of the user. EF.IMPI consist of
NAI TLV data object values see IETF RFC 2486. The NAI shall be encoded to
an octet string according to UTF-8 encoding rules as specified in IETF RFC 3629

./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org --pcscf pcscf.ims.testop.org --ims-hdomain ims.testop.org --impi 1234567895@ims.testop.org

Change-Id: Ic1ccf99b5aa45297ef1e43a43373df603f756379
2021-01-05 11:46:41 +01:00
Supreeth Herle 0c02d8a57b pySim-read.py: Enable reading of EF.IMPU from ISIM
IMS public user identity (IMPU)
As per TS1.103, this EF contains one or more records, with each record able
to hold a public SIP Identity (SIP URI) of the user. EF.IMPI consist of URI TLV data
object values see IETF RFC 3261. The URI shall be encoded to an octet string according
to UTF-8 encoding rules as specified in IETF RFC 3629

Reading of EF.IMPU is achieved by first selecting the ISIM application using its AID.
This is followed by selecting EF.IMPU with File ID - 6f04 in ADF.ISIM

Change-Id: Icf78a564aeaf4254658d3b018ff57dfc4b987e6f
2021-01-05 11:46:41 +01:00
Supreeth Herle 3f67f9c1d3 pySim-read.py: Enable reading of EF.IMPI from ISIM
IMS private user identity (IMPI)
As per TS1.103 version 14.2.0, this EF contains the private user identity of the user.
EF.IMPI consist of NAI TLV Data object.The NAI shall be encoded to an octet string
according to UTF-8 encoding rules as specified in IETF RFC 3629.

Reading of EF.IMPI is achieved by first selecting the ISIM application using its AID.
This is followed by selecting EF.IMPI with File ID - 6f02 in ADF.ISIM

Change-Id: I8d8e76e3f6b9ca7a0be262fee99cd5a397edbefa
2021-01-05 11:46:41 +01:00
Supreeth Herle 28484d03e3 Update ISIM IST table as per the 3GPP TS 31.103 specification V15.6.0
Change-Id: I8f6a96721beb9621ac453715cf2d0303989cfe85
2021-01-05 11:46:41 +01:00
herlesupreeth 1279085f7e sysmoISIM-SJA2: Add support for programming IPv4 address for PCSCF
This commit allows programming of IPv4 or FQDN in EF.PCSCF

Example:

./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid 172.24.15.20 --pcscf 99.100.80.102 --ims-hdomain testims.org

Change-Id: I247a5413e9e5fef6b9d7b6cb8442313e72f9156a
2021-01-05 11:46:41 +01:00
Supreeth Herle c491dc019f sysmoISIM-SJA2: Add support for programming IPv4 address for Home ePDG Identifier
This commit allows programming of IPv4 or FQDN in EF.ePDGId

Example:

./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid 172.24.15.20 --pcscf pcscf.testims.org --ims-hdomain testims.org

Change-Id: I10a708d3e0c3ae398c942e3529e364dfe4bb23e7
2021-01-05 11:46:41 +01:00
Supreeth Herle 43fd03b627 utils.py: Support IPv4 decoding for Address TLV object present in EF.ePDGId and EF.ePDGIdEm
Change-Id: I96c30c1fcc03e50c55e9db7e6a18297a3b1d889d
2021-01-05 11:46:41 +01:00
Supreeth Herle 654eca72c9 utils.py: Support IPv4 encoding for Address TLV object present in EF.ePDGId and EF.ePDGIdEm
Change-Id: Id46a44257d09c98ad5e0b7558e25e2bc52b23978
2021-01-05 11:46:41 +01:00
Supreeth Herle 4779034f9e sysmoISIM-SJA2: Fill unused bytes of Home ePDGId with 'f'
Change-Id: Ia0464f230afcb0f37465d3ed0dfd8f417b53b0c3
2021-01-05 11:46:41 +01:00
Supreeth Herle 79f43dda3d sysmoISIM-SJA2: Add support for programming IMS Home Network Domain Name
As per 3GPP TS 31.103, this EF (DOMAIN) can found under ADF.ISIM at File Id 6f03.

The Home Network Domain Name, i.e. FQDN shall be encoded to an octet string
according to UTF-8 encoding rules as specified in IETF RFC 3629 [27].
The tag value of the Home Network Domain Name TLV data object shall be '80'.

Example:

./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org --pcscf pcscf.testims.org --ims-hdomain testims.org

Change-Id: I3c823203aee88734ae423e4ad73da1027a4eaeed
2021-01-05 11:46:41 +01:00
Supreeth Herle 556b0fe262 utils.py: Add helper method to get type of address (FQDN, IPv4, IPv6)
The function takes address string as input, then validates it and returns the type.
Return: 0x00 (FQDN), 0x01 (IPv4), 0x02 (IPv6), None (Bad address format)

Change-Id: I0fabd4f17bbb11f6bb191c1a9e6276427f9d001f
2021-01-05 11:46:41 +01:00
Supreeth Herle 44e046240e sysmoISIM-SJA2: Disable Service 95, 99, 115 in EF.UST in case ISIM is present
As per TS 31.102 version 15.2.0 Release 15, section 4.2.8, EFUST (USIM Service Table),
Service n°95, n°99 and n°115 shall not be declared available if an ISIM application is present on the UICC.

Change-Id: Id9709746de99585ad31c4e9659323484fda87b14
2021-01-05 11:46:41 +01:00
Supreeth Herle 05b2807168 pySim-read.py: Enable reading of Home Network Domain Name from ISIM
As per TS1.103 version 14.2.0, this EF contains the home operator's network domain name.
The Home Network Domain Name, i.e. FQDN shall be encoded to an octet string
according to UTF-8 encoding rules as specified in IETF RFC 3629

Change-Id: Ia3c68c717d105e10d52a8e9d170480da2ad7d65a
2021-01-05 11:46:40 +01:00
Supreeth Herle cf727f2733 sysmoISIM-SJA2: Add support for programming Proxy Call Session Control Function address
Example:

./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org --pcscf pcscf.testims.org

Change-Id: Ic654baa93e2ecb91ced596b49dde4c1f208ecda2
2021-01-05 10:44:12 +01:00