Commit Graph

88 Commits

Author SHA1 Message Date
Philipp Maier a028c7d7aa pySim-shell: add method to match card profile to card
UICC and old SIM cards can be difficult to tell apart without prior
knowledge of the card. The ATR won't tell if the card is UICC or not.
The only remaining option is to try out if the card is able to handle
UICC APDUs. The same is true for 2G SIM cards. It is not guranteed that
every UICC card will have 2G functionality.

Lets add functionality to match a profile to the currently plugged card
by actively probing it.

Lets also add another profile to distinguish between UICC-only cards and
UICC cards that include SIM functionality.

Change-Id: If090d32551145f75c644657b90085a3ef5bfa691
Related: OS#5274
2021-11-19 13:21:32 +01:00
Philipp Maier 1db33115ea utils: cosmetic: remove stray comment
The comment is already covered by the help string, lets remove it.

Change-Id: Ide2080ddb898441b6af70e32511b33ced23d0023
2021-11-05 16:55:01 +00:00
Philipp Maier 796ca3daf9 commands: do not check SW manually, use send_apdu_checksw()
The transport layer provides a method send_apdu_checksw to send APDUs
and to be sure the SW is the expected one. Given that, there is no need
to verify the SW manually. The exception of send_apdu_checksw will catch
the problem and also display the SW in a human readable form.

Change-Id: I9ce556ac0b7bb21c5c5a27170c32af0152255b79
Related: OS#5275
2021-11-05 16:54:43 +00:00
Vadim Yanitskiy dbd5ed64d7 Python is not C: get rid of unnecessary semicolons
See https://www.python.org/dev/peps/pep-0008/.

Change-Id: I9de3bcd324b0a1b98af761678996eaae85f7f790
Related: OS#5292
2021-11-05 16:22:06 +03:00
Harald Welte 9a75410a88 utils: Fix BER-TLV tag decode for multi-byte tags
We cannot simply skip anything that has 0xFF as first byte to detect
the padding after the end of a TLV object:

0xFF may very well be a valid first octet of a multi-octet TAG:

Tags of private class (11) with constructed (1) payload will have 0xFF
as first octet.

So let's expand the check to only detect padding in case of either only
a single byte FF being left, or two FF following each other [with
whatever suffix].

Change-Id: I5d64ce9ef1d973804daabae0b15c2e2349e6fab9
2021-10-21 14:23:19 +02:00
Harald Welte f0885b1042 utils: Add bertlv_encode_tag()
We so far had decoders for BER-TLV tags, but no encoder yet.

Change-Id: I4183546bed9d6232ddcefad764f4e67afcf8b2ed
2021-05-30 19:27:37 +02:00
Harald Welte 6912b1b67d utils: Add 'raw' version of TLV tag decoders
The existing {comprehension,ber}tlv_parse_tag() functions are
decoding the tag to a high level of detail.  However, all the 3GPP
specs seem to deal with the 'raw' version, i.e something like
0xD1 as a single-byte tag with the class + constructed fields already
shifted next to the actual tag value.

Let's accommodate that with new *_parse_tag_raw() functions.

Change-Id: Ib50946bfb3b3ecd7942c423ac0f98b6c07649224
2021-05-29 22:21:38 +02:00
Harald Welte 9f3b44d6ff utils: COMPREHENSION-TLV support
Change-Id: I8d969382b73fa152ee09c456fa4aee428fb36285
2021-05-29 22:13:56 +02:00
Harald Welte c1475307c8 bertlv_parse_one: Also return remainder after end of TLV
Change-Id: I10ebd87f72ee934561118b768108e5dc76277660
2021-05-25 09:43:13 +02:00
Harald Welte de02718631 add unit tests for BER-TLV encoder/decoder functions
... and while at it resolve a bug in bertlv_parse_len()
discovered by those new tests.

Change-Id: I9f14dafab4f712c29224c4eb25cacab7885e2b68
2021-05-25 09:43:13 +02:00
Philipp Maier be18f2a419 utils: split string formatting from dec_addr_tlv
The function dec_addr_tlv() takes an encoded FQDN or IPv4 address and
fromats it into a human readable string that contains the human readable
form and the encoded hex form. Unfortunately this limits the usecase of
dec_addr_tlv. Lets split the string generation into a separate function
so that we can use dec_addr_tlv universally

Change-Id: Id017b0786089adac4d6c5be688742eaa9699e529
Related: OS#4963
2021-05-23 10:05:50 +00:00
Robert Falkenberg b07a3e9c87 Add codecs for EF_SPN and GSM strings via construct
This will replace the hand-crafted codec for EF_SPN
by a struct definition using the construct library.
Old encoders are updated and kept for API compatibility
but are not used internally anymore.

New data structures:
* Rpad(Adapter): Right-padded bytestring (0xff, adjustable)
* GsmStringAdapter(Adapter): Codec for "SMS default 7-bit
	coded alphabet as defined int TS 23.038" using
	the gsm0338 library.
* GsmString(n): Convenient wrapper of both above

Adjustments:
* utils: update+deprecate old dec_spn(), enc_spn()
* remove refs to deprecated functions

Change-Id: Ia1d3a3835933bac0002b7c52511481dd8094b994
2021-05-10 06:15:39 +02:00
Harald Welte 917d98c1a5 BER-TLV EF support (command, filesystem, shell)
This adds support for a new EF file type: BER-TLV files.  They are
different from transparent and linear fixed EFs in that they neither
operate on a byte stream nor fixed-sized records, but on BER-TLV encoded
objects.  One can specify a tag value, and the card will return the
entire TLV for that tag.

As indicated in the spec, the magic tag value 0x5C (92) will return a
list of tags existing in the file.

Change-Id: Ibfcce757dcd477fd0d6857f64fbb4346d6d62e63
2021-05-04 13:24:07 +02:00
Harald Welte 90441436a0 utils: Introduce CommandSet abstraction
This will allow us to match INS -> name and add more related
bits in the future (e.g. for decoding APDU traces)

Change-Id: I314ff15186dc05778ea12363cac0a310b6c7713c
2021-05-04 13:24:01 +02:00
Harald Welte 3de6ca2d20 utils: Introduce DataObject representation
Represents DataObject (DO) in the sense of ISO 7816-4.  Contrary to
'normal' TLVs where one simply has any number of different TLVs that may
occur in any order at any point, ISO 7816 has the habit of specifying
TLV data but with very specific ordering, or specific choices of tags at
specific points in a stream.  This is represented by DataObjectChoice,
DataObjectCollection and DataObjectSequence classes.

Change-Id: Iac18e7665481c9323cc7d22a3cd93e3da7869deb
2021-05-03 21:45:39 +02:00
Philipp Maier f39a4cb369 utils: specify type of parameter name in enc_spn
Related: OS#4963
Change-Id: I43a1e68afe9e756346bc0cfe8bda4ac665ac6c54
2021-05-03 17:08:37 +02:00
Philipp Maier e7d417955d ts_51_011, utils: fix Access Technology Identifier coding
When the Access Technology Identifier encoder sets the bits for E-UTRAN
it does not respect that bit "100" is also a valid bit combination that
encodes E-UTRAN WB-S1 and E-UTRAN NB-S1. Lets encode this bit
combination if the user is just specifying "E-UTRAN" without further
spefication of WB or NB.

The decoder only looks at bit 14 and decodes "1xx" always to "E-UTRAN".
This is not specific enough. Lets make sure that the decoder is
complementary to the encoder.

Change-Id: Ibfe8883a05f9ad6988d8e212cb9a598229954296
Related: OS#4963
2021-05-03 17:08:37 +02:00
Philipp Maier b919f8bd75 utils: fix dec_xplmn_w_act() and format_xplmn_w_act()
The function dec_xplmn_w_act(), which is also used by
format_xplmn_w_act() is using integer numbers as MCC/MNC representation.
This causes various problems since the information about leading zeros
gets lost.

Change-Id: I57f7dff80f48071ef9a3732ae1088882b127a6d4
2021-05-03 15:08:27 +00:00
Philipp Maier 6c5cd8031d utils: fix mcc/mnc encoding in dec_plmn (EF_PLMNsel)
The dec_plmn function takes an hexstring and returns the decoded MCC and
MNC as integer values. The result is then used by the json encoder in
EF_PLMNsel, which means the json output will contrary to the input, use
integer values instead of strings.

This is not correct since there may be leading zeros (e.g. mnc 01 and
001 both exist are different) which must be retained in order to know
the correct length of the MNC.

Related: OS#4963
Change-Id: I393e04836814d992d2a6d0a4e4e01850976d6e81
2021-05-03 15:07:50 +00:00
Philipp Maier e6f8d683e1 utils: specify paremeters of enc_plmn() as Hexstr
To prevent missunderstandings when using enc_plmn(), specify the input
and return parameters as Hexstr.

Change-Id: I57cf8e2de357650aef2a06fbffc7615ccb2a45b4
Related: OS#4963
2021-05-03 14:19:41 +00:00
Philipp Maier b46cb3ffa2 utils: fix encoding of EF.MSISDN
The encoding of EF.MSISDN is a bit unstrutured. The encoder function
does not return a valid result since it lacks the parameters
Capability/Configuration2 Record Identifier and Extension5 Record
Identifier, which are mandatory but can be set to 0xFF. Also the
encoder gets its input from pySim-shell, so it should have some
more input validation, especially when the user encodes an empty
string. The encoder and decoder function also do not have unit-tests.

Since the encoder now adds the missing two bytes by isself this does
not have to be done manually anymore, so cards.py needs to be
re-aligned.

For pySim-shell.py the encoder is used from ts_51_011.py. Unfortunately
it is used wrongly there. The optional Alpha Identifier is required
here as well.

Related: OS#4963
Change-Id: Iee5369b3e3ba7fa1155facc8fa824bc60e33b55b
2021-04-23 15:52:10 +02:00
Philipp Maier 80ce71f58c pySim-shell: separate export summary with a headline
the export summary is printed after the log entry for the last file
without separation. This is confusing because it looks like if the
summary would refer to the last file only. Lets add a headline to make
clear that the last few lines are the "Export summary"

Change-Id: I90771e525b2b114bdb41a8e90d298ca991c09c3d
Related: OS#4963
2021-04-21 15:11:34 +02:00
Harald Welte c9cdce3e02 fix various typos all over the code
Change-Id: Ic8392a951bf94f67b51e35bed95d0e856f7a9250
2021-04-11 12:20:29 +02:00
Harald Welte 5e749a79ac extend JSONEncoder to serialze 'bytes' style objects as hex strings
This means we can skip a lot of code that manually converts from
bytes to hex before JSON serialization.

Change-Id: I9c9eff0556d9d196e64553b5276e162f69d0c18f
2021-04-10 18:41:15 +02:00
Harald Welte d7a7e17a48 utils.py: Add missing dec_plmn function.
This function is being used e.g. for ADF.USIM/EF.FPLMN entries.

The EF_PLMNsel class also already uses a function by this name, we just
haven't had any actual implementation around.

Change-Id: Iacb45c90bb6491ebb89a477a85ef1f3129b38788
2021-04-06 22:53:18 +00:00
Harald Welte 522555710b utils.py: Add more type annotations
Change-Id: I50a0a07132890af0817f4ff0ce9fec53b7512522
2021-04-04 10:53:36 +02:00
Harald Welte 6e0458dda6 Move init_reader() from utils.py to transport/__init__.py
This avoids a circular dependency when introducing type annotations.

Change-Id: I168597ac14497fb188a15cb632f32452128bc1c6
2021-04-04 10:53:36 +02:00
Harald Welte 236a65f02f cosmetic: fix typo in comment
Change-Id: Iac8b310a470b3ad8dee5f61342fd5acedbbd6e5d
2021-04-03 20:33:31 +00:00
Harald Welte 1e45657e0f filesystem: fix various issues found by mypy
Change-Id: Ib4de80451614712bdf5377a3a5b86156008e2c42
2021-04-02 21:09:40 +02:00
Harald Welte ee3501fc62 Add more documentation to the classes/methods
* add type annotations in-line with PEP484
* convert existing documentation to follow the
  "Google Python Style Guide" format understood by
  the sphinx.ext.napoleon' extension
* add much more documentation all over the code base

Change-Id: I6ac88e0662cf3c56ae32d86d50b18a8b4150571a
2021-04-02 21:08:35 +02:00
Philipp Maier 78e32f2b36 utils: fix sw_match()
The SW_match function takes a given status word and compares it against
a pattern that may contain wildcards (x or ?). This works by creating a
masked version of the SW using a pattern first (each hex digit is
replaced by a wildcard charafter if the pattern has a wildcard in the
same position). Once this is done, the resulting masked version is
compared at the pattern. However, the current implementation can not
work, since it compares the input SW against the pattern to decide
wihich chrafters should be masked. The input SW never contains wildcard
charafters.

Change-Id: I805ad32160fcfcb8628bf919b64f7eee0fe03c7e
Related: OS#4963
2021-03-23 12:17:23 +00:00
Philipp Maier 5d3e2592f7 pySim-shell: add "dir" command.
pysim-shell does not have a convinient way to list the files available
in one directory. Tab completion currently is the only way to obtain a
list of the available files. Lets add a dir command to print a file
list.

Change-Id: Ic06a60e0a0ec54d9bb26e151453ceb49d69e3df7
Related: OS#4963
2021-03-18 17:19:17 +01:00
Philipp Maier 47236500fe utils: add is_hex function to check hex strings
since we have added pySim-shell.py that has a lot of locations where the
user can enter hexadecimal data there is an increased need for input
validation. Lets add a central is_hex function that verifies hex
strings.

Change-Id: Ia29a13c9215357dd2adf141f2ef222c823f8456d
Related: OS#4963
2021-03-12 07:35:37 +00:00
Vadim Yanitskiy c8458e2477 pySim/utils.py: fix 3-digit MNC encoding in enc_plmn()
The bug that was attempted to be fixed in [1] actually was in the
encoding API - pySim.utils.enc_plmn().  According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit (E)HPLMN shall be encoded
as shown below (ASCII-art interpretation):

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 2  |  MNC Digit 1  |
  +---+---+---+---+---+---+---+---+

while pySim.utils.enc_plmn() would produce the following:

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 1  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MNC Digit 2  |
  +---+---+---+---+---+---+---+---+

Initially the _decoding_ API was correct, but then got changed in
[1] to follow buggy pySim's encoding API.  As a result, a (E)HPLMN
programmed with pySim-prog.py would look correct if verified by
pySim-read.py, but the actual file content would be wrong.

This situation shows that our 'program-read-match' build verification
approach alone is insignificant.  The lack of unit test coverage,
at least for the core parts of the project, makes it possible to have
symmetrical bugs in both encoding and decoding API parts unnoticed.

This problem was found while trying to enable dead unit tests in [3].
Change [1] that introduced a symmetrical bug is reverted in [2].

Change-Id: Ic7612502e1bb0d280133dabbcb5cb146fc6997e5
Related: [1] I799469206f87e930d8888367890babcb8ebe23a9
Related: [2] If6bf5383988ad442e275efc7c5a159327d104879
Related: [3] I4d4facfabc75187acd5238ff4d0f26022bd58f82
2021-03-12 01:00:32 +01:00
Vadim Yanitskiy b271be3dc0 Revert "utils.py: Fix for parsing MNC"
This reverts commit bdf3d3597b, which
broke pySim.utils.dec_mnc_from_plmn().  According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit EHPLMN shall be encoded
as shown below (ASCII-art interpretation):

    0   1   2   3   4   5   6   7
  +---+---+---+---+---+---+---+---+
  |  MCC Digit 2  |  MCC Digit 1  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 3  |  MCC Digit 3  |
  +---+---+---+---+---+---+---+---+
  |  MNC Digit 2  |  MNC Digit 1  |
  +---+---+---+---+---+---+---+---+

So the original implementation was correct, and we even had a unit
test for it.  Most likely, the SIM card itself was programmed
incorrectly?

Makes 'testDecMNCfromPLMN_threeDigitMNC' pass again.

Change-Id: If6bf5383988ad442e275efc7c5a159327d104879
2021-03-12 01:00:26 +01:00
Denis 'GNUtoo' Carikli 79f5b6080b Python 2 is deprecated, remove backwards compatibility chunks
pySim has already been migrated to Python 3 in another change [1],
and the build verification has been migrated to Debian 10 with
Python 3.7.  However, there is still some backwards compatibility
code left.  Let's get rid of it.

[1] Ic78da9c03e99f59d142c93394051bbc2751f0205

Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Tweaked-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Change-Id: I430d173535e0cd5bb895b9dfc9070cbc40cfc8ff
2021-03-07 20:33:46 +01:00
Harald Welte ab34fa895e pySim/utils.py: Attempt to support pycryptodpme
This should resolve the following error when using with pycryptodome
instead of pycrypto:

TypeError: new() missing 1 required positional argument: 'mode'

Change-Id: Ibd3ca00d62b864909f5e89e0feb350268157a4ca
Related: OS#5060
2021-03-05 20:39:18 +00:00
Harald Welte eab8d2adf7 fix TypeError in derive_milenage_opc()
In 4f6ca43e1f we started to use
the bytearray type as 'b' type, but PyCrypto insists on getting
a bytes type.

This fixes the following Exception:
TypeError: argument 1 must be read-only bytes-like object, not bytearray

Change-Id: If2a727ed417ffd56c0f7d7b4e9f633d67fde5ced
Closes: OS#5060
2021-03-05 20:38:50 +00:00
Harald Welte 4f6ca43e1f start using python3 bytearray for our b2h/h2b types
The code was written long ago, when the python3 bytearray type
probably didn't exist yet, or was at least not known.  Let's stop
using string types with binary bytes inside, and instead standardize
on two types:
 * bytearray for binary data
 * string for hexadecimal nibbles representing that binary data

Change-Id: I8aca84b6280f9702b0e2aba2c9759b4f312ab6a9
2021-03-03 08:37:50 +01:00
Harald Welte 67d551a443 move SW matching to a generic utility function
This will allow using it outside the transport/__init__.py

Change-Id: Id26dfefa85d91e3b3a23e0049f3b833e29cb1cef
2021-03-02 14:26:32 +01:00
Harald Welte 79b5ba4bdf utils.py: de-couple sanitize_pin_adm from argparse 'opts'
This allows the function to be re-used in other contexts

Change-Id: I116e85acca3aeb0a0c24f74653c500ac2dc1d844
2021-03-02 07:50:23 +01:00
Vadim Yanitskiy 6d5e0c9272 Remove Python shebang from files where it's not needed
Change-Id: I1d08544c37f50416acf8dc30139c572c029790d0
2021-03-01 17:33:46 +01:00
Philipp Maier c8caec2933 utils: catch exceptions during reader initalitation
Failed reader initializations happen frome time to time, mostly because
of messed up commandline arguments. This results in ugly crashes very
often. Lets control this a bit by catching the exception string and
print it.

Change-Id: I313017355da01bbef7c3d3f1899555aadb509319
2021-02-25 15:16:07 +01:00
herlesupreeth bdf3d3597b utils.py: Fix for parsing MNC
This commit fixes the incorrect parsing of MNC from PLMN.
Previously its was parsing PLMN string 130062 as MCC 310 MNC 260,
whereas it should be MCC 310 MNC 026.

(The SIM was programmed with MCC 310 and MNC 026)

Change-Id: I799469206f87e930d8888367890babcb8ebe23a9
2021-02-11 07:02:50 +01:00
Supreeth Herle 43fd03b627 utils.py: Support IPv4 decoding for Address TLV object present in EF.ePDGId and EF.ePDGIdEm
Change-Id: I96c30c1fcc03e50c55e9db7e6a18297a3b1d889d
2021-01-05 11:46:41 +01:00
Supreeth Herle 654eca72c9 utils.py: Support IPv4 encoding for Address TLV object present in EF.ePDGId and EF.ePDGIdEm
Change-Id: Id46a44257d09c98ad5e0b7558e25e2bc52b23978
2021-01-05 11:46:41 +01:00
Supreeth Herle 556b0fe262 utils.py: Add helper method to get type of address (FQDN, IPv4, IPv6)
The function takes address string as input, then validates it and returns the type.
Return: 0x00 (FQDN), 0x01 (IPv4), 0x02 (IPv6), None (Bad address format)

Change-Id: I0fabd4f17bbb11f6bb191c1a9e6276427f9d001f
2021-01-05 11:46:41 +01:00
Supreeth Herle 3b342c2f14 Re-purpose helper method to be used for parsing Address TLV Object in general
The Address TLV object is used in EF.P-CSCF Address, EF.ePDGId and EF.ePDGIdEm.
See 3GPP TS 31.102 version 13.4.0 Release 13, section 4.2.8, 4.2.102 and 4.2.104.

Address TLV Object format
Tag (1 Byte) - '80'
Length (1 Byte)
Address Type (1 Byte) - '00' (FQDN), '01' (IPv4), '02' (IPv6)
Address (Address Length Bytes)

Change-Id: Ifd8a240f6b5c7736e58a8151295c30ec5b32ed5f
2021-01-05 10:06:19 +01:00
herlesupreeth 3a261d31d5 utils.py: Bugfix for parsing non-programmed EFePDGSelection
Change-Id: I3a16af785d8ae9ea8730771367bba2d50690b414
2021-01-05 10:06:19 +01:00
Supreeth Herle 95ec772b61 utils.py: Add helper method to encode ePDGSelection info TLV
Encodes ePDGSelection info TLV so it can be stored at EF.ePDGSelection or EF.ePDGSelectionEm.
See 3GPP TS 31.102 version 15.2.0 Release 15, section 4.2.104 and 4.2.106.

Take original hex string of EF.ePDGSelection or EF.ePDGSelectionEm, MCC,
MNC, ePDG priority for PLMN and ePDG FQDN Format to use for PLMNas input
and outputs the encoded hex string.

Change-Id: Ia7292d33783c770a3bb91b081c671af36bbb907f
2020-10-21 08:32:57 +02:00