The function dec_addr_tlv() takes an encoded FQDN or IPv4 address and
fromats it into a human readable string that contains the human readable
form and the encoded hex form. Unfortunately this limits the usecase of
dec_addr_tlv. Lets split the string generation into a separate function
so that we can use dec_addr_tlv universally
Change-Id: Id017b0786089adac4d6c5be688742eaa9699e529
Related: OS#4963
The method update_record as a "force_len" parameter, which is somewhat
irretatating. Some explainatory comments and a reformat of the if
statement will help to make it more understandable to the api user.
In the non force_len case the method determines the record length from
the select response and throws an exception if the data input does not
match that length. This makes sense if the data input exceeds the
record length of the file but if the data input is less then the record
length the situation is fixable by padding the input with 0xff. This
also a quite common case because in some situation it is not guaranteed
that the data will fill the entire record.
Change-Id: I9a5df0e46c3dd2e87d447c5c01cf15844b0eed07
Related: OS#4963
Even though Card, UsimCard and IsimCard are abstract classes which are
normally only used to inherit from mit may make sense to pre-populate
the name property with some meaningful value.
Change-Id: Id643e1f83718aea073e7200aecbf2db2def8652f
This will replace the hand-crafted codec for EF_SPN
by a struct definition using the construct library.
Old encoders are updated and kept for API compatibility
but are not used internally anymore.
New data structures:
* Rpad(Adapter): Right-padded bytestring (0xff, adjustable)
* GsmStringAdapter(Adapter): Codec for "SMS default 7-bit
coded alphabet as defined int TS 23.038" using
the gsm0338 library.
* GsmString(n): Convenient wrapper of both above
Adjustments:
* utils: update+deprecate old dec_spn(), enc_spn()
* remove refs to deprecated functions
Change-Id: Ia1d3a3835933bac0002b7c52511481dd8094b994
The encoder for EF_SPN is passing the 'spn' parameter (which is a list)
directly to enc_spn without taking it apart first.
Change-Id: I0a405793c8909d4279e634b93dcb76e5cb2963f3
Related: OS#4963
This adds support for a new EF file type: BER-TLV files. They are
different from transparent and linear fixed EFs in that they neither
operate on a byte stream nor fixed-sized records, but on BER-TLV encoded
objects. One can specify a tag value, and the card will return the
entire TLV for that tag.
As indicated in the spec, the magic tag value 0x5C (92) will return a
list of tags existing in the file.
Change-Id: Ibfcce757dcd477fd0d6857f64fbb4346d6d62e63
The Access Mode (AM) and Security Condition (SC) DOs are incredibly
convoluted, so we need a lot of code to properly decode them.
Change-Id: If4f0725a849d41fd93de327ed00996d8179f2b0e
This will allow us to match INS -> name and add more related
bits in the future (e.g. for decoding APDU traces)
Change-Id: I314ff15186dc05778ea12363cac0a310b6c7713c
Represents DataObject (DO) in the sense of ISO 7816-4. Contrary to
'normal' TLVs where one simply has any number of different TLVs that may
occur in any order at any point, ISO 7816 has the habit of specifying
TLV data but with very specific ordering, or specific choices of tags at
specific points in a stream. This is represented by DataObjectChoice,
DataObjectCollection and DataObjectSequence classes.
Change-Id: Iac18e7665481c9323cc7d22a3cd93e3da7869deb
When the Access Technology Identifier encoder sets the bits for E-UTRAN
it does not respect that bit "100" is also a valid bit combination that
encodes E-UTRAN WB-S1 and E-UTRAN NB-S1. Lets encode this bit
combination if the user is just specifying "E-UTRAN" without further
spefication of WB or NB.
The decoder only looks at bit 14 and decodes "1xx" always to "E-UTRAN".
This is not specific enough. Lets make sure that the decoder is
complementary to the encoder.
Change-Id: Ibfe8883a05f9ad6988d8e212cb9a598229954296
Related: OS#4963
The function dec_xplmn_w_act(), which is also used by
format_xplmn_w_act() is using integer numbers as MCC/MNC representation.
This causes various problems since the information about leading zeros
gets lost.
Change-Id: I57f7dff80f48071ef9a3732ae1088882b127a6d4
The dec_plmn function takes an hexstring and returns the decoded MCC and
MNC as integer values. The result is then used by the json encoder in
EF_PLMNsel, which means the json output will contrary to the input, use
integer values instead of strings.
This is not correct since there may be leading zeros (e.g. mnc 01 and
001 both exist are different) which must be retained in order to know
the correct length of the MNC.
Related: OS#4963
Change-Id: I393e04836814d992d2a6d0a4e4e01850976d6e81
To prevent missunderstandings when using enc_plmn(), specify the input
and return parameters as Hexstr.
Change-Id: I57cf8e2de357650aef2a06fbffc7615ccb2a45b4
Related: OS#4963
Previous implementation waits 300ms for response after
each command issued. But many commands finish earlier.
This patch improves the command execution time by frequently
checking for the response to complete (i.e. ends with
OK or ERROR), or the occurence of a timeout (default 200ms).
Timeout can be adapted per command to support long response
times of certain commands like AT+COPS=? (network search)
Execution time benchmark (20 AT commands/responses):
Previous: 6.010s (100.0%)
New code: 0.045s ( 0.7%)
Change-Id: I69b1cbc0a20d54791e5800bf27ebafc2c8606d93
Let's just use the scope limited TemporaryDirectory() instead, so
the temporary directory will be removed by Python automatically.
pySim/filesystem.py:679:16: E0602: Undefined variable 'shutil' (undefined-variable)
Change-Id: I4ea833fd79f4342c33899124379be509ba1e35ed
pySim/transport/__init__.py:86:15: E1101:
Instance of 'LinkBase' has no '_send_apdu_raw' member;
maybe 'send_apdu_raw'? (no-member)
Change-Id: I14fcdceca5d1e35491b6ad98f96b4276b69b2fc1
This method has been removed [1] in Python 3.0:
pySim/cards.py:581:14: E1101: Instance of 'dict' has no 'iteritems' member (no-member)
pySim/cards.py:591:24: E1101: Instance of 'dict' has no 'iteritems' member (no-member)
[1] https://wiki.python.org/moin/Python3.0#Built-In_Changes
Change-Id: Iba7ad9ed2a9b197ecedaaed1c6744fe1c721515a
Fixes the following pylint's warnings:
pySim/cards.py:494:18: E1101: Class '_MagicSimBase' has no '_files' member (no-member)
pySim/cards.py:509:6: E1101: Instance of '_MagicSimBase' has no '_files' member (no-member)
pySim/cards.py:529:26: E1101: Instance of '_MagicSimBase' has no '_files' member (no-member)
pySim/cards.py:537:5: E1101: Instance of '_MagicSimBase' has no '_ki_file' member (no-member)
pySim/cards.py:547:5: E1101: Instance of '_MagicSimBase' has no '_ki_file' member (no-member)
pySim/cards.py:548:8: E1101: Instance of '_MagicSimBase' has no '_ki_file' member (no-member)
pySim/cards.py:559:26: E1101: Instance of '_MagicSimBase' has no '_files' member (no-member)
pySim/cards.py:560:11: E1101: Instance of '_MagicSimBase' has no '_files' member (no-member)
pySim/cards.py:576:14: E1101: Instance of '_MagicSimBase' has no '_files' member (no-member)
Change-Id: I4db9d21258d6e04140962134c540e36631466322
pySim/card_key_provider.py:67:2: E1111:
Assigning result of a function call, where the function
has no return (assignment-from-no-return)
Change-Id: I43bab69f53300fbe837944735cd999fab5405d7a
Some modems may reject AT+CSIM if PDU contains lower
case hex digits [a-f]. Modem response is "ERROR"
without any error code.
This patch converts each PDU to upper case.
Tested with Sierra Wireless EM7565.
Example:
AT+CSIM=14,"00a40004023F00"
ERROR
AT+CSIM=14,"00A40004023F00"
+CSIM: 4,"612F"
OK
Change-Id: I318e36abc7ae975c62d32b7fe0ec949bf5997d13
The json input that is used with EF.MSISDN seems to be somewhat
ambigious. The original code accepts {"msisdn": "+4916012345678"}
only while the output is {"msisdn": [1, 1, "+4916012345678"]}. Lets
add a check and also accept the latter version.
Change-Id: I8f8dd68aac25d3fa3bc1aab06b855f8ec6640258
Related: OS#4963
I always assumed BER-TLV files are transparent EF with BER-TLV contents.
However, this is wrong. ETS TS 102 221 Section 8.2.2.4 specifies them.
TS 102 221 Section 11.3 describes the specific RETRIEVE DATA, SET DATA
commands, which are not yet implemented in pySim.
Change-Id: Ie4701d9f72b05c8a5810e287e55a20f6ea86a574
The encoding of EF.MSISDN is a bit unstrutured. The encoder function
does not return a valid result since it lacks the parameters
Capability/Configuration2 Record Identifier and Extension5 Record
Identifier, which are mandatory but can be set to 0xFF. Also the
encoder gets its input from pySim-shell, so it should have some
more input validation, especially when the user encodes an empty
string. The encoder and decoder function also do not have unit-tests.
Since the encoder now adds the missing two bytes by isself this does
not have to be done manually anymore, so cards.py needs to be
re-aligned.
For pySim-shell.py the encoder is used from ts_51_011.py. Unfortunately
it is used wrongly there. The optional Alpha Identifier is required
here as well.
Related: OS#4963
Change-Id: Iee5369b3e3ba7fa1155facc8fa824bc60e33b55b
Ideally that shared definition would be used by all programs,
rather than copy+pasting it. Unfortunately pySim-{read,prog}
are still using optparse and first need to be converted to
argparse.
Change-Id: If77f53850e1ca65f42cf1dca3e0f460dac1b0d1a
the helpstring of update_record_decoded mentions hex bytes for the data
parameter, but it should be mentioned as abstract json data like in
update_binary_decoded
Change-Id: Ibae2ab49054ac5dd6fcccddd28c98d886403dac9
Related: OS#4963
the export summary is printed after the log entry for the last file
without separation. This is confusing because it looks like if the
summary would refer to the last file only. Lets add a headline to make
clear that the last few lines are the "Export summary"
Change-Id: I90771e525b2b114bdb41a8e90d298ca991c09c3d
Related: OS#4963
Also serves as example for RFU (reserved for future use) fields
which should not always be reset to zero in case they have been
set on the uSIM for some reason.
See pySim/ts_51_011.py, class EF_AD.
* Add definitions for RFU {Flag, Bits, Byte, Bytes}
* Use IntEnum for OP_MODE (convenient auto completion)
* Remove obsolete definitions and imports
* Update test results for all SIMs (opmode strings are shortened)
Change-Id: I65e0a426f80a619fec38856a30e590f0e726b554
This adds support for AUTHENTICATE to the USIM and ISIM application,
based on the newly-introduced 'construct' encoder/decoder support.
Change-Id: Id5697463e29c3dceff98bcf80f5400f7f2bcaa6c
'construct' is a declarative symmetric encoder/decoder for user
specified binary formats. It should come in extremely handy in
tools like pySim.
We start the integration by adding transport methods for transceiving
APDUs with built-in encoding of the command data and decoding of the
response data.
Change-Id: Ibf457aa8b9480a8db5979defcfafd67674303f6c
Use ``--opmode=OPMODE`` in cmdline mode or column ``OPMODE`` in csv mode
to specify OPMODE as listed below.
Details:
The ``EF_AD`` field contains administrative data (AD).
It consists of four bytes ``B1``, ``B2``, ``B3``, ``B4``,
and optionally further bytes for future use.
Previous implementation only sets the MNC field appropriately
(located in `B4`) and sets all other bits/bytes to 0.
However, `B1` also defines the *UE operation mode* (see below).
For type approval operations, such as testing with a test uSIM,
this value could be set to `0x80` rather than `0x00`(= normal operation).
This may unlock some UE capabilities that are restricted in
normal operation mode.
Excerpt from [ETSI TS 131 102, 4.2.18](https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/04.15.00_60/ts_131102v041500p.pdf):
```
B1 - UE operation mode:
Coding:
Initial value
- '00' normal operation.
- '80' type approval operations.
- '01' normal operation + specific facilities.
- '81' type approval operations + specific facilities.
- '02' maintenance (off line).
- '04' cell test operation.
B2 - Additional information:
Coding:
Reserved for future use
B3 - Additional information:
Coding:
- B3.b1: OFM setting (Ciphering Indicator)
- B3.others: Reserved for future use
B4 - Length of MNC in the IMSI:
Coding:
- B4.b4..B4.b1: length: '0010' (= 2) or '0011' (=3)
- B4.others: Reserved for future use
```
**Legend:** Byte X, bit Y: BX.bY
Further reading: https://nickvsnetworking.com/usim-basics/
Change-Id: Ie9040c6b127c268878a0845ed73d0918ec6bbb08
This means we can skip a lot of code that manually converts from
bytes to hex before JSON serialization.
Change-Id: I9c9eff0556d9d196e64553b5276e162f69d0c18f
This allows the user to edit the file/record contents in its
JSON representation inside the standard system text editor.
Change-Id: Icf6a6e8529e7664c5645519fb4bdd55b35f34664
The ``EF_ACC`` field defines the access control class (ACC)
for a subscriber.
Without this patch, the implementation adds padding 1 towards
the most significant bits if the input is shorter than 2 bytes.
However, it should be padded with 0, otherwise additional ACCs
are allocated to the subscriber. (Probably only a single bit
shall be set to 1)
Excerpt from [ETSI TS 131 102, 4.2.15](https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/04.15.00_60/ts_131102v041500p.pdf):
```
EF_ACC: Two bytes: B1, B2
B1.b8...B1.b4: high priority users (class 15...11)
B1.b3: always 0
B1.b2...B1.b2 and B2.b7...B2.b0: normal priority users (class 9...0) - to be evenly distributed across subscribers
```
**Legend:** Byte X, bit Y: BX.bY
Change-Id: I1b8dc01a6c48adad1ed8158de59b12519ed688e9
The function fixup_fcp_proprietary_tlv_map() addes propritary TLV
tags in the range of d0 to ff to the TLV map. However, the spec defines
this range as b7 and b8 of the first tag byte set to 1. This results
in a range from c0 to ff. See also ETSI TS 102 221, section 11.1.1.4.6.0
Change-Id: I8359527c9ff303b257b181b87dc440f27735ece9
Related: OS#4963
This function is being used e.g. for ADF.USIM/EF.FPLMN entries.
The EF_PLMNsel class also already uses a function by this name, we just
haven't had any actual implementation around.
Change-Id: Iacb45c90bb6491ebb89a477a85ef1f3129b38788
the print statements in read_binary_decoded and update_binary_decoded
should have been removed a long time ago.
Change-Id: I9ccc61c426a755fae9008d0717d579fa2da0ef7c
The EF.AD class only had a partial decoder and no encoder before this
patch.
You can now do things like
pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded
{
"ms_operation_mode": "normal_and_specific_facilities",
"specific_facilities": {
"ofm": false
},
"len_of_mnc_in_imsi": 2
}
pySIM-shell (MF/ADF.USIM/EF.AD)> update_binary_decoded '{"ms_operation_mode": "normal_and_specific_facilities", "specific_facilities": {"ofm": false}, "len_of_mnc_in_imsi": 3}'
not quite all that elegant yet, but working at all.
Change-Id: Id2cb66cb26b6bd08befe9f8468b0b0773da842b1
Prior to this patch, any SwMatchError raised within the 'transport'
would not be interpreted.
EXCEPTION of type 'SwMatchError' occurred with message: 'SW match failed! Expected 9000 and got 6982.'
vs (now)
EXCEPTION of type 'SwMatchError' occurred with message: 'SW match failed! Expected 9000 and got 6982: Command not allowed - Security status not satisfied'
Change-Id: I08b7f2b6bd422f7f2f36094bc8a29b187ff882a6
* move existing docs to sphinx / autodoc
* add more api documentation
* improve wording on some exception strings
Change-Id: Ia41e14d643d452d92fc8d3c2fb9c4ac9021402e9
"data" is an awfully generic term. Anything stored on a card is data.
This specific code deals with resolving key/pin material from an
external source.
Change-Id: I4c8e1be3e766f7c0565c07b39d48abf8adc375af
As we can notice during 'export': Some files had been defined
as LinFixed but are Transparent - and vice versa. Let's fix those
an bring our definitions in sync with the specs.
Change-Id: I365ece7b82a1c79b3af87a79ff964d7989362789
When the CardFile hierarchy talks about 'application' it means CardADF.
When the RuntimeState and CardProfile talk about 'application' they mean
a CardApplication.
Let's clarify this in the file names, and make CardADF have an optional
reference to the CardApplication, so that application specific status
word interpretation really works.
Change-Id: Ibc80a41d79dca547f14d5d84f447742e6b46d7ca
* add type annotations in-line with PEP484
* convert existing documentation to follow the
"Google Python Style Guide" format understood by
the sphinx.ext.napoleon' extension
* add much more documentation all over the code base
Change-Id: I6ac88e0662cf3c56ae32d86d50b18a8b4150571a
In Change-Id I848a766e6d00be497c7db905475e0681cce197ac we added a CardDF
instance for DF_5GS. That DF should not have provided a
decode_select_response() method, and instead fall back to that of the
base class, which calls the method of the parent directory (ADF_USIM).
The difference is illustrated below
pySIM-shell (MF/ADF.USIM/EF.IMSI)> select DF.5GS
"622e8202782183025fc0a509800171830400018d088a01058c056611111111c60f90017083010183018183010a83010b"
vs. (with this patch):
pySIM-shell (MF/ADF.USIM)> select DF.5GS
{
"file_descriptor": {
"shareable": true,
"file_type": "df",
"structure": "no_info_given"
},
"file_identifier": "5FC0",
"proprietary_info": {
"uicc_characteristics": "71",
"available_memory": 101640
},
"life_cycle_status_int": "operational_activated",
"security_attrib_compact": "6611111111",
"pin_status_template_do": "90017083010183018183010A83010B"
}
Change-Id: I80612711bbc8c47285a828a0759b20beea6619f1
At the moment we only have a basic version of a verify_chv commnad, but
in order to handle any CHV/PIN related situation we also need commands
to enable, disable, change and unblock CHV.
- fix verify_chv commnad: more distinct parameter names, better help
strings, correct pin code encoding and add external source lookup
- Add unblock_chv, change_chv, enable_chv and disable_chv commands
- add/fix related functions in commands.py
Change-Id: Ic89446e6bd2021095e579fb6b20458df48ba6413
Related: OS#4963
When a record or a binary file is written the card goes throth a full
flash/eeprom write cycle at this location, even when the data does not
change. This can be optimized by reading before writing in order to
compere if the data we are about to write is actually different.
Change-Id: Ifd1b80d3ede15a7caa29077a37ac7cf58c9053f1
Related: OS#4963
It can be hard to manage ADM pins when working with different cards at
the same time. To make this easier, add an automatic way to determine
the ADM pin for each card from a CSV file.
- add a CardData clas model that can be extended to to get the data from
various different sources. For now use CSV-Files. Also add a way how
multiple CardData classes can be registered so that one global get
function can query all registered CardData classes at once.
- automatically check for CSV-File in home directory and use it as
default CardData source unless the user specifies a CSV file via
commandline argument.
- extend the verify_adm command so that it automatically queries the
ADM pin if no argument is given. Also do not try to authenticate if
no ADM pin could be determined.
Change-Id: I51835ccb16bcbce35e7f3765e8927a4451509e77
Related: OS#4963
When the ADF is selected, then this is done by the AID. At the moment
only the first 7 bytes of the AID are used to select the ADF.
sysmo-isim-sja2 tolerates this, but sysmo-usim-sjs1 does not. The Cards
class already has methods to deal with this problem. The method
select_adf_by_aid takes an ADF name and completes the AID from an
internal list. This can be extended to support partial hexadecimal AIDs
as well.
Change-Id: If99b143ae5ff42a889c52e8023084692e709e1b1
Related: OS#4963
The class ShellCommands defined in ADF_USIM overloads useful CommandSet
classes defined in the superclass, making their commands inaccessible.
Also ts_31_102 does not have such a class definition in the ADF_ISIM
class, so lets remove this class.
Change-Id: I0e67c570fc4f17641d990a9cd239632ecf622de3
Related: OS#4963
Some cards may have additional propritary EF files which pySim-shell
does not support. If the user knows the exact FID the file can still be
selected and it is possible to read the file type and memory model from
the select response. This info can be used to create a new file object
at runtime that will work like any other EF/DF.
Change-Id: Iafff97443130f8bb8c5bc68f51d2fe1d93fff07c
Related: OS#4963
The Change I83d718ff9c3ff6aef47930f38d7f50424f9b880f removes the
keyword arguments from the CardProfile class constructor. This requires
us to use the keywords during instantiation since we can not rely on
the position anymore.
Change-Id: Ia62597c59287848662dbbedcc38ba90f183c4aca
The do_update_... functions do always print the returned data. However,
there may be no data. If this is the case than an empty line is printed.
This may cause ugly log output, especially when working with scripts.
Change-Id: Ia9715d46ec957544cfbeea98d2fe15eb74f5b884
Related: OS#4963
Having lists and dictionaries as default argument values is a bad
idea, because the same instance of list/dict will be used by all
objects instantiated using such constructor:
def appendItem(itemName, itemList=[]):
itemList.append(itemName)
return itemList
print(appendItem('notebook'))
print(appendItem('pencil'))
print(appendItem('eraser'))
Output:
['notebook']
['notebook', 'pencil']
['notebook', 'pencil', 'eraser']
Change-Id: I83d718ff9c3ff6aef47930f38d7f50424f9b880f
currently ADF.ISIM and ADF.USIM are always added regardless if there is
a matching application on the card or not. Lets check what applications
are actually installed and add ADF.ISIM and ADF.USIM dynamically.
Change-Id: I42ee23375f98e6322708c1c4db6d65e1425feecd
Related: OS#4963
In the method add_application() the method name should be append()
instead of add().
Change-Id: Ic8ad62567968e09786eac86f219b56a3d3200511
Related: OS#4963
The SW_match function takes a given status word and compares it against
a pattern that may contain wildcards (x or ?). This works by creating a
masked version of the SW using a pattern first (each hex digit is
replaced by a wildcard charafter if the pattern has a wildcard in the
same position). Once this is done, the resulting masked version is
compared at the pattern. However, the current implementation can not
work, since it compares the input SW against the pattern to decide
wihich chrafters should be masked. The input SW never contains wildcard
charafters.
Change-Id: I805ad32160fcfcb8628bf919b64f7eee0fe03c7e
Related: OS#4963
The _scc.veryif_adm() method already does status word checking
internally and also raises an execption should the authentication be
unsuccessful, so we do not have to put an additional status word check +
execition when we use the method from cards.
Change-Id: I785d27e4d49a9cda1a771b56ce5ac9c1f1d1e79a
Related: OS#4963
At the moment we use the send_apdu_checksw() method to send the APDU for
ADM authentication. This method only checks if the command returns with
sw = 9000. If not it raises an exception that the sw is not as expected.
The user may think that this is a problem with thr reader, pcscd or
pySim in the first place and may try multiple times until the card is
permanently locked. A better execption string that also displays the
tries which are left may be helpful.
Change-Id: Icf428831094f8c1045eefaa8cb2b92e6a36b0c13
Related: OS#4963
The file identifier of a file is strictly defined as a two digit
hexadecimal number. Do not allow adding child files that violate this
constraint.
Change-Id: I096907285b742e611d221b03ba067ea2522e7e52
Related: OS#4963
The __main__ function in filesystem.py seems to be some experimental
testcode from the very beginning of pySim-shell. Lets drop it.
Change-Id: I34f459469dfc45711ad0928c83184d7f99e0f5e3
Related: OS#4963
The method add_file of class CardDF does some constraint checking
to the basic file parameters (e.g. fid). Since one might also expect
those checks in the superclass CardFile lets leave a comment to make
the code better understandable.
Change-Id: Iebae28909fe6aade3bd4024112a222819572d735
Related: OS#4963
It is better to use the term "fid" instead of "name" when a reserved FID
is detected.
Change-Id: I054f3b3a156f0164c62610cfde1aec2145c20925
Related: OS#4963
pysim-shell does not have a convinient way to list the files available
in one directory. Tab completion currently is the only way to obtain a
list of the available files. Lets add a dir command to print a file
list.
Change-Id: Ic06a60e0a0ec54d9bb26e151453ceb49d69e3df7
Related: OS#4963
The flags NAMES, FIDS and APPS do not properly distinguish between
applications and normal files. With APPS it is only possible to exclude
or include the selectable applications in a list with NAMES or FIDS, but
it is not possible to get only the application names or identifiers.
- remove the APPS flag
- rename NAMES to FNAMES and make it only normal file related
- add ANAMES and relate it only to application (ADF) names
- add AIDS and relate it only to application identifiers
Change-Id: Id07e0dcbab10cd78c1b78d37319b7b0e5e83b64d
Related: OS#4963
At the moment we can only request pySim-shell to dump a specific record
of a file. However, it may be useful to dump multiple records of a
record oriented file at once.
Change-Id: Id62db2cba4e3dfb6a7b3e6be8b892c16d11a8e3e
Related: OS#4963
The file identifier (and allso application ids for ADFs), are
hexadecimal. We should be case insensitive when accepting hex
identifiers but file names should still be full matched.
Change-Id: Ibe283a108ddc9058af77c823b7222db555e1e0f6
Related: OS#4963
since we have added pySim-shell.py that has a lot of locations where the
user can enter hexadecimal data there is an increased need for input
validation. Lets add a central is_hex function that verifies hex
strings.
Change-Id: Ia29a13c9215357dd2adf141f2ef222c823f8456d
Related: OS#4963
When requesting what DF/EF/ADF are selectable it is useful to have some
control of what we do not want in the resulting list.
Change-Id: Idb50a512bfdbfdf2e98f2ce0e89928cb0ff19f5e
Related: OS#4963
The bug that was attempted to be fixed in [1] actually was in the
encoding API - pySim.utils.enc_plmn(). According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit (E)HPLMN shall be encoded
as shown below (ASCII-art interpretation):
0 1 2 3 4 5 6 7
+---+---+---+---+---+---+---+---+
| MCC Digit 2 | MCC Digit 1 |
+---+---+---+---+---+---+---+---+
| MNC Digit 3 | MCC Digit 3 |
+---+---+---+---+---+---+---+---+
| MNC Digit 2 | MNC Digit 1 |
+---+---+---+---+---+---+---+---+
while pySim.utils.enc_plmn() would produce the following:
0 1 2 3 4 5 6 7
+---+---+---+---+---+---+---+---+
| MCC Digit 2 | MCC Digit 1 |
+---+---+---+---+---+---+---+---+
| MNC Digit 1 | MCC Digit 3 |
+---+---+---+---+---+---+---+---+
| MNC Digit 3 | MNC Digit 2 |
+---+---+---+---+---+---+---+---+
Initially the _decoding_ API was correct, but then got changed in
[1] to follow buggy pySim's encoding API. As a result, a (E)HPLMN
programmed with pySim-prog.py would look correct if verified by
pySim-read.py, but the actual file content would be wrong.
This situation shows that our 'program-read-match' build verification
approach alone is insignificant. The lack of unit test coverage,
at least for the core parts of the project, makes it possible to have
symmetrical bugs in both encoding and decoding API parts unnoticed.
This problem was found while trying to enable dead unit tests in [3].
Change [1] that introduced a symmetrical bug is reverted in [2].
Change-Id: Ic7612502e1bb0d280133dabbcb5cb146fc6997e5
Related: [1] I799469206f87e930d8888367890babcb8ebe23a9
Related: [2] If6bf5383988ad442e275efc7c5a159327d104879
Related: [3] I4d4facfabc75187acd5238ff4d0f26022bd58f82
This reverts commit bdf3d3597b, which
broke pySim.utils.dec_mnc_from_plmn(). According to 3GPP TS 31.102,
which points to TS 24.008, the three-digit EHPLMN shall be encoded
as shown below (ASCII-art interpretation):
0 1 2 3 4 5 6 7
+---+---+---+---+---+---+---+---+
| MCC Digit 2 | MCC Digit 1 |
+---+---+---+---+---+---+---+---+
| MNC Digit 3 | MCC Digit 3 |
+---+---+---+---+---+---+---+---+
| MNC Digit 2 | MNC Digit 1 |
+---+---+---+---+---+---+---+---+
So the original implementation was correct, and we even had a unit
test for it. Most likely, the SIM card itself was programmed
incorrectly?
Makes 'testDecMNCfromPLMN_threeDigitMNC' pass again.
Change-Id: If6bf5383988ad442e275efc7c5a159327d104879
As it turns out, we had this set of unit tests since 2018, but
so far they were not executed during the build verification.
Let's fix this:
* run unittest in discovery mode for all files in 'tests/' (commented out);
* rename this file, so it can be automatically detected and executed;
* properly import the API to be tested.
Currently 2 out of 16 unit tests are failing, so we need to get
them passing first and then uncomment the unittest execution.
Change-Id: I4d4facfabc75187acd5238ff4d0f26022bd58f82
This commit fixes two problems (found by semgrep):
* "'foo' and 'bar' in list" is incorrect, because it's interpreted
as "'foo' and ('bar' in list)". Strings with a non-zero length
evaluate to True, thus it's True if at least 'bar' is present.
* Copy-pasted 'E-UTRAN NB-S1' checked two times.
The first condition is redundant, and the whole block can be
re-implemented using two independent 'if' statements.
Change-Id: Iceb66160cfb571db8879d3810c55d252c763d320
pySim has already been migrated to Python 3 in another change [1],
and the build verification has been migrated to Debian 10 with
Python 3.7. However, there is still some backwards compatibility
code left. Let's get rid of it.
[1] Ic78da9c03e99f59d142c93394051bbc2751f0205
Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Tweaked-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Change-Id: I430d173535e0cd5bb895b9dfc9070cbc40cfc8ff
Unfortunately, Debian ships old Python (3.5 vs 3.8) and old pyscard
(1.9.4 vs 1.9.9). Calling PCSCCardConnection.disconnect() from a
destructor causes warnings about ignored exceptions:
AttributeError: 'NoneType' object has no attribute 'disconnect'
AttributeError: 'NoneType' object has no attribute 'setChanged'
AttributeError: 'NoneType' object has no attribute 'SCardDisconnect'
TypeError: 'NoneType' object is not callable
All these exceptions happen in pyscard's own destructors.
Change-Id: I9c644bc5fe9791b141a30bfc13647d77937a82ee
This should resolve the following error when using with pycryptodome
instead of pycrypto:
TypeError: new() missing 1 required positional argument: 'mode'
Change-Id: Ibd3ca00d62b864909f5e89e0feb350268157a4ca
Related: OS#5060
In 4f6ca43e1f we started to use
the bytearray type as 'b' type, but PyCrypto insists on getting
a bytes type.
This fixes the following Exception:
TypeError: argument 1 must be read-only bytes-like object, not bytearray
Change-Id: If2a727ed417ffd56c0f7d7b4e9f633d67fde5ced
Closes: OS#5060
pySim-prog was nice when there were only 5 parameters on a SIM that we
could program, and where the use case was pretty limited. Today, we
have SIM/USIM/ISIM cards with hundreds of files and even more parameters
to program. We cannot add a command line argument for each file to
pySim-prog.
Instead, this introduces an interactive command-line shell / REPL,
in which one can navigate the file system of the card, read and update
files both in raw format and in decoded/parsed format.
The idea is primarily inspired by Henryk Ploatz' venerable
cyberflex-shell, but implemented on a more modern basis using
the cmd2 python module.
See https://lists.osmocom.org/pipermail/simtrace/2021-January/000860.html
and https://lists.osmocom.org/pipermail/simtrace/2021-February/000864.html
for some related background.
Most code by Harald Welte. Some bug fixes by Philipp Maier
have been squashed.
Change-Id: Iad117596e922223bdc1e5b956f84844b7c577e02
Related: OS#4963
The code was written long ago, when the python3 bytearray type
probably didn't exist yet, or was at least not known. Let's stop
using string types with binary bytes inside, and instead standardize
on two types:
* bytearray for binary data
* string for hexadecimal nibbles representing that binary data
Change-Id: I8aca84b6280f9702b0e2aba2c9759b4f312ab6a9
This method, like select_adf(), only selects a single file ID
and unlike select_path() returns the actual status words returned by the
card.
Change-Id: I8bc86654c6d79f2428e196cc8a401e12d93a676b
In reality, the function is not a simple avstraction around the SELECT
command, but it iterates over a list/path and selects at each element.
Change-Id: I63e01155de4ae47aeed8500708c0eb6580c7b8d1
This allows callers further up the stack to catch the exception and
interpret it in some way (like decoding the number of remaining tries
in case of authentication errors)
Change-Id: Ia59962978745aef7038f750fa23f8dfc820645f4
SJS1 and SJA2 card types don't use the generic verify_adm()
method of the Card base class, so they must override it with their
own methods. Only this way application code can call card.verify_adm()
irrespective of the card type.
Change-Id: I05f7f3280873f006310266867f04a9ce1b0a63af
The SerialSimLink only has an _sl member if serial initalization was
successfull. If we close a serial connection, check if we even have the
_sl member. Otherwise move on silently.
Change-Id: Ic3f3f5e50d780f424da7d0be5733d7167bb7159c
The most common reason for pySim to crash is when it is executed without
commandline parameters. Then pySim will expect a serial reader on
/dev/ttyUSB0 since this is the default. Lets check if /dev/ttyUSB0 even
exists before trying to open it.
Change-Id: I7545c728b531e9a796eee8f80f0b08d4097f8399
Failed reader initializations happen frome time to time, mostly because
of messed up commandline arguments. This results in ugly crashes very
often. Lets control this a bit by catching the exception string and
print it.
Change-Id: I313017355da01bbef7c3d3f1899555aadb509319
This commit fixes the incorrect parsing of MNC from PLMN.
Previously its was parsing PLMN string 130062 as MCC 310 MNC 260,
whereas it should be MCC 310 MNC 026.
(The SIM was programmed with MCC 310 and MNC 026)
Change-Id: I799469206f87e930d8888367890babcb8ebe23a9
This EF contains one or more records, with each record able
to hold a public SIP Identity (SIP URI) of the user. EF.IMPU consist of URI TLV data
object values see IETF RFC 3261. The URI shall be encoded to an octet string according
to UTF-8 encoding rules as specified in IETF RFC 3629
./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org --pcscf pcscf.ims.testop.org --ims-hdomain ims.testop.org --impi 1234567895@ims.testop.org --impu sip:5987654321@ims.testop.org
Change-Id: If10bc2e50eca390b81755b5cc7211e695233612d
This EF contains the private user identity of the user. EF.IMPI consist of
NAI TLV data object values see IETF RFC 2486. The NAI shall be encoded to
an octet string according to UTF-8 encoding rules as specified in IETF RFC 3629
./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org --pcscf pcscf.ims.testop.org --ims-hdomain ims.testop.org --impi 1234567895@ims.testop.org
Change-Id: Ic1ccf99b5aa45297ef1e43a43373df603f756379
IMS public user identity (IMPU)
As per TS1.103, this EF contains one or more records, with each record able
to hold a public SIP Identity (SIP URI) of the user. EF.IMPI consist of URI TLV data
object values see IETF RFC 3261. The URI shall be encoded to an octet string according
to UTF-8 encoding rules as specified in IETF RFC 3629
Reading of EF.IMPU is achieved by first selecting the ISIM application using its AID.
This is followed by selecting EF.IMPU with File ID - 6f04 in ADF.ISIM
Change-Id: Icf78a564aeaf4254658d3b018ff57dfc4b987e6f
IMS private user identity (IMPI)
As per TS1.103 version 14.2.0, this EF contains the private user identity of the user.
EF.IMPI consist of NAI TLV Data object.The NAI shall be encoded to an octet string
according to UTF-8 encoding rules as specified in IETF RFC 3629.
Reading of EF.IMPI is achieved by first selecting the ISIM application using its AID.
This is followed by selecting EF.IMPI with File ID - 6f02 in ADF.ISIM
Change-Id: I8d8e76e3f6b9ca7a0be262fee99cd5a397edbefa
As per 3GPP TS 31.103, this EF (DOMAIN) can found under ADF.ISIM at File Id 6f03.
The Home Network Domain Name, i.e. FQDN shall be encoded to an octet string
according to UTF-8 encoding rules as specified in IETF RFC 3629 [27].
The tag value of the Home Network Domain Name TLV data object shall be '80'.
Example:
./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 -k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 -o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test --msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org --pcscf pcscf.testims.org --ims-hdomain testims.org
Change-Id: I3c823203aee88734ae423e4ad73da1027a4eaeed
The function takes address string as input, then validates it and returns the type.
Return: 0x00 (FQDN), 0x01 (IPv4), 0x02 (IPv6), None (Bad address format)
Change-Id: I0fabd4f17bbb11f6bb191c1a9e6276427f9d001f
As per TS 31.102 version 15.2.0 Release 15, section 4.2.8, EFUST (USIM Service Table),
Service n°95, n°99 and n°115 shall not be declared available if an ISIM application is present on the UICC.
Change-Id: Id9709746de99585ad31c4e9659323484fda87b14
As per TS1.103 version 14.2.0, this EF contains the home operator's network domain name.
The Home Network Domain Name, i.e. FQDN shall be encoded to an octet string
according to UTF-8 encoding rules as specified in IETF RFC 3629
Change-Id: Ia3c68c717d105e10d52a8e9d170480da2ad7d65a
As per 3GPP TS 31.103 version 14.2.0 Release 14, this EF can found under ADF.ISIM at File Id 6f09.
This EF contains one or more Proxy Call Session Control Function addresses.
The first record in the EF shall be considered to be of the highest priority
If ISIM service n°1 and/or service n°5 is available, this file shall be present.
Change-Id: I7a701212c84d3dc5d4c8ccbcf638c97ceda33654
EF_ISIM_ADF_map introduced in this commit maps EF file names in ISIM ADF
to its repective Identifier and serves as a lookup table
Change-Id: I95c8691d9112541c2c0e01857b19681c00f322f2
This commit introduces a ISIM generic Card class which can hold
parameters/functions specific to ISIM application on UICC
Change-Id: I242e679ff2f8831175e76d2fcc5fb285d28bd890
The Address TLV object is used in EF.P-CSCF Address, EF.ePDGId and EF.ePDGIdEm.
See 3GPP TS 31.102 version 13.4.0 Release 13, section 4.2.8, 4.2.102 and 4.2.104.
Address TLV Object format
Tag (1 Byte) - '80'
Length (1 Byte)
Address Type (1 Byte) - '00' (FQDN), '01' (IPv4), '02' (IPv6)
Address (Address Length Bytes)
Change-Id: Ifd8a240f6b5c7736e58a8151295c30ec5b32ed5f
Service 106 and 107 must be set available in order to ME to consider as supported in USIM:
1. ePDG configuration Information support
2. ePDG configuration Information configured
Change-Id: Ica067915b9d06ba67f53da7d628f8bacde1ab80e
If the EF.ePDGSelection is present, it is populated with a single entry with PLMN 1 set
to Home PLMN of USIM, ePDG FQDN format set to Operator Identifier FQDN and ePDG Priority value
set to 1.
Change-Id: I92f3f813afa41ae497ebc0dc2ca73da810f82364
Encodes ePDGSelection info TLV so it can be stored at EF.ePDGSelection or EF.ePDGSelectionEm.
See 3GPP TS 31.102 version 15.2.0 Release 15, section 4.2.104 and 4.2.106.
Take original hex string of EF.ePDGSelection or EF.ePDGSelectionEm, MCC,
MNC, ePDG priority for PLMN and ePDG FQDN Format to use for PLMNas input
and outputs the encoded hex string.
Change-Id: Ia7292d33783c770a3bb91b081c671af36bbb907f
As per TS 31.102, this EF can found under ADF.USIM at File Id 6ff4.
Also, if service n°106 and service n°107 are available, this file shall be present.
Change-Id: I98916e6f5c9791aff63c18a3b16bdfb8ae9b2d36
ePDG selection information TLV data object is made of following elements:
ePDG Selection Information Tag '80' (1 Byte)
Length 5n Note
PLMN 1 (3 Bytes)
ePDG Priority (2 Bytes)
ePDG FQDN format '00' or '01' (1 Byte)
...
PLMN n
...
Note: The length is coded according to ISO/IEC 8825-1 [35]
Note 2: Inconsistency in spec: 3GPP TS 31.102 version 15.2.0 Release 15, 4.2.104
As per spec, Length field value is 5n, where n is number of PLMNs
But, each PLMN entry is made of PLMN (3 Bytes) + ePDG Priority (2 Bytes) + ePDG FQDN format (1 Byte)
Totalling to 6 Bytes, maybe Length should be 6n and not 5n
Change-Id: I0f9f38961a589e3f9a53d2288a3dc6fa71a4b1b0
These functions are replaced by a more generic function hexstr_to_Nbytearr().
And, all occurances of redundant functions are replaced by generic functions
so its safe to remove them.
Change-Id: I7848451b90b35dca29d29f630cdc5405b5e9c19b
This method helps in encoding of Service Tables in EF.SST, EF.UST, EF.EST, EF.IST.
Takes original hex string of EF.SST, EF.UST, EF.EST, EF.IST, Service number,
Service to be enabled or disable flag as input and outputs the modified Service Table.
Change-Id: I0c97317d5a17aa0df720659d021b5cbf7d30edcc
Example:
./pySim-prog.py -p 0 -x 001 -y 01 -s 8988211900000000004 -i 001011234567895 \
-k 8baf473f2f8fd09487cccbd7097c6862 --op 11111111111111111111111111111111 \
-o 8E27B6AF0E692E750F32667A3B14605D -a 85524953 -n isim.test \
--msisdn 0598765432100 --epdgid epdg.epc.mnc001.mcc001.pub.3gppnetwork.org
Note:
1. For Operator Identifier based ePDG FQDN must be in the format
epdg.epc.mnc<MNC>.mcc<MCC>.pub.3gppnetwork.org
2. For Tracking/Location Area Identity based ePDG FQDN must be in format
lac<LAC>.epdg.epc.mnc<MNC>.mcc<MCC>.pub.3gppnetwork.org
and
tac-lb<TAC-low-byte>.tac-hb<TAC-high-byte>.tac.epdg.epc.mnc<MNC>.mcc<MCC>.pub.3gppnetwork.org
and
3. For 5GS Tracking Area Identity based ePDG FQDN using a 3 octet TAC
tac-lb<TAC-low-byte>.tac-mb<TAC-middle-byte>.tac-hb<TAC-high-byte>.5gstac. epdg.epc.mnc<MNC>.mcc<MCC>.pub.3gppnetwork.org
Change-Id: Ia00bfea36c50b6a38a5387d2f8147f25c81b1de4
Basically, the idea is to read all the AIDs on the UICC once
rather than reading each time we want to select an ADF.
The function select_adf_by_aid select the ADF by its AID which
is already populated by read_aids() function
Change-Id: I5e0e87e9cf238922d60fda7a7836e65f91f2c233
If the EF.EHPLMN exists, it contains the "Equivalent Home PLMN List".
The odd part of that list is that it is not just a list of additional
PLMN identities, but if the first digits of the IMSI are *not* listed
in EF.EHPLMN, then the MCC/MNC of the IMSI prefix is suddently no
longer considered the home network, but the subscriber is roaming.
See TS 23.122: "If the HPLMN code derived from the IMSI is not present
in the EHPLMN list, then it shall be treated as a Visited PLMN for PLMN
selection purposes."
Change-Id: I22d96ab4a424ec5bc1fb02f5e80165c646a748d3
This method encodes ePDG Id so it can be stored to EF.ePDGId or EF.ePDGIdEm.
See 3GPP TS 31.102 version 13.4.0 Release 13, section 4.2.102 and 4.2.104.
Resulting hex string is made of tag value + length + address type + address.
tag value for home ePDG identifier is 80
address type: 0x00 (FQDN), 0x01 (IPv4), 0x02 (IPv6), other (Reserved)
ePDG FQDN example: epdg.mnc001.mcc001.3gppnetwork.org
Note: Only FQDN format is supported for now
Change-Id: I864bda5505e9061391a727add294a6e90c50f9ef
The LOCI, PSLOCI and EPSLOCI contain some info,
including the PLMN, added helper functions to
decode it
Change-Id: Ibb513ff7d1dc6d33b354ae26cbd9c390ea3c8efc
the EF files in the USIM ADF are different to the ones int the GSM dir
so added the dictionary to avoid conflicts and hardcoded values
the 'DIR' one was added in ts_51_011.py, not sure if it should be there
as it's not in that standard, but did it for simplicity
Change-Id: I458380bf46b2986662ecdede2551c22cd9be92ba
According to 3GPP TS 27.007, sections 8.17 and 8.18, the modem
may *optionally* provide Generic and/or Restricted SIM Access
to the TE (Terminal Equipment) by means of the AT commands.
This basically means that a modem can act as a card reader.
Generic SIM Access allows the TE to send raw PDUs in the format
as described in 3GPP TS 51.011 directly to the SIM card, while
Restricted SIM Access is more limited, and thus is not really
interesting to us.
This change implements a new transport called ModemATCommandLink,
so using it a SIM card can be read and/or programmed without the
need to remove it from the modem's socket. A downside of this
approach is relatively slow I/O speed compared to PC/SC readers.
Tested with Quectel EC20:
$ ./pySim-read.py --modem-dev /dev/ttyUSB2
Change-Id: I20bc00315e2c7c298f46283852865c1416047bc6
Signed-off-by: Vadim Yanitskiy <axilirator@gmail.com>
By definition, h2s() is supposed to skip padding in the given
hexstring, so it was working fine for 'ff', but not for 'FF'.
Change-Id: I2c5d72a0f7f2796115116737f9f7b5299021f6a3
Signed-off-by: Vadim Yanitskiy <axilirator@gmail.com>
Some of the cards do not implement the erase method that each card
should have. However, having an empty method in each of those classes
does not make too much sense. Lets rather have an erase method in the
superclass (Card) that prints a warning to inform the user that erasing
the spcified card is not supported.
Change-Id: If5add960ec0cab58a01d8f83e6af8cb86ec70a8d
Resetting the contents of a file before re-writing it with parameters
might be helpful when implementing the currently empty erase() methods
of the various card implementations. Lets add two methods, one for
resetting a binary file and one for resetting a specific record in a
record oriented file
Change-Id: I3c3a4ef3d3f358404af307a68a20b7059f1a9e8d
In case try to decode the contents of an uninitalized EF.IMSI, the
function dec_imsi() would crash because it truncates all 0xFF from the
swapped version of the EF.IMSI contents and then accesses the first
element of the buffer. This always works for EF.IMSI contents that
contain valid IMSI data, but if all bytes are set to 0xFF, then no data
is left in the buffer after truncating, so lets check if we even have
bytes left before we move on with the decoding.
Change-Id: I93874a1d7e0b87d39e4b06a5c504643cfabb451c
When writing to files we often just write without making sure that the
actual file contents actually written. Lets add features to do a
read-after-write verification when writing to files.
Change-Id: I7be842004102f4998af376790c97647c107be3d5
Philipp Maier
Harald Welte
Robert Falkenberg
Vadim Yanitskiy