Commit Graph

864 Commits

Author SHA1 Message Date
Philipp Maier ec9cdb73e7 tests: add test script for pySim-trace
pySim-trace has no test coverage yet. Let's use a script to run a
GSAMTAP pcacp through it and check that no exceptions are raised.

Related: OS#6094
Change-Id: Icfabfa7c59968021eef0399991bd05b92467d8d2
2023-07-29 08:56:07 +00:00
Alexander Couzens c8facea845 Fix the remaining functions using the broken Card.update_ust() call
Card.update_ust() got replaced by the file operation ust_update().
In addition to Change-Id I7a6a77b872a6f5d8c478ca75dcff8ea067b8203e

Fixes: f8d2e2ba08 ("split pySim/legacy/{cards,utils} from pySim/{cards,utils}")
Change-Id: Ie6405cae37493a2101e5089a8d11766fbfed4518
2023-07-29 06:23:15 +00:00
Alexander Couzens 2dd59edd74 ARA-M: fix encoding of the PkgRefDO when using aram_store_ref_ar_do
The command wasn't used the correct dict to allow the encoders to work.

Related: OS#6121
Change-Id: Ic2bc179b413a6b139e07e3e55b93ff921cb020a9
2023-07-29 06:23:15 +00:00
Alexander Couzens 760e421be5 utils.py: remove superfluous import from itself
b2h() is already available.

Change-Id: Ied513a08cc8b5091dd467106250f1e6b5067c3a8
2023-07-29 06:21:54 +00:00
Alexander Couzens 6c5c3f8b2b Reimplement ust_service_activate and ust_service_deactivate for USIM/EF.UST
Fixes: f8d2e2ba08 ("split pySim/legacy/{cards,utils} from pySim/{cards,utils}")
Change-Id: I7a6a77b872a6f5d8c478ca75dcff8ea067b8203e
2023-07-28 15:34:40 +00:00
Philipp Maier 8dc2ca2d37 pySim-trace: catch StopIteration exception on trace file end
When the trace file end is reaced, pyShark raises a StopIteration
exception. Let's catch this exception and exit gracefully.

Related: OS#6094
Change-Id: I6ab5689b909333531d08bf46e5dfea59b161a79e
2023-07-28 10:36:52 +02:00
Philipp Maier 162ba3af3e pySim-trace: mark card reset in the trace
The trace log currently does not contain any information about card
resets. This makes the trace difficult to follow. Let's use the
CardReset object to display the ATR in the trace.

Related: OS#6094
Change-Id: Ia550a8bd2f45d2ad622cb2ac2a2905397db76bce
2023-07-28 10:14:19 +02:00
Philipp Maier 1f46f07e3c utils: tolerate uninitialized fields in dec_addr_tlv
TLV fields holding an address may still be uninitialized and hence
filled with 0xff bytes. Lets interpret those fields in the same way as
we interpret empty fields.

Related: OS#6094
Change-Id: Idc0a92ea88756266381c8da2ad62de061a8ea7a1
2023-07-28 10:14:19 +02:00
Philipp Maier 784b947b11 pySim-trace: remove stray debug print
Related: OS#6094
Change-Id: I5f030a8552a84f721bd12ab4751933fc6eeae256
2023-07-28 10:14:19 +02:00
Philipp Maier 407c95520f pySim-trace: add commandline option --show-raw-apdu
The trace log currently only shows the parsed APDU. However, depending
on the problem to investigate it may be required to see the raw APDU
string as well. Let's add an option for this.

Related: OS#6094
Change-Id: I1a3bc54c459e45ed3154479759ceecdc26db9d37
2023-07-28 10:07:35 +02:00
Philipp Maier 791f80a44f construct: add adapter Utf8Adapter to safely interpret utf8 text
Uninitialized Files, File records or fields in a File record or File
usually contain a string of 0xff bytes. This becomes a problem when the
content is normally encoded/decoded as utf8 since by the construct
parser. The parser will throw an expection when it tries to decode the
0xff string as utf8. This is especially a serious problem in pySim-trace
where an execption stops the parser.

Let's fix this by interpreting a string of 0xff as an empty string.

Related: OS#6094
Change-Id: Id114096ccb8b7ff8fcc91e1ef3002526afa09cb7
2023-07-26 17:13:54 +02:00
farhadh fec721fcb1 Fixed `mnc` encoding
According to 3GPP TS 24.008 section 10.5.5.36 PLMN identity of the CN operator

Change-Id: I400435abfa8b67da886fc39c801e1abba39725bf
2023-07-21 11:09:49 +00:00
Philipp Maier 92b9356ed2 runtime: fix lchan deletion in method reset
When we perform a reset while multiple channels are open (this is in
particular the case when parsing real world traces with pySim-trace). To
delete those channels during the reset we iterate over the dictionary
using the keys and delete the channels one by one. However, this must
not be done using the keys as index directly. Python will then throw an
exception: "RuntimeError: dictionary changed size during iteration".

Instead using the keys directly we should cast them into a list and then
using that list for the iteration.

Related: OS#6094
Change-Id: I430ef216cf847ffbde2809f492ee9ed9030343b6
2023-07-21 11:52:10 +02:00
Philipp Maier 7d86fe1d8a apdu/ts_102_221: extract channel number from dict before calling del_lchan
When the method del_lchan is called, closed_channel_nr still contains a dict
that contains the channel number under the key 'logical_channel_number'.
This will lead to an exception. We must extact the channel number from
the dict before we can use it with del_lchan. (See also
created_channel_nr)

Related: OS#6094
Change-Id: I399856bc227f17b66cdb4158a69a35d50ba222a7
2023-07-20 15:50:16 +00:00
Philipp Maier cfb665bb3f pySim-shell: fix verify_adm command
The comman verify_adm does no longer work since the verify_adm method is
no longer available in the card base classes (cards.py). Let's use the
verify_chv method from SimCardCommands instead.

Change-Id: Ic87e1bff221b10d33d36da32b589e2737f6ca9cd
2023-07-20 17:36:08 +02:00
Philipp Maier 3175d61eb2 cards: fix swapped PIN mapping number
The constant for _adm_chv_num is swapped. It should be 0x0A, rather than
0xA0

Change-Id: I5680d2deee855ef316a98058e8c8ff8cf4edbbb2
2023-07-20 17:36:04 +02:00
Harald Welte 38306dfc04 pySim-shell: Add a mode where a pySim-shell cmd can be passed by shell
This adds a new operation mode for pySim-shell, where a single command
can be passed to pySim-shell, which then is executed before pySim-shell
terminates.

Example: ./pySim-shell.py -p0 export --json

Change-Id: I0ed379b23a4b1126006fd8f9e7ba2ba07fb01ada
Closes: OS#6088
2023-07-12 22:05:14 +02:00
Harald Welte 531894d386 move Runtime{State,Lchan} from pySim.filesystem to new pySim.runtime
Those two are really separate concepts, so let's keep them in separate
source code files.

Change-Id: I9ec54304dd8f4a4cba9487054a8eb8d265c2d340
2023-07-12 22:05:14 +02:00
Harald Welte b77063b9b7 pySim/filesystem.py: remove unused class FileData
Change-Id: I62eb446e4995a532227a45c8cc521f5f80535d93
2023-07-12 22:05:14 +02:00
Harald Welte 6ad9a247ef pySim-shell: Iterate over CardApplication sub-classes
Rather than having to know and explicitly list every CardApplication,
let's iterate over the __subclasses__ of the CardApplication base class.

Change-Id: Ia6918e49d73d80acfaf09506e604d4929d37f1b6
2023-07-12 22:05:14 +02:00
Harald Welte 2d5959bf47 ts_102_221: Remove CardProfileUICCSIM
This profile has always been a hack/work-around for the situation that
a classic GSM SIM is not a UICC, and we didn't yet have the concept of
CardProfileAddons yet, so there was no way to probe and add something
to an UICC which was not an application with its own AID/ADF.

Since now we have CardProfileAddons (including one for GSM SIM),
and pySim-trace (the other user of CardProfileUICCSIM) has also switched
over to using CardProfileUICC + addons, we can remove this work-around.

Change-Id: I45cec68d72f2003123da4c3f86ed6a5a90988bd8
2023-07-12 22:05:14 +02:00
Harald Welte 323a35043f Introduce concept of CardProfileAddon
We have a strict "one CardProfile per card" rule.  For a modern UICC
without legacy SIM support, that works great, as all applications
have AID and ADF and can hence be enumerated/detected that way.

However, in reality there are mostly UICC that have legacy SIM, GSM-R
or even CDMA support, all of which are not proper UICC applications
for historical reasons.

So instead of having hard-coded hacks in various places, let's introduce
the new concept of a CardProfileAddon.  Every profile can have any
number of those.  When building up the RuntimeState, we iterate over the
CardProfile addons, and probe which of those are actually on the card.
For those discovered, we add their files to the filesystem hierarchy.

Change-Id: I5866590b6d48f85eb889c9b1b8ab27936d2378b9
2023-07-12 22:05:14 +02:00
Harald Welte f9e2df1296 cdma_ruim: Fix unit tests and actually enable them
As pySim.cdma_ruim was not imported by test_files.py, the unit tests
were apparently never executed and hence didn't pass.  Let's fix both
of those problems.

Change-Id: Icdf4621eb68d05a4948ae9efeb81a007d48e1bb7
2023-07-12 22:05:14 +02:00
Harald Welte 659d7c11ca cards: all UICC should use sel_ctrl="0400" and SIM "0000"
Hence move this from the derived classes into the respective base
classes SimCardBase and UiccCardBase

Change-Id: Iad197c2b560c5ea05c54a122144361de5742aafd
2023-07-12 22:05:14 +02:00
Harald Welte 775ab01a2b cards: cosmetic rename, argument name should be scc, not ssc
ssc = SimCardCommands

Change-Id: I9d690a0a5b9b49ea342728a29b7d4ed10ac31e4e
2023-07-12 22:05:14 +02:00
Harald Welte 172c28eba8 cards: All derived of SimCardBase use CLA=A0; all UiccCardBase use CLA=00
Change-Id: Id61b549f68410631529349ee62b08a102f609405
2023-07-12 22:05:14 +02:00
Harald Welte b314b9be34 ts_31_102, ts_31_103: Move legacy-only code to pySim.legacy
Change-Id: Ifebfbbc00ef0d01cafd6f058a32d243d3696e97e
2023-07-12 22:05:14 +02:00
Harald Welte 57ad38e661 create pySim.legacy.ts_51_011.py and move legacy code there
Those old flat dicts indicating FID to string-name mapping have long
been obsoleted by the pySim.filsystem based classes.

Change-Id: I20ceea3fdb02ee70d8c8889c078b2e5a0f17c83b
2023-07-12 22:05:14 +02:00
Harald Welte a3961298ef pySim/cards: Add type annotations
Change-Id: Id5752a64b59097584301c860ebf74d858ed3d240
2023-07-12 22:05:14 +02:00
Harald Welte f8d2e2ba08 split pySim/legacy/{cards,utils} from pySim/{cards,utils}
There are some functions / classes which are only needed by the legacy
tools pySim-{read,prog}, bypassing our modern per-file transcoder
classes.  Let's move this code to the pySim/legacy sub-directory,
rendering pySim.legacy.* module names.

The long-term goal is to get rid of those and have all code use the
modern pySim/filesystem classes for reading/decoding/encoding/writing
any kind of data on cards.

Change-Id: Ia8cf831929730c48f90679a83d69049475cc5077
2023-07-12 22:03:59 +02:00
Harald Welte 263fb0871c pySim/cards: Split legacy classes away from core SIM + UICC
This introduces an internal split between
* the code that is shared between pySim-shell and legacy tools, which is
  now in the new class hierarchy {Card,SimCard,UiccCard}Base
* the code that is only used by legacy tools,
  which is using the old class names inherited from the *Base above

All users still go through the legacy {Sim,Usim,Isim}Card classes, they
will be adjusted in subsequent patches.

Change-Id: Id36140675def5fc44eedce81fc7b09e0adc527e1
2023-07-12 21:35:17 +02:00
Harald Welte 02a7f7441f filesystem: Support selecting MF from MF
This was currently not handled in build_select_path_to(), resulting in
weird exceptions like 'Cannot determine path from MF(3f00) to MF(3f00)'

Change-Id: I41b9f047ee5dc6b91b487f370f011af994aaca04
2023-07-11 17:50:48 +02:00
Harald Welte 284efda086 pySim-prog: Also accept 18-digit ICCIDs
There are cards with 18-digit ICCIDs, so let's be a bit more tolerant.

Change-Id: I5395daeb2e96987335f6f9bf540c28d516001394
2023-07-11 11:09:00 +02:00
Harald Welte fdcf3c5702 GlobalPlatform ADF.SD: Add command line reference + error message
The get_data shell command didn't have any interactive help / syntax,
and no meaningful error message in case an unknown data object name
was specified by the user.  Let's fix that.

Change-Id: I09faaf5d45118635cf832c8c513033aede1427e5
2023-07-11 08:54:04 +02:00
Harald Welte a1561fe9ae ts_102_222: Remove unneeded imports
Change-Id: I0fc54a042f03ecf707fde81a859c7dd65a7009cc
2023-07-11 08:42:12 +02:00
Harald Welte f9f8d7a294 pySim/transport: Use newly-defined ResTuple type
Let's use the newly-added ResTuple type annotation rather than
open-coding it everywhere.

Change-Id: I122589e8aec4bf66dc2e86d7602ebecb771dcb93
2023-07-11 08:42:12 +02:00
Harald Welte fdb187d7ff pySim/commands.py: Better type annotations
Change-Id: I68081b5472188f80a964ca48d5ec1f03adc70c4a
2023-07-11 08:42:12 +02:00
Harald Welte ab6897c4cd pySim/transport: More type annotations
Change-Id: I62e081271e3a579851a588a4ed7282017e56f852
2023-07-11 08:42:12 +02:00
Harald Welte f5e26ae954 pySim/utils: define 'Hexstr' using NewType
This means Hexstr is no longer an alias for 'str', but a distinct
new type, a sub-class of 'str'.

Change-Id: Ifb787670ed0e149ae6fcd0e6c0626ddc68880068
2023-07-11 08:42:12 +02:00
Harald Welte 2352f2dcdd pySim/tlv.py: Fix TLV_IE_Collection from_dict with nested collections
This is all quite complicated.  In general, the TLV_IE.to_dict() method
obviously is expected to return a dict (with key equal to the snake-case
name of the class, value to the decode IE value).  This single-entry
dict can then be passed back to the from_dict() method to build the
binary representation.

However, with a TLV_IE_Collection, any TLV_IE can occur any number of
times, so we need an array to represent it (dict would need unique key,
which doesn't exist in multiple instances of same TLV IE).  Hence, the
TLV_IE_Collection.to_dict() method actually returns a list of dicts,
rather than a dict itself.  Each dict in the list represents one TLV_IE.

When encoding such a TLV_IE_Collection back from the list-of-dicts, we
so far didn't handle this special case and tried to de-serialize with
a class-name-keyed dict, which doesn't work.

This patch fixes a regression in the aram_store_ref_ar_do pySim-shell
command which got introduced in Change-Id I3dd5204510e5c32ef1c4a999258d87cb3f1df8c8

While we're fixing it, add some additional comments to why things are
how they are.

Change-Id: Ibdd30cf1652c864f167b1b655b49a87941e15fd5
2023-07-11 08:42:12 +02:00
Harald Welte ba955b650e pySim/tlv.py: Don't create an exception from within raise
An invalid variable used in a raise ValueError() would cause a further
exception, depriving the user of a meaningful error message.

Change-Id: I6eb31b91bd69c311f07ff259a424edc58b57529a
2023-07-11 08:42:12 +02:00
Harald Welte 30de9fd8ab TLV_IE_Collection: use snake-style names during from_dict()
The TLV_IE_Collection, just like the individual TLV classes, do
use their snake-style names when converting from binary to dict
using the to_dict() method.  It is inconsistent (and a bug) to
expect the CamelCase names during encoding (from_dict).  After all,
we want the output of to_dict() to be used as input to from_dict().

Change-Id: Iabd1ad98c3878659d123eef919c22ca824886f8a
2023-07-11 08:42:12 +02:00
iw0 f818acd5eb pySim-shell: Unregister profile commands during equip
This avoids error messages about re-registering 'AddlShellCommands' commandsets during 'equip()' in the bulk_script command.

Change-Id: I893bb5ae95f5c6e4c2be2d133754e427bc92a33d
2023-07-09 08:12:28 +00:00
Harald Welte f4a01472bf pySim-shell: Support USIM specific methods/commands on unknown UICC
So far, if no known programmable card (like sysmoISIM) has been found,
we were using the SimCard base class.  However, once we detect an UICC,
we should have switched to the UsimCard class, as otherwise the various
methods called by USIM/ISIM specific commands don't exist and we get
weird 'SimCard' object has no attribute 'update_ust' execptions.

The entire auto-detection and the legacy SimCard / UsimCard classes
are showing the legacy of the code base and should probably be
re-architected.  However, let's fix the apparent bug for now.

Change-Id: I5a863198084250458693f060ca10b268a58550a1
Closes: OS#6055
2023-07-04 21:17:19 +02:00
Harald Welte fa9f348180 ts_31_103: enable encode tests for files containing single TLV IE
Now that we have fixed OS#6073 in the previous commit, we can enable
the so-far disabled encoder tests for EF.{DOMAIN,IMPU,IMPI} and
remove associated FIXMEs.

Change-Id: I79bfc5b77122907d6cc2f75605f9331b5e650286
2023-06-27 09:29:37 +02:00
Harald Welte 579ac3ec0e tlv: Fix IE.from_dict() method
The existing IE.from_dict() method *supposedly* accepts a dict as
input value, but it actually expects the raw decoded value, unless it is
a nested IE.  This is inconsistent in various ways, and results in a bug
visible at a higher layer, such as files like EF.{DOMAIN,IMPI,IMPU},
which are transparent files containing a single BER-TLV IE.

Decoding such files worked, but re-encoding them did not, due to the
fact that we'd pass a dict to the from_dict method, which then gets
assigned to self.decoded and further passed along to any later actual
encoder function like to_bytes or to_tlv.  In that instance, the dict
might be handed to a self._construct which has no idea how to process
the dict, as it expects the raw decoded value.

Change-Id: I3dd5204510e5c32ef1c4a999258d87cb3f1df8c8
Closes: OS#6073
Related: OS#6072
2023-06-27 09:29:37 +02:00
Harald Welte 0ec01504ab cosmetic: Implement cmd2.Settable backwards-compat via wrapper class
Let's avoid too many open-coded if-clauses and simply wrap it in
a compatibility class.

Change-Id: Id234f3fa56fe7eff8e1153d71b9be8a2e88dd112
2023-06-27 09:29:25 +02:00
Harald Welte 985ff31efa work-around what appears to be a pylint bug
smpp.pdu.pdu_types.DataCodingScheme.GSM_MESSAGE_CLASS very much exists,
and I can prove that manually in the python shell.  So let's assume this
is a pylint bug and work around it

pySim/sms.py:72:21: E1101: Instance of 'DataCodingScheme' has no 'GSM_MESSAGE_CLASS' member (no-member)

Change-Id: Iab34bae06940fecf681af9f45b8657e9be8cbc7b
2023-06-27 09:26:28 +02:00
Harald Welte e126872a29 Fix run-editor bug with cmd2 >= 2.0.0 compatibility
In cmd2, the upstream authors decided to rename a method in 2.0.0
without providing a backwards compatibility wrapper.  Let's add that
locally.

Change-Id: Iaa17b93db13ba330551799cce5f0388c78217224
Closes: OS#6071
2023-06-25 08:22:56 +02:00
Harald Welte 721ba9b31f tests: Add new, data-driven OTA tests
Rather than writing one test class with associated method for each
OTA algorithm / test, let's do this in a data-driven way, where new
test cases just have to provide test data, while the code iterates over
it.

Change-Id: I8789a21fa5a4793bdabd468adc9fee3b6e633c25
2023-06-18 10:50:50 +02:00