252 lines
9.6 KiB
Plaintext
252 lines
9.6 KiB
Plaintext
pySim-shell - next generation SIM configuration tool
|
|
====================================================
|
|
:author: Harald Welte <laforge@gnumonks.org>
|
|
:copyright: 2021 by Harald Welte (License: CC-BY-SA)
|
|
:backend: slidy
|
|
:max-width: 45em
|
|
|
|
== why program your own SIM cards?
|
|
|
|
* because you want to operate your own cellular network (2G, 3G, 4G, 5G)
|
|
** whether FOSS (Osmocom CNI, srsLTE, nextepc/open5gs, free5gc, OpenBTS, yateBTS, ...)
|
|
** but also proprietary (e.g. Amarisoft)
|
|
* 2G can use random 3rd party cards, but it's not really convenient in most cases
|
|
* Anything >= 3G requires you to know the card-specific secrete keys
|
|
** in reality that translates to having your own programmable SIM cards
|
|
|
|
== some smartcard terminology
|
|
|
|
* card filesystem
|
|
** *MF* (Master File): The root directory
|
|
** *DF* (Dedicated File): A subdirectory
|
|
** *ADF* (Application Dedicated File): Directory of an application (like USIM, ISIM)
|
|
** *EF* (Entry File): A regulare file
|
|
*** *Transparent EF*: An unstructured file
|
|
*** *Linear Fixed EF*: An file consisting of fixed-length records
|
|
|
|
== specs vs. proprietary
|
|
|
|
* SIM cards are fully specified by a combination of ISU, ETSI and 3GPP specs
|
|
* this covers only the operation after the card has been issued, e.g.
|
|
** reading and writing files accessible without PIN auth, after PIN auth
|
|
** performing GSM authentication or UMTS AKA
|
|
* it does _not_ cover how the card is issued/provisioned
|
|
** secret key material (Ki / K / OP / OPc) is not readable from the card
|
|
** it is an implementation detail on how the card manufacturer writes those to the card
|
|
* this leads to the need for card-specific support / code in software like pySim
|
|
|
|
== pySim "classic"
|
|
|
|
* originally developed in 2009 by Sylvain Munaut
|
|
* Linux/FOSS tool for configuration of SuperSIM/MagicSIM
|
|
* replicated what proprietary vendor tools did to configure IMSI, Ki, ...
|
|
* those ultra-cheap Chinese SIM cards at the time (from the SIM card cloning days of COMP128v1) shipped with binary-only windows programs
|
|
|
|
== pySim "classic" fundamentals
|
|
|
|
* library code
|
|
** _transport_ - how to exchange APDUs with card
|
|
** _commands_ - encoding of ISO7816-4 / TS 51.011 / ETS 102 221 commands
|
|
** _cards_ - classes implementing various different programmable SIM cards
|
|
* executable programs
|
|
** `pySim-prog` for card programming
|
|
** `pySim-read` for card reading
|
|
|
|
== pySim transports
|
|
|
|
* transport - how to exchange APDUs with card
|
|
** _serial_ transport for serial readers (UART Rx+Tx connected to I/O)
|
|
** _pcsc_ transport for pcsc-lite supported readers (e.g. USB CCID)
|
|
* some more exotic options added later on by Vadim Yanitskiy:
|
|
** _calypso_ for using L1CTL to talk to OsmocomBB on Ti Calypso phone
|
|
** _modem_atcmd_ for using AT+CSIM commands talking to a cellular modem
|
|
|
|
== advanced pySim "classic" features
|
|
|
|
* batch programming of cards, using CSV file input
|
|
* even with ability to execute external tool for driving motorized card readers
|
|
|
|
== Limitations of pySim "classic"
|
|
|
|
* supports only very few files/parameters that can be programmed
|
|
** IMSI, MSISDN, Ki and SMSP was sufficient for making classic GSM SIMs work on your own network
|
|
** today, people want to configure VoLTE/IMS parameters, 5G SUCI stuff, etc.
|
|
* adding more fields to pySim-classic makes it unusable: Who wants to specify 250 different fields as command line arguments?
|
|
|
|
== Usability of pySim "classic"
|
|
|
|
* it was designed to program a batch of cards with defaults and random keys
|
|
* there are all kinds of 'automagic' bits happening if not all fields are specified
|
|
* pySim at least originally assumed that you program a full card, and not that you perform incremental small changes
|
|
* no user manual, very limited RADME and wiki instructions
|
|
|
|
== the road to pySim-shell
|
|
|
|
What to do about the user interface?
|
|
|
|
* GUIs are bad, they are not really scriptable/automatizable
|
|
* GUIs are hard to portable, test automatically, etc.
|
|
* people setting up FOSS cellular programs most likely use bash/zsh/tcsh on daily basis, so they are familiar with the concept of interactive shells / CLIs
|
|
* so let's do a CLI / shell / REPL!
|
|
|
|
== pySim-shell and `cmd2`
|
|
|
|
* uses `cmd2` python module for the shell
|
|
** extremely powerful framework for building CLI/shell type interfaces
|
|
** allows us to focus on implementing logic (like individual commands), not the shell itself
|
|
* supports output redirection to local files
|
|
* supports piping output through external programs ("| grep foo")
|
|
* supports `argparse` to define commands and their options
|
|
* supports [persistent] command history
|
|
* allows to dynamically register and unregister commands
|
|
* allows full control over tab completion items.
|
|
|
|
== pySim-shell UI fundamentals
|
|
|
|
* some global commands for things like PIN/CHV verification, enabling, disabling, unblcoking
|
|
* you `select` your way through the card filesystem, like 'cd' on bash/zsh/...
|
|
** tab-completion for [known] file names
|
|
|
|
.Selecting a file
|
|
----
|
|
pySIM-shell (MF/ADF.USIM)> select EF.IMSI
|
|
{
|
|
"file_descriptor": {
|
|
"shareable": true,
|
|
"file_type": "working_ef",
|
|
"structure": "transparent"
|
|
},
|
|
"file_identifier": "6F07",
|
|
"proprietary_info": {
|
|
"proprietary_D0": "20",
|
|
"proprietary_D2": "0F"
|
|
},
|
|
"life_cycle_status_int": "operational_activated",
|
|
"security_attrib_ref_expanded": "6F0603",
|
|
"file_size": 9,
|
|
"short_file_id": "38"
|
|
}
|
|
pySIM-shell (MF/ADF.USIM/EF.IMSI)>
|
|
----
|
|
|
|
== pySim-shell reading/writing
|
|
|
|
* context-sensitive 'action' commands depend on the type of file selected
|
|
** `read_binary`, `update_binary`, ... for transparent EF
|
|
** `read_record`, `update_record`, ... for linear fixed EF
|
|
|
|
.Reading and updating a transparent EF
|
|
----
|
|
pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary
|
|
089910070000400300
|
|
pySIM-shell (MF/ADF.USIM/EF.IMSI)> update_binary 089910070000400301
|
|
pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary
|
|
089910070000400301
|
|
----
|
|
|
|
== pySim-shell decoding/encoding
|
|
|
|
* working with hex-encoded binary data is not very convenient
|
|
* each specified file in the filesystem is represented by a python class
|
|
** class can provide encoder + decoder methods to go from binary to JSON and vice-versa
|
|
* some shell commands combine reading a file from the card + decoding / encoding
|
|
** `read_binary_decoded`, `write_binary_decoded`, ... for transparent EF
|
|
** `read_record_decoded`, `write_record_decoded`, ... for linear fixed EF
|
|
|
|
.Reading and updating a decoded transparent EF
|
|
----
|
|
pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary_decoded
|
|
{
|
|
"imsi": "901700000043000"
|
|
}
|
|
pySIM-shell (MF/ADF.USIM/EF.IMSI)> update_binary_decoded '{"imsi": "901700000043999"}'
|
|
pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary_decoded
|
|
{
|
|
"imsi": "901700000043999"
|
|
}
|
|
----
|
|
|
|
== pySim-shell editing of single parameters
|
|
|
|
* many files contain a lot of parameters
|
|
* entering all of them as a single line in `update_binary_decoded` becomes rather cumbersome
|
|
* `edit_binary_decoded` and `edit_record_decoded` to the rescue
|
|
** reads data from the card
|
|
** decodes it to JSON
|
|
** spawns whatever is your favorite text editor
|
|
** you do your editing, save + exit
|
|
** re-encodes the modified JSON
|
|
** writes the result to the card
|
|
|
|
== pySim-shell `export`
|
|
|
|
* one-shot command to iterate over all [known] files below the current DF and dump them
|
|
* output format is structured in a way that you can use it as a pySim-shell script
|
|
* results in easy method to backup and restore a card
|
|
|
|
NOTE:: only the ETSI/3GPP standard files that are covered by pySim classes are exported. There may be any
|
|
number of proprietary additional files on the card. Particularly K/OP/OPc or OTA keys are not exported this
|
|
way
|
|
|
|
== pySim-shell advanced cmd2 usage
|
|
|
|
* `run_script`: execute any text file as script inside the pySim-shell
|
|
* `run_pyscript`: execute a python script with access to all of the state / classes / methods
|
|
* `ipy`: run an interactive IPython session
|
|
* `py`: run an interactive python interpreter
|
|
** `card` represents the `pySim.cards.Card` instance
|
|
** `rs` represents the `pySim.filesystem.RuntimeState`
|
|
|
|
.Using interactive python shell with access to pySim-shell state
|
|
----
|
|
pySIM-shell (MF)> py
|
|
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
|
|
[GCC 10.2.1 20210110] on linux
|
|
Type "help", "copyright", "credits" or "license" for more information.
|
|
|
|
End with `Ctrl-D` (Unix) / `Ctrl-Z` (Windows), `quit()`, `exit()`.
|
|
Non-Python commands can be issued with: app("your command")
|
|
|
|
>>> rs.selected_file
|
|
<pySim.filesystem.CardMF object at 0x7f97952f3070>
|
|
>>> rs.selected_file.desc
|
|
'Master File (directory root)'
|
|
----
|
|
|
|
|
|
== Other recent pySim developments
|
|
|
|
* introduction of `sphinx` for generating documentation
|
|
* adding API documentation at least for all of the *new* classes/methods that are used to build pySim-shell,
|
|
and which you need to extend to cover more files
|
|
* automatic generation of pySim-shell command reference from `argparse` instances
|
|
* keep your ADM1 pins per ICCID in a CSV file and automatically verify them
|
|
|
|
== pySim-shell status
|
|
|
|
* many, if not most SIM/USIM/ISIM files defined
|
|
* only few of them have JSON decoders and encoders yet
|
|
* works for interactive use cases
|
|
* not yet full feature parity with pySim-prog (no batch programming, ...)
|
|
|
|
== Call for Contributions
|
|
|
|
* pySim-shell is already very usable, but needs your help!
|
|
** most files still need encoder + decoder methods to be implemented
|
|
** we need automatic testing of encoders + decoders
|
|
** we want JSON schema for the JSON representation of each file
|
|
** we want to have batch programming in pySim-shell
|
|
|
|
So if you know a bit of python and care about SIM cards, please send your patches!
|
|
|
|
== Further Reading
|
|
|
|
* https://osmocom.org/projects/pysim/wiki[pySim project page on osmocom.org]
|
|
* https://git.osmocom.org/pysim/about/[pySim source code / git repository]
|
|
* https://media.ccc.de/v/36c3-10737-sim_card_technology_from_a-z[Video of talk "SIM card technology from A-Z"]
|
|
|
|
|
|
== EOF
|
|
|
|
End of File
|