CardKeyProvider: Implement support for column-based transport key encryption

It's generally a bad idea to keep [card specific] key material lying
around unencrypted in CSV files.  The industry standard solution in the
GSMA is a so-called "transport key", which encrypts the key material.

Let's introduce support for this in the CardKeyProvider (and
specifically, the CardKeyProviderCSV) and allow the user to specify
transport key material as command line options to pySim-shell.

Different transport keys can be used for different key materials, so
allow specification of keys on a CSV-column base.

The higher-level goal is to allow the CSV file not only to store
the ADM keys (like now), but also global platform key material for
establishing SCP towards various security domains in a given card.

Change-Id: I13146a799448d03c681dc868aaa31eb78b7821ff
This commit is contained in:
Harald Welte 2024-05-25 10:53:56 +02:00
parent a3d41a147f
commit 1aaf978d9f
4 changed files with 168 additions and 15 deletions

contrib/ Executable file
View File

@ -0,0 +1,79 @@
#!/usr/bin/env python3
# Utility program to perform column-based encryption of a CSV file holding SIM/UICC
# related key materials.
# (C) 2024 by Harald Welte <>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <>.
import sys
import csv
import argparse
from Cryptodome.Cipher import AES
from pySim.utils import h2b, b2h, Hexstr
from pySim.card_key_provider import CardKeyProviderCsv
def dict_keys_to_upper(d: dict) -> dict:
return {k.upper():v for k,v in d.items()}
class CsvColumnEncryptor:
def __init__(self, filename: str, transport_keys: dict):
self.filename = filename
self.transport_keys = dict_keys_to_upper(transport_keys)
def encrypt_col(self, colname:str, value: str) -> Hexstr:
key = self.transport_keys[colname]
cipher =, AES.MODE_CBC, CardKeyProviderCsv.IV)
return b2h(cipher.encrypt(h2b(value)))
def encrypt(self) -> None:
with open(self.filename, 'r') as infile:
cr = csv.DictReader(infile)
cr.fieldnames = [field.upper() for field in cr.fieldnames]
with open(self.filename + '.encr', 'w') as outfile:
cw = csv.DictWriter(outfile, dialect=csv.unix_dialect, fieldnames=cr.fieldnames)
for row in cr:
for key_colname in self.transport_keys:
if key_colname in row:
row[key_colname] = self.encrypt_col(key_colname, row[key_colname])
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('CSVFILE', help="CSV file name")
parser.add_argument('--csv-column-key', action='append', required=True,
help='per-CSV-column AES transport key')
opts = parser.parse_args()
csv_column_keys = {}
for par in opts.csv_column_key:
name, key = par.split(':')
csv_column_keys[name] = key
if len(csv_column_keys) == 0:
print("You must specify at least one key!")
csv_column_keys = CardKeyProviderCsv.process_transport_keys(csv_column_keys)
for name, key in csv_column_keys.items():
print("Encrypting column %s using AES key %s" % (name, key))
cce = CsvColumnEncryptor(opts.CSVFILE, csv_column_keys)

View File

@ -41,6 +41,38 @@ of pySim-shell. If you do not specify a CSV file, pySim will attempt to
open a CSV file from the default location at
`~/.osmocom/pysim/card_data.csv`, and use that, if it exists.
Column-Level CSV encryption
pySim supports column-level CSV encryption. This feature will make sure
that your key material is not stored in plaintext in the CSV file.
The encryption mechanism uses AES in CBC mode. You can use any key
length permitted by AES (128/192/256 bit).
Following GSMA FS.28, the encryption works on column level. This means
different columns can be decrypted using different key material. This
means that leakage of a column encryption key for one column or set of
columns (like a specific security domain) does not compromise various
other keys that might be stored in other columns.
You can specify column-level decryption keys using the
`--csv-column-key` command line argument. The syntax is
`FIELD:AES_KEY_HEX`, for example:
` --csv-column-key SCP03_ENC_ISDR:000102030405060708090a0b0c0d0e0f`
In order to avoid having to repeat the column key for each and every
column of a group of keys within a keyset, there are pre-defined column
group aliases, which will make sure that the specified key will be used
by all columns of the set:
* `UICC_SCP02` is a group alias for `UICC_SCP02_KIC1`, `UICC_SCP02_KID1`, `UICC_SCP02_KIK1`
* `UICC_SCP03` is a group alias for `UICC_SCP03_KIC1`, `UICC_SCP03_KID1`, `UICC_SCP03_KIK1`
* `SCP03_ECASD` is a group alias for `SCP03_ENC_ECASD`, `SCP03_MAC_ECASD`, `SCP03_DEK_ECASD`
* `SCP03_ISDA` is a group alias for `SCP03_ENC_ISDA`, `SCP03_MAC_ISDA`, `SCP03_DEK_ISDA`
* `SCP03_ISDR` is a group alias for `SCP03_ENC_ISDR`, `SCP03_MAC_ISDR`, `SCP03_DEK_ISDR`
Field naming

View File

@ -967,6 +967,8 @@ global_group.add_argument('--script', metavar='PATH', default=None,
help='script with pySim-shell commands to be executed automatically at start-up')
global_group.add_argument('--csv', metavar='FILE',
default=None, help='Read card data from CSV file')
global_group.add_argument('--csv-column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append',
help='per-CSV-column AES transport key')
global_group.add_argument("--card_handler", dest="card_handler_config", metavar="FILE",
help="Use automatic card handling machine")
@ -993,13 +995,18 @@ if __name__ == '__main__':
print("Invalid script file!")
csv_column_keys = {}
for par in opts.csv_column_key:
name, key = par.split(':')
csv_column_keys[name] = key
# Register csv-file as card data provider, either from specified CSV
# or from CSV file in home directory
csv_default = str(Path.home()) + "/.osmocom/pysim/card_data.csv"
if opts.csv:
card_key_provider_register(CardKeyProviderCsv(opts.csv, csv_column_keys))
if os.path.isfile(csv_default):
card_key_provider_register(CardKeyProviderCsv(csv_default, csv_column_keys))
# Init card reader driver
sl = init_reader(opts, proactive_handler = Proact())

View File

@ -10,10 +10,10 @@ the need of manually entering the related card-individual data on every
operation with pySim-shell.
# (C) 2021 by Sysmocom s.f.m.c. GmbH
# (C) 2021-2024 by Sysmocom s.f.m.c. GmbH
# All Rights Reserved
# Author: Philipp Maier
# Author: Philipp Maier, Harald Welte
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@ -29,18 +29,29 @@ operation with pySim-shell.
# along with this program. If not, see <>.
from typing import List, Dict, Optional
from Cryptodome.Cipher import AES
from pySim.utils import h2b, b2h
import abc
import csv
card_key_providers = [] # type: List['CardKeyProvider']
# well-known groups of columns relate to a given functionality. This avoids having
# to specify the same transport key N number of times, if the same key is used for multiple
# fields of one group, like KIC+KID+KID of one SD.
class CardKeyProvider(abc.ABC):
"""Base class, not containing any concrete implementation."""
'IMSI', 'PIN1', 'PIN2', 'PUK1', 'PUK2']
# check input parameters, but do nothing concrete yet
def _verify_get_data(self, fields: List[str] = [], key: str = 'ICCID', value: str = "") -> Dict[str, str]:
@ -53,14 +64,10 @@ class CardKeyProvider(abc.ABC):
dictionary of {field, value} strings for each requested field from 'fields'
for f in fields:
if f not in self.VALID_FIELD_NAMES:
raise ValueError("Requested field name '%s' is not a valid field name, valid field names are: %s" %
(f, str(self.VALID_FIELD_NAMES)))
if key not in self.VALID_FIELD_NAMES:
if key not in self.VALID_KEY_FIELD_NAMES:
raise ValueError("Key field name '%s' is not a valid field name, valid field names are: %s" %
(key, str(self.VALID_FIELD_NAMES)))
(key, str(self.VALID_KEY_FIELD_NAMES)))
return {}
@ -84,19 +91,47 @@ class CardKeyProvider(abc.ABC):
class CardKeyProviderCsv(CardKeyProvider):
"""Card key provider implementation that allows to query against a specified CSV file"""
"""Card key provider implementation that allows to query against a specified CSV file.
Supports column-based encryption as it is generally a bad idea to store cryptographic key material in
plaintext. Instead, the key material should be encrypted by a "key-encryption key", occasionally also
known as "transport key" (see GSMA FS.28)."""
IV = b'\x23' * 16
csv_file = None
filename = None
def __init__(self, filename: str):
def __init__(self, filename: str, transport_keys: dict):
filename : file name (path) of CSV file containing card-individual key/data
transport_keys : a dict indexed by field name, whose values are hex-encoded AES keys for the
respective field (column) of the CSV. This is done so that different fields
(columns) can use different transport keys, which is strongly recommended by
self.csv_file = open(filename, 'r')
if not self.csv_file:
raise RuntimeError("Could not open CSV file '%s'" % filename)
self.filename = filename
self.transport_keys = self.process_transport_keys(transport_keys)
def process_transport_keys(transport_keys: dict):
"""Apply a single transport key to multiple fields/columns, if the name is a group."""
new_dict = {}
for name, key in transport_keys.items():
if name in CRYPT_GROUPS:
for field in CRYPT_GROUPS[name]:
new_dict[field] = key
new_dict[name] = key
return new_dict
def _decrypt_field(self, field_name: str, encrypted_val: str) -> str:
"""decrypt a single field, if we have a transport key for the field of that name."""
if not field_name in self.transport_keys:
return encrypted_val
cipher =[field_name]), AES.MODE_CBC, self.IV)
return b2h(cipher.decrypt(h2b(encrypted_val)))
def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]:
super()._verify_get_data(fields, key, value)
@ -113,7 +148,7 @@ class CardKeyProviderCsv(CardKeyProvider):
if row[key] == value:
for f in fields:
if f in row:
rc.update({f: row[f]})
rc.update({f: self._decrypt_field(f, row[f])})
raise RuntimeError("CSV-File '%s' lacks column '%s'" %
(self.filename, f))