Overview

WTF? Why?
History: Telephony Networks (in Germany)
History: TDM networks
existing TDMoIP technology
OCTOI Protocol
OCTOI Software
Current OCTOI Network
WIP / Future Plans

WTF is this all about?

enable people to run legacy WAN equipment
- Modems
- ISDN-Adapters
- Phones, PBXs
- Telefax
- Frame Relay / HDLC / X.25 / routers
in a distributed fashion, over the internet
allow to run retronetworking tech from mid-1980s to about 2010 at a time where the related transport services are no longer available from public telephone operators / carriers

History: Analog Telephony Networks

analog telephony exited (in Germany) since 1877
analog electrical circuit from end to end
switching initially manually
automatic (electromechanical) switching introduced 1908-1966

History: Digitization inside the network

analog pairs of copper lines
carrier frequency systems (TF-Systeme) over coaxial wire
first digital PCM (pulse code modulation) links between exchanges
digital switching with analog switch matrix (EWS)
digital switching with digital switch matrix (EWSD, S12)
1979: decision to migrate entire German PSTN to digital switching
- estimated completion date in 2018
- actual completion of all 6200 exchanges in 1997

History: ISDN + digitization of subscriber line

German national ISDN (1TR6)
- available publicly from 1988
Euro-ISDN (EDSS1)
- MoU signed in 1993
- widely available by software update of exchange switches in 1994

History: Key properties of PSTN + ISDN

relatively few central offices (compared to modern DSLAM to the curb)
all central offices under battery backup
phantom voltage supply to every subscriber
emergency and other calls fully working during power outage!

History: the migration to DSL

initially [A]DSL as add-on to analog or ISDN using frequency splitter
- DSL in upper frequency band
- analog/ISDN in lower band
- network also has splitters and separate equipment for DSL and telephony
- you could still use your ISDN TAs and modems
today (>= 2018), DSL is used over full frequency spectrum, splitters are gone
- telephony implemented as VoIP service using SIP
- it became impossible to use most old non-voice ISDN/PSTN equipment

Migration paths for ISDN equipment

SIP ATA
- provides analog phone jacks on phone side, speaks SIP to operator
- works fine for voice telephony
- can occasionally work for Fax
- doesn’t work well for Modems

Migration paths for ISDN equipment

Fritzbox with ISDN BRI (S0) port
- terminates Q.921/Q.931 ISDN protocol stack internally
- speaks SIP to operator
- works fine for voice telephony
- can occasionally work for Fax
- doesn’t work well for Modems
- doesn’t work at all for X.25/X.31

Migration paths for ISDN equipment

enterprise grade SIP/ISDN gateways
- available for BRI (S0) and PRI (E1)
- typically to help companies avoid replacing their ISDN PBX
- terminates Q.921/Q.931 ISDN protocol stack internally
- speaks SIP to operator
- works fine for voice telephony
- can occasionally work for Fax
- doesn’t work well for Modems
- doesn’t work at all for X.25/X.31

VoIP: Why does only voice work well

well, it’s called voice over IP, not modem over ip ;)
one problem is audio codecs
- most VoIP uses lossy compression (think of MP3)
- lossy codecs work for voice, but nothing else
- can only be solved if end-to-end VoIP path can be forced to G711 or CLEARMODE
- depending on situation (involved SIP operators, …) this may or may not be possible
other problem is clocking
- impossible to solve / work around

Clocking in synchronous networks vs VoIP

classic analog networks had analog connection end-to-end. There’s constant phase / delay
ISDN networks used PDH and later SDH for transmission
- tight synchronization between the clocks of all central offices
- clocks means both the sample clock of ADCs/DACs as well as bit clock of ISDN PRI/BRI
In VoIP networks everyone uses their own clock source
- typically a normal XTAL in your ATA / FritzBox with as much as 20 ppm tolerance
if clocks between two partners drift, there will be underruns or overruns
- if this happens, you drop samples or insert samples
- this creates phase discontinuity
- modem standards were not designed for this

Modem/ISDN in the VoIP world

VoIP really is designed for voice only
- codec issue can be worked around if you can control codecs end-to-end
- clock issue requires hardware modding your ATA/FritzBox/Gateway
you still have no transparency
- what about ISDN signaling subtleties like HLC/LLC in SETUP?
- what about X.25 over ISDN D-Channel (X.31)

What you really want a transparent way of transporting ISDN/PSTN over IP, not some gateway that’s way too high up the protocol stack.

History: TDM networks

at some point (1970s?), digital PCM technology was introduced between exchanges
- initially PDH, later SDH (which can transport PDH)
- see OsmoDevCall on TDM/PDH/SDH
external interfaces, particularly to subscriber, still analog until mid-1980s
ISDN changed that: Introduced digital subscriber lines
- Basic Rate ISDN (BRI): 2x B, 1x D-Channel
- Primary Rate ISDN (PRI): 30x B, 1x D-Channel

Present: TDM networks mostly dead

Telcos have meanwhile mostly migrated to All-IP telephony
PDH/SDH networks mostly shut down, occasionally still in operation for legacy customers or some internal legacy services. Not actively sold anymore.
You cannot get a real PRI line anymore to connect your ISDN PBX or your Cisco router
Idea: Create community network of people who want to play with ISDN / TDM stuff
Naive approach: Use existing, off-the-shelf TDMoIP devices

Wanted: TDMoIP network

Existing TDMoIP: SAToP / CESoPSN

So we just get some of these and all is good?

Existing TDMoIP: SAToP / CESoPSN

Examples for transparent TDMoIP technologies

SAToP (Structure Agnostic TDM over IP)
- RFC 4553
CESoPSN (Circuit Emulation Service over Packet Switched Network)
- RFC 5086
transport of raw E1 frames over MPLS or UDP
designed for use in LAN or carrier backbone networks
waste a lot of bandwidth, even if TDM circuits are completely idle
- > 2Mbps plus header/packet overhead, bi-directional
typically no support for dynamic IP addresses

Existing TDMoIP: SIGTRAN IUA, EL2TPD

Examples for non-transparent TDMoIP technologies

SIGTRAN IUA (ISDN User Adaptation)
- RFC 3057
- works only for ISDN BRI/PRI with Q.921 as Layer 2 on signaling channel
- uses SCTP as transport (might not pass all middleboxes in public internet)
- no specification how the B-channels are handled, pure signaling solution
Ericsson L2TP / PacketAbis over IP
- proprietary, but FOSS implementation in @osmo-el2tpd@
- makes specific assumption of use of E1 as Ericsson Abis (GSM BTS back-haul)

The case for a new TDMoIP protocol

There’s room for a new protocol based on the following goals:

transparent. Should work for ISDN and other use cases (Frame Relay, Abis, SS7, ATM, …)
efficient. Should not waste a lot of bandwidth on an otherwise idle/unused link
dynamic IP. End-user internet access normally has dynamic IP addresses
nat friendly. Should work through any number of NATs and CG-NAT without special ALG/helper
IPv6 support. Should support IPv6 just like IPv4
authentication. Should have built-in mutual authentication

OCTOI Protocol: Bandwidth

Timeslot Suppression
- Transmitter has history of 1 TDM frame
- Before Tx, to-be-transmitted frame is compared with last frame
- only those timeslots with value != that of last frame are transmitted
Batching
- Batch 32 (up to 40) E1 frames per UDP packet
  - 8000 frames/s / 32 ⇒ 250 packets/s

Result: About 70 kbit/s internet bandwidth use (including UDP + IPv4 overhead) for an idle E1 link

OCTOI Protocol: dynamic IP / NAT-friendly

Classic client/server approach
Server currently requires fixed IP (no STUN/ICE/…)
Clients can have dynamic IPs
All messages (control plane + TDM user plan) in one UDP flow

First supported OCTOI hardware: icE1usb

osmocom icE1usb is an open source hardware/gateware/firmware/driver E1 interface
originally developed for talking to GSM base stations
contains a GPS disciplined oscillator for high-precision bit clock timing
further reading
- https://osmocom.org/projects/e1-t1-adapter/wiki/IcE1usb
- https://media.ccc.de/v/osmodevcon2019-97-software-defined-e1

E1oIP: osmo-e1d + icE1usb

osmo-e1d was the first (libusb, userspace) driver for the icE1usb hardware
traditionally, it sits between icE1usb hardware and applications using E1 like osmo-bsc
instead of a local application, it can now interface icE1usb to OCTOI
- octoi-server mode
- octoi-client mode

First 5 OCTOI users were connected to the hub that way

E1oIP: osmo-e1d + dahdi-trunkdev

having one icE1usb per peer/user doesn’t scale at the hub
this triggered the development of dahdi-trunkdev
- virtual E1 span for DAHDI without any real hardware
- simply provides stream of E1 frames on a character-device
- can be used to implement any virtual TDM interface/protocol
osmo-e1d was extended to open dahdi-trunkdev instead of icE1usb

This has become the only method of connecting to the OCTOI hub in September 2022

Current OCTOI network hub

DIVF: Debian 11 with dahdi-trunkdev, osmo-e1d and yate
2x TE820 8-port E1 cards, attaching to
- icE1usb (clock source)
- Livingston PortMaster 3 RAS
- Cisco AS5400 RAS

Rack view

1U Enclosure with APU, icE1usb, OVP

GPS Antenna mounted on the roof

this is an old antenna mount previously used for a satellite dish

GPS Antenna mounted on the roof

Current OCTOI participants

The hub currently has the following clients / participants:

using icE1usb at hub side
- manawyrm
- gruetzkopf
- roox
- cquirin
- drdeke
- tmwawpl
- tom/sirtux
- tnt

Current OCTOI services

E1 / TDM as "transport" layer
ISDN network as "routing" layer
- hub simulates the network / central office / switch side
- client devices implement the "user" side, just like when attaching to ISDN/PSTN
Services on top of ISDN
- Audio Telephony
- Video Telephony (T-View 100 / H.320)
- ISDN Data Calls
- Analog Modem Calls
- Telefax

OCTOI ISDN / Modem calls

hub currently has a Livingston Portmaster 3 RAS
- up to 30 ISDN data or analog modem calls (up to V.90)
services can be created in RADIUS, identified by Called Party Number
- PPP dial-up locally terminated
- PPP dial-up with forwarding of data via L2TP to remote LAC
- telnet or TCP clear forwarding to remote BBSs
Cisco AS5400 with much higher capacity is waiting to be provisioned

Challenges: Clocking

TDM networks need a stable bitclock at all parts of the network
everyone must transmit and receive at the same rate of bits / frames
we use GPS disciplined oscillators (GPS-DO) to ensure same clock everywhere
- this avoids overruns / underruns resulting in cycle slips that would create phase discontinuity for any modem signals carried over the network

Challenges: Packet Re-ordering

It seems like particularly on DOCSIS there is quite a bit of UDP packet re-ordering
Initially, osmo-e1d used a FIFO and dropped all out-of-order packets
later, we introduced a RIFO (Random In, First Out) to support re-ordering without loss

Challenges: Packet Loss

There is quite a bit of packet loss on the public internet
People probably don’t generally notice much, as most services use TCP or retransmit UDP
Surprisingly, I couldn’t find any comprehensive studies / research papers on packet loss behaviour of consumer internet that are less than 10 years old?
Right now we just accept it exists
Some early thoughts and experiments on FEC (Forward Error Correction)

Challenges: Latency

intercontinental public IP can easily reach >= 200ms RTT
ISDN timers in Q.921 and Q.931 are luckily working just fine over that kind of RTT
some suspicion that the high latency might have negative impact on Fax (T.30 timers) and modems (buffer sizes). I’m personally not yet convinced it is really an issue.

Current Services on top of OCTOI ISDN

Participants can call each other for any ISDN call/bearer type
Central hub provides
- Dial-Up (Modem up to V.90 + ISDN V.120/X.75) access
  - Tons of telnet-reachable BBSs all around the world
  - BTX (an instance of Neu-Ulm courtesy of https://btx-museum.de/)
    - unfortunately currently still crashes ISDN BTX Terminal T-DisplayTel
  - Fax gateway (WIP)
Some other users provide
- Video on demand for ISDN video telephony
- Remote DOOM for ISDN video telephony
- Chiptune audio on demand via voice calls

https://osmocom.org/projects/octoi/wiki/Phonebook

What do I need to connect to the network?

Currently, we only offer PRI (E1) service, so you need
- an icE1usb plus some Linux USB host (Raspi, …)
- some equipment that talks E1, like an enterprise PBX
  - Auerswald COMmander2 Basic with S2M is a popular and widely available choice
  - this PBX then offers analog and BRI ports for your phones / modems / TAs

What will I need to connect to the network?

We’re working on a small, low-power fully integrated device
Ethernet/IP connection towards Internet / OCTOI hub
2x S0 (BRI) ports for your ISDN equipment, or analog phones via a/b TA
First prototype looks promising. Main work pending is the uC software.

Future Plans

ISDN BRI (S/T) access to OCTOI network
- more end-user friendly; many people have S0 equipment and no E1/S2M PBX at home
Frame Relay switch / hub
- second, additional service, completely independent of the current ISDN service
- initial testing has confirmed HDLC / FR over OCTOI works without problems
X.25 network support
- X.25 access via ISDN D-Channel (X.31)
exhibits at hacker and vintage computing events (like VCFB!)

ISDN BRI (S/T) access to OCTOI

BRI (ISDN basic rate aka "S/T" aka "S0") support in OCTOI protocol
BRI hardware with 2x S/T interface and GPS-DO
- unlike icE1usb: Ethernet/IP support, not USB
- complete appliance for OCTOI client mode: no need for computer
- Status: design of first break-out / evaluation board for ISDN BRI side complete
- Software not even started yet. Idea is to use Nuttx on Atmel SAMV71

Longer-Term Future Plans

improve FOSS soft-modem situation (linmodem, spandsp)
investigate somewhat limited V.90 performance so far
support for other Q.931 dialects than DSS1 (like 1TR6 or even NI1)

Thanks

Thanks to

Sylvain "tnt" Munaut for icE1usb and all his help
All of the early OCTOI network participants manawyrm, gruetzkopf, roox, DrDeke, tmwawpl, cquirin
noris.net for sponsoring the co-location of the OCTOI hub
dexter, manawrym and TobleMiner for staffing the Retronetworking exhibit at VCFB

Grafana / Stats

Let’s have a look at the statistics…

Modem call

Let’s demo a modem call

EOF

End of File

OCTOI - Osmocom Community TDM over IP

Overview

WTF is this all about?

History: Analog Telephony Networks

History: Digitization inside the network

History: ISDN + digitization of subscriber line

History: Key properties of PSTN + ISDN

History: the migration to DSL

Migration paths for ISDN equipment

Migration paths for ISDN equipment

Migration paths for ISDN equipment

VoIP: Why does only voice work well

Clocking in synchronous networks vs VoIP

Modem/ISDN in the VoIP world

History: TDM networks

Present: TDM networks mostly dead

Wanted: TDMoIP network

Existing TDMoIP: SAToP / CESoPSN

Existing TDMoIP: SAToP / CESoPSN

Existing TDMoIP: SIGTRAN IUA, EL2TPD

The case for a new TDMoIP protocol

OCTOI Protocol: Bandwidth

OCTOI Protocol: dynamic IP / NAT-friendly

First supported OCTOI hardware: icE1usb

E1oIP: osmo-e1d + icE1usb

E1oIP: osmo-e1d + dahdi-trunkdev

Current OCTOI network hub

Rack view

1U Enclosure with APU, icE1usb, OVP

GPS Antenna mounted on the roof

GPS Antenna mounted on the roof

Current OCTOI participants

Current OCTOI services

OCTOI ISDN / Modem calls

Challenges: Clocking

Challenges: Packet Re-ordering

Challenges: Packet Loss

Challenges: Latency

Current Services on top of OCTOI ISDN

What do I need to connect to the network?

What will I need to connect to the network?

Future Plans

ISDN BRI (S/T) access to OCTOI

Longer-Term Future Plans

Thanks

Further Reading

Grafana / Stats

Modem call

EOF