OCTOI - Osmocom Community TDM over IP
=====================================
:author:	Harald Welte <laforge@gnumonks.org>
:copyright:	2022 by Harald Welte (License: CC-BY-SA)
:backend:	slidy
:max-width:	45em


== Overview

* WTF? Why?
* History: Telephony Networks (in Germany)
* History: TDM networks
* existing TDMoIP technology
* OCTOI Protocol
* OCTOI Software
* Current OCTOI Network
* WIP / Future Plans

== WTF is this all about?

* enable people to run legacy WAN equipment
** Modems
** ISDN-Adapters
** Phones, PBXs
** Telefax
** Frame Relay / HDLC / X.25 / routers
* in a distributed fashion, over the internet
* allow to run retronetworking tech from mid-1980s to about 2010 at a time where the related transport
  services are no longer available from public telephone operators / carriers

// ***********************************************************************
// History: telephony networks
// ***********************************************************************

== History: Analog Telephony Networks

* analog telephony exited (in Germany) since 1877
* analog electrical circuit from end to end
* switching initially manually
* automatic (electromechanical) switching introduced 1908-1966

== History: Digitization inside the network

* analog pairs of copper lines
* carrier frequency systems (TF-Systeme) over coaxial wire
* first digital PCM (pulse code modulation) links between exchanges
* digital switching with analog switch matrix (EWS)
* digital switching with digital switch matrix (EWSD, S12)
* 1979: decision to migrate entire German PSTN to digital switching
** estimated completion date in 2018
** actual completion of all 6200 exchanges in 1997

== History: ISDN + digitization of subscriber line

* German national ISDN (1TR6)
** available publicly from 1988
* Euro-ISDN (EDSS1)
** MoU signed in 1993
** widely available by software update of exchange switches in 1994

== History: Key properties of PSTN + ISDN

* relatively few central offices (compared to modern DSLAM to the curb)
* all central offices under battery backup
* phantom voltage supply to every subscriber
* emergency and other calls fully working during power outage!

== History: the migration to DSL

* initially [A]DSL as add-on to analog or ISDN using frequency splitter
** DSL in upper frequency band
** analog/ISDN in lower band
** network also has splitters and separate equipment for DSL and telephony
** you could still use your ISDN TAs and modems 
* today (>= 2018), DSL is used over full frequency spectrum, splitters are gone
** telephony implemented as VoIP service using SIP
** it became impossible to use most old non-voice ISDN/PSTN equipment

== Migration paths for ISDN equipment

* SIP ATA
** provides analog phone jacks on phone side, speaks SIP to operator
** works fine for voice telephony
** can occasionally work for Fax
** doesn't work well for Modems

== Migration paths for ISDN equipment

* Fritzbox with ISDN BRI (S0) port
** terminates Q.921/Q.931 ISDN protocol stack internally
** speaks SIP to operator
** works fine for voice telephony
** can occasionally work for Fax
** doesn't work well for Modems
** doesn't work at all for X.25/X.31

== Migration paths for ISDN equipment

* _enterprise grade_ SIP/ISDN gateways
** available for BRI (S0) and PRI (E1)
** typically to help companies avoid replacing their ISDN PBX
** terminates Q.921/Q.931 ISDN protocol stack internally
** speaks SIP to operator
** works fine for voice telephony
** can occasionally work for Fax
** doesn't work well for Modems
** doesn't work at all for X.25/X.31

== VoIP: Why does only voice work well

* well, it's called _voice_ over IP, not _modem_ over ip ;)
* one problem is audio codecs
** most VoIP uses lossy compression (think of MP3)
** lossy codecs work for voice, but nothing else
** can only be solved if end-to-end VoIP path can be forced to G711 or CLEARMODE
** depending on situation (involved SIP operators, ...) this may or may not be possible
* other problem is clocking
** impossible to solve / work around

== Clocking in synchronous networks vs VoIP

* classic analog networks had analog connection end-to-end.  There's constant phase / delay
* ISDN networks used PDH and later SDH for transmission
** tight synchronization between the clocks of all central offices
** clocks means both the sample clock of ADCs/DACs as well as bit clock of ISDN PRI/BRI
* In VoIP networks everyone uses their own clock source
** typically a normal XTAL in your ATA / FritzBox with as much as 20 ppm tolerance
* if clocks between two partners drift, there will be underruns or overruns
** if this happens, you drop samples or insert samples
** this creates phase discontinuity
** modem standards were *not* designed for this

== Modem/ISDN in the VoIP world

* VoIP really is designed for voice only
** codec issue can be worked around if you can control codecs end-to-end
** clock issue requires hardware modding your ATA/FritzBox/Gateway
* you still have no transparency
** what about ISDN signaling subtleties like HLC/LLC in SETUP?
** what about X.25 over ISDN D-Channel (X.31)

What you really want a *transparent* way of transporting ISDN/PSTN over IP, not some gateway that's way too high up the protocol stack.

// ***********************************************************************
// History: TDM networks
// ***********************************************************************

== History: TDM networks

* at some point (1970s?), digital PCM technology was introduced between exchanges
** initially PDH, later SDH (which can transport PDH)
** see https://media.ccc.de/v/osmodevcall-20211112-laforge-tdm[OsmoDevCall on TDM/PDH/SDH]
* external interfaces, particularly to subscriber, still analog until mid-1980s
* ISDN changed that: Introduced digital subscriber lines
** Basic Rate ISDN (BRI): 2x B, 1x D-Channel
** Primary Rate ISDN (PRI): 30x B, 1x D-Channel

== Present: TDM networks mostly dead

* Telcos have meanwhile mostly migrated to All-IP telephony
* PDH/SDH networks mostly shut down, occasionally still in operation for legacy customers or some internal
  legacy services. Not actively sold anymore.
* You cannot get a real PRI line anymore to connect your ISDN PBX or your Cisco router
* Idea: Create community network of people who want to play with ISDN / TDM stuff
* Naive approach: Use existing, off-the-shelf TDMoIP devices


== Wanted: TDMoIP network

[graphviz]
----
graph G {
  hub [shape=box, label="TDMoIP hub\ncross-connect\nin public internet"];
  subgraph cluster_1 {
    label="Hobbyist A";
    ad1 [label="Access Device\n"];
    pbx1 [label="PBX"];
    phone1a [label="Phone"];
    modem1b [label="Modem"];
    ta1c [label="ISDN TA"];
    pbx1 -- ad1 [label="E1"];
    phone1a -- pbx1 [label="POTS"];
    modem1b -- pbx1 [label="POTS"];
    ta1c -- pbx1 [label="ISDN-BRI"];
  }
  subgraph cluster_2 {
    label="Hobbyist B";
    ad2 [label="Access Device\n"];
    pbx2 [label="PBX"];
    phone2a [label="Phone"];
    modem2b [label="Modem"];
    ta2c [label="ISDN TA"];
    pbx2 -- ad2 [label="E1"];
    phone2a -- pbx2 [label="POTS"];
    modem2b -- pbx2 [label="POTS"];
    ta2c -- pbx2 [label="ISDN-BRI"];
  }
  subgraph cluster_3 {
    label="Hobbyist C";
    ad3 [label="Access Device\n"];
    pbx3 [label="PBX"];
    phone3a [label="Phone"];
    modem3b [label="Modem"];
    ta3c [label="ISDN TA"];
    pbx3 -- ad3 [label="E1"];
    phone3a -- pbx3 [label="POTS"];
    modem3b -- pbx3 [label="POTS"];
    ta3c -- pbx3 [label="ISDN-BRI"];
  }
  ad1 -- hub [label="TDMoIP\nInternet"];
  ad2 -- hub [label="TDMoIP\nInternet"];
  ad3 -- hub [label="TDMoIP\nInternet"];
}
----

// ***********************************************************************
// Existing TDMoIP technology
// ***********************************************************************

== Existing TDMoIP: SAToP / CESoPSN

So we just get some of these and all is good?

image::RAD_IPmux-1_front.jpg[width=1200,align="center"]

== Existing TDMoIP: SAToP / CESoPSN

Examples for _transparent_ TDMoIP technologies

* SAToP (Structure Agnostic TDM over IP)
** RFC 4553
* CESoPSN (Circuit Emulation Service over Packet Switched Network)
** RFC 5086
* transport of raw E1 frames over MPLS or UDP
* designed for use in LAN or carrier backbone networks
* waste a lot of bandwidth, even if TDM circuits are completely idle
** > 2Mbps plus header/packet overhead, bi-directional
* typically no support for dynamic IP addresses

== Existing TDMoIP: SIGTRAN IUA, EL2TPD

Examples for _non-transparent_ TDMoIP technologies

* SIGTRAN IUA (ISDN User Adaptation)
** RFC 3057
** works only for ISDN BRI/PRI with Q.921 as Layer 2 on signaling channel
** uses SCTP as transport (might not pass all middleboxes in public internet)
** no specification how the B-channels are handled, pure signaling solution
* Ericsson L2TP / PacketAbis over IP
** proprietary, but FOSS implementation in @osmo-el2tpd@
** makes specific assumption of use of E1 as Ericsson Abis (GSM BTS back-haul)

// ***********************************************************************
// OCTOI TDMoIP protocol
// ***********************************************************************

== The case for a new TDMoIP protocol

There's room for a new protocol based on the following goals:

* *transparent*.  Should work for ISDN and other use cases (Frame Relay, Abis, SS7, ATM, ...)
* *efficient*. Should not waste a lot of bandwidth on an otherwise idle/unused link
* *dynamic IP*. End-user internet access normally has dynamic IP addresses
* *nat friendly*. Should work through any number of NATs and CG-NAT without special ALG/helper
* *IPv6 support*. Should support IPv6 just like IPv4
* *authentication*.  Should have built-in mutual authentication

== OCTOI Protocol: Bandwidth

* Timeslot Suppression
** Transmitter has history of 1 TDM frame
** Before Tx, to-be-transmitted frame is compared with last frame
** only those timeslots with value != that of last frame are transmitted
* Batching
** Batch 32 (up to 40) E1 frames per UDP packet
*** 8000 frames/s / 32 => 250 packets/s

Result: About 70 kbit/s internet bandwidth use (including UDP + IPv4 overhead) for an idle E1 link

== OCTOI Protocol: dynamic IP / NAT-friendly

* Classic client/server approach
* Server currently requires fixed IP (no STUN/ICE/...)
* Clients can have dynamic IPs
* All messages (control plane + TDM user plan) in one UDP flow

// ***********************************************************************
// OCTOI Software
// ***********************************************************************

== First supported OCTOI hardware: icE1usb

* osmocom icE1usb is an open source hardware/gateware/firmware/driver E1 interface
* originally developed for talking to GSM base stations
* contains a GPS disciplined oscillator for high-precision bit clock timing
* further reading
** https://osmocom.org/projects/e1-t1-adapter/wiki/IcE1usb
** https://media.ccc.de/v/osmodevcon2019-97-software-defined-e1

image::icE1usb-usb_side.jpg[width=600,align="center"]

== E1oIP: osmo-e1d + icE1usb

* osmo-e1d was the first (libusb, userspace) driver for the icE1usb hardware
* traditionally, it sits between icE1usb hardware and applications using E1 like osmo-bsc
* instead of a local application, it can now interface icE1usb to OCTOI
** `octoi-server` mode
** `octoi-client` mode

First 5 OCTOI users were connected to the hub that way

[graphviz]
----
graph {
  rankdir=LR;
  subgraph cluster_L {
     label="Site L";
     PBX_L [label="PBX"];
     icE1usb_L [label="icE1usb"];
     GW_L [label="GW L"];
     PBX_L -- icE1usb_L [label="E1"];
     icE1usb_L -- GW_L [label="USB"];
  }
  subgraph cluster_R {
     label="Site R";
     PBX_R [label="PBX"];
     icE1usb_R [label="icE1usb"];
     GW_R [label="GW R"];
     PBX_R -- icE1usb_R [label="E1"];
     icE1usb_R -- GW_R [label="USB"];
  }
  Internet;
  GW_L -- Internet [label="IP"];
  GW_R -- Internet [label="IP"];
}
----

== E1oIP: osmo-e1d + dahdi-trunkdev

* having one icE1usb per peer/user doesn't scale at the hub
* this triggered the development of `dahdi-trunkdev`
** virtual E1 _span_ for DAHDI without any real hardware
** simply provides stream of E1 frames on a character-device
** can be used to implement any virtual TDM interface/protocol
* osmo-e1d was extended to open dahdi-trunkdev instead of icE1usb

This has become the only method of connecting to the OCTOI hub in September 2022

[graphviz]
----
graph {
  rankdir=LR;
  subgraph cluster_L {
     label="Site L";
     PBX_L [label="PBX"];
     icE1usb_L [label="icE1usb"];
     GW_L [label="GW L"];
     PBX_L -- icE1usb_L [label="E1"];
     icE1usb_L -- GW_L [label="USB"];
  }
  subgraph cluster_R {
     label="Site R";
     PBX_R [label="PBX\n(Virtual DAHDI)"];
  }
  Internet;
  GW_L -- Internet [label="IP"];
  PBX_R -- Internet [label="IP"];
}
----

// ***********************************************************************
// Current OCTOI Network
// ***********************************************************************

== Current OCTOI network hub

[graphviz]
----
graph G {
  subgraph cluster_colo {
    label = "Co-Located Hub";
    divf [label="DIVF\nCentral Switch",shape="house"];
    icE1usb;
    pm3 [label="Livingston Portmaster 3"];
    as54 [label="Cisco AS5400"];
    icE1usb -- divf [label="E1 (timing master)"];
    pm3 -- divf [label="E1"];
    as54 -- divf [label="7xE1"];
  }
  divf -- laforge [label="TDMoIP"];
  divf -- manawyrm [label="TDMoIP"];
  divf -- gruetzkopf [label="TDMoIP"];
  divf -- roox [label="TDMoIP"];
  divf -- DrDeke [label="TDMoIP"];
  etc [label="other users..."];
  divf -- etc [label="TDMoIP"];
}
----

* DIVF: Debian 11 with `dahdi-trunkdev`, `osmo-e1d` and `yate`
* 2x TE820 8-port E1 cards, attaching to
** icE1usb (clock source)
** Livingston PortMaster 3 RAS
** Cisco AS5400 RAS

== Rack view

image::octoi_hub-colocated.jpg[width=900]

== 1U Enclosure with APU, icE1usb, OVP

image::octoi_hub-ice1usb.jpg[width=1200]

== GPS Antenna mounted on the roof

image::octoi_hub-gpsant.jpg[width=600]

this is an old antenna mount previously used for a satellite dish

== GPS Antenna mounted on the roof

image::octoi_hub-gps.jpg[width=1200]



== Current OCTOI participants

The hub currently has the following clients / participants:

* using icE1usb at hub side
** manawyrm
** gruetzkopf
** roox
** cquirin
** drdeke
** tmwawpl
** tom/sirtux
** tnt

== Current OCTOI services

* E1 / TDM as "transport" layer
* ISDN network as "routing" layer
** hub simulates the network / central office / switch side
** client devices implement the "user" side, just like when attaching to ISDN/PSTN
* Services on top of ISDN
** Audio Telephony
** Video Telephony (T-View 100 / H.320)
** ISDN Data Calls
** Analog Modem Calls
** Telefax

== OCTOI ISDN / Modem calls

* hub currently has a https://osmocom.org/projects/retronetworking/wiki/Livingston_Portmaster_3[Livingston Portmaster 3] RAS
** up to 30 ISDN data or analog modem calls (up to V.90)
* services can be created in RADIUS, identified by Called Party Number
** PPP dial-up locally terminated
** PPP dial-up with forwarding of data via L2TP to remote LAC
** telnet or TCP clear forwarding to remote BBSs
* Cisco AS5400 with much higher capacity is waiting to be provisioned

== Challenges: Clocking

* TDM networks need a stable bitclock at all parts of the network
* everyone must transmit and receive at the same rate of bits / frames
* we use GPS disciplined oscillators (GPS-DO) to ensure same clock everywhere
** this avoids overruns / underruns resulting in cycle slips that would create phase discontinuity for
any modem signals carried over the network

== Challenges: Packet Re-ordering

* It seems like particularly on DOCSIS there is quite a bit of UDP packet re-ordering
* Initially, osmo-e1d used a FIFO and dropped all out-of-order packets
* later, we introduced a RIFO (Random In, First Out) to support re-ordering without loss

== Challenges: Packet Loss

* There is quite a bit of packet loss on the public internet
* People probably don't generally notice much, as most services use TCP or retransmit UDP
* Surprisingly, I couldn't find any comprehensive studies / research papers on packet loss behaviour of
  consumer internet that are less than 10 years old?
* Right now we just accept it exists
* Some early thoughts and experiments on FEC (Forward Error Correction)

== Challenges: Latency

* intercontinental public IP can easily reach >= 200ms RTT
* ISDN timers in Q.921 and Q.931 are luckily working just fine over that kind of RTT
* some suspicion that the high latency might have negative impact on Fax (T.30 timers) and modems (buffer
  sizes). I'm personally not yet convinced it is really an issue.

== Current Services on top of OCTOI ISDN

* Participants can call each other for any ISDN call/bearer type
* Central hub provides
** Dial-Up (Modem up to V.90 + ISDN V.120/X.75) access
*** Tons of telnet-reachable BBSs all around the world
*** BTX (an instance of Neu-Ulm courtesy of https://btx-museum.de/)
**** unfortunately currently still crashes ISDN BTX Terminal T-DisplayTel
*** Fax gateway (WIP)
* Some other users provide
** Video on demand for ISDN video telephony
** Remote DOOM for ISDN video telephony
** Chiptune audio on demand via voice calls

https://osmocom.org/projects/octoi/wiki/Phonebook

== What do I need to connect to the network?

* Currently, we only offer PRI (E1) service, so you need
** an icE1usb plus some Linux USB host (Raspi, ...)
** some equipment that talks E1, like an _enterprise_ PBX
*** Auerswald COMmander2 Basic with S2M is a popular and widely available choice
*** this PBX then offers analog and BRI ports for your phones / modems / TAs

== What will I need to connect to the network?

* We're working on a small, low-power fully integrated device
* Ethernet/IP connection towards Internet / OCTOI hub
* 2x S0 (BRI) ports for your ISDN equipment, or analog phones via a/b TA
* First prototype looks promising. Main work pending is the uC software.

image::osmo-xhfc2su-breakout.jpg[width=1200]

// ***********************************************************************
// Future Plans
// ***********************************************************************

== Future Plans

* ISDN BRI (S/T) access to OCTOI network
** more end-user friendly; many people have S0 equipment and no E1/S2M PBX at home
* Frame Relay switch / hub
** second, additional service, completely independent of the current ISDN service
** initial testing has confirmed HDLC / FR over OCTOI works without problems
* X.25 network support
** X.25 access via ISDN D-Channel (X.31)
* exhibits at hacker and vintage computing events (like VCFB!)

== ISDN BRI (S/T) access to OCTOI

* BRI (ISDN basic rate aka "S/T" aka "S0") support in OCTOI protocol
* BRI hardware with 2x S/T interface and GPS-DO
** unlike icE1usb: Ethernet/IP support, not USB
** complete _appliance_ for OCTOI client mode: no need for computer
** Status: design of first break-out / evaluation board for ISDN BRI side complete
** Software not even started yet. Idea is to use Nuttx on Atmel SAMV71

== Longer-Term Future Plans

* improve FOSS soft-modem situation (linmodem, spandsp)
* investigate somewhat limited V.90 performance so far
* support for other Q.931 dialects than DSS1 (like 1TR6 or even NI1)


== Thanks

Thanks to

* Sylvain "tnt" Munaut for icE1usb and all his help
* All of the early OCTOI network participants manawyrm, gruetzkopf, roox, DrDeke, tmwawpl, cquirin
* noris.net for sponsoring the co-location of the OCTOI hub
* dexter, manawrym and TobleMiner for staffing the Retronetworking exhibit at VCFB

== Further Reading

* https://osmocom.org/projects/octoi/wiki[OCTOI Project Homepage/Wiki]
* https://osmocom.org/projects/octoi/wiki/Proposed_efficient_TDMoIP[OCTOI Protocol Description]
* https://osmocom.org/projects/retronetworking/wiki[Retronetworking Wiki]
* https://osmocom.org/projects/osmo-e1d/wiki/Wiki[osmo-e1d software]
* https://osmocom.org/projects/octoi/wiki/Proposed_efficient_TDMoIP[icE1usb hardware]


== Grafana / Stats

Let's have a look at the statistics...


== Modem call

Let's demo a modem call 

== EOF

End of File
