laforge-slides/2022/knf_kongress2022-octoi/octoi.adoc

598 lines
19 KiB
Plaintext

OCTOI - Osmocom Community TDM over IP
=====================================
:author: Harald Welte <laforge@osmocom.org>
:copyright: 2022 by Harald Welte (License: CC-BY-SA)
:backend: slidy
:max-width: 45em
== Overview
* WTF? Why?
* History: Telephony / TDM networks
* existing TDMoIP technology
* OCTOI Protocol
* OCTOI Software
* Current OCTOI Network
* WIP / Future Plans
== Introduction
Some people collect + operate old computers (_retrocomputing_)
...
But what's a 1980s or 1990s computer without it's contemporary networking technology?
== Zyxel V.32bis Modem
image::zyxel.jpg[width=1000,align="center"]
== ELSA MicroLink ISDN TA
image::ELSA_MicroLink_ISDN_2ab_front.jpg[width=800,align="center"]
== TELES S0/16.3 passiver ISDN-Adapter
image::teles_s0_16_3.jpg[width=1000,align="center"]
== T-DisplayTel ISDN-BTX-Telefon
image::displaytel.jpg[width=1000,align="center"]
== T-View 100 ISDN-Videotelefon
image::tview100.jpg[width=800,align="center"]
== WTF is this all about?
* enable people to run legacy WAN equipment
** Modems
** ISDN-Adapters
** Phones, PBXs
** Telefax
** Frame Relay / HDLC / X.25 / routers
* in a distributed fashion, over the internet
* allow to run retronetworking tech from mid-1980s to about 2010 at a time where the related transport
services are no longer available from public telephone operators / carriers
== Migration paths for ISDN equipment
* SIP ATA
** provides analog phone jacks on phone side, speaks SIP to operator
** works fine for voice telephony
** can occasionally work for Fax
** doesn't work well for Modems
== Migration paths for ISDN equipment
* Fritzbox with ISDN BRI (S0) port
** terminates Q.921/Q.931 ISDN protocol stack internally
** speaks SIP to operator
** works fine for voice telephony
** can work for Fax
** doesn't work well for Modems
** doesn't work at all for X.25/X.31
== Migration paths for ISDN equipment
* _enterprise grade_ SIP/ISDN gateways
** available for BRI (S0) and PRI (E1)
** typically to help companies avoid replacing their ISDN PBX
** terminates Q.921/Q.931 ISDN protocol stack internally
** speaks SIP to operator
** works fine for voice telephony
** can work for Fax
** doesn't work well for Modems
** doesn't work at all for X.25/X.31
== VoIP: Why does only voice work well
* well, it's called _voice_ over IP, not _modem_ over ip ;)
* one problem is audio codecs
** most VoIP uses lossy compression (think of MP3)
** lossy codecs work for voice, but nothing else
** can only be solved if end-to-end VoIP path can be forced to G711 or CLEARMODE
** depending on situation (involved SIP operators, ...) this may or may not be possible
* other problem is clocking
** impossible to solve / work around
== Clocking in synchronous networks vs VoIP
* classic analog networks had analog connection end-to-end. There's constant phase / delay
* ISDN networks used PDH and later SDH for transmission
** tight synchronization between the clocks of all central offices
** clocks means both the sample clock of ADCs/DACs as well as bit clock of ISDN PRI/BRI
* In VoIP networks everyone uses their own clock source
** typically a normal XTAL in your ATA / FritzBox with as much as 20 ppm tolerance
* if clocks between two partners drift, there will be underruns or overruns
** if this happens, you drop samples or insert samples
** this creates phase discontinuity
** modem standards were *not* designed for this
== Modem/ISDN in the VoIP world
* VoIP really is designed for voice only
** codec issue can be worked around if you can control codecs end-to-end
** clock issue requires hardware modding your ATA/FritzBox/Gateway
* you still have no transparency
** what about ISDN signaling subtleties like HLC/LLC in SETUP?
** what about X.25 over ISDN D-Channel (X.31)
What you really want a *transparent* way of transporting ISDN/PSTN over IP, not some gateway that's way too high up the protocol stack.
// ***********************************************************************
// History: TDM networks
// ***********************************************************************
== History: Analog Telephony Networks
* analog telephony exited (in Germany) since 1877
* analog electrical circuit from end to end
* switching initially manually
* automatic (electromechanical) switching introduced 1908-1966
== History: Digitization inside the network
* analog pairs of copper lines
* carrier frequency systems (TF-Systeme) over coaxial wire
* first digital PCM (pulse code modulation) links between exchanges
* digital switching with analog switch matrix (EWS)
* digital switching with digital switch matrix (EWSD, S12)
* 1979: decision to migrate entire German PSTN to digital switching
** estimated completion date in 2018
** actual completion of all 6200 exchanges in 1997
== History: TDM networks
* at some point (1970s?), digital PCM technology was introduced between exchanges
** initially PDH, later SDH (which can transport PDH)
** see https://media.ccc.de/v/osmodevcall-20211112-laforge-tdm[OsmoDevCall on TDM/PDH/SDH]
* external interfaces, particularly to subscriber, still analog until mid-1980s
* ISDN changed that: Introduced digital subscriber lines
** Basic Rate ISDN (BRI): 2x B, 1x D-Channel
*** German: Basisanschluss (Anlagenanschluss, Mehrgeraeteanschluss)
** Primary Rate ISDN (PRI): 30x B, 1x D-Channel
*** German: Primaermultiplexer / S2M
== Present: TDM networks mostly dead
* Telcos have meanwhile mostly migrated to All-IP telephony
* PDH/SDH networks mostly shut down, occasionally still in operation for legacy customers or some internal
legacy services. Not actively sold anymore.
* You cannot get a real PRI line anymore to connect your ISDN PBX or your Cisco router
* Idea: Create community network of people who want to play with ISDN / TDM stuff
* Naive approach: Use existing, off-the-shelf TDMoIP devices
== Wanted: TDMoIP network
[graphviz]
----
graph G {
hub [shape=box, label="TDMoIP hub\ncross-connect\nin public internet"];
subgraph cluster_1 {
label="Hobbyist A";
ad1 [label="Access Device\n"];
pbx1 [label="PBX"];
phone1a [label="Phone"];
modem1b [label="Modem"];
ta1c [label="ISDN TA"];
pbx1 -- ad1 [label="E1"];
phone1a -- pbx1 [label="POTS"];
modem1b -- pbx1 [label="POTS"];
ta1c -- pbx1 [label="ISDN-BRI"];
}
subgraph cluster_2 {
label="Hobbyist B";
ad2 [label="Access Device\n"];
pbx2 [label="PBX"];
phone2a [label="Phone"];
modem2b [label="Modem"];
ta2c [label="ISDN TA"];
pbx2 -- ad2 [label="E1"];
phone2a -- pbx2 [label="POTS"];
modem2b -- pbx2 [label="POTS"];
ta2c -- pbx2 [label="ISDN-BRI"];
}
subgraph cluster_3 {
label="Hobbyist C";
ad3 [label="Access Device\n"];
pbx3 [label="PBX"];
phone3a [label="Phone"];
modem3b [label="Modem"];
ta3c [label="ISDN TA"];
pbx3 -- ad3 [label="E1"];
phone3a -- pbx3 [label="POTS"];
modem3b -- pbx3 [label="POTS"];
ta3c -- pbx3 [label="ISDN-BRI"];
}
ad1 -- hub [label="TDMoIP\nInternet"];
ad2 -- hub [label="TDMoIP\nInternet"];
ad3 -- hub [label="TDMoIP\nInternet"];
}
----
// ***********************************************************************
// Existing TDMoIP technology
// ***********************************************************************
== Existing TDMoIP: SAToP / CESoPSN
So we just get some of these and all is good?
image::RAD_IPmux-1_front.jpg[width=1000,align="center"]
== Existing TDMoIP: SAToP / CESoPSN
Examples for _transparent_ TDMoIP technologies
* SAToP (Structure Agnostic TDM over IP)
** RFC 4553
* CESoPSN (Circuit Emulation Service over Packet Switched Network)
** RFC 5086
* transport of raw E1 frames over MPLS or UDP
* designed for use in LAN or carrier backbone networks
* waste a lot of bandwidth, even if TDM circuits are completely idle
** > 2Mbps plus header/packet overhead, bi-directional
* typically no support for dynamic IP addresses
== Existing TDMoIP: SIGTRAN IUA, EL2TPD
Examples for _non-transparent_ TDMoIP technologies
* SIGTRAN IUA (ISDN User Adaptation)
** RFC 3057
** works only for ISDN BRI/PRI with Q.921 as Layer 2 on signaling channel
** uses SCTP as transport (might not pass all middleboxes in public internet)
** no specification how the B-channels are handled, pure signaling solution
* Ericsson L2TP / PacketAbis over IP
** proprietary, but FOSS implementation in @osmo-el2tpd@
** makes specific assumption of use of E1 as Ericsson Abis (GSM BTS back-haul)
// ***********************************************************************
// OCTOI TDMoIP protocol
// ***********************************************************************
== The case for a new TDMoIP protocol
There's room for a new protocol based on the following goals:
* *transparent*. Should work for ISDN and other use cases (Frame Relay, Abis, SS7, ATM, ...)
* *efficient*. Should not waste a lot of bandwidth on an otherwise idle/unused link
* *dynamic IP*. End-user internet access normally has dynamic IP addresses
* *nat friendly*. Should work through any number of NATs and CG-NAT without special ALG/helper
* *IPv6 support*. Should support IPv6 just like IPv4
* *authentication*. Should have built-in mutual authentication
== OCTOI Protocol: Bandwidth
* Timeslot Suppression
** Transmitter has history of 1 TDM frame
** Before Tx, to-be-transmitted frame is compared with last frame
** only those timeslots with value != that of last frame are transmitted
* Batching
** Batch 32 (up to 40) E1 frames per UDP packet
*** 8000 frames/s / 32 => 250 packets/s
Result: About 70 kbit/s internet bandwidth use (including UDP + IPv4 overhead) for an idle E1 link
== OCTOI Protocol: dynamic IP / NAT-friendly
* Classic client/server approach
* Server currently requires fixed IP (no STUN/ICE/...)
* Clients can have dynamic IPs
* All messages (control plane + TDM user plan) in one UDP flow
// ***********************************************************************
// OCTOI Software
// ***********************************************************************
== First supported OCTOI hardware: icE1usb
* osmocom icE1usb is an open source hardware/gateware/firmware/driver E1 interface
* originally developed for talking to GSM base stations
* contains a GPS disciplined oscillator for high-precision bit clock timing
* further reading
** https://osmocom.org/projects/e1-t1-adapter/wiki/IcE1usb
** https://media.ccc.de/v/osmodevcon2019-97-software-defined-e1
image::icE1usb-usb_side.jpg[width=550,align="center"]
== E1oIP: osmo-e1d + icE1usb
* osmo-e1d was the first (libusb, userspace) driver for the icE1usb hardware
* traditionally, it sits between icE1usb hardware and applications using E1 like osmo-bsc
* instead of a local application, it can now interface icE1usb to OCTOI
** `octoi-server` mode
** `octoi-client` mode
First 5 OCTOI users were connected to the hub that way
[graphviz]
----
graph {
rankdir=LR;
subgraph cluster_L {
label="Site L";
PBX_L [label="PBX"];
icE1usb_L [label="icE1usb"];
GW_L [label="GW L"];
PBX_L -- icE1usb_L [label="E1"];
icE1usb_L -- GW_L [label="USB"];
}
subgraph cluster_R {
label="Site R";
PBX_R [label="PBX"];
icE1usb_R [label="icE1usb"];
GW_R [label="GW R"];
PBX_R -- icE1usb_R [label="E1"];
icE1usb_R -- GW_R [label="USB"];
}
Internet;
GW_L -- Internet [label="IP"];
GW_R -- Internet [label="IP"];
}
----
== E1oIP: osmo-e1d + dahdi-trunkdev
* having one icE1usb per peer/user doesn't scale at the hub
* this triggered the development of `dahdi-trunkdev`
** virtual E1 _span_ for DAHDI without any real hardware
** simply provides stream of E1 frames on a character-device
** can be used to implement any virtual TDM interface/protocol
* osmo-e1d was extended to open dahdi-trunkdev instead of icE1usb
This has become the only method of connecting to the OCTOI hub in September 2022
[graphviz]
----
graph {
rankdir=LR;
subgraph cluster_L {
label="Site L";
PBX_L [label="PBX"];
icE1usb_L [label="icE1usb"];
GW_L [label="GW L"];
PBX_L -- icE1usb_L [label="E1"];
icE1usb_L -- GW_L [label="USB"];
}
subgraph cluster_R {
label="Site R";
PBX_R [label="PBX\n(Virtual DAHDI)"];
}
Internet;
GW_L -- Internet [label="IP"];
PBX_R -- Internet [label="IP"];
}
----
// ***********************************************************************
// Current OCTOI Network
// ***********************************************************************
== Current OCTOI network hub
[graphviz]
----
graph G {
subgraph cluster_colo {
label = "Co-Located Hub";
divf [label="DIVF\nCentral Switch",shape="house"];
icE1usb;
pm3 [label="Livingston Portmaster 3"];
as54 [label="Cisco AS5400"];
icE1usb -- divf [label="E1 (timing master)"];
pm3 -- divf [label="E1"];
as54 -- divf [label="7xE1"];
}
divf -- laforge [label="TDMoIP"];
divf -- manawyrm [label="TDMoIP"];
divf -- gruetzkopf [label="TDMoIP"];
divf -- roox [label="TDMoIP"];
divf -- DrDeke [label="TDMoIP"];
etc [label="other users..."];
divf -- etc [label="TDMoIP"];
}
----
* DIVF: Debian 11 with `dahdi-trunkdev`, `osmo-e1d` and `yate`
* 2x TE820 8-port E1 cards, attaching to
** icE1usb (clock source)
** Livingston PortMaster 3 RAS
** Cisco AS5400 RAS
== Rack view
image::octoi_hub-colocated.jpg[width=900]
== 1U Enclosure with APU, icE1usb, OVP
image::octoi_hub-ice1usb.jpg[width=1100]
== GPS Antenna mounted on the roof
image::octoi_hub-gpsant.jpg[width=600]
this is an old antenna mount previously used for a satellite dish
== GPS Antenna mounted on the roof
image::octoi_hub-gps.jpg[width=1200]
== Current OCTOI participants
The hub currently has the following clients / participants:
* using icE1usb at hub side
** laforge
** manawyrm
** gruetzkopf
** roox
** cquirin
** drdeke
** tmwawpl
** tom/sirtux
** tnt
** marrold
== Current OCTOI services
* E1 / TDM as "transport" layer
* ISDN network as "routing" layer
** hub simulates the network / central office / switch side
** client devices implement the "user" side, just like when attaching to ISDN/PSTN
* Services on top of ISDN
** Audio Telephony
** Video Telephony (T-View 100 / H.320)
** ISDN Data Calls
** Analog Modem Calls
** Telefax
== OCTOI ISDN / Modem calls
* hub currently has a https://osmocom.org/projects/retronetworking/wiki/Livingston_Portmaster_3[Livingston Portmaster 3] RAS
** up to 30 ISDN data or analog modem calls (up to V.90)
* services can be created in RADIUS, identified by Called Party Number
** PPP dial-up locally terminated
** PPP dial-up with forwarding of data via L2TP to remote LAC
** telnet or TCP clear forwarding to remote BBSs
* Cisco AS5400 with much higher capacity is waiting to be provisioned
== Challenges: Clocking
* TDM networks need a stable bitclock at all parts of the network
* everyone must transmit and receive at the same rate of bits / frames
* we use GPS disciplined oscillators (GPS-DO) to ensure same clock everywhere
** this avoids overruns / underruns resulting in cycle slips that would create phase discontinuity for
any modem signals carried over the network
== Challenges: Packet Re-ordering
* It seems like particularly on DOCSIS there is quite a bit of UDP packet re-ordering
* Initially, osmo-e1d used a FIFO and dropped all out-of-order packets
* later, we introduced a RIFO (Random In, First Out) to support re-ordering without loss
== Challenges: Packet Loss
* There is quite a bit of packet loss on the public internet
* People probably don't generally notice much, as most services use TCP or retransmit UDP
* Surprisingly, I couldn't find any comprehensive studies / research papers on packet loss behaviour of
consumer internet that are less than 10 years old?
* Right now we just accept it exists
* Some early thoughts and experiments on FEC (Forward Error Correction)
== Challenges: Latency
* intercontinental public IP can easily reach >= 200ms RTT
* ISDN timers in Q.921 and Q.931 are luckily working just fine over that kind of RTT
* some suspicion that the high latency might have negative impact on Fax (T.30 timers) and modems (buffer
sizes). I'm personally not yet convinced it is really an issue.
== Current Services on top of OCTOI ISDN
* Participants can call each other for any ISDN call/bearer type
* Central hub provides
** Dial-Up (Modem up to V.90 + ISDN V.120/X.75) access
*** Tons of telnet-reachable BBSs all around the world
*** BTX (an instance of Neu-Ulm courtesy of https://btx-museum.de/)
**** unfortunately currently still crashes ISDN BTX Terminal T-DisplayTel
*** Fax gateway (WIP)
* Some other users provide
** Video on demand for ISDN video telephony
** Remote DOOM for ISDN video telephony
** Chiptune audio on demand via voice calls
https://osmocom.org/projects/octoi/wiki/Phonebook
== What do I need to connect to the network?
* Currently, we only offer PRI (E1) service, so you need
** an icE1usb plus some Linux USB host (Raspi, beaglebone, ...)
** some equipment that talks E1, like an _enterprise_ PBX
*** Auerswald COMmander2 Basic with S2M is a popular and widely available choice
*** this PBX then offers analog and BRI ports for your phones / modems / TAs
== The setup used to connect KNF-Kongress
* 19" Version of Auerswald COMmander2 Basic PBX
* sufficient space internally to put Beaglebone + icE1usb inside
image::octoi-event-pbx.jpg[width=1100]
== What will I need to connect to the network?
* We're working on a small, low-power fully integrated device
* Ethernet/IP connection towards Internet / OCTOI hub
* 2x S0 (BRI) ports for your ISDN equipment, or analog phones via a/b TA
* First prototype looks promising. Main work pending is the uC software.
image::osmo-xhfc2su-breakout.jpg[width=1100]
// ***********************************************************************
// Future Plans
// ***********************************************************************
== Future Plans
* ISDN BRI (S/T) access to OCTOI network
** more end-user friendly; many people have S0 equipment and no E1/S2M PBX at home
* Frame Relay switch / hub
** second, additional service, completely independent of the current ISDN service
** initial testing has confirmed HDLC / FR over OCTOI works without problems
* X.25 network support
** X.25 access via ISDN D-Channel (X.31)
* exhibits at hacker and vintage computing events (like VCFB!)
== ISDN BRI (S/T) access to OCTOI
* BRI (ISDN basic rate aka "S/T" aka "S0") support in OCTOI protocol
* BRI hardware with 2x S/T interface and GPS-DO
** unlike icE1usb: Ethernet/IP support, not USB
** complete _appliance_ for OCTOI client mode: no need for computer
** Status: design of first break-out / evaluation board for ISDN BRI side complete
** Software not even started yet. Idea is to use Nuttx on Atmel SAMV71
== Longer-Term Future Plans
* improve FOSS soft-modem situation (linmodem, spandsp)
* investigate somewhat limited V.90 performance so far
* support for other Q.931 dialects than DSS1 (like 1TR6 or even NI1)
* dive into H.320 / H.221 to have software endpoint for ISDN video phones
== Thanks
Thanks to
* Sylvain "tnt" Munaut for icE1usb and all his help
* All of the early OCTOI network participants manawyrm, gruetzkopf, roox, DrDeke, tmwawpl, cquirin
* noris.net for sponsoring the co-location of the OCTOI hub
== Further Reading
* https://osmocom.org/projects/octoi/wiki[OCTOI Project Homepage/Wiki]
* https://osmocom.org/projects/octoi/wiki/Proposed_efficient_TDMoIP[OCTOI Protocol Description]
* https://osmocom.org/projects/retronetworking/wiki[Retronetworking Wiki]
* https://osmocom.org/projects/osmo-e1d/wiki/Wiki[osmo-e1d software]
* https://osmocom.org/projects/octoi/wiki/Proposed_efficient_TDMoIP[icE1usb hardware]
== Grafana / Stats
Let's have a look at the statistics...
== Modem call
Let's demo a modem call
== EOF
End of File