doc: Introduce documentation for osmo-trx-ipc and its IPC interface
Related: SYS#6861 Change-Id: Id6863731f9398720030b16efaaf559e05f2444ed
This commit is contained in:
parent
8fd51cb7c9
commit
c5f623f966
|
|
@ -17,6 +17,7 @@ edge[dir=back, arrowtail=empty]
|
|||
7[label = "{UHDDevice|...}"]
|
||||
8[label = "{LMSDevice|...}"]
|
||||
9[label = "{USRPDevice|...}"]
|
||||
10[label = "{IPCDevice|...}"]
|
||||
|
||||
2->3[arrowtail=odiamond]
|
||||
3->4[constraint=false]
|
||||
|
|
@ -25,6 +26,7 @@ edge[dir=back, arrowtail=empty]
|
|||
6->7
|
||||
6->8
|
||||
6->9
|
||||
6->10
|
||||
}
|
||||
----
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,301 @@
|
|||
[[ipc_if]]
|
||||
== osmo-trx-ipc IPC Interface
|
||||
|
||||
This interface is the one used by _osmo_trx_ipc_ backend to communicate to a
|
||||
third party process in charge of driving the lowest layer device-specific bits
|
||||
(from now on the Driver).
|
||||
|
||||
It consists of a set of Unix Domain (UD) sockets for the control plane, plus a
|
||||
shared memory region for the data plane.
|
||||
|
||||
Related code can be found in the
|
||||
https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc[Transceiver52M/device/ipc/]
|
||||
directory in _osmo-trx.git_.
|
||||
|
||||
If you are a potential driver implementator, the
|
||||
various primitives and data structures are publicly available in header file
|
||||
https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h].
|
||||
|
||||
=== Control plane
|
||||
|
||||
Control plane protocol is transmitted over Unix Domain (UD) sockets using
|
||||
message based primitives. Each primitive has a type identified by an integer,
|
||||
and each type of primitive has a number of extra attributes attached to it. The
|
||||
IPC interface consists of 2 types of UD sockets:
|
||||
|
||||
* _Master_ UD socket: One per osmo-trx-ipc process.
|
||||
|
||||
* _Channel_ UD socket: One for each channel managed by osmo-trx-ipc process.
|
||||
|
||||
The _Driver_ is in all cases expected to take the server role when creating UD
|
||||
sockets, while _osmo-trx-ipc_ takes the client role and connects to sockets
|
||||
provided by the driver.
|
||||
|
||||
=== Master UD socket
|
||||
|
||||
During startup, _osmo-trx-ipc_ will try connecting to the _Driver_ Master UD
|
||||
socket located in the path provided by its own (VTY) configuration. As a result,
|
||||
it means the _Driver_ process must be running and listening on the Master UD
|
||||
socket before _osmo-trx-ipc_ is started, otherwise _osmo-trx-ipc_ will fail and
|
||||
exit.
|
||||
|
||||
Once connected, _osmo-trx-ipc_ will submit a `GREETING_REQ` message primitive
|
||||
announcing the maximum supported protocol version (first version ever is `1`,
|
||||
increasing over time).
|
||||
|
||||
The _Driver_ shall then answer in `GREETING_CNF` message primitive with its own
|
||||
maximum supported version (`<=` version received), providing 0 if none is
|
||||
supported.
|
||||
|
||||
If _osmo-trx-ipc_ receives back the requested version, then both sides agreed
|
||||
on the protocol version to use.
|
||||
If _osmo-trx-ipc_ receives back a lower version, it shall decide to continue
|
||||
with version negotiation using a lower version, until a supported version or 0
|
||||
is received. If finally 0 is received, _osmo-trx-ipc_ will disconnect and exit
|
||||
with failure.
|
||||
|
||||
Once the version is negotiated (`v1` as of current date), _osmo-trx-ipc_ will
|
||||
ask for device information and available characeristics to the _Driver_ using
|
||||
the `INFO_REQ` message primitive.
|
||||
|
||||
The _Driver_ shall then answer with a `INFO_CNF` message
|
||||
containing information, such as:
|
||||
|
||||
* String containing device description
|
||||
|
||||
* Available reference clocks,
|
||||
|
||||
* {rx,tx} I/Q scaling factors
|
||||
|
||||
* Maximum number of channels supported
|
||||
|
||||
* for each channel:
|
||||
|
||||
** List of available {rx,tx} paths/antennas.
|
||||
|
||||
** {min,max}{rx,tx} gains
|
||||
|
||||
** Nominal transmit power
|
||||
|
||||
All the information received from the _Driver_ during `INFO_CNF` will be used by
|
||||
_osmo-trx-ipc_ to decide whether it can fullfil the requested configuration from
|
||||
the user, and proceed to open the device, or exit with a failure (for instance
|
||||
number of channels, referece clock or tx/rx antenna selected by the user cannot
|
||||
be fullfilled).
|
||||
|
||||
_osmo-trx-ipc_ will then proceed to open the device and do an initial
|
||||
configuration using an `OPEN_REQ` message, where it will provide the _Driver_
|
||||
with the desired selected configuration (such as number of channels, rx/tx
|
||||
paths, clock reference, bandwidth filters, etc.).
|
||||
|
||||
The _Driver_ shall then configure the device and send back a `OPEN_CNF` with:
|
||||
|
||||
* `return_code` integer attribute set to `0` on success or `!0` on error.
|
||||
|
||||
* Name of the Posix Shared Memory region where data plane is going to be
|
||||
transmitted.
|
||||
|
||||
* One path for each channel, containing the just-created UD socket to manage
|
||||
that channel (for instance by taking Master UD socket path and appending
|
||||
`_$chan_idx`).
|
||||
|
||||
* Path Delay: this is the loopback path delay in samples (= used as a timestamp
|
||||
offset internally by _osmo-trx-ipc_), this value contains the analog delay as
|
||||
well as the delay introduced by the digital filters in the fpga in the sdr
|
||||
devices, and is therefore device type and bandwidth/sample rate dependant. This
|
||||
can not be omitted, wrong values will lead to a _osmo-trx-ipc_ that just doesn't
|
||||
detect any bursts.
|
||||
|
||||
Finally, _osmo-trx-ipc_ will connect to each channel's UD socket (see next
|
||||
section).
|
||||
|
||||
Upon _osmo-trx-ipc_ closing the UD master socket connection, the _Driver_ shall
|
||||
go into _closed_ state: stop all processing and instruct the device to power
|
||||
off.
|
||||
|
||||
TIP: See
|
||||
https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]
|
||||
for the detailed definition of all the related message primitives and data
|
||||
types for this socket.
|
||||
|
||||
=== Channel UD Socket
|
||||
|
||||
This socket can be used by _osmo-trx-ipc_ to start/stop data plane processing or
|
||||
change channel's parameters such as Rx/Tx Frequency, Rx/Tx gains, etc.
|
||||
|
||||
A channel can be either in _started_ or _stopped_ state. When a channel is
|
||||
created (during `OPEN_REQ` in the Master UD Socket), it's by default in
|
||||
_stopped_ state. `START_REQ` and `STOP_REQ` messages control this state, and
|
||||
eventual failures can be reported through `START_CNF` and `STOP_CNF` by the
|
||||
_Driver_.
|
||||
|
||||
The message `START_REQ` instructs the _Driver_ to start processing data in the
|
||||
data plane. Similary, `STOP_REQ` instructs the _Driver_ to stop processing data
|
||||
in the data plane.
|
||||
|
||||
Some parameters are usually changed only when the channel is in stopped mode,
|
||||
for instance Rx/Tx Frequency.
|
||||
|
||||
TIP: See
|
||||
https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]
|
||||
for the detailed definition of all the related message primitives and data
|
||||
types for this socket.
|
||||
|
||||
=== Data Plane
|
||||
|
||||
Data plane protocol is implemented by means of a ring buffer structure on top of
|
||||
Posix Shared Memory (see `man 7 shm_overview`) between _osmo-trx-ipc_ process
|
||||
and the _Driver_.
|
||||
|
||||
The Posix Shared Memory region is created and its memory structure prepared by
|
||||
the _Driver_ and its name shared with _osmo-trx-ipc_ during _OPEN_CNF_ message
|
||||
in the Master UD Socket from the Control Plane. Resource allocation for the
|
||||
shared memory area and cleanup is up to the ipc server, as is mutex
|
||||
initialization for the buffers.
|
||||
|
||||
==== Posix Shared Memory structure
|
||||
|
||||
[[fig-shm-structure]]
|
||||
.General overview of Posix Shared Memory structure
|
||||
[graphviz]
|
||||
----
|
||||
digraph hierarchy {
|
||||
node[shape=record,style=filled,fillcolor=gray95]
|
||||
edge[dir=back, arrowtail=empty]
|
||||
|
||||
SHM[label = "{Posix Shared Memory region|+ num_chans\l+ Channels[]\l}"]
|
||||
CHAN0[label = "{Channel 0|...}"]
|
||||
CHAN1[label = "{Channel 1|...}"]
|
||||
CHANN[label = "{Channel ...|}"]
|
||||
STREAM0_UL[label = "{UL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"]
|
||||
STREAM0_DL[label = "{DL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"]
|
||||
STREAM1_UL[label = "{UL Stream|...}"]
|
||||
STREAM1_DL[label = "{DL Stream|...}"]
|
||||
STREAMN_UL[label = "{UL Stream|...}"]
|
||||
STREAMN_DL[label = "{DL Stream|...}"]
|
||||
BUF_0DL0[label = "{DL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"]
|
||||
BUF_0DLN[label = "{DL Sample Buffer ....|...}"]
|
||||
BUF_0UL0[label = "{UL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"]
|
||||
BUF_0ULN[label = "{UL Sample Buffer ...|...}"]
|
||||
|
||||
SHM->CHAN0
|
||||
SHM->CHAN1
|
||||
SHM->CHANN
|
||||
|
||||
CHAN0->STREAM0_DL
|
||||
CHAN0->STREAM0_UL
|
||||
STREAM0_DL->BUF_0DL0
|
||||
STREAM0_DL->BUF_0DLN
|
||||
STREAM0_UL->BUF_0UL0
|
||||
STREAM0_UL->BUF_0ULN
|
||||
|
||||
CHAN1->STREAM1_UL
|
||||
CHAN1->STREAM1_DL
|
||||
|
||||
CHANN->STREAMN_UL
|
||||
CHANN->STREAMN_DL
|
||||
}
|
||||
----
|
||||
|
||||
The Posix Shared Memory region contains an array of _Channels_.
|
||||
|
||||
Each _Channel_ contains 2 Streams:
|
||||
|
||||
* Downlink _Stream_
|
||||
|
||||
* Uplink _Stream_
|
||||
|
||||
Each _Stream_ handles a ring buffer, which is implemented as:
|
||||
|
||||
* An array of pointers to _Sample Buffer_ structures.
|
||||
|
||||
* Variables containing the number of buffers in the array, as well as the
|
||||
maximum size in samples for each Sample Buffer.
|
||||
|
||||
* Variables containing `next_read` and `next_write` _Sample Buffer_ (its index
|
||||
in the array of pointers).
|
||||
|
||||
* Unnamed Posix semaphores to do the required locking while using the ring
|
||||
buffer.
|
||||
|
||||
Each _Sample Buffer_ contains:
|
||||
|
||||
* A `timestamp` variable, containing the position in the stream of the first
|
||||
sample in the buffer
|
||||
|
||||
* A `data_len` variable, containing the amount of samples available to process
|
||||
in the buffer
|
||||
|
||||
* An array of samples of size specified by the stream struct it is part of.
|
||||
|
||||
==== Posix Shared Memory format
|
||||
|
||||
The Posix Shared memory region shall be formatted applying the following
|
||||
considerations:
|
||||
|
||||
* All pointers in the memory region are encoded as offsets from the start
|
||||
address of the region itself, to allow different processes with different
|
||||
address spaces to decode them.
|
||||
|
||||
* All structs must be force-aligned to 8 bytes
|
||||
|
||||
* Number of buffers must be power of 2 (2,4,8,16,...) - 4 appears to be plenty
|
||||
|
||||
* IQ samples format: One (complex) sample consists of 16bit i + 16bit q, so the
|
||||
buffer size is number of IQ pairs.
|
||||
|
||||
* A reasonable per-buffer size (in samples) is 2500, since this happens to be
|
||||
the ususal TX (downlink) buffer size used by _osmo-trx-ipc_ with the b210 (rx
|
||||
over-the-wire packet size for the b210 is 2040 samples, so the larger value of
|
||||
both is convenient).
|
||||
|
||||
TIP: See
|
||||
https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]
|
||||
for the detailed definition of all the objects being part of the Posix Shared
|
||||
memory region structure
|
||||
|
||||
==== Posix Shared Memory procedures
|
||||
|
||||
The queue in the shared memory area is not supposed to be used for actual
|
||||
buffering of data, only for exchange, so the general expectation is that it is
|
||||
mostly empty. The only exception to that might be minor processing delays, and
|
||||
during startup.
|
||||
|
||||
Care must be taken to ensure that only timed waits for the mutex protecting it
|
||||
and the condition variables are used, in order to ensure that no deadlock occurs
|
||||
should the other side die/quit.
|
||||
|
||||
Thread cancellation should be disabled during reads/writes from/to the queue. In
|
||||
general a timeout can be considered a non recoverable error during regular
|
||||
processing after startup, at least with the current timeout value of one second.
|
||||
|
||||
Should over- or underflows occur a corresponding message should be sent towards
|
||||
_osmo-trx-ipc_.
|
||||
|
||||
Upon **read** of `N` samples, the reader does something like:
|
||||
|
||||
. Acquire the semaphore in the channel's stream object.
|
||||
|
||||
. Read `stream->next_read`, if `next_read==next_write`, become blocked in
|
||||
another sempahore (unlocking the previous one) until writer signals us, then
|
||||
`buff = stream->buffers[next_read]`
|
||||
|
||||
. Read `buff->data_len` samples, reset the buffer data (`data_len=0`),
|
||||
increment `next_read` and if read samples is `<N`, continue with next buffer
|
||||
until `next_read==next_write`, then block again or if timeout elapsed, then we
|
||||
reach conditon buffer underflow and `return len < N`.
|
||||
|
||||
. Release the semaphore
|
||||
|
||||
Upon **write** of `N` samples, the writer does something like:
|
||||
|
||||
. Acquire the semapore in the channel's stream object.
|
||||
|
||||
. Write samples to `buff = stream->buffers[next_write]`. If `data_len!=0`,
|
||||
signal `buffer_overflow` (increase field in stream object) and probably
|
||||
increase next_read`.
|
||||
|
||||
. Increase `next_write`.
|
||||
|
||||
. If `next_write` was `== next_read`, signal the reader through the other
|
||||
semaphore that it can continue reading.
|
||||
|
|
@ -71,3 +71,103 @@ with a memory buffer. In this mode, data written to the USRP is actually stored
|
|||
in a buffer, and read commands to the USRP simply pull data from this buffer.
|
||||
This was very useful in early testing, and still may be useful in testing basic
|
||||
Transceiver and radioInterface functionality.
|
||||
|
||||
|
||||
[[backend_ipc]]
|
||||
=== `osmo-trx-ipc` Inter Process Communication backend
|
||||
|
||||
This OsmoTRX model provides its own Inter Process Communication (IPC) interface
|
||||
to drive the radio device driver (from now on the Driver), allowing for third
|
||||
party processes to implement the lowest layer device-specific bits without being
|
||||
affected by copyleft licenses of OsmoTRX.
|
||||
|
||||
For more information on such interface, see section <<ipc_if>>.
|
||||
|
||||
[[fig-backend-ipc]]
|
||||
.Architecture with _osmo-trx-ipc_ and its IPC _Driver_
|
||||
[graphviz]
|
||||
----
|
||||
digraph G {
|
||||
rankdir=LR;
|
||||
MS0 [label="MS"];
|
||||
MS1 [label="MS"];
|
||||
OsmoTRX [label="osmo-trx-ipc", color=red];
|
||||
BTS;
|
||||
|
||||
subgraph cluster_ipc_driver {
|
||||
label = "IPC Driver";
|
||||
color=red;
|
||||
RE [label = "Radio Equipment"];
|
||||
REC [label="Radio Equipment Controller"];
|
||||
RE->REC;
|
||||
}
|
||||
|
||||
REC->OsmoTRX [label="IPC Interface", color=red];
|
||||
|
||||
MS0->RE [label="Um"];
|
||||
MS1->RE [label="Um"];
|
||||
OsmoTRX->BTS [label="bursts over UDP"];
|
||||
|
||||
}
|
||||
----
|
||||
|
||||
A sample config file for this OsmoTRX model can be found in _osmo-trx.git_ https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/doc/examples/osmo-trx-ipc/osmo-trx-ipc.cfg[doc/examples/osmo-trx-ipc/osmo-trx-ipc.cfg]
|
||||
|
||||
In the config file, the following VTY command can be used to set up the IPC UD Master Socket _osmo-trx-ipc_ will connect to at startup:
|
||||
|
||||
.Example: _osmo-trx-ipc_ will connect to UD Master Socket /tmp/ipc_sock0 upon startup
|
||||
----
|
||||
dev-args ipc_msock=/tmp/ipc_sock0
|
||||
----
|
||||
|
||||
==== ipc-device-test
|
||||
|
||||
When built with `--with-ipc --with-uhd` configure options, _osmo-trx.git_ will
|
||||
build the test program called _ipc-driver-test_. This program implements the
|
||||
_Driver_ side of the osmo-trx-ipc interface (see <<ipc_if>> for more
|
||||
information) on one side, and also interacts internally with UHD (eg B210 as
|
||||
when using osmo-trx-uhd).
|
||||
|
||||
You can use this small program as a reference to:
|
||||
|
||||
* Test and experiment with _osmo-trx-ipc_.
|
||||
|
||||
* Write your own IPC _Driver_ connecting to osmo-trx-ipc.
|
||||
|
||||
[[fig-backend-ipc-device-test]]
|
||||
.Architecture with _osmo-trx-ipc_ and ipc-device-test as IPC _Driver_
|
||||
[graphviz]
|
||||
----
|
||||
digraph G {
|
||||
rankdir=LR;
|
||||
MS0 [label="MS"];
|
||||
MS1 [label="MS"];
|
||||
SDR;
|
||||
ipc_device_test[label = "ipc-device-test", color=red];
|
||||
OsmoTRX [label="osmo-trx-ipc", color=red];
|
||||
BTS;
|
||||
|
||||
MS0->SDR [label="Um"];
|
||||
MS1->SDR [label="Um"];
|
||||
SDR->ipc_device_test [label="UHD"];
|
||||
ipc_device_test->OsmoTRX [label="IPC Interface", color=red];
|
||||
OsmoTRX->BTS [label="bursts over UDP"];
|
||||
}
|
||||
----
|
||||
|
||||
The code for this app is found here:
|
||||
|
||||
* https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/ipc-driver-test.h[Transceiver52M/device/ipc/ipc-driver-test.h]
|
||||
|
||||
* https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/ipc-driver-test.c[Transceiver52M/device/ipc/ipc-driver-test.c]
|
||||
|
||||
Those files use the server-side (_Driver_ side) code to operate the Posix Shared
|
||||
Memory region implemented in files `shm.c`, `shm.h`, `ipc_shm.c` and `ipc_shm.h`
|
||||
in the same directory.
|
||||
|
||||
Most of the code in that same directory is deliverately released under a BSD
|
||||
license (unlike most of _osmo-trx.git_), allowing third parties to reuse/recycle
|
||||
the code on their implemented _Driver_ program no matter it being proprietary or
|
||||
under an open license. However, care must be taken with external dependencies,
|
||||
as for instance shm.c uses the talloc memory allocator, which is GPL licensed
|
||||
and hence cannot be used in a proprietary driver.
|
||||
|
|
@ -35,6 +35,8 @@ include::./common/chapters/vty_cpu_sched.adoc[]
|
|||
|
||||
include::./common/chapters/trx_if.adoc[]
|
||||
|
||||
include::{srcdir}/chapters/ipc_if.adoc[]
|
||||
|
||||
include::./common/chapters/port_numbers.adoc[]
|
||||
|
||||
include::./common/chapters/bibliography.adoc[]
|
||||
|
|
|
|||
Loading…
Reference in New Issue