maxsharabayko / srt-xtransmit Goto Github PK

Howdy,

I'm trying to use srt-xtransmit to see how fast I can send data via srt. When I do, I get actual sending and receiving rates around ~1100 mbps, but the estimated bandwidth is around ~3000 mbps. Via iperf3, I get ~4300 mbps. Why is the actual rate so much lower than the estimated bandwidth? Is there anything I can/should do to raise the actual rate to meet/approach the available bandwidth?

From file mode:

Here are the commands I used (IPs redacted). It's just the defaults recommended by the README but with --sendrate set to 1200mbps for live mode and --num set to 10,000,000 instead of 1000 for file mode.

Live mode:
Sender: srt-xtransmit generate "srt://<ip>:<port>?transtype=live&rcvbuf=1000000000&sndbuf=1000000000" --msgsize 1316 --sendrate 1200Mbps --duration 10s --statsfile stats-snd.csv --statsfreq 100ms
Receiver: srt-xtransmit receive "srt://:<port>?transtype=live&rcvbuf=1000000000&sndbuf=1000000000" --msgsize 1316 --statsfile stats-rcv.csv --statsfreq 100ms

File mode:
Sender: ./srt-xtransmit generate "srt://<ip>:<port>?transtype=file&messageapi=1&payloadsize=1456&rcvbuf=1000000000&sndbuf=1000000000&fc=800000" --msgsize 1456 --num 10000000 --statsfile stats-snd.csv --statsfreq 1s
Receiver: ./srt-xtransmit receive "srt://:<port>?transtype=file&messageapi=1&payloadsize=1456&rcvbuf=1000000000&sndbuf=1000000000&fc=800000" --msgsize 1456 --statsfile stats-rcv.csv --statsfreq 1s

Here are all the stats files:
stats-rcv_1200mbps.csv
stats-snd_1200mbps.csv
stats-rcvFile10M.csv
stats-sndFile10M.csv

Reproduced with SRT v1.4.1 and later. Previous versions were not checked.
srt-xtransmit master Jan 28, 2020 939416e

Receiver:

srt-xtransmit.exe receive srt://:4200 -v --statsfreq 100ms --statsfile rcv-stats.csv

Sender:

srt-xtransmit.exe generate "srt://127.0.0.1:4200" --sendrate 1Mbps --duration 10s -v

When stats on the receiver are disabled, the isuue is not reproduced.
Receiver:

srt-xtransmit.exe receive srt://:4200 -v --statsfreq 100ms --statsfile rcv-stats.csv

Change timestamp data format to '%Y-%m-%d' (or maybe '%d-%m-%Y'). At the current moment, it's '%Y-%d-%m'. Document this in readme.

I'm having some trouble getting socket groups working. Following the instructions at https://github.com/maxsharabayko/srt-xtransmit/blob/master/docs/socket-groups.md, I compiled with -DENABLE_BONDING=ON. When I run the provided example generate command for multiple ports:

./srt-xtransmit generate srt://127.0.0.1:55555?grouptype=backup&weight=2 srt://127.0.0.1:55556?weight=1 --enable-metrics --reconnect

I get the following output, whether or not the corresponding receive command is running:

[1] 2085 -bash: srt://127.0.0.1:55556?weight=1: No such file or directory
14:56:47.531200 [W] CONN Invalid URI query option
'grouptype=backup (not recognized)!

Example implementation:
SRT Packet Reordering - Python script.

As of srt-xtransmit checks if a correct URI query was provided, but only reports a warning.
Instead, it is better to report an error and do not start data transmission, because the user may miss the warning, especially if ran from some scripts that suppress std error.

Current behavior when rexmit_algo=1 is given instead of retransmitalgo=1:

./bin/srt-xtransmit generate "srt://:4200?rexmit_algo=1"
18:06:15.779625 [W] SOCKET::SRT srt://:4200: Ignoring socket option 'rexmit_algo=1' (not recognized)!

TODO

• generate and receive subcommands report an error. PR #27.

• route "srt://:4200?modest=caller" srt://4201 -v tries to connect destination first, then check source and report an error.

srt-xtransmit was used to generate a 7 Mbps stream (on a link with 4% data loss)
However, the data analysis with lib-tcpdump-processing showed values way off 7 Mbps.

./srt-xtransmit generate "srt://192.168.2.2:4200&latency=100" \
--msgsize 1316 -v --sendrate 7Mbps --duration 30s

SRT Data payload:          6.665 Mbps
SRT Data overhead:         1.824%
SRT Data lost:             0.000%
SRT Data rexmit overhead:  10.723%
SRT ACK overhead:          0.286%
SRT ACKACK overhead:       0.286%
SRT NAK overhead:          0.162%
===========================================
SRT overall overhead:      11.325%
SRT Retransmitted:         10.531% of original packets
including:
    retransmitted twice:   0.291% of original packets
    retransmitted more:    3.304% of original packets

Data Payload + Data overhead sums up to 6.7866 Mbps

An additional between flip & flop (no packet loss) showed following using iptraf-ng

Incoming rates: 7256.78 kbps
which is devided by 1024:
Incoming rates: 7.0867 Mbps

Maybe I got a misunderstanding of what the rates out of the lib-tcpdump-processing mean, but the difference is quite large.

There were several requests to send received data to multiple outputs.

See SRT issues 396, 574.

srt-xtransmit route udp://:4200 --output srt://:4201 srt://:4202 udp://192.168.0.1:4500 srt://192.168.1.15:4300

Handle transmission of zero-length files.

Currently, metrics do not take into account byte order, supposing little-endian machines.

TODO

• Add byte swapping on big-endian platforms.

Hi,
I use the following commands to get a udp stream:
.\srt-xtransmit.exe route -i udp://:8866 -o "srt://:8867" -v
or
.\srt-xtransmit.exe receive "udp://:8866" --enable-metrics --metricsfile udp-metrics.csv --metricsfreq 1s
And I use the following commands to send streams to them:
.\srt-xtransmit.exe generate "udp://remote_ip:8866" --sendrate 5Mbps --duration 120s --enable-metrics
or
ffmpeg -i D:\video.mp4 -f mpegts "udp://127.0.0.1:8866"
But no communication is established between the receiver and the sender.
If I use srt instead of udp, the connection is established easily, but I need to use udp.

RFC 8593: Video Traffic Models for RTP Congestion Control Evaluations.
See also: Syncodecs project.

With every new connection, the CUnitQueue capacity keeps increasing.

Steps to Reproduce

srt-xtransmit receive srt://:4200?groupconnect=1 --reconnect -v

srt-xtransmit generate srt://ip:4200 srt://ip:4200 --sendrate 5Mbps --duration 5s --reconnect -v

Reason

Memory grows upon reconnection because sender breaks the connection and establishes a new one almost immediately due to low RTT. Hence it starts sending packets over those member links. Meanwhile, the receiver is still reading packets from a group. A new group connection is in the backlog and is not yet accepted by the application. So all those packets are stored in the receiver buffer for quite some time.

(normal flow, about 60 packets in the RCV buffer)
18:06:48.811000/T27156!W:SRT.br: @452069845 RCV buffer readMessage m_iMaxPosOff 59

(T0 - meanwhile reconnection happens in the background)
18:06:48.829000/T22900*E:SRT.gm: addGroup: @1525811665
18:06:48.830000/T22900.N:SRT.gm: group/add: adding member @452069842 into group $1525811665
18:06:48.830000/T22900.N:SRT.cn: @452069842 connection on listener @452069846 (127.0.0.1:4200) from peer @1044177265 (127.0.0.1:57913)
18:06:48.831000/T22900.N:SRT.qr: @452069842: SOCKET pending for connection - ADDING TO RCV QUEUE/MAP
18:06:48.831000/T22900.N:SRT.gm: group/add: adding member @452069840 into group $1525811665
18:06:48.832000/T22900.N:SRT.cn: @452069840 connection on listener @452069846 (127.0.0.1:4200) from peer @1044177264 (127.0.0.1:57914)
18:06:48.833000/T22900.N:SRT.qr: @452069840: SOCKET pending for connection - ADDING TO RCV QUEUE/MAP
18:06:48.833000/T22900.N:SRT.qr: CUnitQueue::state: Capacity 256 units, 108 in use.

(T+10ms - previous group members are broken)
18:06:48.843000/T22900.N:SRT.qr: @452069845: SOCKET broken, REMOVING FROM RCV QUEUE/MAP.
18:06:48.843000/T22900.N:SRT.qr: @452069843: SOCKET broken, REMOVING FROM RCV QUEUE/MAP.
18:06:48.869000/T27156!W:SRT.br: @452069845 RCV buffer readMessage m_iMaxPosOff 38

(T+60ms members *40 and *42 are receiving packets, the group is not accepted by the app yet)
18:06:48.888000/T22900!W:SRT.br: @452069840 RCV buffer insert m_iMaxPosOff 19
18:06:48.889000/T22900!W:SRT.br: @452069842 RCV buffer insert m_iMaxPosOff 19
18:06:48.931000/T27156!W:SRT.br: @452069845 RCV buffer readMessage m_iMaxPosOff 9

(T+245ms  - still receiving, but not reading, capacity of the memory pool has to be increased)
18:06:49.074000/T22900!W:SRT.br: @452069842 RCV buffer insert m_iMaxPosOff 107
18:06:49.074000/T22900!W:SRT.br: @452069840 RCV buffer insert m_iMaxPosOff 107

(T+1010ms - now removing members)
18:06:49.808000/T22900.N:SRT.qr: CUnitQueue::increase: Capacity 1024 + 128 new units, 922 in use.
18:06:49.824000/T22900!W:SRT.br: @452069840 RCV buffer insert m_iMaxPosOff 463
18:06:49.824000/T22900!W:SRT.br: @452069842 RCV buffer insert m_iMaxPosOff 463
18:06:49.839000/T28584.N:SRT.gm: group/remove: removing member @452069843 from group $1525811668
18:06:49.839000/T28584.N:SRT.gm: group/remove: removing member @452069845 from group $1525811668
18:06:49.839000/T27156*E:SRT.gr: grp/recv: $1525811668: ABANDONING: opened=false connected=false
18:06:49.840194 [W] RECEIVE read::recv: Connection does not exist
18:06:49.885000/T22900!W:SRT.br: @452069840 RCV buffer insert m_iMaxPosOff 493
18:06:49.886000/T22900!W:SRT.br: @452069842 RCV buffer insert m_iMaxPosOff 493

(T+1788ms - finally application accepts new group connection and starts reading)
18:06:50.617041 [I] SOCKET::SRT @452069846 (srt://:4200) Accepted connection @1525811665. TSBPD Latency RCV 120ms, peer 0ms. KM state INVALID (RCV UNSECURED, SND UNSECURED). PB key length: -1. Cryptomode INVALID. Stream ID: not set.
18:06:50.617934 [I] RECEIVE Latency, us: avg n/a, min n/a, max n/a. Jitter: 0us. Delay Factor: 1us. Pkts: rcvd 0, reordered 0 (dist 0), lost 0, MD5 err 0, bad len 0.
18:06:50.620000/T27156!W:SRT.br: @452069842 CRcvBuffer.readMessage() : m_iMaxPosOff 800
18:06:50.624000/T27156!W:SRT.br: @452069842 CRcvBuffer.readMessage() : m_iMaxPosOff 700

(initial end-to-end delay on reconnection is 1.776 seconds instead of 120 ms)
18:06:51.626057 [I] RECEIVE Latency, us: avg 125384, min 124575, max 1776108. Jitter: 96us. Delay Factor: 1651344us. Pkts: rcvd 1259, reordered 0 (dist 0), lost 0, MD5 err 0, bad len 0.

Possible Solution

If further connections are possible move the reading from a socket to a separate thread instantiated upon every connection. This way the second connection would trigger a new receiving thread, the receiving would happen earlier and in parallel with the previous receiving.

UDP socket options to consider for URI params:

• SO_REUSEADDR
• SO_REUSEPORT

I am creating a UDP->SRT transmitter stream with ROUTE, this is my command:
Transmitter: ./srt-xtransmit route -i "udp://127.0.0.1:6001?adapter=127.0.0.1" -o "srt:// :9000" -v
Receiner: ./srt-xtransmit route -i "srt://public_ip:9000" -o "udp://127.0.0.1:6001?adapter=127.0.0.1" -v
The flow starts according to the ROUTE [SRC->DST] Started log, but nothing is transmitted, and I do not end the connection on the receiver side, the sender does not close the connection, remaining ROUTE [SRC- >DST] Started. I would like to know if it is possible to perform UDP->SRT flow with ROUTE or is it some bug.

When collecting SRT csv statistics, if the filepath contains the folder name _some_folder/filename.csv and this folder does not exist, srt-xtransmit does not handle this situation: 1. There is no error and it starts transmission, but 2. No folder is created and as a result no csv file is created.

I think it would be useful to create the directory in this case.

All the settings can be passed via a config file in the TOML format.

https://github.com/CLIUtils/CLI11/tree/v1.9.0

It would be nice if generate and receive had an option that would make them output events in a structured format, so another program could tell what's going on. The most useful events I can think of are:

• connect attempt started (caller)
• started listening (listener)
• connection established (both)
• connect attempt aborted (caller)
• connection rejected (listener)
• disconnected (both)

Newline-delimited JSON is probably the nicest to consume, perhaps with a structure like:

{
  "event": "CONNECT_REJECTED",
  "detail": {"reason": "incorrect passphrase"},
  "timestamp": "2020-02-04T16:45:05+00:00"
}

Check if IPv6 works well and fix bugs if any.

• a potential seg fault there if user uses ipv6 and accepts a "connection" assuming it is from an ipv4 address
  

  srt-xtransmit/xtransmit/srt_socket.cpp

  Line 137 in c197de3

  sockaddr_in scl;

There should be an option to print an info on the connection established.

Possible command-line flag:
--show-conninfo

What to show?

• SRTO_PBKEYLEN
• SRTO_CRYPTO_MODE
• ...

Is it possible to create stable releases for package managers (like myself) that are updated in-sync with Git-pushes?