Flow control for datagram flows about draft-pauly-quic-datagram HOT 7 CLOSED

There was a discussion on the mailing list about this, "flow control and DATAGRAM".

From that, one of Martin thomson's points was:

>>[Tommy] Having a separate outstanding data limit for the DATAGRAM "stream" is an interesting solution to >> the space. It would then have the nice property of not looking like traditional flow control. It could >> even be measured in number of frames, rather than bytes (depending on what the limiting factors are).

[Martin]
This is something I alluded to in my email.  I don't think that it
improves things much, other than perhaps insulating streams from the
excesses of DATAGRAM frames and vice versa.

If you have a limit, then you need to ensure that both endpoints agree
on its value eventually.  With DATAGRAM, you might think that it's
possible to add a new frame that expresses how much data was sent
(that is the sum of sizes of DATAGRAM frames).  That would run afoul
of reordering problems, where the limit is exceeded when delayed
DATAGRAM frames that are already counted, arrive after the accounting
frame.

You could limit the count to just lost DATAGRAM frames.  That would
allow a receiver to repair its view of how much was used.  However
that runs afoul of two problems: acknowledgments aren't reliable, and
packets can be declared lost and still delivered.  In both cases, the
sender can believe that frames didn't arrive and so increase their
report of "lost bytes" too much.  The receiver ends up with an
inflated view of what was consumed (as opposed to the value being
short).

If you wanted to solve this problem, you would probably need to have
per-frame tracking, which makes this a lot less lightweight than I
think we want.

from draft-pauly-quic-datagram.

Also comments from Ian:

If you add flow control, there's little benefit over a unidirectional stream, so adding flow control fundamentally breaks this design.

My goal is to use this for cases when I want to use a single QUIC connection to replace DTLS or RTP + something reliable.  WebRTC is the obvious use case, but not the only one.  DTLS and RTP don't have flow control, so adding it just make the application mapping a lot more complex.

from draft-pauly-quic-datagram.

I went through the email archive related to this discussion.

I see why we probably don't want flow control for datagram frames. But having a connection level flow control is just the same problem. How would a receiver decide the value of MAX_DATA that should take into account the datagram frames?

from draft-pauly-quic-datagram.

Okay, so based on our discussion, I think we should make the following changes:

• Whenever a sender detects a gap in ACKs that includes DATAGRAM frames, the sender MUST remove the size of the lost DATAGRAM frames' content from its counter tracking the peer's received data (what it uses to send BLOCKED)

• Whenever a receiver accepts a packet for which it has previously sent an ACK gap that contains DATAGRAM frames, it MAY drop the contents if it is out of room, and MUST NOT increase its counter of received application bytes.

from draft-pauly-quic-datagram.

There was a clear lack of interest in including datagrams in connection flow at today's meeting. If one wanted to to include them in flow control of some sort(connection or datagram-specific), it would require clear rules both peers could agree upon about what DATAGRAMs counted towards flow control. Lost(or abandoned) packets containing DATAGRAMS must not count towards flow control because the peer doesn't know they exist.

In order to accomplish this, I believe the most practical solution is to specify the max reordering threshold in packet number space before DATAGRAMs will no longer count towards flow control. Note that they could still be delivered in this case, but they would not count towards flow control. Ideally they would be acknowledged, but implementations would have to still apply the reordering threshold.

This requires very tight coupling between the congestion control and ACK processing and flow control, which may be difficult or highly unappealing to some implementations, but it's worth understanding what the potential options are.

from draft-pauly-quic-datagram.

I agree that including datagrams in flow control (connection or datagram flow) would require a more intensive ACK processing to determine lost packets. Also, the receiver would need to send the ACKs similar to reliable QUIC.
But if we are already doing congestion control based on ACKs and timeouts, then we could leverage it for flow control (to determine lost packets) as well.

Adding flow control will regulate data being sent by the sender and it would possibly reduce packet drops at receiver if the application is not reading. And we can always set a higher flow control limit for datagrams to allow receiving of time sensitive data.

from draft-pauly-quic-datagram.

For now, we're resolving this based on consensus in the room in Prague with #9.

from draft-pauly-quic-datagram.