to terminating it with a CLOSE_WEBTRANSPORT_SESSION capsule that has an error code of 0 and an empty error string.  6. [[RFC editor: please remove this section before publication.]]

This is mostly based on what Chrome currently does in production.
Chair: discussed during editor's meeting, plan is to remove text about 3 PTO and note that delivering other open streams is best-effort. Also add rationale and mention that this aligns with what QUIC does - this message has similar delivery guarantees as the close_connection in QUIC
A common use (at least for non-pooled WebTransport connections) will be to close the underlying QUIC connection when closing the WebTransport session. This is inherently racy: The sender will send both a CLOSEWEBTRANSPORTSESSION capsule and a QUIC CONNECTIONCLOSE frame (unless you wait for the peer's FIN on the request stream, but why would you?). It's undetermined which one of the two will be received / processed first by the receiver. This is not a problem if the sender uses the same error code and error message for the CONNECTIONCLOSE frame and for the CLOSEWEBTRANSPORTSESSION capsule. Assuming the QUIC stack properly exposes these values on its API, the receiver will be able to learn error code and error message. Should we add that as a recommendation, or even a requirement?
Chair: discussed during editor's meeting, see URL

to terminating it with a CLOSE_WEBTRANSPORT_SESSION capsule that has an error code of 0 and an empty error string. <ins> In some scenarios, an endpoint might want to send a CLOSE_WEBTRANSPORT_SESSION with detailed close information and then immediately close the underlying QUIC connection.  If the endpoint were to do both of those simultaneously, the peer could potentially receive the CONNECTION_CLOSE before receiving the CLOSE_WEBTRANSPORT_SESSION, thus never receiving the application error data contained in the latter.  To avoid this, the endpoint SHOULD wait until all of the data on the CONNECT stream is acknowledged before sending the CONNECTION_CLOSE; this gives CLOSE_WEBTRANSPORT_SESSION properties similar to that of the QUIC CONNECTION_CLOSE mechanism as a best-effort mechanism of delivering application close metadata. </ins> 6. [[RFC editor: please remove this section before publication.]]


4.2. WebTransport endpoints can initiate bidirectional streams by opening <del> an HTTP/3 bidirectional stream and sending an HTTP/3 frame with type "WEBTRANSPORT_STREAM" (type=0x41).  The format of the frame SHALL be the frame type, followed by the session ID, encoded as a variable- length integer, followed by the user-specified stream data (fig-bidi- client).  The frame SHALL last until the end of the stream. </del>  HTTP/3 does not by itself define any semantics for server-initiated bidirectional streams.  If WebTransport setting is negotiated by both

Also ban sending it anywhere that's not the beginning of the stream. Potentially also and
cc NAME
I'm curious how anyone has got handling of the WEBTRANSPORTSTREAM frame working based on strict following of the specs. WT over H3 says: H3 says in URL Type (i), Length (i), Frame Payload (..), } snip WT over h3 says of the frame: So it appears that WT is trying to omit the length field to imply an Indefinate length, but H3 doesn't allow that. I'd expect a standard H3 frame parser to read the first N bytes of the session ID and incorrectly parse that as the length. I don;t think you can special case this frame, because your breaking the H3 contract that unknown frame types have a correct length that allows them to be read and discarded. One option (lets call it A) would be to say that has a length field of MAXVAL and then create a new caveat that if you're a WebTransport supporting endpoint, you don't need to create a connection error the stream if it is FIN'd before that length is reached. However, that has the unfortunate problem of consuming 8 bytes at the start of every stream. Another option (B) might be to say that the frame is fixed length as so: Type (i) = 0x41, Length (i), Session ID (i), } Meaning the frame only contains the Session ID field, and the length of that frame is truly the length of Session ID - satisfying the general H3 requirements. Then after this frame, you switch into WebTransport bidi streams as we seem to be going towards on . However, H3 endpoints that read this frame and discard it, would then move on to parsing the subsequent bytes as frames (when the sender never intended that) and bad stuff might happen.
We already discussed this in . The expectation for unknown H3 frames to have length does not apply here: WebTransport support is explicitly negotiated via SETTINGS, meaning that by the time someone sends WEBTRANSPORT_STREAM, both peers know they support it.
Thanks for reminding of that. Given the discussion on , if ee go that way I don't think we should bother calling this an HTTP/3 frame. For all intents and purposes it isn't. Rather, we could make it clearer that WebTransport is a semantic extension to H3 that permits client bidirectional streams to start with a special byte sequence that changes the stream purpose and meaning.
It needs to be registered in the h3 frame type IANA registry to avoid conflicts though
You could seek to reserve the codepoint to help prevent unintended interop problems, we already do that e.g. URL
At the end of the day calling this an infinite-length-frame or a non-frame-that-is-reserved-in-the-list-of-frames ends up being the same thing implementation-wise. My personal two cents is that infinite-length frame is easier to explain and reason about.
I disagree. With the proposal on , we're making a non-frame that can't have anything before or after it. This is a very special thing that sticks out and I doubt is likely to be replicated. So I'd like to hear what other implementers have done or would do. My implementation that reads from streams already has branches that go into frame parsing mode or not depending on the stream type. If I were to add the code, I'd model it as a different stream type that is only supported when WebTransport is enabled and it wouldn't touch my frame parsing part at all. edit: (added comment) given HTTP/3 already has the notion of stream types, I don't know why we need to invent another special concept in HTTP/3 to support WebTransport.
Chair: discussed in editor's meeting, NAME thinks this is resolved by . NAME do you agree?
Sorry I missed that PR as I was on holiday. I believe that channels the spirit of my PR , and addresses the original issue here I think the text could continue to have some editorial improvements to better highlight how much of a divergence this is for request streams, but lets track the separately.
Awesome, thanks Lucas! Closing based on that comment. For the remaining editorial bits, please file a separate issue or PR :-)
The draft says: It is unclear if it is allowed to send other frames before sending the WEBTRANSPORT_STREAM frame. I can't think of any currently defined HTTP/3 frame that would make sense to send here, but that might not be the case with future extensions. At the very least, it would be nice of grease frames could be sent.
touches on this issue for unidirectional WT streams - eg: should they be defined as a sequence of TLV frames followed by a WT TV frame which converts the stream to unframed?
What is the value in sending grease frames on these streams? Specifically to allow for future extensions that send other frames before WEBTRANSPORT_STREAM?
I can imagine sending a PRIORITYUPDATE or something of that nature, though it's not clear to me it's worth the effort compared top just not using any frames on the stream before WEBTRANSPORTSTREAM.
Chair: discussed at IETF 114, consensus in the room to be consistent between unidirectional and bidirectional require WEBTRANSPORTSTREAM to be the first frame on a bidirectional stream this also disallows extension frames before WEBTRANSPORTSTREAM
I wasn't in the room but following up on the meeting recording, I could support this proposal. However, I don't understand how we make the WEBTRANSFORM_STREAM actually do what we describe here robustly, so I opened
WT uni streams are currently defined as This was awkward to implement, and asymmetric with how client-initiated bidi WT streams are implemented (and now server-initiated bidi -- with a frame). I'd like to propose we change WT uni streams to or perhaps
Not discussed at IETF 111. I don't see the point in trying to match this up with bidi streams. All this achieves is wasting bytes for the frame type field.
I tend to think the current design is fine as well. If you think about the first value of the unidirectional stream being a frame type, not a stream type, then I think they do conceptually align with the Bidi WT streams?
Paging in context... I think my reason for suggesting this was that my parser always wants to consume the stream-type when it dispatches a unidirectional stream, but given the similarities to the other WT stream types, I wanted to use the generic frame parser after dispatch. I made it work by not consuming the type-byte in this special case. A side effect of defining it as is that one could more easily add extension frames before the TV frame in the future, at the cost of an extra byte now. I can live with it as is too.
NAME Should I add this to the agenda for the next meeting, or should we just leave this as-is and close this issue?
Given the relation to the issue just opened ^ covering a similar topic on bidi streams (eg: which frames are allowed before TV), maybe we should have a brief discussion? I think they should behave similarly. I have the context paged in now before the meeting. It sounds like NAME suggested Grease frames as a possible use case, in addition to "future extensions" suggested here. Maybe a frame for signaling the initial priority in the bidi-case is plausible? Grease frames may be marginal in this context (you can't send one on a QPACK stream either), and any extension needs to be enabled via SETTINGS (or capsules or headers?) anyways so said extension can change the format of WT streams. I'll counter that the cost (1 byte per stream) is low and it's easier to bake in the flexibility from the start rather than bolt it on later. I'll reiterate that I can live with the current scheme.
I think that the unidirectional design (having a dedicated stream type) is the much cleaner solution. There is greasing for unidirectional stream types in HTTP/3, so I don't think we need to do anything here. I wish we also had stream types for bidirectional streams in HTTP/3, so that ship has sailed.
NAME - I read your position on the other issue and assumed you wanted future extensions to be able to send frames before the WT stream became unframed. If that's true, I don't understand why you would want that capability for bidi streams but not uni streams.
At IETF 114, we discussed uni/bidi streams and there was consensus in the room to be consistent between unidirectional and bidirectional. See
Chair: understanding is that this is let's keep the related discussion going on in which is still open

4.2. WebTransport endpoints can initiate bidirectional streams by opening  <ins> an HTTP/3 bidirectional stream and then immediately sending a special signal value 0x41, followed by the associated session ID, both encoded as a variable-length integer; the rest of the stream is the application payload of the WebTransport stream (fig-bidi-client). This document registers the special signal value 0x41 as a WEBTRANSPORT_STREAM frame type.  While it is registered as an HTTP/3 frame type to avoid collisions, WEBTRANSPORT_STREAM is not a proper HTTP/3 frame, as it lacks length; it is an extension of HTTP/3 frame syntax that MUST be supported by any peer negotiating "SETTINGS_ENABLE_WEBTRANSPORT".  Any attempt to use WEBTRANSPORT_STREAM as a frame type outside of the very first byte of the stream MUST be treated as a connection error of type H3_FRAME_ERROR. </ins> HTTP/3 does not by itself define any semantics for server-initiated bidirectional streams.  If WebTransport setting is negotiated by both


The following entry is added to the "HTTP/3 Frame Type" registry established by HTTP3: <del> The "WEBTRANSPORT_STREAM" frame allows HTTP/3 client-initiated bidirectional streams to be used by WebTransport: </del>  0x41

This updates the IANA considerations section for Frame Type to match URL, by including "server-initiated" in the type of bidirectional streams allowed by the frame.

The following entry is added to the "HTTP/3 Frame Type" registry established by HTTP3:  <ins> The "WEBTRANSPORT_STREAM" frame allows HTTP/3 client-initiated and server-initiated bidirectional streams to be used by WebTransport: </ins> 0x41


closing the connection.  The message takes up the remainder of the capsule, and its length MUST NOT exceed 1024 bytes. <del> A CLOSE_WEBTRANSPORT_SESSION capsule MUST be followed by a FIN on the sender side.  If any additional stream data is received on the CONNECT stream after CLOSE_WEBTRANSPORT_SESSION, the stream MUST be reset with code H3_MESSAGE_ERROR.  The recipient MUST close the stream upon receiving a FIN.  If the sender of CLOSE_WEBTRANSPORT_SESSION does not receive a FIN after some time, it SHOULD send STOP_SENDING on the CONNECT stream. </del>  Cleanly terminating a CONNECT stream without a CLOSE_WEBTRANSPORT_SESSION capsule SHALL be semantically equivalent

After sending the CLOSEWEBTRANSPORTSESSION capsule and sending the FIN on the stream, it would make sense to also stop reading on the stream (by sending a STOP_SENDING). Should that be the recommended behavior in the spec?
Chair: discussed at IETF 114, consensus in the room to allow but not require sending STOP_SENDING
The spec currently reads: So it currently recommends setting a timer and then issuing a STOPSENDING. I guess the request here is to send indicate the sender of CLOSEWEBTRANSPORTSESSION may send the STOPSENDING immediately. I'm going to remove the bit about the timer since it's not really relevant? If the closing endpoint wants to wait to see if there's any other messages in flight on the CONNECT stream that's fine. The CLOSEWEBTRANSPORTSESSION + FIN + must-respond-with-FIN is an implicit STOP_SENDING.
Fix the typo, raise an issue for the other thing.

closing the connection.  The message takes up the remainder of the capsule, and its length MUST NOT exceed 1024 bytes.  <ins> An endpoint that sends a CLOSE_WEBTRANSPORT_SESSION capsule MUST immediately send a FIN.  The endpoint MAY send a STOP_SENDING to indicate it is no longer reading from the CONNECT stream.  The recipient MUST close the stream upon receiving a FIN.  If any additional stream data is received on the CONNECT stream after receiving a CLOSE_WEBTRANSPORT_SESSION capsule, the stream MUST be reset with code H3_MESSAGE_ERROR. </ins> Cleanly terminating a CONNECT stream without a CLOSE_WEBTRANSPORT_SESSION capsule SHALL be semantically equivalent


4.1.  A datagram is a sequence of bytes that is limited in size (generally to the path MTU) and is not expected to be transmitted reliably.  The general goal for WebTransport datagrams is to be similar in behavior it can send.  The size SHOULD be derived from the result of performing path MTU discovery. <del> 4.2. </del>  A unidirectional stream is a one-way reliable in-order stream of bytes where the initiator is the only endpoint that can send data.  A In addition to flow controlling stream data, the creation of new streams is flow controlled as well: an endpoint may only open a limited number of streams until the peer explicitly allows creating <del> more streams. </del>  5.

Currently, in the W3C API draft, we cite the HTTP/3 draft and RFC 9000 directly. This does not work with WebTransport over HTTP/2, hence we need for the overview draft to provide a common set of operations that both H3 and H2 versions of the draft are expected to implement.
cc NAME NAME NAME NAME
How does this square with the set of features that we list in the overview draft -- right now it looks like we'll just have two lists. Do we want to do anything to relate them / break things down as being part of each feature? (Not saying that's strictly necessary, just curious if we'd thought about it)
Chair: discussed at 113, request for folks to review this PR
Thanks everyone for the comments! I've revised the PR; please take another look. By features, you mean the requirements in the section above that? Those are somewhat different in nature, I feel. The features are the list of things that I should be able to do with a WebTransport, whereas the requirements are on how it is supposed to work (e.g. you must use TLS, etc).
(no hats) overall this LGTM, my comments are minor

4.1. <ins> Any WebTransport protocol SHALL provide the following operations on the session: Create a new WebTransport session given a URI RFC3986 and the origin RFC6454 of the requester. Terminate the session while communicating to the peer an unsigned 32-bit error code and an error reason string of at most 1024 bytes.  The error code and string are optional; the default values are 0 and "". Any WebTransport protocol SHALL provide the following events: Indicates that the WebTransport session has been terminated, either by the peer or by the local networking stack, and no user data can be exchanged on it any further.  If the session has been terminated as a result of the peer performing the "terminate a session" operation above, a corresponding error code and an error string can be provided. 4.2. </ins> A datagram is a sequence of bytes that is limited in size (generally to the path MTU) and is not expected to be transmitted reliably.  The general goal for WebTransport datagrams is to be similar in behavior it can send.  The size SHOULD be derived from the result of performing path MTU discovery.  <ins> In the WebTransport model, all of the outgoing and incoming datagrams are placed into a size-bound queue (similar to a network interface card queue). Any WebTransport protocol SHALL provide the following operations on the session: Enqueues a datagram to be sent to the peer.  This can potentially result in the datagram being dropped if the queue is full. Dequeues an incoming datagram, if one is available. Returns the largest size of the datagram that a WebTransport session is expected to be able to send. 4.3. </ins> A unidirectional stream is a one-way reliable in-order stream of bytes where the initiator is the only endpoint that can send data.  A In addition to flow controlling stream data, the creation of new streams is flow controlled as well: an endpoint may only open a limited number of streams until the peer explicitly allows creating  <ins> more streams.  From the perspective of the client, this is presented as a size-bounded queue of incoming streams. Any WebTransport protocol SHALL provide the following operations on the session: Creates an outgoing unidirectional stream; this operation may block until the flow control of the underlying protocol allows for it to be completed. Creates an outgoing bidirectional stream; this operation may block until the flow control of the underlying protocol allows for it to be completed. Removes a stream from the queue of incoming unidirectional streams, if one is available. Removes a stream from the queue of incoming unidirectional streams, if one is available. Any WebTransport protocol SHALL provide the following operations on an individual stream: Add bytes into the stream send buffer.  The sender can also indicate a FIN, signalling the fact that no new data will be send on the stream.  Not applicable for incoming unidirectional streams. Removes bytes from the stream receive buffer.  FIN can be received together with the stream data.  Not applicable for outgoing unidirectional streams. Sends a signal to the peer that the write side of the stream has been aborted.  Discards the send buffer; if possible, no currently outstanding data is transmitted or retransmitted.  An unsigned 8-bit error code can be supplied as a part of the signal to the peer; if omitted, the error code is presumed to be 0. Sends a signal to the peer that the read side of the stream has been aborted.  Discards the receive buffer; the peer is typically expected to abort the corresponding send side in response.  An unsigned 8-bit error code can be supplied as a part of the signal to the peer. Any WebTransport protocol SHALL provide the following events for an individual stream: Indicates that the peer has aborted the corresponding receive side of the stream.  An unsigned 8-bit error code from the peer may be available. Indicates that the peer has aborted the corresponding send side of the stream.  An unsigned 8-bit error code from the peer may be available. Indicates that no further data will be transmitted or retransmitted on the local send side, and that the FIN has been sent.  Data Recvd implies that aborting send-side is a no-op. </ins> 5.


Abstract <del> Coding is a reliability mechanism that is distinct and separated from the loss detection of congestion controls.  Using coding can be a useful way to better deal with tail losses or with networks with non- congestion losses.  However, coding mechanisms should not hide congestion signals.  This memo offers a discussion of how coding and congestion control can coexist.  This document can help the comparison of FEC schemes by identifying at which level they are operating with respect to the transport congestion control. </del>  This document is the product of the Coding for Efficient Network Communications Research Group (NWCRG).  The scope of the document is <del> end-to-end communications: coding for tunnels is out-of-the scope of the document. </del>  1. <del> There are cases where deploying coding improves the quality of the transmission.  As an example, it may take time for the server to detect tail losses and this would impact the experience of applications using short flows.  Another example are networks where non-congestion losses are persistent and prevent a sender from exploiting the link capacity. </del>  Coding is a reliability mechanism that is distinct and separated from the loss detection of congestion controls.  RFC5681 defines TCP as a <del> loss-based congestion control; because coding repairs such losses, blindly applying it may easily lead to an implementation that also hides a congestion signal to the sender.  It is important to ensure that such information hiding does not occur. </del>  <del> This memo offers a discussion of how coding and congestion control can coexist.  One objective is to help the research community consider congestion control aspects when proposing and comparing coding solutions. </del>  This document represents the collaborative work and consensus of the Coding for Efficient Network Communications Research Group (NWCRG); <del> it is not an IETF product and is not a standard.  The proposed recommendations apply for coding at the transport or application layer.  Coding for tunnels is out of scope for the document. </del>  2. fig:sep-channel presents the notations that will be used in this document and introduce the Congestion Control (CC) and Forward Erasure Correction (FEC) channels.  The Congestion Control channel <del> carries data packets (from a server to a client) and potential </del>  information signaling the packets that have been received (from the client to the server).  The Forward Erasure Correction channel <del> carries coded packets (from the server to the client) and a potiential information signaling the packets that have been repaired </del>  (from the client to the server).  It is worth pointing out that there are cases where these channels are not separated. conceptually separate: fig:sep-entities provides more details than fig:sep-channel by <del> focusing on the server side.  The inputs to FEC (data to work upon, and signaling from the receiver about losses and/or repaired blocks) are distinct from the inputs to CC.  The latter calculates a sending rate or window from network measurements, and it takes the data to send as input, sometimes along with application requirements such as upper/lower rate bounds, periods of quiescence, or a priority. </del>  It is not clear that the ACK signals feeding into a congestion control algorithm are useful to FEC in their raw form, and vice versa - information about repaired blocks may be quite irrelevant to a CC <del> algorithm.  However, there can be meaningful other interactions (indicated by the horizontal double arrow) between the two entities, usually as a result of their operation rather than by relaying their own raw inputs. </del>  3. <del> This section describes the scope of the document. </del>  3.1. <del> The document focuses on reliable transmissions. 3.2. The document focuses on end-to-end coding, i.e. cases where coding is added at the server and client end points.  The discussions should then consider fairness with non-coding solutions. 3.3. sec:notations presents that FEC and CC can be seen as two separate channels.  In practice, implementations may mix the signals that are exchanged on these channels.  This document discusses how FEC and CC relate.  The discussion can help in avoiding comparing FEC schemes that do not operate at the same level (with respect to the CC).  [NK: We may want to later add that we provide recommendations - but for the moment, we may just list the server side coding solutions in the layering approach without providing recommendations for each case] 4. </del> fig:fec-above illustrates the FEC above the transport case. fig:fec-above presents an architecture where FEC is on top of the <del> transport.  The coded packets are sent within what the congestion </del>  window allows. The advantage of this approach is that the FEC does not contribute to adding congestion in the network. <del> The drawback of this approach is that it is useless if the transport guarantees data delivery by retransmitting packets. </del>  <del> 5. </del>  fig:fec-in illustrates the FEC within the transport case. fig:fec-in presents an architecture where FEC is within the <del> transport.  The coded packets are sent within what the congestion </del>  window allows, such as in CTCP.  Examples of the solution would be <del> sending coded packets when there is no more data to transmit or preferably send coded packets instead of the following packets in the send buffer. </del>  The advantage of this approach is that it can enable conjoint optimization between the CC and the FEC.  Moreover, the transmission <del> of coded packets does not add congestion in potentially congested </del>  networks but help repair lost packets (such as tail losses). The drawback of this approach is that it may not result in much gains as opposed to classical CC retransmissions mechanisms and it costs <del> bandwidth that could have been exploited to transmit non coded data packets.  The coding ratio needs to be carefully designed. </del>  <del> 6. </del>  fig:fec-below illustrates the FEC below the transport case. fig:fec-below presents an architecture where FEC is below the <del> transport.  The coded packets are sent on top of what is allowed by </del>  the congestion control.  Examples of the solution could be adding a given percentage of the congestion window as supplementary packets or <del> sending a given amount of coded packets at a given rate.  The </del>  redundancy flow can be decorrelated from the congestion control that manages source packets: a secondary congestion control could be introduced underneath the FEC layer, such as in coupled congestion already congested networks.  The coding ratio needs to be carefully designed. <del> 7. </del>  This memo includes no request to IANA. <del> 8. </del>  <del> There is no security considerations required for this document. </del>

Never say what follows: There is no security considerations required for this document. There are security considerations in most documents, because it does change something (which is fortunate) and therefore it can introduce threats. Here CC and FEC coding rate could be leveraged to create DoS. Although it's not specific to this document, saying the opposite is not appropriate.
Done in PR
I appreciate to see these three sections, but I think they should be grouped as subsections of a single one entitled: "Recommendations depending on the CC versus FEC layering" or something like that. And adding a small introduction text would be fine too. There's really an ambiguity about what is coded data: does it include source data packets or not? And why data rather than packets? Have a look at our terminology RFC and stick with it. I do not agree about the following wording: The drawback of this approach is that it is useless if the transport guarantees data delivery by retransmitting packets. This is only a drawback if one is stupid enough to do FEC protection on top of a reliable transport protocol. I'd rather say: Of course, this approach requires that that the transport protocol does not include a reliability (e.g., based on lost packet retransmition). This approach is perfectly fine with UDP which is pretty usual. This is suggested in the figure ("sending rate or window"), but not detailed in text. There is preferably a feedback between CC and FEC entities.
Covered in PR
Section 3.1 is pretty concise. What type of "transmission"? In intro you say that tunnels are out of scope, which should be clarified and justified here. Do you restrict yourself to transport protocols (where CC is usually found)? Do you want to extend to application protocols as well (with the difficulty of defining what is an application protocol, since we see more and more protocols on top of UDP). Do you consider multipaths? Do you keep multicast as out of scope as there is a dedicated CC that is pretty different? A new idea is introduced here, that of "fairness with non-coding solutions". It should prominently mentioned in abstract and introduction as well. I'm surprised to see "Section 3.3. Objective of the document" at this moment in the document. Please put it before. The first two sentences about mixing the CC and FEC channels should also be moved at the end of section 2, it does not belong to the objectives.
About fairness: I'd rather say that this only concerns the case in section 6, FEC below the transport, and should be discussed only there.
This section may not be relevant if we move all the discussions of the scope in the Introduction. This is what has been done in PR
Mention of fairness has been removed and the present focus of the draft is unicast single path transfer.
A detail: arrows are missing in fig. 1. Or perhaps it is indicated by the use of { and }, but in that case it's not very clear. Using plural is also preferable. A detail: there is no "coded data" but "coded packets".It is said: "signaling from the receiver about losses and/or repaired blocks" The use of "blocks" here is erroneous, we talk about packets. I'd also suggest that you change the mention "information about repaired ratio" in fig. 2 by what you say here, in the text. What exact information is given back to the FEC entity really depends on the use-case, but I like the way it's written in the text: it's some information about what is lost and what is recovered thanks to FEC coding. It is said: However, there can be meaningful other interactions (indicated by the horizontal double arrow) between the two entities, There is no horizontal double arrow! And I cannot really decide whether entities refer to client/server or CC/FEC. typo: potiential => potential
Thanks. Updated in PR
Minor comment: It is said (2nd sentence): ... for the server to detect tail losses... Make it clear that in this document, the server sends data. Ambiguous: One objective is to help the research community consider congestion control aspects when proposing and comparing coding solutions. I'd rather say: Another objective is to encourage the research community to also consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems. Rationals: this is another objective; comparing FEC Schemes without considering congestion control makes sense if the goal is to compare those schemes only.
Thanks. This is covered in PR
I do not understand: This document can help the comparison of FEC schemes by identifying at which level they are operating with respect to the transport congestion control. What does "the comparison of FEC schemes" refer to? Between one another? I guess this is not the intent (and this is explained later on). Global comment, title including: using "coding" without any further information, is ambiguous. I'd prefer FEC coding or erasure correction coding, at least in the abstract/introduction.
The intent is indeed to help the comparison of different FEC schemes. There are cases where a FEC scheme A hide losses to a congestion control, while a FEC scheme B do not. Comparing A and B is not fair. The comment on using "coding" is dealt with in PR

Abstract  <ins> FEC coding is a reliability mechanism that is distinct and separated from the loss detection of congestion controls.  Using FEC coding can be a useful way to better deal with transfer tail losses or with networks with non-congestion losses.  However, FEC coding mechanisms should not hide congestion signals.  This memo offers a discussion of how FEC coding and congestion control can coexist.  This document can help the comparison of FEC schemes by identifying at which level they are operating with respect to the transport congestion control. </ins> This document is the product of the Coding for Efficient Network Communications Research Group (NWCRG).  The scope of the document is  <ins> end-to-end communications: FEC coding for tunnels is out-of-the scope of the document. </ins> 1.  <ins> There are cases where deploying FEC coding improves the quality of the transmission.  As an example, when a server transmits data to a client, it may take time for the server to detect tail losses (i.e. tail loss can refer to losses at the tail end of transactions).  This would impact the experience of applications using short flows. Another example are networks where non-congestion losses are persistent and prevent a sender from exploiting the link capacity. </ins> Coding is a reliability mechanism that is distinct and separated from the loss detection of congestion controls.  RFC5681 defines TCP as a  <ins> loss-based congestion control; because FEC coding repairs such losses, blindly applying it may easily lead to an implementation that also hides a congestion signal to the sender.  It is important to ensure that such information hiding does not occur. </ins>  <ins> FEC coding and congestion control can be seen as two separate channels.  In practice, implementations may mix the signals that are exchanged on these channels.  This memo offers a discussion of how FEC coding and congestion control can coexist.  Another objective is to encourage the research community to also consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems.  That being said, this document does not aim at proposing FEC coding solutions characterization guidelines: comparing FEC Schemes without considering congestion control can be relevant if the goal is to compare those schemes only. The proposed document considers FEC coding at the transport or application layer for an end-to-end unicast data transfer.  The typical application scenario that is considered in the current version of the document is a client browsing the web.  This memo may be extended to cases with multiple paths. </ins> This document represents the collaborative work and consensus of the Coding for Efficient Network Communications Research Group (NWCRG);  <ins> it is not an IETF product and is not a standard.  The document follows the terminology proposed in the taxonomy document RFC8406. </ins> 2. fig:sep-channel presents the notations that will be used in this document and introduce the Congestion Control (CC) and Forward Erasure Correction (FEC) channels.  The Congestion Control channel  <ins> carries source packets (from a server to a client) and potential </ins> information signaling the packets that have been received (from the client to the server).  The Forward Erasure Correction channel  <ins> carries repair packets (from the server to the client) and a potential information signaling the packets that have been repaired </ins> (from the client to the server).  It is worth pointing out that there are cases where these channels are not separated. conceptually separate: fig:sep-entities provides more details than fig:sep-channel by  <ins> focusing on the server side.  Some elements are introduced: 'network measurements' (input control plane for the transport including CC): refers to the information a ongestion control can obtain from a network (e.g.  TCP can estimate the latency, the loss rate and bottleneck). 'requirements' (input control plane for the transport including CC): refers to application requirements such as upper/lower rate bounds, periods of quiescence, or a priority. 'sending rate (or window)' (output control plane for the transport including CC): refers to the rate at which a congestion control decides to transmit packets, based on 'network measurements'. 'signaling about losses and/or repaired packets' (input control plane for the FEC): refers to the information a FEC server can obtain from a FEC client about the performance of the FEC solution as seen from the client. 'coding rate' (output control plane for the FEC): refers to the coding rate that is used by the FEC solution. 'source and/or repair packets' (data plane for both the FEC and the CC): refers to the packets that are transmitted.  There can only be source packets (if the coding rate is 0), repair packets (if the solution decides not to send the original source packets) or a mix of both. The inputs to FEC (packet to work upon, and signaling from the receiver about losses and/or repaired packets) are distinct from the inputs to CC.  The latter calculates a sending rate or window from network measurements, and it takes the packet to send as input, sometimes along with application requirements such as upper/lower rate bounds, periods of quiescence, or a priority. </ins> It is not clear that the ACK signals feeding into a congestion control algorithm are useful to FEC in their raw form, and vice versa - information about repaired blocks may be quite irrelevant to a CC  <ins> algorithm. </ins> 3.  <ins> This section discusses how FEC and CC can relate in different cases (FEC above the transport, FEC within the transport, FEC below the transport). </ins> 3.1.  fig:fec-above illustrates the FEC above the transport case. fig:fec-above presents an architecture where FEC is on top of the  <ins> transport.  The repair packets are sent within what the congestion </ins> window allows. The advantage of this approach is that the FEC does not contribute to adding congestion in the network.  <ins> This approach requires that the transport protocol does not include a reliability (e.g., based on lost packet retransmition).  This approach is relevant with UDP. </ins>  <ins> 3.2. </ins> fig:fec-in illustrates the FEC within the transport case. fig:fec-in presents an architecture where FEC is within the  <ins> transport.  The repair packets are sent within what the congestion </ins> window allows, such as in CTCP.  Examples of the solution would be  <ins> sending repair packets when there is no more data to transmit or preferably send repair packets instead of the following packets in the send buffer. </ins> The advantage of this approach is that it can enable conjoint optimization between the CC and the FEC.  Moreover, the transmission  <ins> of repair packets does not add congestion in potentially congested </ins> networks but help repair lost packets (such as tail losses). The drawback of this approach is that it may not result in much gains as opposed to classical CC retransmissions mechanisms and it costs  <ins> bandwidth that could have been exploited to transmit source packets. The coding ratio needs to be carefully designed. </ins>  <ins> 3.3. </ins> fig:fec-below illustrates the FEC below the transport case. fig:fec-below presents an architecture where FEC is below the  <ins> transport.  The repair packets are sent on top of what is allowed by </ins> the congestion control.  Examples of the solution could be adding a given percentage of the congestion window as supplementary packets or  <ins> sending a given amount of repair packets at a given rate.  The </ins> redundancy flow can be decorrelated from the congestion control that manages source packets: a secondary congestion control could be introduced underneath the FEC layer, such as in coupled congestion already congested networks.  The coding ratio needs to be carefully designed.  <ins> 4. </ins> This memo includes no request to IANA.  <ins> 5. FEC and CC schemes can contribute to DoS attacks.  This is not specific to this document. </ins>  <ins> More information TBD. </ins>


1. There are cases where deploying FEC coding improves the quality of <del> the transmission.  As an example, when a server transmits data to a client, it may take time for the server to detect tail losses (i.e. tail loss can refer to losses at the tail end of transactions).  This would impact the experience of applications using short flows. Another example are networks where non-congestion losses are persistent and prevent a sender from exploiting the link capacity. </del>  Coding is a reliability mechanism that is distinct and separated from the loss detection of congestion controls.  RFC5681 defines TCP as a fig:sep-channel presents the notations that will be used in this document and introduce the Congestion Control (CC) and Forward Erasure Correction (FEC) channels.  The Congestion Control channel <del> carries source packets (from a server to a client) and potential </del>  information signaling the packets that have been received (from the <del> client to the server).  The Forward Erasure Correction channel carries repair packets (from the server to the client) and a </del>  potential information signaling the packets that have been repaired <del> (from the client to the server).  It is worth pointing out that there are cases where these channels are not separated. </del>  Inside a host, the CC and FEC entities can be regarded as conceptually separate: fig:sep-entities provides more details than fig:sep-channel by <del> focusing on the server side.  Some elements are introduced: </del>  'network measurements' (input control plane for the transport including CC): refers to the information a ongestion control can decides to transmit packets, based on 'network measurements'. 'signaling about losses and/or repaired packets' (input control <del> plane for the FEC): refers to the information a FEC server can obtain from a FEC client about the performance of the FEC solution as seen from the client. </del>  'coding rate' (output control plane for the FEC): refers to the coding rate that is used by the FEC solution.

I propose some rewording and talk about sender and receiver instead of server and client. The sender is not always the server and it could be confusing to assume the sender as being a server (e.g. in ssh it is the inverse). Let me know if you disagree on this.
Au contraire, I disliked the client-server thing from the beginning and thought we should move it to sender-receiver some day! So thanks a LOT for doing this!
LGTM, thanks for this!

1. There are cases where deploying FEC coding improves the quality of  <ins> the transmission.  As an example, it may take time for the sender to detect tail losses (i.e. tail loss can refer to losses at the tail end of transactions).  This would impact the experience of applications using short flows.  Another example are networks where non-congestion losses are persistent and prevent a sender from exploiting the link capacity. </ins> Coding is a reliability mechanism that is distinct and separated from the loss detection of congestion controls.  RFC5681 defines TCP as a fig:sep-channel presents the notations that will be used in this document and introduce the Congestion Control (CC) and Forward Erasure Correction (FEC) channels.  The Congestion Control channel  <ins> carries source packets (from a sender to a receiver) and potential </ins> information signaling the packets that have been received (from the  <ins> receiver to the sender).  The Forward Erasure Correction channel carries repair packets (from the sender to the receiver) and a </ins> potential information signaling the packets that have been repaired  <ins> (from the receiver to the sender).  It is worth pointing out that there are cases where these channels are not separated. </ins> Inside a host, the CC and FEC entities can be regarded as conceptually separate: fig:sep-entities provides more details than fig:sep-channel by  <ins> focusing on the sender-side.  Some elements are introduced: </ins> 'network measurements' (input control plane for the transport including CC): refers to the information a ongestion control can decides to transmit packets, based on 'network measurements'. 'signaling about losses and/or repaired packets' (input control  <ins> plane for the FEC): refers to the information a FEC sender can obtain from a FEC receiver about the performance of the FEC solution as seen from the receiver. </ins> 'coding rate' (output control plane for the FEC): refers to the coding rate that is used by the FEC solution.


The proposed document considers FEC coding at the transport or application layer for an end-to-end unicast data transfer.  The typical application scenario that is considered in the current <del> version of the document is a client browsing the web.  This memo may be extended to cases with multiple paths. </del>  This document represents the collaborative work and consensus of the Coding for Efficient Network Communications Research Group (NWCRG); Inside a host, the CC and FEC entities can be regarded as conceptually separate: <del> fig:sep-entities provides more details than fig:sep-channel by focusing on the sender-side.  Some elements are introduced: </del>  'network information' (input control plane for the transport including CC): refers not only to the network information that is including CC): refers to the rate at which a congestion control decides to transmit packets, based on 'network information'. <del> 'signaling about losses and/or repaired packets' (input control plane for the FEC): refers to the information a FEC sender can obtain from a FEC receiver about the performance of the FEC solution as seen from the receiver. </del>  'coding rate' (output control plane for the FEC): refers to the coding rate that is used by the FEC solution. inputs to CC.  The latter calculates a sending rate or window from network information, and it takes the packet to send as input, sometimes along with application requirements such as upper/lower <del> rate bounds, periods of quiescence, or a priority. It is not clear that the ACK signals feeding into a congestion control algorithm are useful to FEC in their raw form, and vice versa - information about repaired blocks may be quite irrelevant to a CC algorithm. </del>  3. <del> This section discusses how FEC and CC can relate in different cases (FEC above the transport, FEC within the transport, FEC below the transport). </del> 3.1. <del> fig:fec-above illustrates the FEC above the transport case. </del>  <del> fig:fec-above presents an architecture where FEC is on top of the transport.  The repair packets are sent within what the congestion window allows. The advantage of this approach is that the FEC does not contribute to adding congestion in the network. </del>  While CC is in principle independent of other transport functions such as reliability, we note that CC is often embedded in reliable <del> transfer protocols (e.g.  TCP).  This approach requires that the </del>  transport protocol does not implement a fully reliable data transfer <del> service (e.g., based on lost packet retransmission).  UDP is an example of a protocol for which this approach is relevant. </del>  <del> 3.2. </del>  <del> fig:fec-in illustrates the FEC within the transport case. </del>  fig:fec-in presents an architecture where FEC is within the transport.  The repair packets are sent within what the congestion <del> window allows, such as in CTCP.  Examples of the solution would be sending repair packets when there is no more data to transmit or preferably send repair packets instead of the following packets in the send buffer. </del>  The advantage of this approach is that it can enable conjoint optimization between the CC and the FEC.  Moreover, the transmission bandwidth that could have been exploited to transmit source packets. The coding ratio needs to be carefully designed. <del> 3.3. </del>  <del> fig:fec-below illustrates the FEC below the transport case. </del>  fig:fec-below presents an architecture where FEC is below the transport.  The repair packets are sent on top of what is allowed by <del> the congestion control.  Examples of the solution could be adding a given percentage of the congestion window as supplementary packets or sending a given amount of repair packets at a given rate.  The redundancy flow can be decorrelated from the congestion control that manages source packets: a secondary congestion control could be introduced underneath the FEC layer.  The separate congestion control mechanisms could be made to work together while adhering to priorities, as in coupled congestion control for RTP media I-D.ietf- rmcat-coupled-cc.  Another possibility would be to exploit a lower than best-effort congestion control RFC6297 for repair packets. </del>  The advantage of this approach is that it can result in performance gains when there are persistent transmission losses along the path. already congested networks.  The coding ratio needs to be carefully designed. <del> 4. </del>  This memo includes no request to IANA. <del> 5. </del>  FEC and CC schemes can contribute to DoS attacks.  This is not specific to this document.

Looks good to me !
I see a major scoping problem in these sections. What does this document really aim at? It is said earlier: This document discusses how FEC and CC relate. Here we see architectural discussions on advantages/drawbacks instead. There is also the additional idea of how it relates to transport reliability mechanism which is (in theory) out of scope. I'd like to see more info about what concretely can be done (examples). The case of QUIC and what was already proposed here to let FEC and CC work together is pretty interesting. I also think the application (as data source) is potentially involved in the big picture: an application generating a real-time data flow is not the same as an application generating background traffic. In the first case we could end up with situations where it's just impossible to carry traffic by adding a certain redundancy, given the network conditions. It means that without adapting the data flow there is probably no solution. In the second case (which includes François's experiments with tail losses)
item 1: I think this is now addressed by careful phrasing in the para that mentions reliability, in PR . Item 2: I agree but I'd greatly prefer if we could postpone this to the next version. I think we should include multipath considerations too, so this will be quite a bit of work. Item 3: I agree... someone should write something :)
Item 1 addressed by PR Open for next submission I'd like to see more info about what concretely can be done (examples). The case of QUIC and what was already proposed here to let FEC and CC work together is pretty interesting. I also think the application (as data source) is potentially involved in the big picture: an application generating a real-time data flow is not the same as an application generating background traffic. In the first case we could end up with situations where it's just impossible to carry traffic by adding a certain redundancy, given the network conditions. It means that without adapting the data flow there is probably no solution. In the second case (which includes François's experiments with tail losses)
I am not sure I completely get the last item though. More specifically, I don't know what to say about this in this draft about the relations between coding and the congestion control. We could state that the application can provide information about importance or deadline concerning its data, but I think the optimal FEC pattern , with or without congestion, will likely depend on the application's requirements in the end ? Does anybody have a clearer idea on what we could state on this ? Let's discuss this here before writing ?
I think you're trying to find a solution, whereas we're only describing a situation. The latter is simpler: we can basically repeat what Vincent has told us: "an application generating a real-time data flow is not the same as an application generating background traffic. In the first case we could end up with situations where it's just impossible to carry traffic by adding a certain redundancy, given the network conditions. It means that without adapting the data flow there is probably no solution." Just point it out.
Looks good to me. Thank you !
The application requirements aspect have been covered by PR
The current version of hte draft seems to go for a more "server" side discussion. We should not forget to speak about the client.
I agree - but maybe delay this until the next version of the draft?
I have pushed some content with some more details on the server side solutions. I think we may be soon ready to either submit this version or had a small section on the receiver side. It can be very simple IMHO: " The receiver must not use CC channel signals to acknowledge the recovery of data packets with the coding schemes. The information on recovered packets must be reported on the FEC channel."
I agree with both approaches. I'd vote for shipping it as-is (better to have a more useful discussion on the receiver side - surely we can come up with something more for the next version of the draft?)
Thanks for the feedback - I would vote the same. Any other thoughts ?
This is done in PR
LGTM modulo a minor suggestion, but that was from a quick glance only... can't dedicate more time this round, sorry!

The proposed document considers FEC coding at the transport or application layer for an end-to-end unicast data transfer.  The typical application scenario that is considered in the current  <ins> version of the document is a client browsing the web or watching a live video.  This memo may be extended to cases with multiple paths. </ins> This document represents the collaborative work and consensus of the Coding for Efficient Network Communications Research Group (NWCRG); Inside a host, the CC and FEC entities can be regarded as conceptually separate:  <ins> fig:sep-entities-srv and fig:sep-entities-clt provide more details than fig:sep-channel.  Some elements are introduced: </ins> 'network information' (input control plane for the transport including CC): refers not only to the network information that is including CC): refers to the rate at which a congestion control decides to transmit packets, based on 'network information'.  <ins> 'signaling repaired packets' (input control plane for the FEC): refers to the information a FEC sender can obtain from a FEC receiver about the performance of the FEC solution as seen from the receiver. </ins> 'coding rate' (output control plane for the FEC): refers to the coding rate that is used by the FEC solution. inputs to CC.  The latter calculates a sending rate or window from network information, and it takes the packet to send as input, sometimes along with application requirements such as upper/lower  <ins> rate bounds, periods of quiescence, or a priority.  It is not clear that the ACK signals feeding into a congestion control algorithm are useful to FEC in their raw form, and vice versa - information about repaired blocks may be quite irrelevant to a CC algorithm. The choice of the adequate transport layer may be related to application requirements: In the case of an unreliable data transfer, the transport layer may implement a non-reliable transport layer (e.g.  UDP or DCCP RFC4340 or a partially reliable transport protocol such as SCTP with partial reliability RFC3758).  Depending on the amount of redundancy and network conditions, there could be cases where it ends up being impossible to carry traffic. In the case of a reliable data transfer, the transport layer may implement a retransmission mechanism to guarantee the reliability of the file transfer (e.g.  TCP).  Depending on how the FEC and CC functions are scheduled (FEC above CC, FEC in CC, FEC below CC), the impact of a reliable transport on the FEC reliability mechanisms is different. </ins> 3.  3.1.  <ins> fig:fec-above present an architecture where FEC is on top of the transport. </ins>  <ins> 3.2. </ins> While CC is in principle independent of other transport functions such as reliability, we note that CC is often embedded in reliable  <ins> transfer protocols (e.g.  TCP).  This approach may require that the </ins> transport protocol does not implement a fully reliable data transfer  <ins> service (e.g., based on lost packet retransmission), otherwise this could lead to spurious retransmissions.  UDP is an example of a protocol for which this approach is relevant. </ins>  <ins> The advantage of this approach is that the FEC does not contribute to adding congestion in the network.  When a congestion control is implementedd at the transport layer, the repair packets are sent within what the congestion window allows. </ins>  <ins> 4. 4.1. </ins> fig:fec-in presents an architecture where FEC is within the transport.  The repair packets are sent within what the congestion  <ins> window allows, such as in CTCP. 4.2. </ins> The advantage of this approach is that it can enable conjoint optimization between the CC and the FEC.  Moreover, the transmission bandwidth that could have been exploited to transmit source packets. The coding ratio needs to be carefully designed.  <ins> Examples of the solution would be sending repair packets when there is no more data to transmit or preferably send repair packets instead of the following packets in the send buffer. 5. </ins>  <ins> 5.1. </ins> fig:fec-below presents an architecture where FEC is below the transport.  The repair packets are sent on top of what is allowed by  <ins> the congestion control. 5.2. </ins> The advantage of this approach is that it can result in performance gains when there are persistent transmission losses along the path. already congested networks.  The coding ratio needs to be carefully designed.  <ins> Examples of the solution could be adding a given percentage of the congestion window as supplementary packets or sending a given amount of repair packets at a given rate.  The redundancy flow can be decorrelated from the congestion control that manages source packets: a secondary congestion control could be introduced underneath the FEC layer.  The separate congestion control mechanisms could be made to work together while adhering to priorities, as in coupled congestion control for RTP media I-D.ietf-rmcat-coupled-cc.  Another possibility would be to exploit a lower than best-effort congestion control RFC6297 for repair packets. 6. </ins> This memo includes no request to IANA.  <ins> 7. </ins> FEC and CC schemes can contribute to DoS attacks.  This is not specific to this document.


carries source packets from a sender to a receiver, and packets signaling information about the network (number of packets received vs. lost, ECN marks, etc.) from the receiver to the sender.  The <del> Forward Erasure Correction channel carries repair packets (from the </del>  sender to the receiver) and potential information signaling which packets have been repaired (from the receiver to the sender).  It is worth pointing out that there are cases where these channels are not including CC): refers to the rate at which a congestion control decides to transmit packets, based on 'network information'. <del> 'signaling repaired packets' (input control plane for the FEC): </del>  refers to the information a FEC sender can obtain from a FEC receiver about the performance of the FEC solution as seen from the receiver. 'coding rate' (output control plane for the FEC): refers to the coding rate that is used by the FEC solution. <del> 'source and/or repair packets' (data plane for both the FEC and the CC): refers to the packets that are transmitted.  There can only be source packets (if the coding rate is 0), repair packets (if the solution decides not to send the original source packets) or a mix of both. </del>  The inputs to FEC (packets to work upon, and signaling from the <del> receiver about losses and/or repaired packets) are distinct from the </del>  inputs to CC.  The latter calculates a sending rate or window from network information, and it takes the packet to send as input, sometimes along with application requirements such as upper/lower 3.2. <del> While CC is in principle independent of other transport functions such as reliability, we note that CC is often embedded in reliable transfer protocols (e.g.  TCP).  This approach may require that the transport protocol does not implement a fully reliable data transfer service (e.g., based on lost packet retransmission), otherwise this could lead to spurious retransmissions.  UDP is an example of a protocol for which this approach is relevant. </del>  <del> The advantage of this approach is that the FEC does not contribute to adding congestion in the network.  When a congestion control is implementedd at the transport layer, the repair packets are sent within what the congestion window allows. </del>  4. 4.1. fig:fec-in presents an architecture where FEC is within the <del> transport.  The repair packets are sent within what the congestion </del>  window allows, such as in CTCP. 4.2. The advantage of this approach is that it can enable conjoint optimization between the CC and the FEC.  Moreover, the transmission <del> of repair packets does not add congestion in potentially congested </del>  networks but helps repair lost packets (such as tail losses). <del> The drawback of this approach is that it may not result in much gains as opposed to classical retransmission mechanisms and it costs bandwidth that could have been exploited to transmit source packets. The coding ratio needs to be carefully designed. Examples of the solution would be sending repair packets when there is no more data to transmit or preferably send repair packets instead of the following packets in the send buffer. </del>  5. 5.1. <del> fig:fec-below presents an architecture where FEC is below the transport.  The repair packets are sent on top of what is allowed by the congestion control. </del>  5.2.  The advantage of this approach is that it can result in performance gains when there are persistent transmission losses along the path. designed. Examples of the solution could be adding a given percentage of the <del> congestion window as supplementary packets or sending a given amount of repair packets at a given rate.  The redundancy flow can be </del>  decorrelated from the congestion control that manages source packets: <del> a secondary congestion control could be introduced underneath the FEC layer.  The separate congestion control mechanisms could be made to work together while adhering to priorities, as in coupled congestion control for RTP media I-D.ietf-rmcat-coupled-cc.  Another possibility would be to exploit a lower than best-effort congestion control RFC6297 for repair packets. </del>  6. <del> This memo includes no request to IANA. </del>  7.  FEC and CC schemes can contribute to DoS attacks.  This is not specific to this document.

My 2 cents and it’s probably not worth more: I am following the evolution of this draft and while the issues are starting to be well covered this is RG document so while I approve of the changes it would be nice to highlight the research that is still needed. mjm Marie-José Montpetit, Ph.D. EMAIL
Regarding Marie-José's comment: I agree, and I think that we could approach this by trying to find at least one published example of research per category - ideally the "best" per category. This would show, wherever we find something, what the SoA is - research could focus on improving this SoA. Also, it would probably give us some holes - e.g., I don't think there's any much research on (e2e!) FEC "below the transport", especially using multiple CC instances as hinted on in the draft. These holes would let us identify missing research.
You should define what you mean by fairness. I remember endless discussions about fairness in NC with Bob Briscoe amongst others. Marie-José Montpetit, Ph.D. EMAIL
I have corrected most of the comments and issues. I have added two issues to discuss the discussion points opened by Marie-José. How about we merge this and have a new fresh start ? (from which it will be easier for you to create branches, etc.) I guess, looking at the amount of modifications, NAME : may you merge if OK with you?
Ah sorry, I see the comment now, asking me to merge - ok, then I click these 3 "commit"s and then I merge! Doing it now!
Emmanuel made a comment on partial reliability. Should we need a new subsection 6.X to discuss this ?
I don't get this: Emmanuel's comment was about partial order, not partial reliability. Quote: One important thing to consider is reordering and not only reliability alone, we could also consider Partial Order Connection (POC) and Partial Order Transport Service (see Paul Amer works). Spurious retransmission is not only linked to reliability but reordering.
If this issue ends up being about ordering, and not partial reliability: From francoismichel (copied over from a PR): There is something we do not mention here concerning the reliable transport that deliver data in-order : with a reliable transport, the data are delivered in order. I can see two possible cases: Above a single TCP connection: the repair packets won't ever be delivered to the applications if there is a loss occurring before them. They are thus useless as they are delivered once the loss is receovered. Above multiple streams (e.g. QUIC streams or one TCP connection for the source and one for the repair symbols): the repair symbols are less effective: if there is a hole in the data (ex: seq 1 and 3 are received but not 2),you will need to consider 3 a lost too as it won't be delivered to the upper layer.
An answer from me to this: there are transports that can deliver data out-of-order. SCTP can do that, and it's exposed in the interface of TAPS ( draft-ietf-taps-interface ). Multi-streaming is closely related to that: if you take out-of-order message delivery and just add stream identifiers to ensure that, per stream, messages are delivered in order, you have it. This means that FEC could perhaps be applied across streams in case of a multi-streaming transport. I think this just complicates the discussion, let's not go there in the draft.
Thank you for summing up the discussion here. I think we already quite an important number of new sections in this document - we may leave the topic on out-of-order packets on the side and focus on what we have at the moment.
Ok, but do we agree that this is about "partial ordering", not "partial reliability"? In this case, we should probably rename this issue to avoid further confusion.
Text proposed in PR
Should we consider the impact of cross trafic that uses congestion-controled transport protocol?
done in PR
There should be a discussion on the amount of redundancy of the FEC e.g. FEC above transport : including redundancy reduces goodput for large files transfers e.g. FEC in transport : including redundancy reduces goodput for large files transfers but can be adapted depending on the CWND size e.g. FEC below transport : including redundancy adds congestions without reducing goodput
From Dave Oran (IETF108): One thing it might be good to cover is how to couple increasing transmission rate with increasing/decreasgin coding rate. It may be interesting to use increasing coding rate as a "probe" to more safely increase transmission rate. Has this been looked at?
Done in PR
More related work, examples of FEC research work? Papers? This kind of thing? (I agree, just clarifying if this is what this issue is asking for)
Started in PR
The title of the document is "Coding and congestion control in transport" but we only present a "FEC Channel" that only carries repair packets and some signaling about the repaired packets. I like the idea of two separated channels a lot, but I think we also should integrate source packets as part of the FEC channel, and maybe rename the FEC channel to "Coding Channel" and "packets" to "symbols" when we talk about the coding channel ? Here is what I propose concretely: The CC channel stays unchanged, it carries packets The FEC channel becomes a Coding channel and carries symbols: Source Symbols and Repair Symbols. I think the approach there is more general: one network packet can contain several Source/Repair Symbols (similarly to the coding-for-quic draft). I think this helps to isolate both CC and FEC channels from each others as the FEC/Coding channel only plays with Symbols while the CC channel plays with packets. When I read the current design, it makes me think that the FEC channel needs the source packets from the CC channel to decode the received repair packets but this is shown nowhere in the graph. By considering different entities (packets VS symbols that are in fact a view on the sent/received data), we decorrelate more intuitively the FEC and the CC channels. As a consequence, we will only talk about symbols when it concerns the Coding channel and packets when it concerns the CC channel. What do you think ? I do not think this is an urging change by the way.
Hi, Coding channel refers to physical layer and when combined with the term symbol, it also refers to physical operations which are not those at packet level. We stated in RFC8406 that Generally speaking, a Packet is the unit of data that is sent in the Packet Erasure Channel, while a Symbol is the unit of data that is manipulated during the encoding and decoding operations. So we allow the use of symbol only when it corresponds to non yet transmitted data. As soon as transmitted, it becomes a packet. EL
Hello, symbol, it also refers to physical operations which are not those at packet level. Hmm, in that case, renaming the channel this way would make it more confusing I agree, sorry for that then :-) transmitted data. As soon as transmitted, it becomes a packet. I think the definition I proposed here still fits with the symbol definition you described though. A packet can contain more than 1 source/repair symbol(s) and the source symbols can be different than what is actually sent on the wire (e.g. source symbols can contain padding that is not actually sent on the wire or can contain metadata that are not sent on the wire either). So I still think that it matches more the definition of symbols than packets. All the real packets received by a protocol can be processed and transformed into source symbols before being passed to the FEC Channel. (sorry in advance if I have misconceptions on this matter)
Yes in that context it's fine and we stick to RFC8406.
Done in PR
DO: Fine, but you miss the opposite case: how many repair packets are you sending that turn out to be useless? It's a matter of measuring the usefulness of repair traffic. It gives an extra degree of freedom as you may want to play at the congestion control level or at the coding rate level. NK: Good point, that's the kind of topic we need to gather in the document. There are several corner cases that are very important. This point is about how you add coding inside congestion control. With this claim, we want to be generic.
I think this document ought to remove the confusion between coding and congestion control that folks seem to have. So: "It gives an extra degree of freedom as you may want to play at the congestion control level or at the coding rate level." sounds like it's based on this confusion. And: "This point is about how you add coding inside congestion control." doesn't seem right either.
Indeed - the introduction of the coding and congestion channels could help. I may need to be reflected in the rest of the document.
Maybe we could emphasize on the fact that coding is a reliability mechanism and is distinct and separated from the loss detection and congestion control mechanisms. If all we care about is bandwidth in a general unicast scenario, sending a-priori repair packets should not help more than Selective-Repeat ARQ. In the best case, it should perform as well as a SR-ARQ if the packet is lost (without additional gain), and will simply waste bandwidth compared to SR-ARQ if the packet is not lost. I can identify one case where doing FEC can help for bandwidth: when the buffer are full or very limited and cannot be in-order delivered to the application because of head-of-line blocking. Sending FEC could help to unblock the peer's receive buffer sooner. But this has nothing to do with congestion control.
I think this kind of discussion should be in the document.
I agree with NAME 's idea of unblocking the receiver. Also, in the other issue, he mentioned the big disadvantage (delay) coming from tail losses. These things should now be added to the document - now it's structured such that it could be added better, in the right place. Other thoughts that were brought up in discussions, and should go into the doc - both are about the FEC-below-transport case: 1) ECN: From NAME : There is more detail about possibilities to use ECN for FEC-below-transport in Emmanuel's presentation: URL (slide 4) 2) special environments: and we had this discussion in a PR related to FEC-below-transport: NAME wrote: and I answered: .... and about the FEC-above-transport case slide 2 reminds me that this is obviously useless if the transport repairs all losses, as with TCP! This is worth mentioning in the FEC-above-transport case - we could mention that unreliable congestion-controlled transports (e.g. RMCAT cc's, or DCCP) are better to consider here than fully reliable ones such as TCP.
I am unsure where this issue is going and what we can integrate in the draft then. I would say that identifying useless repair packets is the work of the reliability mechanism (there is a clear similarity with spurious retransmissions detection) and in my opinion has nothing to do with congestion control, so what can we integrate in the draft ?
I think this discussion is covered in PR
Thanks for the PR ! :-)There are two small suggestions that I made... not sure they're committed yet? Else just press "commit" 3 times please :)

carries source packets from a sender to a receiver, and packets signaling information about the network (number of packets received vs. lost, ECN marks, etc.) from the receiver to the sender.  The  <ins> Forward Erasure Correction channel carries repair symbols (from the </ins> sender to the receiver) and potential information signaling which packets have been repaired (from the receiver to the sender).  It is worth pointing out that there are cases where these channels are not including CC): refers to the rate at which a congestion control decides to transmit packets, based on 'network information'.  <ins> 'signaling repaired symbols' (input control plane for the FEC): </ins> refers to the information a FEC sender can obtain from a FEC receiver about the performance of the FEC solution as seen from the receiver. 'coding rate' (output control plane for the FEC): refers to the coding rate that is used by the FEC solution.  <ins> 'source and/or repair symbols' (data plane for both the FEC and the CC): refers to the data that is transmitted.  The sender can decide to send source symbols only (meaning that the coding rate is 0), repair symbols only (if the solution decides not to send the original source packets) or a mix of both. </ins> The inputs to FEC (packets to work upon, and signaling from the  <ins> receiver about losses and/or repaired symbols) are distinct from the </ins> inputs to CC.  The latter calculates a sending rate or window from network information, and it takes the packet to send as input, sometimes along with application requirements such as upper/lower 3.2.  <ins> The advantage of this approach is that the FEC overhead does not contribute to congestion in the network.  When congestion control is implemented at the transport layer, the repair symbols are sent within what the congestion window allows.  This approach can result in improved quality of experience for latency sensitive and non reliable applications. </ins>  <ins> This approach requires that the transport protocol does not implement a reliable data transfer service (e.g., based on lost packet retransmission).  UDP is an example of a protocol for which this approach is relevant.  For reliable transfers, the usage of coding does not guarantee better performance and would mainly reduce goodput for large files transfers. This discussion section is extended in sec:fairness. </ins> 4. 4.1. fig:fec-in presents an architecture where FEC is within the  <ins> transport.  The repair symbols are sent within what the congestion </ins> window allows, such as in CTCP. 4.2. The advantage of this approach is that it can enable conjoint optimization between the CC and the FEC.  Moreover, the transmission  <ins> of repair symbols does not add congestion in potentially congested </ins> networks but helps repair lost packets (such as tail losses).  <ins> For reliable transfers, including redundancy reduces goodput for large files transfers but the amount of repaired symbols can be adapted, e.g. depending on the congestion window size.  There is a trade-off between the cost in bandwidth that could have been exploited to transmit source packets and the advantages that are brought by transmitting repair symbols (e.g. unlocking the receive buffer if this is limiting).  The coding ratio needs to be carefully designed.  For small files, sending repair symbols when there is no more data to transmit or preferably sending repair symbols instead of the following packets in the send buffer could help to reduce the transfer time.  This is because it could avoid a silent period between the transmission of the last packet and firing the retransmission of lost packets. This discussion section is extended in sec:fairness. </ins> 5. 5.1.  <ins> fig:fec-below presents an architecture where FEC is applied end-to- end below the transport layer, but above the link layer.  Note that it is common to apply FEC at the link layer, in which it contributes to the total capacity that a link exposes to upper layers.  This application of FEC is out of scope of this document.  The repair symbols are sent on top of what is allowed by the congestion control. </ins> 5.2. <ins> In this case, including redundancy adds congestion without reducing goodput but leads to potential fairness issues.  Indeed, the effective bitrate is higher than the CC's computed fair share due to the sending of repair symbols and the losses are hidden to the transport which may cause a problem for loss-based congestion controls, but is not a problem for delay-based ones. </ins> The advantage of this approach is that it can result in performance gains when there are persistent transmission losses along the path. designed. Examples of the solution could be adding a given percentage of the  <ins> congestion window as supplementary symbols or sending a given amount of repair symbols at a given rate.  The redundancy flow can be </ins> decorrelated from the congestion control that manages source packets:  <ins> a separate congestion control could be introduced to manage the amount of repaired packets to transmit on the FEC channel.  The separate congestion control instances could be made to work together while adhering to priorities, as in coupled congestion control for RTP media RFC8699 in case all traffic can be assumed to take the same path, or otherwise with a multipath congestion window coupling mechanism as in Multipath TCP RFC6356.  Another possibility would be to exploit a lower than best-effort congestion control RFC6297 for repair symbols. This discussion section is extended in sec:fairness. </ins> 6.  <ins> The objective of this section is to further detail some aspects that have been expressed in previous discussion subsections. 6.1. The objective of this section is to further discuss how adding coded flows impacts on non-coded flows when they share the same bottleneck. This section assumes that the non-coded flows respond to congestion signals from the network. 6.1.1. The addition of coding within the flow do not impact on how the coded and non-coded flows interact.  This interaction would mainly depend on the congestion controls embedded in each host. 6.1.2. The addition of coding within the flow may impact on the congestion control mechanism and hide congestion losses.  Specific interaction between congestion controls and coding schemes can be proposed (see subsec:cc-recov-interaction, subsec:cc-nc-interaction and subsec:cc- useless-interaction).  If no specific interaction is introduced, the coding scheme may hide congestion losses to the congestion controller and the description of subsub:fec_below may apply. 6.1.3. In this case, the coding scheme may hide congestion losses to the congestion controller.  There are cases where this can drastically reduce the goodput of the non-coded flows.  Depending on the congestion control, it may be possible to indicate that a packet has been recovered, e.g. using ECN flags, to reduce the impact on the non-coded flows (see subsec:cc-recov-inter-below). 6.2. The objective of this subsection is to destribe potential interactions between the congestion control and the recovered symbols. 6.2.1. The congestion control may not be able to differentiate repair symbols from actual source packets.  The relevance of adding coding at the application layer is related to the needs of the application. For real-time applications, this approach may reduce the amount of retransmission.  The usage of a non-reliable transport is more adequate in this case. 6.2.2. If the two FEC and CC channels are decoupled, the endpoint may exploit specific protocol for each of the channel.  The channels may be coupled and one single protocol may be exploited.  In both cases, the receiver can differenciate source packets and repair symbols. The receiver may indicate the amount of source packets that have been received and the amount of repaired symbols that were actually useful in the recovery process of packets. QUIC may require a specific type of frame to transport such information between receiver and sender. 6.2.3. The congestion control may not know what is going on in the network underneath and whether a coding scheme is introduced or not.  The congestion control may behave as if no coding sheme is introduced. The only way for a coding channel to indicate that symbols have been recovered is to exploit existing signaling.  An example would be a receiver to indicate that a packet has been recovered by using ECN signals of TCP. 6.3. This section discuss to what extent interaction between the congestion control and the coding rates are possible. 6.3.1. The coding rate applied at the application layer mainly depends on the available capacity given by the congestion control underneath. Adapting the coding rate to the minimum required data rate of the application may help to reduce packet losses and improve quality of experience. 6.3.2. In this case, there is an important flexibility in the trade-off between reducing goodput, helping recovering from transmission and congestion losses, being fair in respect with non-coded flows.  Such as explained in subsec:cc-recov-inter-in, the receiver may be able to indicate the sender the amount of packets that have been received or recovered.  The sender may exploit this information to tune the coding ratio.  As one example of flexibility of this case, coupling an increasing transmission rate with an increasing or decreasing coding rate could be envisioned.  A server may use increasing coding rate as a probe of the channel capacity and adapt the transmission rate of the congestion control. 6.3.3. The coding rate in this case can be tuned depending on the amount of recovered symbols and the rate of which data are transmitted by the sender.  The coding scheme is not aware of the congestion control implementation, making it hard for the coding scheme to apply the relevant coding rate. 6.4. There are cases where useless repaired symbols may be transmitted. These have an impact on the network load and may reduce the goodput of the flow without concrete gains. 6.4.1. In this case, the discussion depends on application needs.  The only case where adding useless repaired symbols does not result in reduced goodput is when the application needs a limited amount of goodput (e.g.  VoIP traffic).  In this case, the useless repaired symbols would only have an impact on the amount of data generated in the network. 6.4.2. The sender may be able to exploit the information given by the receiver to reduce the amount of useless repair packets and the resulting goodput reduction. 6.4.3. In this case, the useless repaired symbols only have an impact on the load of the network without actual gain for the coded flow. </ins> 7. <ins> This memo includes no request to IANA. 8. </ins> FEC and CC schemes can contribute to DoS attacks.  This is not specific to this document.


amount of redundancy to add, as a function of the transport layer protocol and application requirements. <del> For the FEC within transport case, the addition of coding may hide congestion losses and proposing methods to signal such events to end point could be propose. </del>  For the FEC below transport case, there are opportunities for introducing interaction between congestion control and coding schemes proposing and comparing FEC coding solutions in communication systems.  As one example, this draft proposes discussions on the impact of the proposed FEC solution on the congestion control, <del> especially loss-based congestion control such as CUBIC. </del>  8.

Added some precisions on the research section. When authors improve throughput by hiding the congestion signal, I propose to compare using FEC to hide the congestion signal with replacing the current congestion control by one that similarly ignores the loss signal without using any FEC. Concerning the solutions proposing to disambiguate the ACKs from the recoveries, I cited QUIC-FEC. We should also cite TENTET. NAME do you have a reference of an article about TENTET ? Do not hesitate to suggest changes before merging.
That looks very fine to me! We may just be missing the reference to tentet before merging the branch

amount of redundancy to add, as a function of the transport layer protocol and application requirements.  <ins> For the FEC within transport case, recovering lost symbols may hide congestion losses to the congestion control.  Some existing solutions already propose to disambiguate acked packets from rebuilt packets QUIC-FEC.  New signalling methods and FEC-recovery-aware congestion controls could be proposed. </ins> For the FEC below transport case, there are opportunities for introducing interaction between congestion control and coding schemes proposing and comparing FEC coding solutions in communication systems.  As one example, this draft proposes discussions on the impact of the proposed FEC solution on the congestion control,  <ins> especially loss-based congestion control such as CUBIC.  When a research work aims at improving the throughput by hiding the packet loss signal from the congestion control, the authors should 1) discuss the advantages of using the proposed FEC solution compared to replacing the congestion control by one that ignores a portion of the encountered losses, 2) critically discuss the impact of hiding packet loss from the congestion control mechanism. </ins> 8.


control, it may be possible to signal to the congestion control mechanism that there was congestion (loss) even when a packet has been recovered, e.g. using ECN, to reduce the impact on the non-coded <del> flows (see subsec:cc-recov-inter-below). </del>  6.3. recovered is to exploit existing signaling that is understood by the congestion control mechanism.  An example would be to indicate to a TCP sender that a packet has been recovered (i.e., congestion has <del> occurred), by using ECN signaling. </del>  6.4.

There are two possibilities either: "Point-to-point transmetting method based on the use of an erasure coding scheme and TCP/IP protocol", ISAE-SUPAERO (2019) EP 3 432 500 A1, WO 2019015931 (A1). URL "On the joint use of TCP and Network Coding", Emmanuel Lochin, IETF'100 NWCRG session, November 2017. URL
Since this is IPR protected content I gather the IPR declaration is up to date?
Yes Le mer. 28 oct. 2020 à 10:40, patibulon a écrit :
done in PR

control, it may be possible to signal to the congestion control mechanism that there was congestion (loss) even when a packet has been recovered, e.g. using ECN, to reduce the impact on the non-coded  <ins> flows (see subsec:cc-recov-inter-below and TENTET). </ins> 6.3. recovered is to exploit existing signaling that is understood by the congestion control mechanism.  An example would be to indicate to a TCP sender that a packet has been recovered (i.e., congestion has  <ins> occurred), by using ECN signaling TENTET. </ins> 6.4.


application requirements: In the case of an unreliable data transfer, the transport layer <del> may implement a non-reliable transport layer (e.g.  UDP or DCCP </del>  RFC4340 or a partially reliable transport protocol such as SCTP with partial reliability RFC3758).  Depending on the amount of redundancy and network conditions, there could be cases where it repair lost packets (such as tail losses). For reliable transfers, including redundancy reduces goodput for <del> large file transfers but the amount of repaired symbols can be adapted, e.g. depending on the congestion window size.  There is a trade-off between the cost in capacity used to transmit source packets and the benefits brought out by transmitting repair symbols (e.g. unlocking the receive buffer if this is limiting).  The coding ratio needs to be carefully designed.  For small files, sending repair symbols when there is no more data to transmit could help to reduce the transfer time.  In general, sending repair symbols could avoid a silent period between the transmission of the last packet in the send buffer and 1) firing the retransmission of lost packets, or 2) the transmission of new packets. </del>  This discussion section is extended in sec:fairness. coupled and one single protocol may be exploited.  In both cases, the receiver can differentiate source packets and repair symbols.  The receiver may indicate both the number of source packets received and <del> repaired symbols that were actually useful in the recovery process of </del>  packets. <del> QUIC may require a specific type of frame to transport such information between receiver and sender. </del> 6.3.3. The congestion control may not know what is going on in the network 6.5. <del> There are cases where useless repaired symbols may be transmitted. </del>  These impact on the network load and may reduce the goodput of the flow without concrete gains. 6.5.1. In this case, the discussion depends on application needs.  The only <del> case where adding useless repaired symbols does not result in reduced </del>  goodput is when the application needs a limited amount of goodput <del> (e.g., VoIP traffic).  In this case, the useless repaired symbols would only impact the amount of data generated in the network. </del>  6.5.2. The sender may exploit the information given by the receiver to <del> reduce the number of useless repaired symbols and the resulting goodput reduction. </del>  6.5.3. <del> In this case, the useless repaired symbols only impact the load of the network without actual gain for the coded flow. </del>  7.

Small changes, nothing urging, should be easy to merge if you are ok with it :-)

application requirements: In the case of an unreliable data transfer, the transport layer  <ins> may provide a non-reliable transport service (e.g.  UDP or DCCP </ins> RFC4340 or a partially reliable transport protocol such as SCTP with partial reliability RFC3758).  Depending on the amount of redundancy and network conditions, there could be cases where it repair lost packets (such as tail losses). For reliable transfers, including redundancy reduces goodput for  <ins> large file transfers but the amount of repair symbols can be adapted, e.g. depending on the congestion window size.  There is a trade-off between the cost in capacity used to transmit source packets and the benefits brought out by transmitting repair symbols (e.g. unlocking the receive buffer if this is limiting).  The coding ratio needs to be carefully designed.  For small files, sending repair symbols when there is no more data to transmit could help to reduce the transfer time.  In general, sending repair symbols could avoid a silent period between the transmission of the last packet in the send buffer and 1) firing the retransmission of lost packets, or 2) the transmission of new packets. </ins> This discussion section is extended in sec:fairness. coupled and one single protocol may be exploited.  In both cases, the receiver can differentiate source packets and repair symbols.  The receiver may indicate both the number of source packets received and  <ins> repair symbols that were actually useful in the recovery process of </ins> packets.  6.3.3. The congestion control may not know what is going on in the network 6.5.  <ins> There are cases where useless repair symbols may be transmitted. </ins> These impact on the network load and may reduce the goodput of the flow without concrete gains. 6.5.1. In this case, the discussion depends on application needs.  The only  <ins> case where adding useless repair symbols does not result in reduced </ins> goodput is when the application needs a limited amount of goodput  <ins> (e.g., VoIP traffic).  In this case, the useless repair symbols would only impact the amount of data generated in the network. </ins> 6.5.2. The sender may exploit the information given by the receiver to  <ins> reduce the number of useless repair symbols and the resulting goodput reduction. </ins> 6.5.3.  <ins> In this case, the useless repair symbols only impact the load of the network without actual gain for the coded flow. </ins> 7.