7.8. <del> When an ACK frame is received that establishes loss of all in-flight packets sent over a long enough period of time, the network is considered to be experiencing persistent congestion.  Commonly, this can be established by consecutive PTOs, but since the PTO timer is reset when a new ack-eliciting packet is sent, an explicit duration must be used to account for those cases where PTOs do not occur or are substantially delayed.  The rationale for this threshold is to enable a sender to use initial PTOs for aggressive probing, as TCP does with Tail Loss Probe (TLP; see RACK), before establishing persistent congestion, as TCP does with a Retransmission Timeout (RTO; see RFC5681).  The RECOMMENDED value for kPersistentCongestionThreshold is 3, which is approximately equivalent to two TLPs before an RTO in TCP. </del>  <del> The persistent congestion period SHOULD NOT start until there is at least one RTT sample.  Prior to an RTT sample, the duration cannot be correctly calculated.  Waiting for one RTT sample also avoids spuriously declaring persistent congestion when the initial RTT is larger than the actual RTT. </del>  <del> This duration is computed as follows: </del>  <del> Unlike the PTO computation in pto, persistent congestion includes the </del>  max_ack_delay irrespective of the packet number spaces in which losses are established. <del> The following example illustrates how persistent congestion can be established.  Assume: </del>  Consider the following sequence of events: packet 9 is received at t = 12.2. The congestion period is calculated as the time between the oldest <del> and newest lost packets: 8 - 1 = 7.  The duration for establishing persistent congestion is: 2 * 3 = 6.  Because the threshold was reached and because none of the packets between the oldest and the newest lost packets were acknowledged, the network is considered to have experienced persistent congestion. While this example shows the occurrence of PTOs, they are not required for persistent congestion to be established. When persistent congestion is established, the sender's congestion window MUST be reduced to the minimum congestion window (kMinimumWindow).  This response of collapsing the congestion window on persistent congestion is functionally similar to a sender's response on a Retransmission Timeout (RTO) in TCP (RFC5681) after Tail Loss Probes (TLP; see RACK). </del>  7.9.

I started with trying to do small fixes, but quickly found that restructuring a bit and clarifying the details of what the sender was precisely supposed to do was the better than trying to be terse. This adds a couple of normatives, which may be unnecessary, and I'm happy to take suggestions on those.
It would be necessary to use much resource to implement the detection of persistent congestion described here because of "across all packet spaces". So, let me ask a basic question. What is the motivation to define persistent congestion across all packet spaces? Isn't it good enough to limit persistent congestion only to 1-RTT?
NAME I'm not sure if the amount of state an endpoint needs to retain changes depending on if persistent congestion is a connection-wide event. FWIW, we had some discussion regarding the complexity in , and my take is URL
I have implemented this algorithm based on the pseudocode in . As I said in URL, it was not difficult for Haskell quic to implement this since the data structure was able to be reused. However, I'm not sure if it is easy to implement this algorithm for all implementations and benefits outweighs its complexity.
NAME let's merge this one. Then we can talk about the pseudocode in .
This resolves a design issue, so flagging it so that it doesn't get merged accidentally.
NAME : This is part of the consensus call, so we need to wait for that to complete.
We resolved with , but I realized that it has a problem. The text and the pseudo-code don't agree. The text says: Since the starts with a packet that was sent in the past, this statement means that persistent congestion only includes packets that were sent after an RTT sample was taken. The pseudo code says: This says that the persistent congestion can only be established on the second (or later) ack that yields an RTT sample. This allows for packets that were sent before the first RTT sample was taken to be included in the persistent congestion period. I had convinced myself that the pseudo-code was adequate, but I missed the fact that the text was not aligned.
I see two options. Option 1: Implementing what the text says will require a new variable that records the next PN when the first RTT sample is taken. Option 2: Changing the text to match the pseudo-code will look something like this:
An example might help: 1: Pkt 10, 11, 12 sent 2: PTO, Pkt 13, 14 sent 3: PTO, Pkt 15, 16 sent 4: Ack 10 received. 5: Ack 16 received. At 5, the pseudo-code would declare persistent congestion if . The text however wouldn't, since there was no RTT sample during the persistent congestion period, t1 to t3.
Option 1 and 2 have a behavior difference with the following implication: With option 2, a sender might mark persistent congestion with fewer PTOs than if the sender had a more accurate RTT estimate. This is arguably not a significant concern, since: the sender will still have sent at least two packets in that duration and marked them lost, since you need two ack-eliciting packets to be lost to declare persistent congestion; and the number of packets sent is also sensitive to how the application sends, and a sender cannot rely on that anyway. Option 2 has the benefit that it is simpler to implement. On balance, I think Option 2 is better, though it does make this a design change.
Thanks for the example, I was unclear on the issue up until then, though I think a slightly stronger example might be: Ack 14 received, since that's only a single PTO. I think 1 aligns more with fully fixing the original issue, since it seems like your example is a case we agreed we would not want to declare as persistent congestion in . That being said, it makes the pseudocode more complex. Persistent congestion is a new concept, so I suspect implementations will vary anyway.
I agree that making the change to the pseudo-code seems to be the more correct solution. That being said, I'm not planning to implement persistent congestion detection during the early phases of the handshake (but only after confirmation of the handshake), since I don't see clear benefits of the added complexity of cross-packet-number-space persistent congestion detection.
NAME I personally think that's fine, though I'll note that TCP is less aggressive than that, which is why I think we should describe this as something that covers the handshake as well.
Given what I've heard, I think this is heading in an editorial direction(fix the pseudocode), though that needs a PR.
Yes, I'm not wedded to the design change, and am happy for this to be an editorial one. I'll write up a PR.
Should this become proposal-ready?
NAME : Since this is a pseudo-code issue, it should be an editorial change (the pseudo code is in the appendix). will fix this issue, but that is waiting on , which is included in the latest consensus call.
Persistent congestion has to start with an ack-eliciting packet. The section on persistent congestion isn't very clear about this point. This is because a non-ack-eliciting packet won't be followed up with any probes. It is therefore possible to send one packet and then not send anything else for an extended period. After this, the loss of just the next packet will cause persistent congestion. Note that persistent congestion can be caused by the loss of just two packets after multiple PTOs have occurred, but the common case is that you have at least three packet, or four if you send two packets on PTO.
Would it make sense to change the persistent congestion condition to require at least 3 packets to be lost, in addition to the time threshold that we already have?
FWIW we have had some discussion around this in , and it seemed to me that there were preference to not do that.
I implemented this as requiring ack-eliciting on the basis that once you are two PTOs out, it's not that bad that you hit persistent congestion, but I wouldn't object to that.
I can at least change this to ack-eliciting -- that's a simple editorial change. That was the intent anyway. If we want to change the way persistent congestion is established, that's a bigger issue, so let's separate that issue from this one.
My understanding is that it is not, but it's unclear.
It can't be, since the congestion window is a global property, and declaring persistent congestion results in a collapse of the congestion window.
It shouldn't be, because congestion is a network/connection-level property. That said, NAME and I were discussing this offline, and I'm now coming to terms with the fact that it's hard to do this globally across packet number spaces. Basically we have no current model to track packet transmissions across packet number spaces, and implementations do not typically do this either. I think this is hard to do across PN spaces... or at least, I've not been able to figure out a reliable way to do it. Based on what I've been able to come up with so far, I suspect we might want to disable persistent congestion until the handshake is confirmed. That is a departure from TCP, but I believe that is still ok. In practice, we have a congestion window reduction that occurs on losses in this period, and since the handshake will involve only a few packets in flight, losses should cause the cwnd to reduce to a small enough value. Thoughts?
NAME I think that it is actually easy; see URL
(NAME Could you move the discussion on that from the PR to this issue?)
NAME Sure. The following is what was written in URL One trivial way of implementing is as follows: Retain the entries in sentmap (the list of packets being sent) for at least 3 PTO. When receiving an acknowledgement, mark a hole in the sentmap for the packet being acked. When a loss is detected in a packet number space, check if that loss spans across more than 3 PTO, by traversing the sentmap. Then, if did, check the sentmap of other packet number spaces (by traversing through those sentmaps) to see if any ACK-eliciting packet sent in that period has been acked. This approach might sound too trivial, as it is O(N) where N is the number of packets being lost (or in case there are multiple PN spaces in action, the number of inflight packets in other PN spaces). But in practice, I think this approach is sufficient in terms of performance, because N would be like 3 for the current packet number space, and because we would not be sending that many packets on Initial and Handshake packet number spaces.
In , NAME states: It does not require additional state. In QUIC, there is an expectation that packet numbers would be used sequentially, starting from zero. Therefore, a "gap" can be detected when iterating through the sentmap and if the packet number increases (or decreases) by more than one. Note that endpoints are recommended to treat receipt of an acknowledgment for a packet it did not send as a connection error of type PROTOCOLVIOLATION, if it is able to detect the condition_ (). Therefore, when intentionally skipping a packet number, it is a good idea to have that recorded in the sentmap, and close the connection when the peer acks that skipped packet number. To paraphrase, it is possible determine if a packet has been acked, by looking for a gap between the packet numbers within the list of unacknowledged packets.
I don't think this is trivial, quite the opposite actually. A "hole" would be extra state (note that just the absence of a packet at a certain PN does not constitute a hole, since you could've skipped that PN to mitigate the optimistic ACK attack), and that additional state has to be cleaned up. In my implementation, when a packet is acknowledged, I know I can delete it and get rid of all state associated with it. Deviating from this, just to be able to do do persistent congestion detection across packet number spaces, seems like a lot of hassle to me. As NAME described, any packet loss during the handshake will half the congestion window, and with , it will even reduce the congestion window to half the bytes in flight (which will commonly be less than the cwnd during the handshake). Simple Reno-style congestion control will therefore lead to behavior that is very similar to actually doing persistent congestion detection across packet number spaces, without all the additional complexity.
There was a comment reorder. NAME I think your question has been answered in my previous comment. NAME Going back to the spec discussion, even if it is the case that some implementations might have difficulty in implementing persistent congestion detection covering multiple packet number spaces, that alone does not mean that our recommendation should deviate from what TCP does. I would very much prefer phrasing the discussion around if we should have a MAY allowing endpoints skip persistent congestion detection during the handshake.
There are different ways to implement packet number skipping for optimistic ACK defense, which don't involve creating fake packets in the sentmap. Those ways would make it impossible to use the "hole method" without creating additional state for already acknowledged packets. No matter what we do, our recommendation will deviate from what TCP does. Going back to the example you posted in URL, the acknowledgement of the Handshake packet would already trigger an RTO in TCP. In QUIC, we can't do that, because the packets live in different packet number spaces, and an ACK can only ever declare loss in its own packet number space. So the question cannot be if we want to deviate from what TCP does. We already do, and there's no way around it. Instead, we need to ask how much additional complexity we want to introduce to handle this corner case. I'd argue that the benefit is extremely limited, and we should therefore pick the simplest solution: only run persistent congestion detection for packets sent after handshake confirmation.
For some of the things, we have to deviate from TCP, because there is no way around it. But for this particular case, it is easy for some if not most of the QUIC stacks. Then, there's no ground to "recommend" something that deviates from TCP.
fixes this by clarifying that persistent congestion is across PN spaces. Since the resolution in introduces normative language, I would mark this as design.
A sender sends a tail of 10 packets, packet number 1 to 10. It now receives an acknowledgement for packets 5 and 10, which establishes the loss of all other packets. It's not clear how to perform the check for persistent congestion now. Several options come to mind: check if the period between 6 and 9 is longer than 3PTO check if either the period between packet 1 and 4 or the period between packet 6 and 9 is longer than 3PTO The ACK doesn't establish loss of all tail packets, so there's no persistent congestion. In an offline discussion, both NAME and NAME expressed preferences for different options here. I'm not sure what the right thing to do is here, but at least the draft should be clearer on what exactly establishes persistent congestion and what doesn't.
Consider the case where the sender receives an acknowledgement for packets from 5 to 10, and the period between 1 and 4 was longer than 3*PTO. IIUC, this is clearly a persistent congestion. The original example is strictly worse than that, and I think it would make sense to declare persistent congestion in the case too. In other words, I think that 3 is incorrect. That said, I do not have a strong opinion regarding if we should specifically recommend either of 1 or 2, considering that the logic (or the time threshold) being used to detect persistent congestion is merely a recommendation.
FWIW, we had some discussion regarding the design rationale behind persistent congestion in , and it seemed to me that it was mostly about the amount of time the sender cannot make any progress.
seems also related
I think we should keep it simple, which IMO is do nothing. Or if we want to add a recommendation, we can say that you start with the largest lost packet and try to detect Persistent congestion until either you have found 1 occurrence or reached the end of list.
This is fixed in , and is an editorial fix.
This is much clearer, thanks.One more suggestion

7.8.  <ins> When a sender establishes loss of all in-flight packets sent over a long enough duration, the network is considered to be experiencing persistent congestion. </ins>  <ins> 7.8.1. </ins>  <ins> The persistent congestion duration is computed as follows: </ins>  <ins> Unlike the PTO computation in pto, this duration includes the </ins> max_ack_delay irrespective of the packet number spaces in which losses are established.  <ins> This duration allows a sender to send as many packets before establishing persistent congestion, including some in response to PTO expiration, as TCP does with Tail Loss Probes (RACK) and a Retransmission Timeout (RFC5681). The RECOMMENDED value for kPersistentCongestionThreshold is 3, which is approximately equivalent to two TLPs before an RTO in TCP. This design does not use consecutive PTO events to establish persistent congestion, since application patterns impact PTO expirations.  For example, a sender that sends small amounts of data with silence periods between them restarts the PTO timer every time it sends, potentially preventing the PTO timer from expiring for a long period of time, even when no acknowledgments are being received. The use of a duration enables a sender to establish persistent congestion without depending on PTO expiration. 7.8.2. A sender establishes persistent congestion on receiving an acknowledgement if at least two ack-eliciting packets are declared lost, and: all packets, across all packet number spaces, sent between these two send times are declared lost; the duration between the send times of these two packets exceeds the persistent congestion duration (pc-duration); and a prior RTT sample existed when both packets were sent. The persistent congestion period SHOULD NOT start until there is at least one RTT sample.  Before the first RTT sample, a sender arms its PTO timer based on the initial RTT (pto-handshake), which could be substantially larger than the actual RTT.  Requiring a prior RTT sample prevents a sender from establishing persistent congestion with potentially too few probes. Since network congestion is not affected by packet number spaces, persistent congestion SHOULD consider packets sent across packet number spaces.  A sender that does not have state for all packet number spaces or an implementation that cannot compare send times across packet number spaces MAY use state for just the packet number space that was acknowledged. When persistent congestion is declared, the sender's congestion window MUST be reduced to the minimum congestion window (kMinimumWindow), similar to a TCP sender's response on an RTO (RFC5681). 7.8.3. The following example illustrates how a sender might establish persistent congestion.  Assume: </ins> Consider the following sequence of events: packet 9 is received at t = 12.2. The congestion period is calculated as the time between the oldest  <ins> and newest lost packets: 8 - 1 = 7.  The persistent congestion duration is: 2 * 3 = 6.  Because the threshold was reached and because none of the packets between the oldest and the newest lost packets were acknowledged, the network is considered to have experienced persistent congestion. While this example shows PTO expiration, they are not required for persistent congestion to be established. </ins> 7.9.


Upon receipt of a GOAWAY frame, if the client has already sent requests with a Stream ID greater than or equal to the identifier <del> received in a GOAWAY frame, those requests will not be processed. Clients can safely retry unprocessed requests on a different connection.  A client that is unable to retry requests loses all requests that are in flight when the server closes the connection. </del>  Requests on Stream IDs less than the Stream ID in a GOAWAY frame from the server might have been processed; their status cannot be indicated ID if these requests were not processed. If a server receives a GOAWAY frame after having promised pushes <del> with a Push ID greater than or equal to the identifier received in a GOAWAY frame, those pushes will not be accepted. </del>  Servers SHOULD send a GOAWAY frame when the closing of a connection is known in advance, even if the advance notice is small, so that the

Ambiguous description with for user agents in : It'd be better to remove or clarify rather than ambiguously allowing user agents with . A bit misleading expression with in This sentence looks to readers like that clients MAY send same requests on another connection even when the server keeps canceling or something regardless of clients' requests. There should be additional sentences no matter if it's the expected behavior or not. Some phrases in look verbose and confusingThe link in is wrong: should be replaced with .
These are separate issues, so let's look at them separately: This is deliberately vague, and the reason is explicitly stated. They don't affect the operation of the protocol, but user agents are allowed to make use of the associations in implementation-specific ways. This reflects HTTP/2, where some browsers use the association and some do not. Here again, I'm not seeing the problem. The requests have been canceled -- effectively "unsent" on this connection. If the client still needs to make those requests, they can't send them on this connection, but they MAY open a new connection. "received in a" -> "contained in the" would help here, unless you had a suggestion for making it tighter. The final issue is unrelated to these drafts, and is something the Tools server does. If you see an incorrect reference in the text draft or Editor's Copy HTML, please do point them out.

Upon receipt of a GOAWAY frame, if the client has already sent requests with a Stream ID greater than or equal to the identifier  <ins> contained in the GOAWAY frame, those requests will not be processed.  Clients can safely retry unprocessed requests on a different connection.  A client that is unable to retry requests loses all requests that are in flight when the server closes the connection. </ins> Requests on Stream IDs less than the Stream ID in a GOAWAY frame from the server might have been processed; their status cannot be indicated ID if these requests were not processed. If a server receives a GOAWAY frame after having promised pushes  <ins> with a Push ID greater than or equal to the identifier contained in the GOAWAY frame, those pushes will not be accepted. </ins> Servers SHOULD send a GOAWAY frame when the closing of a connection is known in advance, even if the advance notice is small, so that the


processed.  An endpoint MUST NOT include delays that it does not control when populating the ACK Delay field in an ACK frame.  13.2.7. ACK frames MUST only be carried in a packet that has the same packet frame.  Unlike the packet number in the QUIC long or short header, the value in an ACK frame is not truncated. <del> A variable-length integer representing the time delta in microseconds between when the largest acknowledged packet, as indicated in the Largest Acknowledged field, was received and the ACK frame was sent.  The ACK Delay field carries this encoded time difference.  It is decoded by multiplying the value in the ACK Delay field by 2 to the power of the value of the ack_delay_exponent transport parameter set by the sender of the ACK frame; see transport-parameter-definitions.  Scaling in this fashion allows for a larger range of values with a shorter encoding at the cost of lower resolution.  Because the receiver doesn't use the value in the ACK Delay field of Initial and Handshake packets, a sender SHOULD send a value of 0. </del>  A variable-length integer specifying the number of Gap and ACK Range fields in the frame.

Problem statement: At the moment, the draft states that ack delay is a variable-length integer representing the time delta in microseconds between when the ACK frame was sent and when the largest acknowledged packet was received by this peer. (). I think that the definition is incorrect. Consider the case of a slow endpoint that needs to spend 10ms for processing one packet. When the true RTT of the network is 50ms, the RTT estimate of the peer would become 50ms, as latestrtt would be 60ms and by this definition, the slow endpoint would report an ackdelay of 10ms. This estimate of 50ms is too aggressive, as the peer cannot expect to see an ACK before 60ms. IIUC, we have thought that the issue is negligible as we have a mechanism using minrtt, for detecting and ignoring excessive ackdelay values being reported. But, as stated in URL, I am becoming skeptical if that defense works well when the path is being utilized, as the true RTT when the path is congested tends to be greater than when the path is not congested, and because minrtt would normally reflect the value of the latter. In case of the example above, the defense does not work if the non-congested RTT of that path was 25ms and if minrtt reflected that value. Solution: The definition of ack delay should be changed to: the amount of time an endpoint withheld an ack from the moment when it would have sent an immediate ack.
This came up in and the resolution was along the lines of what you're describing, so I think this is a matter of clarifying the text.
NAME Thank you for pointing that out. So glad to know that this is an editorial problem.
From URL So I think the text is there, but maybe a reference to this section from section 19.3 and some editorial cleanup would help.
We've had a long standing requirement that stream (see Section 3.2.1) by each peer By the way I interpret this, I put a check in my implementation that doesn't permit any frame until SETTINGS is received. Am I interpreting things correctly? It would be more consistent to allow reserved frame types to appear at any point on any stream that carries frames. However, I'm fine with this requirement as is but it means there is a little more effort needed to handle reserved frame types across streams.
In our implementation we are interpreting this in the same way: any frame on the control stream received before SETTINGS is a connection error. Even reserved frame types. In general the way we handle this is by discarding frame types that are forbidden rather than checking frame types that are allowed on each stream. Here is a snippet from our code. In this way we avoid the extra effort needed to handle reserved frame types that you mentioned, if I am reading your statement correctly. Basically, some frame types are explicitly disallowed. All other frame types are equal, reserved or non-reserved. Do you foresee an extension possibly benefiting from sending a frame before settings ?
Thanks NAME you made me realise I asked the question wrong. This is about reserved and non-core frames. Section 7 says: extensible protocol elements. Implementations MUST discard frames ... I have no strong use case for delivering frames before SETTINGS. But I worry it falls into a slight grey area that might affect interop. I think the correct answer is that the control stream is special case and we should clarify that in Section 7.
My preference goes to retaining the requirement as-is. This is the way H2 has worked, and I am not sure if there is a practical reason to change. FWIW, we have a similar requirement for PRIORITY frame too, which I'm also fine with: A PRIORITY frame received after other frames on a request stream MUST be treated as a connection error of type HTTPUNEXPECTEDFRAME ().
Ooh yeah, that's a good catch too. Table 1 in section 4 seems to actually highlight this behaviour with the notation. And we sort of have some text that captures this in section 4: We could add tweaks or further guidance to this paragraph ( in a PR that also that makes things explicitly clear .
On receipt, reserved is irrelevant -- it's just an unknown frame type. The reservation is simply to permit senders to generate unknown frame types without accidental collision to a real extension the client might support. So the question is whether "ignoring" unknown frame types means pretending that they weren't even there, and therefore the subsequent frame might still be the first frame on the stream. I'm inclined to say no -- you take no action on a frame you don't recognize, but that doesn't mean that the next frame was the first frame.
NAME Thanks a lot! LGTM.Nit: larger and lower imply a comparison. Otherwise, LGTM.

processed.  An endpoint MUST NOT include delays that it does not control when populating the ACK Delay field in an ACK frame. <ins> Since the acknowledgement delay is not used for Initial and Handshake packets, the ACK Delay field in acknowledgements for those packet types SHOULD be set to 0. </ins> 13.2.7. ACK frames MUST only be carried in a packet that has the same packet frame.  Unlike the packet number in the QUIC long or short header, the value in an ACK frame is not truncated.  <ins> A variable-length integer encoding the acknowledgement delay in microseconds; see host-delay.  It is decoded by multiplying the value in the field by 2 to the power of the ack_delay_exponent transport parameter sent by the sender of the ACK frame; see transport-parameter-definitions.  Compared to simply expressing the delay as an integer, this encoding allows for a larger range of values within the same number of bytes, at the cost of lower resolution. </ins> A variable-length integer specifying the number of Gap and ACK Range fields in the frame.


Endpoints use transport parameters to authenticate the negotiation of connection IDs during the handshake; see cid-auth.  7.4.1. Using 0-RTT depends on both client and server using protocol

Call out that since transport parameters are not ALPN specific, when the client offers several, that application protocols should not put restrictions on them that might back them into a corner for future versions.
NAME should take a look.
Currently when a client offers multiple ALPNs when connecting to a QUIC server, it still only offers a single QUIC transport parameters set. This implicitly requires (assumes?) that the single TP set is (must be) valid for every protocol version offered. The same is true for QUIC versions when version negotiation is added. We should allow for the client to offer different settings for different QUIC version or ALPNs (or the corresponding combinations?). If we made a change here, I'd also prefer the solution to not be dependent on any further TLS changes, as this would make the solution much more disruptive. It has been suggested on Slack that the ALPS TLS extension might be good for this, but the fact that it would require TLS changes is a non-starter for QUIC v1. One possible solution for fixing this is to modify the format of the existing TPs to a list of { quicversion, alpn, TPs }, and allow for quicversion to be zero and/or alpn to be empty to indicate unrestricted applicability of these TPs (to all versions/ALPNs). App protocol v1 does not allow server initiated bidi-streams, and it's explicitly an error to set an initial stream count greater than zero. App protocol v2 now has a use for these streams and requires the client to send a count greater than zero.A server supports two related protocols, P1 and P2, but each protocol has a slight different interpretation of unidirectional streams, and require different initial counts to be set.QUIC version 2 wishes to remove or redefine a particular transport parameter. If we don't take a change/fix for this in version 1, it restricts the kinds of changes to TPs we can make in future QUIC versions.
I don't think we should make this change at this point. Regarding separate ALPNs over QUICv1, we can solve this by saying that protocols running over QUIC MUST NOT enforce maximum values for transport parameters. For example HTTP/3 does not use server-initiated bidirectional streams, but there is absolutely no reason for HTTP/3 to require the corresponding transport parameter to be zero. Regarding multiple QUIC versions, I don't understand the problem. The packet containing the transport parameters is a long header that carries the QUIC version, so endpoints just need to interpret transport parameters based on the QUIC version they were sent in.
I agree with NAME I'm not in love with the current transport parameters, but I don't see an abundant reason to change them at this point. I similarly don't think they are actually that coupled to TLS or the QUIC version. As David notes, the interpretation is inherently QUIC-version dependent. We could, for example, completely ditch transport parameters in the TLS extension for QUIC v2 and instead opt for a TRANSPORT_PARAMETERS frame.
For the multiple QUIC versions, my understanding of the direction of version negotiation was that the client would use one version, but include the list of supported versions in its TPs. Then the server could negotiate that to a different QUIC version than the one used in the client initial packet. In that scenario, it would require the TP set to be valid for all versions that the server could negotiate to.
NAME I agree there are other possible solutions for version 2, but as you point out, they likely require a TLS stack change, which I am trying to avoid. I guess this could all be pushed to an extension as well. A new transport parameter that encodes the list of version/ALPN specific parameters... I just don't know how you'd handle the extension being optional.
Also, I still don't have a solution for the "related protocol" example above without adding extra cross-protocol negotiation logic, where the server app knows that the client was offering P1 and P2, and therefore understands how to consume the "combined" TP set in a protocol specific way. OR, I'd have to completely merge the protocols and do the negotiation at the app layer.
NAME it sounds like you're referring to , but I don't think your understanding is correct. In that scenario, the client sends a QUICv1 Initial/ClientHello, and then the server can reply with a QUICv2 Initial/ServerHello. That doesn't require the transport parameter meaning to be the same, the server just needs to know that it's using the v1 interpretation, and the client needs to use the v2 interpretation.
I agree to what NAME and NAME say. In addition to what they have stated, I'd point out that endpoints can loosen the settings by sending frames after the handshake is complete. Most transport parameters are merely used to communicate the initial values. And for other parameters that communicate the final value (e.g., maxackdelay), we can add frames for changing the behavior as we find out the needs.
NAME I guess my line of thinking was that instead of just sending v1 (and indicating support for v2) and the server deciding how to migrate the v1 TPs to v2, the client would be able to send both v1 and v2 TPs (or v1 plus whatever extra v2 requires). Then the server doesn't have to migrate the TPs at all. This might even allow for a greater range of compatible versions. Without this, v1 and v2 could be incompatible because of a change in how a TP is used. I guess if we wanted to go that route, the different TPs could be part of the version negotiation extension draft. NAME applications can theoretically decide how to interpret an initial TP value however it wants. Not allowing for the app to pick different initial values based on the protocol version being used restricts what the app can do. If that's the direction folks want to go, we should probably at least mention this so that protocols take this into account when designing their v1, so they don't back themselves into a corner.
Adding some editorial changes to QUICv1 to reflect this discussion sounds like a fine resolution to me
This reminds me of the conversation about putting SETTINGS into TP, which prompted the creation of the ALPS proposal. I'd like to see a future version of QUIC do a better job with both TPs and application settings, but it's way too late to be making such a substantial change to QUIC v1 at this point.
If someone wants to propose an editorial PR we can consider it as an editorial issue. Failing that I sense we can live with the document as is.
I'm not seeing an action for me in this discussion. I will of course review any PR that someone proposes, but it seems like the minefield is sufficiently well mapped out already.
I'll look into adding some warning text later today.
Discussed at IETF 108. Proposed resolution are editorial changes.

Endpoints use transport parameters to authenticate the negotiation of connection IDs during the handshake; see cid-auth. <ins> Application Layer Protocol Negotiation (ALPN; see ALPN) allows clients to offer multiple application protocols during connection establishment.  The transport parameters that a client includes during the handshake apply to all application protocols that the client offers.  Application protocols can recommend values for transport parameters, such as the initial flow control limits. However, application protocols that set constraints on values for transport parameters could make it impossible for a client to offer multiple application protocols if these constraints conflict. </ins> 7.4.1. Using 0-RTT depends on both client and server using protocol


7.3. <del> While in slow start, QUIC increases the congestion window by the number of bytes acknowledged when each acknowledgment is processed, resulting in exponential growth of the congestion window. </del>  <del> QUIC exits slow start upon loss or upon increase in the ECN-CE counter.  When slow start is exited, the congestion window halves and the slow start threshold is set to the new congestion window.  QUIC re-enters slow start any time the congestion window is less than the slow start threshold, which only occurs after persistent congestion is declared. </del>  7.4. <del> Slow start exits to congestion avoidance.  Congestion avoidance uses an Additive Increase Multiplicative Decrease (AIMD) approach that increases the congestion window by one maximum packet size per congestion window acknowledged.  When a loss or ECN-CE marking is detected, NewReno halves the congestion window, sets the slow start threshold to the new congestion window, and then enters the recovery period. </del>  7.5. <del> A recovery period is entered when loss or ECN-CE marking of a packet is detected in congestion avoidance after the congestion window and slow start threshold have been decreased.  A recovery period ends when a packet sent during the recovery period is acknowledged.  This is slightly different from TCP's definition of recovery, which ends when the lost packet that started recovery is acknowledged. </del>  The recovery period aims to limit congestion window reduction to once per round trip.  Therefore during recovery, the congestion window remains unchanged irrespective of new losses or increases in the ECN- CE counter. <del> When entering recovery, a single packet MAY be sent even if bytes in flight now exceeds the recently reduced congestion window.  This speeds up loss recovery if the data in the lost packet is retransmitted and is similar to TCP as described in Section 5 of RFC6675.  If further packets are lost while the sender is in recovery, sending any packets in response MUST obey the congestion window limit. </del> 7.6. During the handshake, some packet protection keys might not be connection flow control limits and so no advantage is gained by doing so. <del> Endpoints choose the congestion controller that they use.  Though congestion controllers generally treat reports of ECN-CE markings as equivalent to loss (RFC8311), the exact response for each controller could be different.  Failure to correctly respond to information about ECN markings is therefore difficult to detect. </del>  9.

Includes some changes suggested in (see below) ISSUE: /When a loss or ECN-CE marking is detected, NewReno halves the congestion window, sets the slow start threshold to the new congestion window, and then enters the recovery period./ The requirement is that TCP needs to reduce after CE, The RFC series does not now say it needs to halve, it could for example follow the reduction method specified in RFC8511. e.g. /When a loss or ECN-CE marking is detected, the sender must reduce the cwnd. NewReno halves the congestion window, sets the slow start threshold to the new congestion window, and then enters the recovery period. [RFC8511] specifies an alternate cwnd reduction./ ISSUE: Similar comment in 8.3: /Though congestion controllers generally treat reports of ECN-CE markings as equivalent to loss [RFC8311], the exact response for each controller could be different. / This does not seem correct. Could I suggest: /Congestion controllers respond to reports of ECN-CE by reducing their rate. Markings can be treated as equivalent to loss [RFC3168], but other responses can be specified (e.g. [RFC8511]) [RFC8311]. /
This was filed a while ago (before WGLC ended), didn't want to drop it.
Editorial, design?
There is a design issue here regarding the initial congestion window that I would like to see split out into a new issue (which can be marked design). The rest appear to be things we can address with editorial changes.
NAME I moved the IW text into , though I'm not sure I think it's worth revisiting.
Thanks :) I agree that it isn't worth re-opening, but we did sign on to at least discuss whether it is worth discussing.
Do we have a plan for this one?
So, is there a plan? (Is there even anything left to do?)
Some of these are addressed already, others are in an accidental commit(), and for some there's no need to address them in the text. The rest are in
As it stands, the recovery draft does not require the congestion window to be reduced on loss. The only part that talks about this is in Section 7.4, which says: We need to require this reduction for the controller to be a NewReno controller.
To clarify, this isn't a change in what we're describing, you just want to add normative language?

7.3.  <ins> While in slow start, a NewReno sender increases the congestion window by the number of bytes acknowledged when each acknowledgment is processed, resulting in exponential growth of the congestion window. </ins>  <ins> A sender MUST exit slow start and enter recovery when a loss is detected or when the ECN-CE counter reported by the peer increases. A sender re-enters slow start any time the congestion window is less than the slow start threshold, which only occurs after persistent congestion is declared. </ins> 7.4.  <ins> A NewReno sender in congestion avoidance uses an Additive Increase Multiplicative Decrease (AIMD) approach that MUST increase the congestion window by no more than one maximum datagram size per congestion window acknowledged. A sender MUST exit congestion avoidance and enter recovery when a loss is detected or when the ECN-CE counter reported by the peer increases. </ins> 7.5.  <ins> A NewReno sender enters the recovery period when it detects loss or an ECN-CE mark is received.  A recovery period ends when a packet sent during the recovery period is acknowledged. On entering a recovery period, a sender MUST set the slow start threshold to half the value of the congestion window at the moment that loss is detected.  The congestion window MUST be set to the reduced value of the slow start threshold before exiting the recovery period. Implementations MAY set the congestion window immediately on entering a recovery period or use other mechanisms, such as Proportional Rate Reduction (PRR), to reduce it more gradually.  If the congestion window is reduced immediately, a single packet can be sent prior to reduction.  This speeds up loss recovery if the data in the lost packet is retransmitted and is similar to TCP as described in Section 5 of RFC6675. </ins> The recovery period aims to limit congestion window reduction to once per round trip.  Therefore during recovery, the congestion window remains unchanged irrespective of new losses or increases in the ECN- CE counter.  7.6. During the handshake, some packet protection keys might not be connection flow control limits and so no advantage is gained by doing so.  <ins> Endpoints choose the congestion controller that they use.  Congestion controllers respond to reports of ECN-CE by reducing their rate, but the response may vary.  Markings can be treated as equivalent to loss (RFC3168), but other responses can be specified, such as (RFC8511) or (RFC8311). </ins> 9.


10.1.1. An endpoint that sends packets close to the effective timeout risks <del> having them be discarded at the peer, since the peer might enter its draining state before these packets arrive. </del>  An endpoint can send a PING or another ack-eliciting frame to test the connection for liveness if the peer could time out soon, such as application wishes to avoid losing state that has been associated with an open connection, but does not expect to exchange application data for some time.  With this option, an endpoint could send a PING <del> frame periodically to defer an idle timeout; see frame-ping. </del>  Application protocols that use QUIC SHOULD provide guidance on when deferring an idle timeout is appropriate.  Unnecessary sending of

Idle timeout doesn't enter the draining period. Sending PING will reset both idle timeouts for both endpoints.
if it is expecting response data, but does not have or is unable to send application data. However, you can't necessarily reset your own idle_timeout that way because it only is reset when you haven't already sent an ack-eliciting packet after the last received packet. If this is about resetting the peer, then maybe editorial. If it is about resetting yourself, then this may be design.
This is both idle timeouts. The idea behind the rules regarding the local reset of the idle timer is that if you send an ack-eliciting packet, you will get an acknowledgment. You are required to send probes until you do. So you only reset the timeout on the first.
says that after the idle timeout expires, "the connection is silently closed and its state is discarded." However, 10.2.1 says that the peer "might enter its draining state." Based on 10.2, I would not expect there to be a draining state; the connection is simply gone, and you would get a stateless reset. Is the draining state used after the endpoints time out, or not?
I need to find the issue, but we explicitly decided to not drain after idle timeout.
resolved this.
Couple of nits, but LGTM.

10.1.1. An endpoint that sends packets close to the effective timeout risks  <ins> having them be discarded at the peer, since the idle timeout period might have expired at the peer before these packets arrive. </ins> An endpoint can send a PING or another ack-eliciting frame to test the connection for liveness if the peer could time out soon, such as application wishes to avoid losing state that has been associated with an open connection, but does not expect to exchange application data for some time.  With this option, an endpoint could send a PING  <ins> frame (frame-ping) periodically, which will cause the peer to restart its idle timeout period.  Sending a packet containing a PING frame restarts the idle timeout for this endpoint also if this is the first ack-eliciting packet sent since receiving a packet.  Sending a PING frame causes the peer to respond with an acknowledgment, which also restarts the idle timeout for the endpoint. </ins> Application protocols that use QUIC SHOULD provide guidance on when deferring an idle timeout is appropriate.  Unnecessary sending of


to include at least 38 bits of data (or 5 bytes, less the two fixed bits). <del> A minimum size of 21 bytes does not guarantee that a stateless reset is difficult to distinguish from other packets if the recipient requires the use of a connection ID.  To prevent a resulting stateless reset from being trivially distinguishable from a valid packet, all packets sent by an endpoint SHOULD be padded to at least 22 bytes longer than the minimum connection ID that the endpoint might use.  An endpoint that sends a stateless reset in response to a packet that is 43 bytes or less in length SHOULD send a stateless reset that is one byte shorter than the packet it responds to. </del>  These values assume that the Stateless Reset Token is the same length as the minimum expansion of the packet protection AEAD.  Additional

For stateless resets, we recommend a minimum packet size on ALL packets, based on the maximum length of connection IDs that might be generated by the same endpoint. This isn't exactly clear, but nor do I think we need to include the fully worked logic that leads to it. This attempts to split the difference.
difficult to distinguish from other packets if the recipient requires the use of a connection ID. To prevent a resulting stateless reset from being trivially distinguishable from a valid packet, all packets sent by an endpoint SHOULD be padded to at least 22 bytes longer than the minimum connection ID that the endpoint might use. I don't quite understand how this text works because I don't get to control the CIDs that I send with.
Would this be clearer if it said: ?
I feel like maybe I'm having trouble understanding the reasoning here. Do you mind walking me through it?
The logic is a little turgid and I think we were reluctant to show our working. The sender of a stateless reset can't know how big a connection ID the peer might have requested it use. This makes packets potentially distinguishable a peer wanted the largest connection ID size (20) and the packet size drops below 41 bytes. The sender of a stateless reset is required to reduce packet size for the stateless reset, unless it is counting somehow. So, if an endpoint wants to ensure that a peer can generate an indistinguishable stateless reset, assuming that its peer is following the size reduction rule, and assuming that the reduction is just one byte (because the stateless reset sender also wants it to be indistinguishable), then it needs to send packets of at least 42 bytes in the general case. But if it knows that it would never ever ask for a connection ID larger than N, then it can instead send packets of size N+22 and the goal is accomplished without sending larger packets all the time.

to include at least 38 bits of data (or 5 bytes, less the two fixed bits).  <ins> The resulting minimum size of 21 bytes does not guarantee that a stateless reset is difficult to distinguish from other packets if the recipient requires the use of a connection ID.  To achieve that end, the endpoint SHOULD pad all packets it sends to at least 22 bytes longer than the minimum connection ID that it might request the peer to include in packets that the peer sends.  This ensures that any stateless reset sent by the peer is indistinguishable from a valid packet sent to the endpoint.  An endpoint that sends a stateless reset in response to a packet that is 43 bytes or shorter SHOULD send a stateless reset that is one byte shorter than the packet it responds to. </ins> These values assume that the Stateless Reset Token is the same length as the minimum expansion of the packet protection AEAD.  Additional


different than what was sent earlier.  Sending these new packets with the same packet number is likely to compromise the packet protection for those packets because the same key and nonce could be used to <del> protect different content. </del>  A server acknowledges the use of a Retry packet for a connection using the retry_source_connection_id transport parameter; see

Fixes issue .
I know we have consensus and are ready to merge this PR, but... have we considered what error code the server shall use? Implementing this, the easiest for me is INVALID_TOKEN. Is that the obvious intent here?
INVALIDTOKEN seems wrong to me. Why not PROTOCOLVIOLATION?
(I first raised this issue on the .)_ There is the following in the transport draft: We should allow the server to abort the connection the the client violates this . The server can do so by including the packet number into the Retry token and comparing against it when the client sends packets that carry the token.
I think that this might as well run through the design process. It's borderline, because this is about enforcement of an existing MUST, which you might argue comes with an implicit MAY enforce, but we do need to agree on the requirement level. Note that this isn't completely trivial to implement. If a servers responds to packet 1 with Retry, it has to ignore packet 0 if it arrives with no token (or a NEW_TOKEN token). That's already a requirement, but a poor implementation of this requirement will fail, likely with low enough probability that the error won't be detected, especially when it is hidden by Alt-Svc.
I agree this isn't trivial to enforce. Which is why I favor "If you detect this, you SHOULD close the connection.", rather than any recommendation on detecting a violation.
Aside from it not being trivial to enforce, I am not sure how much I like the idea of enforcing correctness for Initial packets. And that is because injection of Initial packets is the way for middleboxes to disrupt the connection. IIRC, we have allowed endpoints to ignore suspicious Initial packets, and I tend to think that we do not need to suggest that an endpoint can (IMO needlessly) strict.
How would the server know to ignore packet 0 in a Stateless Retry? I believe it would respond with another Retry packet instead.
If it's not trivial, it's verging on trivial. The "right" way to enforce this, as Dmitri notes, is to include the client's packet number in the Retry token. Upon receipt of any token, if it is a retry token, the server checks whether the containing packet number is less than or equal to that stored in the token. If it's not a retry token, no check is needed. This is completely stateless, so if packets get reordered, nothing breaks. As to the point of modifying Initial packets, that's true. An attacker that can modify packets can always cause the handshake to fail; this is just another way. To use this for disruption by injecting a packet, the attacker has to observe the client's Initial, the server's Retry packet, and then fabricate an Initial with a lower packet number. Not a high bar, but an attacker that can observe the packets in both directions and inject packets can break things in many other ways, too. I'm comfortable with MAY enforce (which I think is already implicit in a MUST NOT) at least, if people think one more integer encoded in the token is too high a bar for a stronger recommendation.
I'm also fine with MAY. Honestly, I'm also fine with can. I agree with Mike's comments that there are so many ways to interrupt the handshake at this stage, this is just another one, and likely not the easiest by far.
I agree that it is unlikely that servers detecting this behavior would help MITM attackers. My concern was mostly about principles regarding what we write down in spec. That said, I would not oppose to using a MAY here. It is already implied by the MUST NOT on the client-side, so to me it is an editorial change.
Based on the discussion of the issue during consensus call, I'd like to mark this as within 24 hours. And have someone create a spin-off issue for a general-sense review of use of MAYs, if they believe there is a strong need for it. (see URL).
As pointed out by NAME on the mailing list, adds little value to the spec. Every MUST can be enforced by aborting the connection. I'm not convinced that adding a "The peer MAY abort the connection if this MUST is violated" after every MUST increases the clarity of the spec.
As with NAME my preference here is for parsimony (I would feel different if there were a SHOULD). However, I'm not going to go to the mat as long as we add a statement somewhere that you are generally permitted to respond to every MUST violation by closing the connection; if we do that this text is just superfluous.
As stated in URL, detecting MUST violations for Handshake and ApplicationData packets is fine because they are protected by the secrets, but not necessarily for unprotected information. Therefore, it makes sense to have a MAY here. My limited experience tells me that we have different customs between WGs. I tend to believe that document produced by TLS WG assume that a MUST implies "peer MAY detect." And that's fine because TLS does not care about attacks against the underlying transport, and hence any violation is considered as misbehavior of the peer. In contrast, MUST in the recovery draft is not enforceable at all. HTTP is in a middle ground, HTTP/2 has many MAY detect rules for misbehaving peers. That might be due to the history of HTTP being a very lax protocol. I know that the differences are confusing, but I think they all have reasons, and am perfectly fine with keeping status-quo.
NAME Do we have any MUST in the (QUIC) spec for which it's forbidden (i.e. the peer MUST NOT) to enforce it? I don't think so.
In my opinion, instead of a piecemeal solution as suggested here, it would make more sense to add a general statement like the following:
That seems to be the core of the issue that people are suggesting we open.
NAME One example that I can immediately recall is the handling of Version Negotiation packet. An endpoint (including client) is forbidden to respond to a Version Negotiation packet with a Version Negotiation packet, but the server is forbidden to recognize Version Negotiation packets (i.e. we have "MUST discard"). Re writing down the principles, my +1 goes to opening a new issue.
In the interest of not getting tied up on this, I'm declaring consensus to merge the PR. I ask one of the folks in favour of taking a broad brush look, to open an issue to continue that discussion and track any proposals that get made to do so. Hello, In a couple of threads on Slack, Lars reminded me of an interesting MUST NOT in the transport draft: " A client MUST NOT reset the packet number for any packet number space " after processing a Retry packet; Section 17.2.3 contains more " information on this. This made me wonder whether the server should check that the client indeed did not reset the packet number. This is possible to do by including the packet number into the Retry token and then comparing this value with the number of the packet that carries the token back. I could not find such SHOULD in the draft. Should it be included? E.g. "The server SHOULD abort the connection if it detects that packet numbers were reset upon Retry." Dmitri.

different than what was sent earlier.  Sending these new packets with the same packet number is likely to compromise the packet protection for those packets because the same key and nonce could be used to  <ins> protect different content.  A server MAY abort the connection if it detects that the client reset the packet number. </ins> A server acknowledges the use of a Retry packet for a connection using the retry_source_connection_id transport parameter; see


Frame types that were used in HTTP/2 where there is no corresponding HTTP/3 frame have also been reserved (iana-frames).  These frame <del> types MUST NOT be sent, and receipt MAY be treated as an error of type H3_FRAME_UNEXPECTED. </del>  8.

We currently say: This text has two failings: It doesn't specify whether the error is a stream error or a connection error In , it was argued that this violation is easily detected, and perhaps warrants a "MUST error" instead of a "MAY error"
+1 to both changes from my perspective.
This sounds like a design issue. Since this spins off from URL and URL I'm going to assume from past discussion that there is sufficient interest in discussing this.
MUST respond with a connection error SGTM. Frames are hop-by-hop (compared to HTTP messages being end-to-end), therefore we can enforce this.
Thanks, NAME This had shades of the tunneling discussion before, and I wasn't sure on which side of that fence you'd land.
NAME Thank you for checking. As stated above, I think we can be strict regarding HTTP frames (especially the types), even though I tend to think that we need to be lax about HTTP messages (and how they are terminated).

Frame types that were used in HTTP/2 where there is no corresponding HTTP/3 frame have also been reserved (iana-frames).  These frame  <ins> types MUST NOT be sent, and their receipt MUST be treated as a connection error of type H3_FRAME_UNEXPECTED. </ins> 8.


the byte limit to account for the smaller 8 byte overhead of UDP compared to the 20 byte overhead for TCP.  Prior to validating the client's address, the server can be further limited by the anti-amplification limit as specified in Section 8.1 of QUIC-TRANSPORT.  Though the anti-amplification limit can prevent

by specifying that IW should be updated if the max packet size decreases in order to complete the handshake.
Broken off from Gorry's comments in per NAME request
Recovery says "QUIC begins every connection in slow start with the congestion window set to an initial value." There's no mention of updating the initial congestion window, so at the beginning of the connection is currently the only time it's set. QUIC advises setting do not fragment, and UDP typically gets dropped instead of fragmented in my experience even without the flag set, so I believe that simplifies the situation. Given that, along the lines of (iii), I can imagine one potential issue is if the initial window is set to a large value based on some past information and then the MTU is decreased in order to allow the handshake to complete. Currently there's nothing saying that the IW should be re-calculated if the max packet size decreases during the handshake, and adding one would be along the lines of your second suggested option. But I'm also fine with removing the twice max datagram size clause, but that seems like a larger departure from existing RFCs, which makes me hesitant. URL
I think that - as a recommendation - this is fine. Like everything else we recommend in this draft, it is almost certainly not congruent with what implementations will do and that is OK. My suggestion is that those who want to look into this further do so and document their results in a draft that will be pursued independent of QUIC. At some point we have to say no to further refinement and that point has passed.
QUIC seems to me to propose the first IETF CC that with a larger IW for datagrams,. If people in future do happen to use jumbo Ethernet frames, the case Ian describes is sufficiently gross that QUIC needs to suggest/require a remedy. I think I'd be happy with either of Iain's 2 simple proposals.
Gorry you wrote "Retransmission with a more appropiate PMTU does not change the IW, which then sends too many segments/packets." However, if you retransmit because you have experienced loss in the initial round, don't you anyway have to decrease your congestion window? About this RFC6928 says: "Furthermore, to limit any negative effect that a larger initial window may have on links with limited bandwidth or buffer space, implementations SHOULD fall back to RFC 3390 for the restart window (RW) if any packet loss is detected during either the initial window or a restart window, and more than 4 KB of data is sent." So understanding would be that you are actually already supposed to kind or "reset" the initial window, no?
FWIW, I don't have a strong opinion on what to do here, and I'm personally fine with doing nothing as well, but I wrote up the PR that seems most inline with existing RFCs from my reading.
I agree with Mirja on what TCP should do, and that such paths are recommended to reset IW to RW. That seems safe. My point was slightly different, we could consider for QUIC that the reduction might allow IW (cwnd) to be recomputed based on the actual PMTU. The PR seems to be heading this direction.

the byte limit to account for the smaller 8 byte overhead of UDP compared to the 20 byte overhead for TCP. <ins> If the maximum datagram size changes during the connection, the initial congestion window SHOULD be recalculated with the new size. If the maximum datagram size is decreased in order to complete the handshake, the congestion window SHOULD be set to the new initial congestion window. </ins> Prior to validating the client's address, the server can be further limited by the anti-amplification limit as specified in Section 8.1 of QUIC-TRANSPORT.  Though the anti-amplification limit can prevent


CONNECTION_CLOSE.  These cause a recipient to send an acknowledgment; see sending-acknowledgements. <del> A packet that does not increase the largest received packet number for its packet number space (packet-numbers) by exactly one.  A packet can arrive out of order if it is delayed, if earlier packets are lost or delayed, or if the sender intentionally skips a packet number. </del> An entity that can participate in a QUIC connection by generating, receiving, and processing QUIC packets.  There are only two types of endpoint in QUIC: client and server. when the endpoint sends an ACK frame in response to other events. In order to assist loss detection at the sender, an endpoint SHOULD <del> send an ACK frame immediately on receiving an ack-eliciting packet that is out of order.  The endpoint SHOULD NOT continue sending ACK frames immediately unless more ack-eliciting packets are received out of order.  If every subsequent ack-eliciting packet arrives out of order, then an ACK frame SHOULD be sent immediately for every received ack-eliciting packet. </del>  Similarly, packets marked with the ECN Congestion Experienced (CE) codepoint in the IP header SHOULD be acknowledged immediately, to

I believe this was the existing intent, so marking this editorial.
What are the downsides of not doing this and generally having logic that triggers an immediate ACK if any out-of-order packet is received?
Also, how would you recommend implementing this logic, assuming you already have the general logic? Keep a flag and only trigger the immediate ACK if that is ?
The code that sends ACK frames would track the largest received ack-eliciting packet to determine if packets are out of order, not the largest received packet. This is similar to PTO, which is armed from the largest ack-eliciting packet, not the largest packet.
The text is already very clear that if you receive an ACK-only packet out of order, you MUST NOT send an ACK. The goal of this PR is to clarify that ACK-only packets don't count when deciding when ack-eliciting packets are out-of-order.
Followup on my , I think what we need to do is to clarify that the scope of out-of-order packets is limited to ack-eliciting packets. We can say that a out-order packet is defined as "a newly received ack-eliciting packet carrying a packet number that is either smaller than those of any previously-received ack-eliciting packets, or is more than one greater than any of those packets."
How do you tell if the absent packets are ack-eliciting? I mean, most scenarios we use this definition are for the receiver, which can't know whether a missing packet is ack-eliciting.
NAME I've left my two cents regarding that concern on the .
NAME if I'm understanding correctly, the concern is that an implementation that sends ACK-only packets immediately but paces others will routinely have ACK-only packets skipping ahead of ACK-eliciting packets. If you react to all reordering, you'll send a lot of unneeded immediate ACKs when interacting with such an implementation.
Yeah, that makes sense. Thanks.
NAME has it right. It doesn't cause a correctness issue, but it could result in a large number of unnecessary ACK-only packets being sent.
I ended up largely taking NAME suggestion from URL, and no longer relying on out-of-order, but didn't remove the definition entirely. I can if people prefer.
Thanks NAME removed.
The existing text mostly covers this, but I don't think it's always as clear as it could be. If one is doing various offloads(pacing/crypto/etc) and those cause the sender to decide to send a batch of packets at once, with release times that are over a few milliseconds, the sender should not have to wait until all those packets are released to send an ACK. A simple example at the receiver is an ACK arrives with PN 10, then PN 9 arrives. I do not believe PN 9 should not be considered 'out-of-order' and trigger an immediate ACK. Recovery is designed with this case in mind, but when re-reading this text, I thought it was unclear. The simple way to think about this is "ACK-only packets don't affect acknowledgement behavior", which matches with TCP and most of our existing text. I think this will end up being editorial and I believe the only clarification necessary is but I'd appreciate other peoples thoughts on whether this is something worth thinking about and whether there are any other changes necessary.
I share the view that this has been the intent, and that it is editorial. Loss recovery is an interaction between sender sending data vs. receiver sending acks. We might have two endpoints doing loss recovery at the same time, when both sides are sending data. But that should not affect how loss recovery works.
In URL, NAME asks: I think we need to think about two possibilities: an ack-eliciting packet overtaking ack-only packet, and an ack-only packet overtaking an ack-eliciting packet. If the former happens, the receiver can do nothing but send an immediate ack, because it cannot tell if the yet-missing packet is ack-only. But in the opposite case, the receiver can tell that it was an ack-only packet, because it has already received that packet. IIUC, NAME point is that we'd frequently see the latter when pacing is used, and therefore that we should address the issue. IMO, this issue is an artifact of us using one packet number space for loss recovery in both directions. The problem cannot be cleanly solved unless we use different packet number spaces for ack-eliciting packets and ack-only packets. That said, I do think that it is a good idea to minimize the impact of the problem, and that the proposed clarification works fine in that respect.
That's a reasonable analysis of this situation. That is also consistent with what our stack implements. That is, we only ever send an immediate acknowledgement when we receive ack-eliciting packets. I can't see how you would do otherwise, frankly. However, I think that attempting to define "out-of-order" is the problem. I just searched the transport document for this and we don't use the definition. Therefore, I suggest that we say what we mean when it comes to describing how to immediately acknolwedge packets. That is, Or something like that. The out-of-order definition can go.
Good thought on getting rid of 'out-of-order', that should help.
I was about to say we could simplify the second bullet, but that would miss the case of: N-3: last received ack-eliciting packet N-2: lost N-1: ACK-only packet N: Newly received We want to send an immediate ACK on receipt of N, but not N-1. Nor do we want one on the late receipt of N-2.
border-Borderline design, let's run the process for visibility
Thanks for writing this down.NAME Thank you for all your work.

CONNECTION_CLOSE.  These cause a recipient to send an acknowledgment; see sending-acknowledgements.  An entity that can participate in a QUIC connection by generating, receiving, and processing QUIC packets.  There are only two types of endpoint in QUIC: client and server. when the endpoint sends an ACK frame in response to other events. In order to assist loss detection at the sender, an endpoint SHOULD  <ins> generate and send an ACK frame without delay when it receives an ack- eliciting packet either: when the received packet has a packet number less than another ack-eliciting packet that has been received, or when the packet has a packet number larger than the highest- numbered ack-eliciting packet that has been received and there are missing packets between that packet and this packet. </ins> Similarly, packets marked with the ECN Congestion Experienced (CE) codepoint in the IP header SHOULD be acknowledged immediately, to


CONNECTION_CLOSE frames sent in multiple packet types can be coalesced into a single UDP datagram; see packet-coalesce. <del> An endpoint might send a CONNECTION_CLOSE frame in an Initial packet or in response to unauthenticated information received in Initial or Handshake packets.  Such an immediate close might expose legitimate connections to a denial of service.  QUIC does not include defensive measures for on-path attacks during the handshake; see handshake-dos. However, at the cost of reducing feedback about errors for legitimate peers, some forms of denial of service can be made more difficult for an attacker if endpoints discard illegal packets rather than terminating a connection with CONNECTION_CLOSE.  For this reason, endpoints MAY discard packets rather than immediately close if errors are detected in packets that lack authentication. </del>  An endpoint that has not established state, such as a server that detects an error in an Initial packet, does not enter the closing

This sentence structure was unnecessarily byzantine.
is hard to read. [A D] [A D]] [A D]] ...

CONNECTION_CLOSE frames sent in multiple packet types can be coalesced into a single UDP datagram; see packet-coalesce.  <ins> An endpoint can send a CONNECTION_CLOSE frame in an Initial packet. This might be in response to unauthenticated information received in Initial or Handshake packets.  Such an immediate close might expose legitimate connections to a denial of service.  QUIC does not include defensive measures for on-path attacks during the handshake; see handshake-dos.  However, at the cost of reducing feedback about errors for legitimate peers, some forms of denial of service can be made more difficult for an attacker if endpoints discard illegal packets rather than terminating a connection with CONNECTION_CLOSE. For this reason, endpoints MAY discard packets rather than immediately close if errors are detected in packets that lack authentication. </ins> An endpoint that has not established state, such as a server that detects an error in an Initial packet, does not enter the closing


An endpoint MUST NOT send a packet if it would cause bytes_in_flight (see vars-of-interest) to be larger than the congestion window, <del> unless the packet is sent on a PTO timer expiration; see pto. </del>  7.1.

We may want to revert this once lands
~ still allows you to reduce the cwnd on entering recovery, so I think this text is still good.~ Now that I've seen your changes to , reverting this makes sense.
It might pay to reconcile: {{vars-of-interest}}) to be larger than the congestion window, unless the packet is sent on a PTO timer expiration; see {{pto}}. And now exceeds the recently reduced congestion window. Now, you might implement the latter using PTO machinery, but that text doesn't really suggest that in any way.
Where does this second sentence come from? I'm pretty sure I haven't implemented anything like this in quic-go. Should I?
That sentence is the the definition of the . We haven't implemented that either, and I was wondering if it truly helps.
For TCP this second case if important because as long as you don't retransmit the lost packet you will not get any ACK that acknowledges new data. I guess we don't need that for QUIC.
The second sentence was added later in , and the MUST wasn't updated, so I'll fix that. Chromium currently implements PRR(RFC6937), which is a more complex variant of this which is designed to keep the ACK clock going. I thought we discussed adding an informative reference to PRR when we added this text, and I'm not sure why we didn't.
seems to be the wrong issue you are pointed to...
Good catch, it's
Thanks. Simple :)

An endpoint MUST NOT send a packet if it would cause bytes_in_flight (see vars-of-interest) to be larger than the congestion window,  <ins> unless the packet is sent on a PTO timer expiration (see pto) or when entering recovery (see recovery-period). </ins> 7.1.


A decoder can limit the amount of state memory used for blocked streams by setting an appropriate value for the maximum number of blocked streams.  In HTTP/3, this is realized by setting an <del> appropriate value for the QPACK_BLOCKED_STREAMS parameter.  An encoder can limit the amount of state memory by only using as many blocked streams as it wishes to support; no signaling to the decoder is required. </del>  The amount of temporary memory consumed by an encoder or decoder can be limited by processing header fields sequentially.  A decoder

Per discussion on Slack, expanding this discussion in the Security Considerations to be more accurate. Editorial, since these are purely statements about the properties of the design.
This is true, but this is also obvious, at least once you start implementing the encoder. Too, there is the natural limit on dynamic references: it is the function of the product of the number of allowed streams and the number of entries in the dynamic table (which itself is a function of the table size). I am concerned that this new verbiage puts the wrong thought into the head of the implementer; we should not encourage this. Limiting the number of dynamic table references (done out of memory conservation zeal) affects compression performance directly -- it worsens it. Perhaps we can tack on the following sentence to the end of the paragraph:
I'm OK with that, though I would qualify with "compression performance", as performance has other dimensions that the extra state might impinge upon.
Another thing to note is that sometimes dynamic table references do not have to have their own objects in the encoder. For examples, they are in ls-qpack. (Everyone reading this in 2022: you're welcome)
Yeah, compression performance qualification is a good suggestion.

A decoder can limit the amount of state memory used for blocked streams by setting an appropriate value for the maximum number of blocked streams.  In HTTP/3, this is realized by setting an  <ins> appropriate value for the QPACK_BLOCKED_STREAMS parameter.  Streams which risk becoming blocked consume no additional state memory on the encoder. An encoder allocates memory to track all dynamic table references in unacknowledged field sections.  An implementation can directly limit the amount of state memory by only using as many references to the dynamic table as it wishes to track; no signaling to the decoder is required.  However, limiting references to the dynamic table will reduce compression effectiveness. </ins> The amount of temporary memory consumed by an encoder or decoder can be limited by processing header fields sequentially.  A decoder


As keys at a given encryption level become available to TLS, TLS indicates to QUIC that reading or writing keys at that encryption <del> level are available.  These events are not asynchronous; they always occur immediately after TLS is provided with new handshake bytes, or after TLS produces handshake bytes. </del>  TLS provides QUIC with three items as a new encryption level becomes available: A Key Derivation Function (KDF) These values are based on the values that TLS negotiates and are used <del> by QUIC to generate packet and header protection keys (see packet- protection and header-protect). </del>  If 0-RTT is possible, it is ready after the client sends a TLS ClientHello message or the server receives that message.  After The output ciphertext, C, of the AEAD is transmitted in place of P. Some AEAD functions have limits for how many packets can be encrypted <del> under the same key and IV (see for example AEBounds).  This might be lower than the packet number limit.  An endpoint MUST initiate a key update (key-update) prior to exceeding any limit set for the AEAD that is in use. </del>  5.4. 6.6. <del> Key updates MUST be initiated before usage limits on packet protection keys are exceeded.  For the cipher suites mentioned in this document, the limits in Section 5.5 of TLS13 apply.  TLS13 does not specify a limit for AEAD_AES_128_CCM, but the analysis in ccm- bounds shows that a limit of 2^23 packets can be used to obtain the same confidentiality protection as the limits specified in TLS. </del>  The usage limits defined in TLS 1.3 exist for protection against attacks on confidentiality and apply to successful applications of packet that cannot be authenticated, allowing multiple forgery attempts. <del> Endpoints MUST count the number of received packets that fail authentication for each set of keys.  If the number of packets that fail authentication with the same key exceeds a limit that is specific to the AEAD in use, the endpoint MUST stop using those keys. Endpoints MUST initiate a key update before reaching this limit.  If a key update is not possible, the endpoint MUST immediately close the connection.  Applying a limit reduces the probability that an attacker is able to successfully forge a packet; see AEBounds and ROBUST. Due to the way that header protection protects the Key Phase, packets that are discarded are likely to have an even distribution of both Key Phase values.  This means that packets that fail authentication will often use the packet protection keys from the next key phase.  It is therefore necessary to also track the number of packets that fail authentication with the next set of packet protection keys.  To avoid exhaustion of both sets of keys, it might be necessary to initiate two key updates in succession. For AEAD_AES_128_GCM, AEAD_AES_256_GCM, and AEAD_CHACHA20_POLY1305, the limit on the number of packets that fail authentication is 2^36. Note that the analysis in AEBounds supports a higher limit for the AEAD_AES_128_GCM and AEAD_AES_256_GCM, but this specification recommends a lower limit.  For AEAD_AES_128_CCM, the limit on the number of packets that fail authentication is 2^23.5; see ccm-bounds. </del>  These limits were originally calculated using assumptions about the limits on TLS record size.  The maximum size of a TLS record

This clarifies the TLS interface by explaining that TLS only generates outputs (handshake bytes or keying material) in response to inputs. Previously, the only inputs we acknowledged were the handshake bytes, but this recognizes two more: the signal from the client to start handshaking, and - for async validation - a signal that a certificate is OK or not.
Weren't we going to say that the traffic keys might get generated asynchronously in ? I wonder how this PR interacts with . I see this PR calling out three types of events (i.e. generation of CH, input of handshake bytes, completion of certificate validation) as the "inputs," then claim that TLS provides the keys "synchronously" in relation to such events. That works as a definition, but I am not sure if it is easy to comprehend. I think my preference would be to talk about (a)synchronicity from the viewpoint of a transport. It would be easier to say that TLS might provide keys "asynchronously" from what the transport provides as input, then point out that key generation (possibly in relation to certificate verification) might take some time.
To answer both questions, I am planning on integrating the text from as well. That would cover the case where the TLS stack has to disappear for some time before it produces new outputs, however that manifests.
NAME NAME I've rebased this based on the changes in . Would you mind checking my work?
The label should be instead of .
Thanks NAME fixed.
The process for interacting with TLS does not acknowledge the possibility that progress in the handshake might depend on inputs other than CRYPTO frames. In particular, endpoints might need to perform additional checking to decide that the TLS handshake is complete. The most important example being the processing that occurs for certificate validation. This is often an asynchronous process, with OCSP and other muck, so recognizing that explicitly in the draft is worthwhile. This might help tease out one of the causes of the problem discussed in , but it doesn't lead to a solution.
Q: Does this indicate that the TLS changes involved in may be editorial? I'd like to focus on the transport and recovery changes to deal with the issue, and it'd be nice if the tls changes were separate from my perspective.
Why isn't adequate for this? Specifically, isn't it enough to change the stance from synchronous to asynchronous?
I would prefer to explain how certificate validation can take time and block availability of keys.
Should this become proposal-ready?
NAME : This is an editorial issue, so no.
Thanks looks great!

As keys at a given encryption level become available to TLS, TLS indicates to QUIC that reading or writing keys at that encryption  <ins> level are available. The availability of new keys is always a result of providing inputs to TLS.  TLS only provides new keys after being initialized (by a client) or when provided with new handshake data. However, a TLS implementation could perform some of its processing asynchronously.  In particular, the process of validating a certificate can take some time.  While waiting for TLS processing to complete, an endpoint SHOULD buffer received packets if they might be processed using keys that aren't yet available.  These packets can be processed once keys are provided by TLS.  An endpoint SHOULD continue to respond to packets that can be processed during this time. After processing inputs, TLS might produce handshake bytes, keys for new encryption levels, or both. </ins> TLS provides QUIC with three items as a new encryption level becomes available: A Key Derivation Function (KDF) These values are based on the values that TLS negotiates and are used  <ins> by QUIC to generate packet and header protection keys; see packet- protection and header-protect. </ins> If 0-RTT is possible, it is ready after the client sends a TLS ClientHello message or the server receives that message.  After The output ciphertext, C, of the AEAD is transmitted in place of P. Some AEAD functions have limits for how many packets can be encrypted  <ins> under the same key and IV; see aead-limits.  This might be lower than the packet number limit.  An endpoint MUST initiate a key update (key-update) prior to exceeding any limit set for the AEAD that is in use. </ins> 5.4. 6.6.  <ins> This document sets usage limits for AEAD algorithms to ensure that overuse does not give an adversary a disproportionate advantage in attacking the confidentiality and integrity of communications when using QUIC. </ins> The usage limits defined in TLS 1.3 exist for protection against attacks on confidentiality and apply to successful applications of packet that cannot be authenticated, allowing multiple forgery attempts.  <ins> QUIC accounts for AEAD confidentiality and integrity limits separately.  The confidentiality limit applies to the number of packets encrypted with a given key.  The integrity limit applies to the number of packets decrypted within a given connection.  Details on enforcing these limits for each AEAD algorithm follow below. Endpoints MUST count the number of encrypted packets for each set of keys.  If the total number of encrypted packets with the same key exceeds the confidentiality limit for the selected AEAD, the endpoint MUST stop using those keys.  Endpoints MUST initiate a key update before sending more protected packets than the confidentiality limit for the selected AEAD permits.  If a key update is not possible or integrity limits are reached, the endpoint MUST stop using the connection and only send stateless resets in response to receiving packets.  It is RECOMMENDED that endpoints immediately close the connection with a connection error of type AEAD_LIMIT_REACHED before reaching a state where key updates are not possible. For AEAD_AES_128_GCM and AEAD_AES_256_GCM, the confidentiality limit is 2^25 encrypted packets; see gcm-bounds.  For AEAD_CHACHA20_POLY1305, the confidentiality limit is greater than the number of possible packets (2^62) and so can be disregarded.  For AEAD_AES_128_CCM, the confidentiality limit is 2^23.5 encrypted packets; see ccm-bounds.  Applying a limit reduces the probability that an attacker can distinguish the AEAD in use from a random permutation; see AEBounds, ROBUST, and GCM-MU. In addition to counting packets sent, endpoints MUST count the number of received packets that fail authentication during the lifetime of a connection.  If the total number of received packets that fail authentication within the connection, across all keys, exceeds the integrity limit for the selected AEAD, the endpoint MUST immediately close the connection with a connection error of type AEAD_LIMIT_REACHED and not process any more packets. For AEAD_AES_128_GCM and AEAD_AES_256_GCM, the integrity limit is 2^54 invalid packets; see gcm-bounds.  For AEAD_CHACHA20_POLY1305, the integrity limit is 2^36 invalid packets; see AEBounds.  For AEAD_AES_128_CCM, the integrity limit is 2^23.5 invalid packets; see ccm-bounds.  Applying this limit reduces the probability that an attacker can successfully forge a packet; see AEBounds, ROBUST, and GCM-MU. Future analyses and specifications MAY relax confidentiality or integrity limits for an AEAD. </ins> These limits were originally calculated using assumptions about the limits on TLS record size.  The maximum size of a TLS record


receiver's buffer capacity for the connection, by limiting the total bytes of stream data sent in STREAM frames on all streams. <del> Senders MUST not send data in excess of either limit. </del>  A receiver sets initial limits for all streams by sending transport parameters during the handshake (transport-parameters).  A receiver

Noticed while reading text for a different issue; this appears to be the only instance of this construction in the documents.

receiver's buffer capacity for the connection, by limiting the total bytes of stream data sent in STREAM frames on all streams.  <ins> Senders MUST NOT send data in excess of either limit. </ins> A receiver sets initial limits for all streams by sending transport parameters during the handshake (transport-parameters).  A receiver


MAX_DATA frame, which indicates the maximum of the sum of the absolute byte offsets of all streams.  A receiver maintains a cumulative sum of bytes received on all streams, which is used to <del> check for flow control violations.  A receiver might use a sum of bytes consumed on all streams to determine the maximum data limit to be advertised. </del>  Once a receiver advertises a limit for the connection or a stream, it MAY advertise a smaller limit, but this has no effect.

(Broken out of .) >A receiver maintains a cumulative sum of It would have helped me, had this explained what was intended by “control violations” - I think it means to detect when the sender sends more than permitted by the flow control limit. I think the control part comes in a later sentence? (Is this measured against the largest advertised offset, or the current offset? … given what is written about reducing the advertised value?)

MAX_DATA frame, which indicates the maximum of the sum of the absolute byte offsets of all streams.  A receiver maintains a cumulative sum of bytes received on all streams, which is used to  <ins> check for violations of the advertised connection or stream data limits.  A receiver might use a sum of bytes consumed on all streams to determine the maximum data limit to be advertised. </ins> Once a receiver advertises a limit for the connection or a stream, it MAY advertise a smaller limit, but this has no effect.


ack_delay_exponent, max_ack_delay, initial_source_connection_id, original_destination_connection_id, preferred_address, retry_source_connection_id, and stateless_reset_token.  The client <del> MUST use the server's new values in the handshake instead, and absent new values from the server, the default value. </del>  A client that attempts to send 0-RTT data MUST remember all other transport parameters used by the server.  The server can remember

:
(Broken out of .) what does ‘absent’ mean in this clause, could you reword?

ack_delay_exponent, max_ack_delay, initial_source_connection_id, original_destination_connection_id, preferred_address, retry_source_connection_id, and stateless_reset_token.  The client  <ins> MUST use the server's new values in the handshake instead; if the server does not provide new values, the default value is used. </ins> A client that attempts to send 0-RTT data MUST remember all other transport parameters used by the server.  The server can remember


When migrating, the new path might not support the endpoint's current sending rate.  Therefore, the endpoint resets its congestion <del> controller, as described in migration-cc. </del>  The new path might not have the same ECN capability.  Therefore, the endpoint verifies ECN capability as described in ecn.

(Broken out of .) I assert the new path may not have the same RTT, because the path could traverse a different set of network devices or be subject to different policies that influence forwarding behaviour. These are not currently listed here, but are the basis for some later text in 9.3.1. Can we mention here?
Is the RTT not included in the reset of the congestion controller?
Section 9.4 reads: "The capacity available on the new path might not be the same as the old path. Packets sent on the old path MUST NOT contribute to congestion control or RTT estimation for the new path."
I sent a PR to add "and RTT estimate" to the sentence NAME quoted to make this completely clear.

When migrating, the new path might not support the endpoint's current sending rate.  Therefore, the endpoint resets its congestion  <ins> controller and RTT estimate, as described in migration-cc. </ins> The new path might not have the same ECN capability.  Therefore, the endpoint verifies ECN capability as described in ecn.


8.4. <del> On receiving a PATH_CHALLENGE frame, an endpoint MUST respond immediately by echoing the data contained in the PATH_CHALLENGE frame in a PATH_RESPONSE frame. </del>  An endpoint MUST NOT send more than one PATH_RESPONSE frame in response to one PATH_CHALLENGE frame; see retransmission-of-

This is the maximal version of a clarification of this text.
WFM, and resolves the original NiT.
(Broken out of .) Of course it may not be able to immediately send anything:-) , but I suspect this needs to not delay?
These frames aren't subject to flow control, so the only reason it would be unable to send something is congestion control. The frame that it echos back needs to be enqueued for sending; if it takes some time to actually hit the network, that's fine.
NAME
Seems resolved by the PR
For the benefit of others reviewing this: This seems to refer to .
This LG, but so did the old text.

8.4.  <ins> On receiving a PATH_CHALLENGE frame, an endpoint MUST respond by echoing the data contained in the PATH_CHALLENGE frame in a PATH_RESPONSE frame.  An endpoint MUST NOT delay transmission of a packet containing a PATH_RESPONSE frame unless constrained by congestion control. </ins> An endpoint MUST NOT send more than one PATH_RESPONSE frame in response to one PATH_CHALLENGE frame; see retransmission-of-