by the endpoints and network load.  An endpoint's loss detection and congestion control mechanisms however need to be tolerant of this delay at the peer.  An endpoint signals its tolerance to its peer <del> using an ACK-FREQUENCY frame, shown below: </del>  Following the common frame format described in Section 12.4 of QUIC- <del> TRANSPORT, ACK-FREQUENCY frames have a type of 0xXX, and contain the </del>  following fields: A variable-length integer representing the sequence number <del> assigned to the ACK-FREQUENCY frame by the sender to allow </del>  receivers to ignore obsolete frames, see multiple-frames. A variable-length integer representing the maximum number of ack- value smaller than the "min_ack_delay" advertised by this endpoint is invalid. <del> Receipt of invalid values in an ACK-FREQUENCY frame MUST be treated </del>  as a connection error of type FRAME_ENCODING_ERROR. <del> ACK-FREQUENCY frames are ack-eliciting.  However, their loss does not </del>  require retransmission. <del> An endpoint MAY send ACK-FREQUENCY frames multiple times during a </del>  connection and with different values. An endpoint will have committed a "max_ack_delay" value to the peer, which specifies the maximum amount of time by which the endpoint will <del> delay sending acknowledgments.  When the endpoint receives an ACK- FREQUENCY frame, it MUST update this maximum time to the value </del>  proposed by the peer in the Update Max Ack Delay field. 5. <del> An endpoint can send multiple ACK-FREQUENCY frames, and each one of </del>  them can have different values.  An endpoint MUST use a sequence <del> number of 0 for the first ACK-FREQUENCY frame it constructs and </del>  sends, and a strictly increasing value thereafter. <del> An endpoint MUST allow reordered ACK-FREQUENCY frames to be received </del>  and processed, see Section 13.3 of QUIC-TRANSPORT. <del> On the first received ACK-FREQUENCY frame in a connection, an </del>  endpoint MUST immediately record all values from the frame.  The sequence number of the frame is recorded as the largest seen sequence number.  The new Packet Tolerance and Update Max Ack Delay values MUST be immediately used for delaying acknowledgements; see sending. <del> On a subsequently received ACK-FREQUENCY frame, the endpoint MUST </del>  check if this frame is more recent than any previous ones, as follows: 6. <del> Prior to receiving an ACK-FREQUENCY frame, endpoints send </del>  acknowledgements as specified in Section 13.2.1 of QUIC-TRANSPORT. <del> On receiving an ACK-FREQUENCY frame and updating its recorded </del>  "max_ack_delay" and "Packet Tolerance" values (multiple-frames), the endpoint MUST send an acknowledgement when one of the following conditions are met: On sending an update to the peer's "max_ack_delay", an endpoint can use this new value in later computations of its Probe Timeout (PTO) period; see Section 5.2.1 of QUIC-RECOVERY.  The endpoint MUST <del> however wait until the ACK-FREQUENCY frame that carries this new </del>  value is acknowledged by the peer. Until the frame is acknowledged, the endpoint MUST use the greater of computing the PTO period.  Doing so avoids spurious PTOs that can be caused by an update that increases the peer's "max_ack_delay". <del> While it is expected that endpoints will have only one ACK-FREQUENCY </del>  frame in flight at any given time, this extension does not prohibit having more than one in flight.  Generally, when using "max_ack_delay" for PTO computations, endpoints MUST use the maximum

Also rename ACK-FREQUENCY to ACK_FREQUENCY to
Use instead of following naming convention in the transport draft.

by the endpoints and network load.  An endpoint's loss detection and congestion control mechanisms however need to be tolerant of this delay at the peer.  An endpoint signals its tolerance to its peer  <ins> using an ACK_FREQUENCY frame, shown below: </ins> Following the common frame format described in Section 12.4 of QUIC-  <ins> TRANSPORT, ACK_FREQUENCY frames have a type of 0xAF, and contain the </ins> following fields: A variable-length integer representing the sequence number  <ins> assigned to the ACK_FREQUENCY frame by the sender to allow </ins> receivers to ignore obsolete frames, see multiple-frames. A variable-length integer representing the maximum number of ack- value smaller than the "min_ack_delay" advertised by this endpoint is invalid.  <ins> Receipt of invalid values in an ACK_FREQUENCY frame MUST be treated </ins> as a connection error of type FRAME_ENCODING_ERROR.  <ins> ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not </ins> require retransmission.  <ins> An endpoint MAY send ACK_FREQUENCY frames multiple times during a </ins> connection and with different values. An endpoint will have committed a "max_ack_delay" value to the peer, which specifies the maximum amount of time by which the endpoint will  <ins> delay sending acknowledgments.  When the endpoint receives an ACK_FREQUENCY frame, it MUST update this maximum time to the value </ins> proposed by the peer in the Update Max Ack Delay field. 5.  <ins> An endpoint can send multiple ACK_FREQUENCY frames, and each one of </ins> them can have different values.  An endpoint MUST use a sequence  <ins> number of 0 for the first ACK_FREQUENCY frame it constructs and </ins> sends, and a strictly increasing value thereafter.  <ins> An endpoint MUST allow reordered ACK_FREQUENCY frames to be received </ins> and processed, see Section 13.3 of QUIC-TRANSPORT.  <ins> On the first received ACK_FREQUENCY frame in a connection, an </ins> endpoint MUST immediately record all values from the frame.  The sequence number of the frame is recorded as the largest seen sequence number.  The new Packet Tolerance and Update Max Ack Delay values MUST be immediately used for delaying acknowledgements; see sending.  <ins> On a subsequently received ACK_FREQUENCY frame, the endpoint MUST </ins> check if this frame is more recent than any previous ones, as follows: 6.  <ins> Prior to receiving an ACK_FREQUENCY frame, endpoints send </ins> acknowledgements as specified in Section 13.2.1 of QUIC-TRANSPORT.  <ins> On receiving an ACK_FREQUENCY frame and updating its recorded </ins> "max_ack_delay" and "Packet Tolerance" values (multiple-frames), the endpoint MUST send an acknowledgement when one of the following conditions are met: On sending an update to the peer's "max_ack_delay", an endpoint can use this new value in later computations of its Probe Timeout (PTO) period; see Section 5.2.1 of QUIC-RECOVERY.  The endpoint MUST  <ins> however wait until the ACK_FREQUENCY frame that carries this new </ins> value is acknowledged by the peer. Until the frame is acknowledged, the endpoint MUST use the greater of computing the PTO period.  Doing so avoids spurious PTOs that can be caused by an update that increases the peer's "max_ack_delay".  <ins> While it is expected that endpoints will have only one ACK_FREQUENCY </ins> frame in flight at any given time, this extension does not prohibit having more than one in flight.  Generally, when using "max_ack_delay" for PTO computations, endpoints MUST use the maximum


Starting a connection up quickly without inducing excess queue is important for latency reduction, for both short and long flows. <del> The sender often needs frequent acknowledgments during this phase; see slow start and hystart. </del>  Congestion controllers that are purely window based and strictly adherent to packet conservation, such as the one defined in QUIC- flows, congestion controllers that are not window-based, such as BBR, can perform well with very few acknowledgements per RTT. <del> New sender startup mechanisms, such as paced chirping, will need a way for the sender to increase the frequency of acknowledgements when fine-grained feedback is required. </del>  QUIC-TRANSPORT currently specifies a simple delayed acknowledgement mechanism that a receiver can use: send an acknowledgement for every

by removing references to slow start, hystart, and paced chirping. Slow start and BBR startup aren't quite the same as NAME mentioned, hystart isn't specified for QUIC, and paced chirping is not an IETF draft to my knowledge.
"Starting a connection up quickly without inducing much queue is important for latency reduction, for both short and long flows. The sender often needs frequent acknowledgments during this phase; see slow start and hystart." There's no description of them in this doc. Slow start is in the recovery draft, but hystart is not, so we'd have to reference a non-QUIC RFC?
Maybe paced chirping too.
An added nit to this: congestion controllers like BBR do not have a slow start / hystart phase exactly, but are still sensitive to ACKs early in the flow during "Startup". With a BBR congestion controller the ACKing interval early in the flow has a measurable effect on the time it takes to find the bottleneck bandwidth. More generally I suspect this is true of pretty much every congestion controller that's trying to find an optimal congestion window. Given that, perhaps we need more general language here about the benefit of ACKs early in a flow, and then referencing slow start and hystart specifically?
BBR has 'Startup', which is quite similar to slow start, so maybe flow startup a better general purpose term? I think we can remove the reference to hystart entirely, since I wouldn't expect decreasing ACK frequency to have a negative impact on it, particularly given QUIC's explicit ack delay.

Starting a connection up quickly without inducing excess queue is important for latency reduction, for both short and long flows.  <ins> The sender often needs more frequent acknowledgments during this phase. </ins> Congestion controllers that are purely window based and strictly adherent to packet conservation, such as the one defined in QUIC- flows, congestion controllers that are not window-based, such as BBR, can perform well with very few acknowledgements per RTT.  <ins> New sender startup mechanisms will need a way for the sender to increase the frequency of acknowledgements when fine-grained feedback is required. </ins> QUIC-TRANSPORT currently specifies a simple delayed acknowledgement mechanism that a receiver can use: send an acknowledgement for every


value smaller than the "min_ack_delay" advertised by this endpoint is invalid.  Receipt of invalid values in an ACK_FREQUENCY frame MUST be treated as a connection error of type FRAME_ENCODING_ERROR. 5. An endpoint can send multiple ACK_FREQUENCY frames, and each one of <del> them can have different values.  An endpoint MUST use a sequence number of 0 for the first ACK_FREQUENCY frame it constructs and sends, and a strictly increasing value thereafter. </del>  An endpoint MUST allow reordered ACK_FREQUENCY frames to be received and processed, see Section 13.3 of QUIC-TRANSPORT. 6.1. <del> As specified in Section 13.3.1 of QUIC-TRANSPORT, endpoints SHOULD send an acknowledgement immediately on receiving a reordered ack- eliciting packet, unless the peer has sent a "disable_ack_on_reordering" transport parameter, described below: This optional transport parameter is sent by an endpoint that is reordering tolerant or expects the connection to experience reordering.  An endpoint that receives this transport parameter MUST NOT make the exception of sending an immediate acknowledgement when reordering is observed.  This parameter is a zero-length value, and is encoded as per Section 18 of QUIC- TRANSPORT. </del>  6.2.

Maybe it would be more consistent if this was a field on the ACK_FREQUENCY frame?
I agree. Not only for consistency but also for practical benefits. Firstly, in many cases, the sender will only be able to tell if the path is going to have lots of reorders. Allowing the sender to alter the client behavior mid-connection would be beneficial, as that would allow the sender reduce the ACK rate when it disables packet number-based loss detection. The second use case is for clustered servers sending different streams from different hosts. Packets being sent from such clustered servers would have reorder when there are multiple servers handling streams, but not when there is only server handling streams (consider the case where only one stream, or only one HTTP request is open). In such deployments, it is beneficial to have the capability of turning off reordering detection mid-connection.
Let's assume here that peers might adjust their reordering threshold based on the amount of reordering they're seeing, i.e. that they will increase it from the default value of 3 to a higher value if they detect a lot of spurious retransmissions when using 3. In that case, would it maybe make sense to define a threshold when to send an ACK for a reordered packet? For example, let's say packet N is missing, but N+1 is received. If you're using a high reordering threshold, you wouldn't care to receive an ACK for this event immediately. However, you'd still care to get an ACK when N+10 is received, because in that case your packet number-based loss detection will be able to quickly repair the loss of packet N.
NAME I agree that that's a possibility. At the same time, it could be well the case that simply turning off reordering detection and solely relying on the time-based loss recovery would be sufficient.
NAME I can't really tell what makes sense from the point of a congestion control algorithm, but I'm wondering if it makes sense to specify something here that would allow a wide range of algorithms to get the kind of information that they deem useful. If we made that field a varint instead of a bool (then we can also avoid the question of how we would encode a bool), you could still effectively disable ACKs for reordered packets by sending a very large value for that field, if you want to disable packet number-based loss recovery.
When NAME and I were discussing this, we thought it was best to not make this part of the mechanism overly complex, hence the transport param. Unfortunately, we have very little(or no) deployment experience in this area, so the TP was more of a future-proofing mechanism than one based on past experience. If we were going to add a field, I'd suggest a max_reordering field where 0 indicates any reordering is tolerated and any other values indicate a max packet number gap before an immediate ACK is sent? If you want immediate ACK upon reordering, you'd send 1. That only adds a byte to the frame size in almost all cases, but I think should work well in a wide variety of use cases. My main concern is whether the implementation complexity increases more than the value it's providing?
I think there's value in making this a field in the frame instead of a TP... as NAME noted, we used a TP because that seemed like the simplest approach, not necessarily the most flexible one. I agree with NAME that allowing for communicating a "threshold" has the most room for play with reordering robustness in the future. I'm not convinced that it'll get all that used, but if the receiver behavior is not much more complicated to implement, then I'm ok with specifying that. So, here's a shot at defining receiver behavior. A receiver has to measure the degree of reordering and then decide whether to ack immediately or not. This degree of reordering can be defined as follows: the distance of the received packet from the largestreceived (for that PN space). Basically: A couple of examples to see if this works out. Let's say LR is the largestreceived packet number. A reordering distance of 2 means that LR+2 and larger or LR-2 and smaller will trigger an immediate ack. This seems ok. All distances greater than 2 also seem ok. Reordering distance of 1 is a bit tricky. This distance means that if LR+1 is received, that triggers an immediate ack. Similarly, anything smaller than or equal to LR-1 will also trigger immediate ack (anything smaller than the largest received). The current reordering recommendation in the transport draft will have the receiver acking immediately on receiving LR-1, and we should be able to cover that case, so we should be able to express this value. But this is a bit wonky, since LR+1 is basically the next expected, and this reordering value effectively turns into . It would be nice to avoid this problem. We can do that by using Ian's suggestion of special casing the value 1: "any reordering, as defined as a PN that does not increase the LR by 1, will trigger an ack." Similarly, special casing the value 0 for disable immediate acking on reordering is useful. Having worked through this, it doesn't seem that terrible. I'm ok with varint or single byte. I agree with NAME that the most usefulness will be in 1 byte, but I can see the value in leaving it open and uniform. Thoughts?
So the final algorithm becomes:
I'd rather have a varint for consistency, even if it's almost always a single byte.
NAME thank you for picking up my suggestion and writing the pseudo code. I'm not sure if just looking at the largest received is sufficient to properly cover all cases. Assume you receive packet , packet . At this point . Now you receive . I would have assumed that it makes sense now to send an immediate ACK to help with the recovery of , however, the algorithm doesn't capture that case. Furthermore, I was wondering about the negative branch (where ). This branch can only be taken if we sent an ACK earlier for exceeding the . Assuming that ACK was received, the packet in question would already have been declared lost, so in that case sending an immediate ACK wouldn't make any difference. Is my understanding correct that the negative branch exists for the case in which the original ACK was lost? Encoding-wise, we might be able to simplify things a bit replacing by (where a of 1 would be a of 2). Then we could avoid special-casing the value of 1. I'm also wondering how many immediate ACKs you're supposed to send. Currently, the draft has some quite complicated conditions for this: Do we need some similar advice here? Or do we need to make it configurable?
NAME on the point about how many immediate ACKs to send, I believe we should only send one, so I have an issue() and PR() to change the text you quoted. In terms of the 'negative branch' as you're describing it, the idea is that it's very possible the packet has not yet been declared lost yet, even though the ACK has arrived. If they're using this reordering tolerant setting, it seems likely they're tolerant to more reordering than a typical sender. For example, it could be because they have adaptive packet threshold loss detection and the threshold has been increased substantially due to observing past reordering.
We definitely need to have a few people ship this reordering tolerant feature before we finalize this draft. I'm quite confident in everything else in the draft, but I'm fairly unsure about exactly how this can and should be used. I'm also not yet sure packet threshold is actually better than time. It seems more appealing, but I think it depends on the use case. I'm also concerned it adds non-trivial implementation complexity. Implementations already have a maxackdelay and a number of packets before sending an immediate ACK, so this is just adding a mechanism to change those thresholds during the connection. However, this reordering feature may end up substantially changing the code to send ACKs. I'd be pretty sad if a bunch of implementations didn't implement the draft because they didn't want to deal with the extra complexity.
I agree, NAME which is why I wanted to go through and see if we could define simple enough behavior that is useful. I don't think we want to be perfect with the detection, but reasonable. I'll think of simplifications to this, but we want to make sure that we dont make this unnecessarily ornate. I am concerned that this threshold becomes another footgun, since we really don't have a good recommendation for how to set it. The primary property we want is a dynamic way to indicate to the peer to not send immediate acks on reordering. Simply moving the TP to the frame will give us that. I am increasingly leaning towards just calling this a boolean instead of a threshold. I will plan to write that up by tomorrow. Any strong objections?
Boolean as a frame field works perfectly fine for my use case.
I prefer "Ingnore Reordering" to "Ignore Order", though I admit the latter is slightly shorter. One suggestion on making the text slightly more proscriptive.

value smaller than the "min_ack_delay" advertised by this endpoint is invalid. <ins> An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01 (representing "true").  This field can be set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer observes reordering (reordering). </ins> Receipt of invalid values in an ACK_FREQUENCY frame MUST be treated as a connection error of type FRAME_ENCODING_ERROR. 5. An endpoint can send multiple ACK_FREQUENCY frames, and each one of  <ins> them can have different values in all fields.  An endpoint MUST use a sequence number of 0 for the first ACK_FREQUENCY frame it constructs and sends, and a strictly increasing value thereafter. </ins> An endpoint MUST allow reordered ACK_FREQUENCY frames to be received and processed, see Section 13.3 of QUIC-TRANSPORT. 6.1.  <ins> As specified in Section 13.3.1 of QUIC-TRANSPORT, endpoints are expected to send an acknowledgement immediately on receiving a reordered ack-eliciting packet.  This extension modifies this behavior. If the endpoint has not yet received an ACK_FREQUENCY frame, or if the most recent frame received from the peer has an "Ignore Order" value of "false" (0x00), the endpoint MUST immediately acknowledge any subsequent packets that are received out of order. If the most recent ACK_FREQUENCY frame received from the peer has an "Ignore Order" value of "true" (0x01), the endpoint does not make this exception.  That is, the endpoint MUST NOT send an immediate acknowledgement in response to packets received out of order, and instead continues to use the peer's "Packet Tolerance" and "max_ack_delay" thresholds for sending acknowledgements. </ins> 6.2.


A variable-length integer representing the minimum amount of time in microseconds by which the endpoint can delay an <del> acknowledgement.  Values of 0 and 2^24 or greater are invalid, and </del>  receipt of these values MUST be treated as a connection error of type PROTOCOL_VIOLATION.

I think minackdelay=0 should be allowed? It means there is no delay sending ack, which is a current behavior.
0 is equivalent to 1 as a value for the "Packet Tolerance" field I believe, so I agree that declaring it invalid seems unnecessary. Also, 0 and 1 are extremely similar given this is in microseconds, and 1 is valid.
0 and 1 are different, since 0 is immediate, and 1 involves a timer. Though, yeah, it seems silly to disallow immediate.

A variable-length integer representing the minimum amount of time in microseconds by which the endpoint can delay an  <ins> acknowledgement.  Values of 2^24 or greater are invalid, and </ins> receipt of these values MUST be treated as a connection error of type PROTOCOL_VIOLATION.


as a connection error of type FRAME_ENCODING_ERROR. ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not <del> require retransmission. </del>  An endpoint MAY send ACK_FREQUENCY frames multiple times during a connection and with different values.

If there's a frame with a larger sequence number, there's no need to retransmit.
Thanks, looks good ;-)

as a connection error of type FRAME_ENCODING_ERROR. ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not  <ins> require retransmission if an ACK_FREQUENCY frame with a larger Sequence Number value has been sent. </ins> An endpoint MAY send ACK_FREQUENCY frames multiple times during a connection and with different values.


in microseconds by which the endpoint can delay an acknowledgement.  Values of 2^24 or greater are invalid, and receipt of these values MUST be treated as a connection error of <del> type PROTOCOL_VIOLATION. </del>  An endpoint's min_ack_delay MUST NOT be greater than the its max_ack_delay.  Endpoints that support this extension MUST treat receipt of a min_ack_delay that is greater than the received <del> max_ack_delay as a connection error of type PROTOCOL_VIOLATION.  Note that while the endpoint's max_ack_delay transport parameter is in milliseconds (Section 18.2 of QUIC-TRANSPORT), min_ack_delay is specified in microseconds. </del>  This Transport Parameter is encoded as per Section 18 of QUIC- TRANSPORT.

Use TRANSPORTPARAMETERERROR for communicating errors in TP, rather than using PROTOCOL_VIOLATION.

in microseconds by which the endpoint can delay an acknowledgement.  Values of 2^24 or greater are invalid, and receipt of these values MUST be treated as a connection error of  <ins> type TRANSPORT_PARAMETER_ERROR. </ins> An endpoint's min_ack_delay MUST NOT be greater than the its max_ack_delay.  Endpoints that support this extension MUST treat receipt of a min_ack_delay that is greater than the received  <ins> max_ack_delay as a connection error of type TRANSPORT_PARAMETER_ERROR.  Note that while the endpoint's max_ack_delay transport parameter is in milliseconds (Section 18.2 of QUIC-TRANSPORT), min_ack_delay is specified in microseconds. </ins> This Transport Parameter is encoded as per Section 18 of QUIC- TRANSPORT.


eliciting packets after which the receiver sends an acknowledgement.  A value of 1 will result in an acknowledgement being sent for every ack-eliciting packet received.  A value of 0 <del> is invalid. </del>  A variable-length integer representing an update to the peer's "max_ack_delay" transport parameter (Section 18.2 of QUIC- TRANSPORT).  The value of this field is in microseconds.  Any value smaller than the "min_ack_delay" advertised by this endpoint <del> is invalid. </del>  An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01 (representing "true").  This field can be set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer observes reordering (reordering). <del> Receipt of invalid values in an ACK_FREQUENCY frame MUST be treated as a connection error of type FRAME_ENCODING_ERROR. </del>  ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission if an ACK_FREQUENCY frame with a larger

At the moment, the specification requires the use of FRAMEENCODINGERROR when an endpoint receives an ACKFREQUENCY frame with the value of Update Max Ack Delay field being smaller than . That contradicts from the practice of the core draft, that restricts the use of FRAMEENCODINGERROR to syntactic (i.e. encoding) errors. This PR fixes the problem by changing the error code to be used in that case to PROTOCOLVIOLATION.

eliciting packets after which the receiver sends an acknowledgement.  A value of 1 will result in an acknowledgement being sent for every ack-eliciting packet received.  A value of 0  <ins> is invalid.  Receipt of an invalid value MUST be treated as a connection error of type FRAME_ENCODING_ERROR. </ins> A variable-length integer representing an update to the peer's "max_ack_delay" transport parameter (Section 18.2 of QUIC- TRANSPORT).  The value of this field is in microseconds.  Any value smaller than the "min_ack_delay" advertised by this endpoint  <ins> is invalid.  Receipt of an invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION. </ins> An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01 (representing "true").  This field can be set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer observes reordering (reordering).  <ins> Receipt of any other value MUST be treated as a connection error of type FRAME_ENCODING_ERROR. </ins> ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission if an ACK_FREQUENCY frame with a larger


using an ACK_FREQUENCY frame, shown below: Following the common frame format described in Section 12.4 of QUIC- <del> TRANSPORT, ACK_FREQUENCY frames have a type of 0xAF, and contain the </del>  following fields: A variable-length integer representing the sequence number

We don't need to change the frame type on each draft version, but I figured that it'd be safer.

using an ACK_FREQUENCY frame, shown below: Following the common frame format described in Section 12.4 of QUIC-  <ins> TRANSPORT, ACK_FREQUENCY frames have a type of 0xaf, and contain the </ins> following fields: A variable-length integer representing the sequence number


max_ack_delay transport parameter is in milliseconds (Section 18.2 of QUIC-TRANSPORT), min_ack_delay is specified in microseconds.  This Transport Parameter is encoded as per Section 18 of QUIC- TRANSPORT.

Clearly declares that the TP only works in one direction.
To me it seems that the draft can be interpreted in the following two ways. It would be great if there was a clarification regarding which is correct: TP is a declaration that the endpoint will recognize ACKFREQUENCY frames. The receiver of the TP can send ACKFREQUENCY frames regardless of it sending TP or not. Use of delayed acks are negotiated by both endpoints sending TP. ACK_FREQUENCY frames can be used only when the both endpoints send the TP. I assume that the intent to be the latter, but I'm not sure.
Interesting, I actually interpreted it to be the former. I think it is reasonable for an implementation to accept ACK_FREQUENCY without having any intention of sending them itself. I.e. I don't see any reason it has to be mutually negotiated.
I agree that it's totally fine to consider the use being allowed independently for each direction. Though, for the ease of testing, I might prefer the use being negotiated mutually. It's a bit complex to have two flags; one that emits the minackdelay TP and another that controls the recognition of that TP sent by the peer. Endpoints that are not interested in sending ACKFREQUENCY frames can just send the minackdelay TP ~with the value set to maxack_delay~.
NAME -- that's not quite right. If this were unilateral, that would mean that an endpoint that receives the TP would not need to advertise the TP. That's a bit confusing, since the endpoint that sends the TP intends to use the ACK_FREQUENCY frame, which it wouldn't then be able to do. The most straightforward interpretation is the second one that NAME proposes.
I think it can be quite natural depending on how you see it. We can define the TP as a proclamation by an endpoint to respect ACK_FREQUENCY frames. I would also point out that most (maybe all) transport parameters are unilateral declarations of endpoint properties, rather than a negotiation. That said, with either of the approaches, it is impossible for an endpoint to declare support for multiple versions. I'm not sure if that's required, but it might be necessary. If we want to support multiple versions, then, we can either: Say that each endpoint unilaterally declares it's capabilities (or supported versions); then the peer chooses what to use and sends the chosen capability (or the version) using a frame, much like how ALPN is being selected. Say that the support for delayed-ack is negotiated by both endpoints. Each endpoints send a list of supported versions, and we define a logic for choosing one from the lists provided by both endpoints. Personally, I think I prefer the former, especially if we are to negotiate use of multiple extensions at once. But the necessity is still unclear.
Ah. I clearly misread what NAME was saying. Yeah, this does align with the rest of the TPs, and it also aligns well with the intent of this TP -- each endpoint has to declare that it will respect and act on an ACK_FREQUENCY frame when one is received. I like the unilateral declaration of capabilites. I don't think we will need to support multiple versions in the longer term, but it's worth supporting if it isn't difficult to do. And in this case, I think it naturally aligns well with the intent of the frame.
There's been no change on this in a while, so I'm planning to merge NAME PR soon unless someone speaks up.

max_ack_delay transport parameter is in milliseconds (Section 18.2 of QUIC-TRANSPORT), min_ack_delay is specified in microseconds. <ins> The min_ack_delay transport parameter is a unilateral indication of support for receiving ACK_FREQUENCY frames.  If an endpoint sends the transport parameter, the peer is allowed to send ACK_FREQUENCY frames independent of whether it also sends the min_ack_delay transport parameter or not. </ins> This Transport Parameter is encoded as per Section 18 of QUIC- TRANSPORT.


On the first received ACK_FREQUENCY frame in a connection, an endpoint MUST immediately record all values from the frame.  The sequence number of the frame is recorded as the largest seen sequence <del> number.  The new ACK-eliciting Threshold and Update Max Ack Delay </del>  values MUST be immediately used for delaying acknowledgements; see sending. acknowledgements as specified in Section 13.2.1 of QUIC-TRANSPORT. On receiving an ACK_FREQUENCY frame and updating its recorded <del> "max_ack_delay" and "ACK-eliciting threshold" values (multiple- </del>  frames), the endpoint MUST send an acknowledgement when one of the following conditions are met: Since the last acknowledgement was sent, the number of received ack-eliciting packets is greater than or equal to the recorded <del> "ACK-eliciting threshold". </del>  Since the last acknowledgement was sent, "max_ack_delay" amount of time has passed. "Ignore Order" value of "true" (0x01), the endpoint does not make this exception.  That is, the endpoint MUST NOT send an immediate acknowledgement in response to packets received out of order, and <del> instead continues to use the peer's "ACK-eliciting threshold" and </del>  "max_ack_delay" thresholds for sending acknowledgements. 6.2.

Part of We still need to decide whether to change 'Update Max Ack Delay' to 'Max Ack Delay'.

On the first received ACK_FREQUENCY frame in a connection, an endpoint MUST immediately record all values from the frame.  The sequence number of the frame is recorded as the largest seen sequence  <ins> number.  The new Ack-Eliciting Threshold and Update Max Ack Delay </ins> values MUST be immediately used for delaying acknowledgements; see sending. acknowledgements as specified in Section 13.2.1 of QUIC-TRANSPORT. On receiving an ACK_FREQUENCY frame and updating its recorded  <ins> "max_ack_delay" and "Ack-Eliciting Threshold" values (multiple- </ins> frames), the endpoint MUST send an acknowledgement when one of the following conditions are met: Since the last acknowledgement was sent, the number of received ack-eliciting packets is greater than or equal to the recorded  <ins> "Ack-Eliciting Threshold". </ins> Since the last acknowledgement was sent, "max_ack_delay" amount of time has passed. "Ignore Order" value of "true" (0x01), the endpoint does not make this exception.  That is, the endpoint MUST NOT send an immediate acknowledgement in response to packets received out of order, and  <ins> instead continues to use the peer's "Ack-Eliciting Threshold" and </ins> "max_ack_delay" thresholds for sending acknowledgements. 6.2.


maximum value it can represent, the endpoint MUST use the largest representable value instead. <del> A variable-length integer representing an update to the peer's "max_ack_delay" transport parameter (Section 18.2 of QUIC- TRANSPORT).  The value of this field is in microseconds.  Any value larger than 16,383,000 microseconds is invalid, as specified for the "max_ack_delay" transport parameter.  Any value smaller than the "min_ack_delay" advertised by this endpoint is invalid. Receipt of an invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION. </del>  An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01 which specifies the maximum amount of time by which the endpoint will delay sending acknowledgments.  When the endpoint receives an ACK_FREQUENCY frame, it MUST update this maximum time to the value <del> proposed by the peer in the Update Max Ack Delay field. </del>  5. On the first received ACK_FREQUENCY frame in a connection, an endpoint MUST immediately record all values from the frame.  The sequence number of the frame is recorded as the largest seen sequence <del> number.  The new Ack-Eliciting Threshold and Update Max Ack Delay </del>  values MUST be immediately used for delaying acknowledgements; see sending.

Fixes the remainder of not fixed by
This is good enough. You have a conflict to resolve, but let's merge this.

maximum value it can represent, the endpoint MUST use the largest representable value instead.  <ins> A variable-length integer representing the value to which the endpoint requests the peer update its "max_ack_delay" (Section 18.2 of QUIC-TRANSPORT).  The value of this field is in microseconds, unlike the 'max_ack_delay' transport parameter, which is in milliseconds.  Sending a value smaller than the "min_ack_delay" advertised by the peer is invalid.  Receipt of an invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION. </ins> An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01 which specifies the maximum amount of time by which the endpoint will delay sending acknowledgments.  When the endpoint receives an ACK_FREQUENCY frame, it MUST update this maximum time to the value  <ins> proposed by the peer in the Request Max Ack Delay field. </ins> 5. On the first received ACK_FREQUENCY frame in a connection, an endpoint MUST immediately record all values from the frame.  The sequence number of the frame is recorded as the largest seen sequence  <ins> number.  The new Ack-Eliciting Threshold and Request Max Ack Delay </ins> values MUST be immediately used for delaying acknowledgements; see sending.


A variable-length integer representing the minimum amount of time in microseconds by which the endpoint can delay an <del> acknowledgement.  Values of 2^24 or greater are invalid, and receipt of these values MUST be treated as a connection error of type TRANSPORT_PARAMETER_ERROR. </del>  An endpoint's min_ack_delay MUST NOT be greater than its max_ack_delay.  Endpoints that support this extension MUST treat A variable-length integer representing an update to the peer's "max_ack_delay" transport parameter (Section 18.2 of QUIC- TRANSPORT).  The value of this field is in microseconds.  Any <del> value smaller than the "min_ack_delay" advertised by this endpoint is invalid.  Receipt of an invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION. </del>  An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01

Does it make sense to specify any maximum values for any of the ACK frequency frame fields, such as for Packet Tolerance? Does a value of 4 billion make sense? Can I use a in my implementation to store this locally? Is there any expectation around the sizes of these fields? If there is no explicit max, what if I implicitly apply one? I'm working on implementing this and I'm trying to figure out how small (byte size) I can make the variables I use to store the values on the connection.
Update Max Ack Delay will be limited by the Max Ack Delay limit once lands. That leaves Packet Tolerance, which I agree is reasonable to limit to a value smaller than 2^62-1 locally. We could add a comment like "Implementations that have a lower limit than 2^62-1 on the Packet Tolerance MUST set treat it as though the maximum supported Packet Tolerance value has been received."
Slight tweak: "If an endpoint receives a Packet Tolerance value that is larger than the maximum value it can represent, the endpoint MUST use the largest representable maximum value instead."
If I understand correctly, the maximum value of the transport parameter is supposed to ensure that the value is smaller or equal than the transport parameter. According to the maximum value of is . However, the maximum value of is .
We're going to remove the 2^24 - 1 limit and just rely on the MUST that minackdelay can't be greater than maxackdelay, which implies the 2^14-1 limit.
QUIC Transport specifies a maximum value of 2^14 ms (roughly 16s) for the transport parameter. Does this limit apply to values sent in the field of the ACK_FREQUENCY frame, or is any value up to 2^62 microseconds allowed here? Either way, the draft might benefit from clarifying the rules here.
The editors do not have a strong opinion, and we currently say nothing about the limit being changed by this extension, so we're inclined to keep the limit as is and add some text clarifying that the extension doesn't change the limit. We're open to arguments otherwise if people have them.

A variable-length integer representing the minimum amount of time in microseconds by which the endpoint can delay an  <ins> acknowledgement. </ins> An endpoint's min_ack_delay MUST NOT be greater than its max_ack_delay.  Endpoints that support this extension MUST treat A variable-length integer representing an update to the peer's "max_ack_delay" transport parameter (Section 18.2 of QUIC- TRANSPORT).  The value of this field is in microseconds.  Any  <ins> value larger than 16,383,000 microseconds is invalid, as specified for the "max_ack_delay" transport parameter.  Any value smaller than the "min_ack_delay" advertised by this endpoint is invalid. Receipt of an invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION. </ins> An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01


acknowledgement.  A value of 1 will result in an acknowledgement being sent for every ack-eliciting packet received.  A value of 0 is invalid.  Receipt of an invalid value MUST be treated as a <del> connection error of type FRAME_ENCODING_ERROR. </del>  A variable-length integer representing an update to the peer's "max_ack_delay" transport parameter (Section 18.2 of QUIC-

Does it make sense to specify any maximum values for any of the ACK frequency frame fields, such as for Packet Tolerance? Does a value of 4 billion make sense? Can I use a in my implementation to store this locally? Is there any expectation around the sizes of these fields? If there is no explicit max, what if I implicitly apply one? I'm working on implementing this and I'm trying to figure out how small (byte size) I can make the variables I use to store the values on the connection.
Update Max Ack Delay will be limited by the Max Ack Delay limit once lands. That leaves Packet Tolerance, which I agree is reasonable to limit to a value smaller than 2^62-1 locally. We could add a comment like "Implementations that have a lower limit than 2^62-1 on the Packet Tolerance MUST set treat it as though the maximum supported Packet Tolerance value has been received."
Slight tweak: "If an endpoint receives a Packet Tolerance value that is larger than the maximum value it can represent, the endpoint MUST use the largest representable maximum value instead."

acknowledgement.  A value of 1 will result in an acknowledgement being sent for every ack-eliciting packet received.  A value of 0 is invalid.  Receipt of an invalid value MUST be treated as a  <ins> connection error of type FRAME_ENCODING_ERROR.  If an endpoint receives an ACK-eliciting threshold value that is larger than the maximum value it can represent, the endpoint MUST use the largest representable value instead. </ins> A variable-length integer representing an update to the peer's "max_ack_delay" transport parameter (Section 18.2 of QUIC-


"max_ack_delay" for PTO computations, endpoints MUST use the maximum of the current value and all those in flight.  8. There are tradeoffs inherent in a sender sending an ACK_FREQUENCY

by allowing implementations to not add maxackdelay to the PTO calculation if enough ack-eliciting packets are in flight.
NAME Does Jana's longer explanation help or would you like more clarification?
With this extension, an immediate ACK is expected if more than the "Packet Tolerance" ack-eliciting packets are outstanding. Given that, it may be a worthwhile optimization to allow the maxackdelay to not be included in the PTO computation in some cases. If we're not going to allow that, I think we should explain why. URL This split off , but I think the question of whether maxackdelay always needs to be included in PTO should be separate from whether QUIC would benefit from an ACK-pull mechanism of some sort. That being said, if a mechanism did exist, we add details about when ACK-pull allowed maxackdelay to not be included.
This is a clever optimization, I agree it's worth some text.
I'm not sure I understand the motivation for this (I'm also having trouble following the issue). Can we add some more explanation here?one suggestion, but lgtm otherwise.

"max_ack_delay" for PTO computations, endpoints MUST use the maximum of the current value and all those in flight. <ins> When the number of in-flight ack-eliciting packets is larger than the ACK-Eliciting Threshold, an endpoint can expect that the peer will not need to wait for its "max_ack_delay" period before sending an acknowledgement.  In such cases, the endpoint MAY therefore exclude the peer's 'max_ack_delay' from its PTO calculation.  Note that this optimization requires some care in implementation, since it can cause premature PTOs under packet loss when "ignore_order" is enabled. </ins> 8. There are tradeoffs inherent in a sender sending an ACK_FREQUENCY


independent of whether it also sends the min_ack_delay transport parameter or not.  Endpoints MUST NOT remember the value of the min_ack_delay transport parameter they received.  Consequently, ACK_FREQUENCY frames cannot be sent in 0-RTT packets, as per Section 7.4.1 of QUIC-TRANSPORT.

I found an interesting regression when implementing the delayed ack extension. The extension allows the sender of the data to control the max ack delay and the ack gap. In general that's a good idea, but not for the 2 RTT of the connection. Once the client obtains the 1RTT key, it will start receiving data sent by the server in "false start". At that point the client has assessed the RTT of the connection, but the server has not. The "max ack delay" sent by the server is a guess based on statistics. It will typically be longer than recommended values like RTT/4. The effect is that the first ACKs are sent too late, which has a cascading effect with some congestion control algorithms. My fix was to use the values locally computed by the client until the session is confirmed and the server can send accurate "ack frequency" frames. Not sure whether we should have text about that in the draft.
I'm not completely clear on the details here, so hopefully you can clarify. Is the server sending an ACK_FREQUENCY frame in 0.5RTT before it has any RTT samples? If so, yes I would say that's a bad idea and we should add a SHOULD NOT to advise against that. And the regression is that slow start is slowed down substantially vs the default behavior of sending an ACK every 2 packets? Or is there another regression? Possibly this issue is a variation on about recommendations and potential pitfalls of this extension frame?
At 0.5 RTT, the server will start sending data frames: HTTP settings for example, or maybe responses to client requests in 0-RTT packets. As you say, the server should not send an ACK FREQUENCY frame in 0.5RTT before it has any RTT samples. Let's assume it doesn't. The client knows that Min Ack Delay has been negotiated. In that case, my first implementation of the extension was completely deferring to the server, and just setting Ack Delay Max to the parameter advertised by the server. After all, the draft does says that the ACK FREQUENCY frame carries an update to the peer's "maxackdelay" transport parameter. That was be by the book. But it was also causing regressions in a number of performance tests, because the value in the TP is typically an upper bound. The acks were sent too late, the connection was ramping up too slowly.
Thanks, presumably the client is setting "Packet Tolerance" to a value larger than 2 in its ACK_FREQUENCY frame? If so, that makes sense. In the "Motivations" section, the existing draft says: "Starting a connection up quickly without inducing excess queue is important for latency reduction, for both short and long flows. The sender often needs more frequent acknowledgments during this phase." But maybe there needs to be more cautionary text later in the draft, and less text in "Motivations"?
Summarizing editors' discussion: the issue here seems to be that in an interpretation of the mechanism, the client could change its behavior after receiving but before receiving an ACK-FREQUENCY frame. This should not happen... the goal of is to negotiate future use of ACK-FREQUENCY frames. Until an ACK-FREQUENCY frame is received, an endpoint continues with RFC 9000 behavior. We will add text to this effect. NAME does this work for you?
NAME yes, that makes sense. I will check that the picoquic code does that.
Works for me.

independent of whether it also sends the min_ack_delay transport parameter or not. <ins> Receiving a min_ack_delay transport parameter indicates that the peer might send ACK_FREQUENCY frames in the future.  Until an ACK_FREQUENCY frame is received, receiving this transport parameter does not cause the endpoint to change its acknowledgement behavior. </ins> Endpoints MUST NOT remember the value of the min_ack_delay transport parameter they received.  Consequently, ACK_FREQUENCY frames cannot be sent in 0-RTT packets, as per Section 7.4.1 of QUIC-TRANSPORT.


independent of whether it also sends the min_ack_delay transport parameter or not.  This Transport Parameter is encoded as per Section 18 of QUIC- TRANSPORT.

and And by implication, cannot send ACK_FREQUENCY in 0-RTT packets.
The transport I-D 31 : This information is missing from .
FWIW, I tend to think that this issue is coupled with , and that endpoints do not need to, due to the reasons being explained on that issue.
Fixed by
See Table in of the transport draft.
We recommend endpoints to ACK Initial and Handshake packets immediately (see URL). I do not think we need to change that, and therefore my view is that use of ACK_FREQUENCY frames can (and should) be limited to 1-RTT.
Agreed, I think we just need to specify that.
Just a note that, new TPs are required to clarify if the TP is to be remembered (URL). Based on the understanding that the use of the ACKFREQUENCY frame will be limited to 1-RTT packets, I assume that remembering the minack_delay TP will be prohibited.
Good point NAME I do not believe minackdelay should be remembered, since it has no use prior to handshake completion.
I agree that ACKFREQUENCY should not apply to Initial and Handshake packets. That doesn't mean that we need to limit ACKFREQUENCY to 1-RTT packets though, it could apply to 0-RTT packets as well, if peers are required to remember the value of the TP. The client could then send an ACK_FREQUENCY in 0-RTT. I don't have a strong preference in either direction, but preventing a flood of ACKs for 0-RTT packets sounds like it could potentially be useful.
Considering our track record of allowing most frames to be sent in 0-RTT, I am fine with allowing the transmission of ACKFREQUENCY frames in 0-RTT. OTOH, I do not think we should require endpoints to retain minackdelay TP across resumptions. That would essentially prohibits client from sending ACKFREQUENCY frames in 0-RTT.
NAME Your two statements seem to contradict each other. If we allow ACKFREQUENCY in 0-RTT, peers need to remember the value of the TP. If peers are not required to remember the value of the TP, ACKFREQUENCY can't be sent in 0-RTT, otherwise a client would risk a PROTOCOL_VIOLATION if a server disabled the extension.
NAME That's a good point. And as you might have already observed, even if the client remembered minackdelay, it might still not be possible to send an ACKFREQUENCY frame in 0-RTT. That's because maxackdelay would be unknown. I think we are reaching consensus that clients would not send ACKFREQUENCY frames in 0-RTT, even if the specification allowed that?
Now I'm a bit confused. In my mind, this is how this extension works (describing it using "client" and "server" for added clarity, it works the same way in the opposite direction): The client declares a maxackdelay in the transport parameters. When generating acknowledgements, the server will then make sure that every ACK is generated within that time frame. When the client sends an ACKFREQUENCY frame, it instructs the server to discard the previous maxackdelay value and use the one encoded in the frame instead. Assuming that my understanding is correct, the fact that the client hasn't received the server's maxackdelay should be irrelevant when considering whether we want to send this frame in 0-RTT packets. Furthermore, the fact that the ClientHello has to be received before the server can decode any 0-RTT packet would guarantee that the server has processed the client's transport parameters before it is able to process any ACKFREQUENCY frame.
NAME and I chatted about this; summarizing. An endpoint's ACKFREQUENCYFRAME updates the received maxackdelay. This makes all situations where an endpoint sends an ACKFREQUENCYFRAME without receiving the peer's maxackdelay problematic. Since a client does not remember the peer's maxackdelay, sending an ACKFREQUENCYFRAME that updates it makes little sense. We should disallow ACKFREQUENCYFRAMEs in 0-RTT packets because a client does not have the base value which it wants to update with this frame until 1-RTT keys are established.
And we agree with NAME that is intertwined with this issue, since if the transport parameter is not remembered, you can't send the ACK_FREQUENCY frame.
My preference here is "application" packet number space. This means that the transport parameter needs to be remembered. It also means that - because maxackdelay is not remembered - that you have to avoid exceeding the maxackdelay default (and permit that variance as a server if you set a lower value for maxackdelay).
My real issue is whether the frames might appear before the TP negotiation has concluded. The client should not send ACK FREQUENCY frames before that. I think this naturally precludes sending ACK FREQUENCY in 0-RTT, but there might be corner cases in which the client knows the result of the TP negotiation before it has access to the 1-RTT keys -- for example if some of the crypto processing is going on in a crypto offload engine. If we are firm on negotiation, then we don't need to make a statement about 0RTT vs 1RTT.
NAME I'm not sure if that follows the design of RFC 9000. For frames that can be sent in 0-RTT, RFC 9000 requires endpoints to remember and reuse the value of the Transport Parameters:
Another potential option here in support of the basic design: this document could require that the value of maxackdelay be remembered alongside minackdelay, (only)? for the generation of ACK_FREQUENCY frames.
NAME Would you mind elaborate why your implementation cannot prohibit frames being sent in 0-RTT packets from being affected by TPs other than remembered ones? RFC 9000 states that when sending frames in 0-RTT packets, a client MUST only use remembered transport parameters, and that the definition of a new transport parameter MUST specify whether storing the transport parameter for 0-RTT is mandatory, optional, or prohibited (). So it seems to me that, for Transport parameters that are prohibited to be remembered, QUIC stacks are expected to be capable of prohibiting those parameters to be used when sending frames using 0-RTT packets. It seems to me that you are against us building extensions on that guidance.
I think there is broad support for not sending ACK_FREQUENCY frames before the negotiation completes. In response to the comment from NAME this requires that "storing the transport parameter for 0-RTT is prohibited." If we do that, we also satisfy the requirement stated by NAME that 0-RTT processing be pretty much the same as 1-RTT processing: there is no new difference between 0-RTT and 1-RTT, in both cases the transport can only send the frame if the peer has explicitly accepted them.
Thanks NAME and NAME I agree that if we prohibit storing minackdelay, ACKFREQUENCY can't be sent in 0-RTT, and I believe that should solve NAME issue? Storing BOTH minackdelay and maxack_delay is a direction I'd really rather avoid.
This works to ensure that there are no surprises for servers. But it doesn't prevent clients from sending the frame in 0-RTT. Our stack will apply transport parameters after receiving them. This is before handshake completion, so if we are still sending 0-RTT, we might send the frame. I don't believe that we will in practice, due to the logic we use for sending it. That logic is something that might change, so there are no strong guarantees. I do think that this is a good solution, I just want to make the consequences clear.
NAME If we are to prohibit minackdelay from being remembered, I do not think we can allow them to be sent in 0-RTT. Requoting from RFC 9000, when sending frames in 0-RTT packets, a client MUST only use remembered transport parameters, and that the definition of a new transport parameter MUST specify whether storing the transport parameter for 0-RTT is mandatory, optional, or prohibited (). Am I missing something? Also, I would point out that, even for remembered TPs, QUIC stacks are required to refrain from using the newly received ones in 0-RTT packets. For example, if the remembered value of MAXSTREAMS is 0 and the newly received value is 4, an endpoint is forbidden to open any stream in 0-RTT even after receiving the transport parameters. It is only when you switch to using 1-RTT packets that the newly received transport parameter is applied. For your implementation, wouldn't it be possible to use the same latch for ackfrequency?
Oh, whoops. We don't have that latch. We'll probably open 4 streams if initialmaxstreams_xxx is 4 in the handshake and we remembered 0. In any case, we can probably add a latch. It might even make the code simpler.
Thank you for the PR. Looks good, below is nothing more than an editorial URL of suggestions, but lgtm

independent of whether it also sends the min_ack_delay transport parameter or not. <ins> Endpoints MUST NOT remember the value of the min_ack_delay transport parameter they received.  Consequently, ACK_FREQUENCY frames cannot be sent in 0-RTT packets, as per Section 7.4.1 of QUIC-TRANSPORT. </ins> This Transport Parameter is encoded as per Section 18 of QUIC- TRANSPORT.


receipt of a min_ack_delay that is greater than the received max_ack_delay as a connection error of type PROTOCOL_VIOLATION.  Note that while the endpoint's max_ack_delay transport parameter is in <del> milliseconds (Section 18.2 in QUIC-TRANSPORT), min_ack_delay is </del>  specified in microseconds. 4. is invalid. A variable-length integer representing an update to the peer's <del> "max_ack_delay" transport parameter (Section 18.2 in QUIC- </del>  TRANSPORT).  The value of this field is in microseconds.  Any <del> value smaller than the min_ack_delay advertised by this endpoint </del>  is invalid. <del> Receipt of invalid values in an ACK-FREQUENCY frame MUST be treated as a connection error of type PROTOCOL_VIOLATION. </del>  ACK-FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission. <del> An endpoint MUST NOT send an empty ACK-FREQUENCY frame.  Endpoints MUST treat receipt of an empty ACK-FREQUENCY frame as a connection error of type PROTOCOL_VIOLATION. </del> An endpoint MAY send ACK-FREQUENCY frames multiple times during a connection and with different values. <del> An endpoint will have committed a max_ack_delay value to the peer, </del>  which specifies the maximum amount of time by which the endpoint will delay sending acknowledgments.  When the endpoint receives an ACK- FREQUENCY frame, it MUST update this maximum time to the value 7. <del> TBD. </del>  8. TBD.

merging.
If it doesn't change it, we should say so. If it allows it to be reduced, then we need to understand what the limits are(ie: it can't be lowered to 1us presumably?). Admittedly, the way we're using maxackdelay should be safe to assuming a 0 value, since the SRTT will accrue the excess.

receipt of a min_ack_delay that is greater than the received max_ack_delay as a connection error of type PROTOCOL_VIOLATION.  Note that while the endpoint's max_ack_delay transport parameter is in  <ins> milliseconds (Section 18.2 of QUIC-TRANSPORT), min_ack_delay is </ins> specified in microseconds. 4. is invalid. A variable-length integer representing an update to the peer's  <ins> "max_ack_delay" transport parameter (Section 18.2 of QUIC- </ins> TRANSPORT).  The value of this field is in microseconds.  Any  <ins> value smaller than the "min_ack_delay" advertised by this endpoint </ins> is invalid.  <ins> Receipt of an empty ACK-FREQUENCY frame or invalid values in an ACK- FREQUENCY frame MUST be treated as a connection error of type PROTOCOL_VIOLATION. </ins> ACK-FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission.  An endpoint MAY send ACK-FREQUENCY frames multiple times during a connection and with different values.  <ins> An endpoint will have committed a "max_ack_delay" value to the peer, </ins> which specifies the maximum amount of time by which the endpoint will delay sending acknowledgments.  When the endpoint receives an ACK- FREQUENCY frame, it MUST update this maximum time to the value 7.  <ins> On sending an update to the peer's "max_ack_delay", an endpoint can use this new value in later computations of its Probe Timeout (PTO) period; see Section 5.2.1 of QUIC-RECOVERY.  The endpoint MUST however wait until the ACK-FREQUENCY frame that carries this new value is acknowledged by the peer. Until the frame is acknowledged, the endpoint MUST use the greater of the current "max_ack_delay" and the value that is in flight when computing the PTO period.  Doing so avoids spurious PTOs that can be caused by an update that increases the peer's "max_ack_delay". While it is expected that endpoints will have only one ACK-FREQUENCY frame in flight at any given time, this extension does not prohibit having more than one in flight.  Generally, when using "max_ack_delay" for PTO computations, endpoints MUST use the maximum of the current value and all those in flight. </ins> 8. TBD. <ins> 9. TBD. </ins>


invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION. <del> An 8-bit field representing a boolean truth value.  This field MUST have the value 0x00 (representing "false") or 0x01 (representing "true").  This field can be set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer observes reordering (reordering). Receipt of any other value MUST be treated as a connection error of type FRAME_ENCODING_ERROR. </del>  ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission if an ACK_FREQUENCY frame with a larger

Fixes issue .
Thanks for the PR!
Section 4 explains what all the fields are mean first and then explains the exception cases. It seems like this was done the opposite way for Ignore order. I propose we change the current text: To:
SGTM, want to send me a PR or should I?

invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION.  <ins> An 8-bit field representing a boolean truth value.  This field is set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer observes reordering (reordering).  The value of this field MUST be 0x00 (representing "false") or 0x01 (representing "true").  Receipt of any other value MUST be treated as a connection error of type FRAME_ENCODING_ERROR. </ins> ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission if an ACK_FREQUENCY frame with a larger


ACK-FREQUENCY frames have a type of 0xXX, and contain the following fields:  A variable-length integer representing the maximum number of ack- eliciting packets after which the receiver sends an acknowledgement.  A value of 1 will result in an acknowledgement 5. On receiving an ACK-FREQUENCY frame, an endpoint will have an updated <del> maximum delay for sending an acknowledgement, and a packet threshold as well.  The endpoint MUST send an acknowledgement when either of the two thresholds are met. loss and batch section describes exceptions to this strategy. </del>  An endpoint is expected to bundle acknowledgements when possible. Every time an acknowledgement is sent, bundled or otherwise, all 6. An endpoint can send multiple ACK-FREQUENCY frames, and each one of <del> them can have different values. If a received ACK-FREQUENCY frame is the first one in this connection, the endpoint MUST immediately record any values from the frame and start using them.  The endpoint MUST also record the packet number of the enclosing packet. If a received ACK-FREQUENCY frame is not the first one in this connection, the endpoint MUST check if this is a more recent ACK- FREQUENCY frame than any previous ones, as follows: If the enclosing packet number is greater than the recorded one, the endpoint MUST immediately replace old recorded state with values received in this frame.  The endpoint MUST also replace the value of the recorded packet number with that of the enclosing packet. If the enclosing packet number is not greater than the recorded one, the endpoint MUST ignore this ACK-FREQUENCY frame. </del>  7.

merging.
We currently use PNs for ordering, but that runs afoul of requirements at the receiver specified in .

ACK-FREQUENCY frames have a type of 0xXX, and contain the following fields: <ins> A variable-length integer representing the sequence number assigned to the ACK-FREQUENCY frame by the sender, see multiple- frames. </ins> A variable-length integer representing the maximum number of ack- eliciting packets after which the receiver sends an acknowledgement.  A value of 1 will result in an acknowledgement 5. On receiving an ACK-FREQUENCY frame, an endpoint will have an updated  <ins> "max_ack_delay" and a "Packet Tolerance" threshold for sending an acknowledgement.  The endpoint MUST send an acknowledgement when either of the two thresholds are met.  That is, an endpoint MUST send an acknowledgement when one of the following conditions are met: Since the last acknowledgement was sent, the number of received ack-eliciting packets is greater than or equal to "Packet Tolerance". Since the last acknowledgement was sent, "max_ack_delay" amount of time has passed. loss and batch section describes exceptions to this strategy. </ins> An endpoint is expected to bundle acknowledgements when possible. Every time an acknowledgement is sent, bundled or otherwise, all 6. An endpoint can send multiple ACK-FREQUENCY frames, and each one of  <ins> them can have different values.  An endpoint MUST use a sequence number of 0 for the first ACK-FREQUENCY frame it constructs and sends, and a strictly increasing value thereafter. An endpoint MUST allow reordered ACK-FREQUENCY frames to be received and processed, see Section 13.3 of QUIC-TRANSPORT. On the first received ACK-FREQUENCY frame in a connection, an endpoint MUST immediately record all values from the frame.  The sequence number of the frame is recorded as the largest seen sequence number.  The new Packet Tolerance and Update Max Ack Delay values MUST be immediately used for delaying acknowledgements; see sending. On subsequently received ACK-FREQUENCY frames, the endpoint MUST check if this is a more recent frame than any previous ones, as follows: If the enclosing packet number is not greater than the largest one seen so far, the endpoint MUST ignore this frame. If the sequence number in the frame is greater than the largest one seen so far, the endpoint MUST immediately replace old recorded state with values received in this frame.  The endpoint MUST start using the new values immediately for delaying acknowledgements; see sending.  The endpoint MUST also replace the recorded sequence number. </ins> 7.


receivers to ignore obsolete frames, see multiple-frames. A variable-length integer representing the maximum number of ack- <del> eliciting packets the recipient of this frame can receive before sending an immediate acknowledgment.  A value of 0 will result in an immediate acknowledgement whenever an ack-eliciting packet received.  If an endpoint receives an ACK-eliciting threshold value that is larger than the maximum value it can represent, the endpoint MUST use the largest representable value instead. </del>  A variable-length integer representing the value to which the endpoint requests the peer update its "max_ack_delay"

In , I noted that the text around a maximum value is a little squishy. It says: The problem here is not about representing values. We have to assume that 62-bit values can be represented easily by anything that can do QUIC. The real problem is that there is a value beyond which performance tanks badly. Endpoints might want to set a limit for this to prevent that from happening. Why not instead just say that endpoints can set a limit for this threshold? That means that we might need to provide some advice around setting the limit. For instance, maybe we might suggest that a value that exceeds the current congestion window (measured in MTUs) might be too high.
The receiver doesn't know the peer's congestion window, so that might be tricky. If there's a value beyond which performance tanks, isn't that an error on the part of the sender of the ACK_FREQUENCY frame? I agree one could use this frame to degrade performance, but the sender could also just set a max CWND of 1 to degrade performance, which would be a lit simpler.
I agree that providing advice on setting a limit is tricky and there are other ways to destroy performance. The worst case is actually when both the threshold and the max ack delay are unreasonably large.
We could bound the delay to the SRTT as measured locally. It's not guaranteed to be the same value the sender measures, but it could be close enough to mitigate large issues. That being said, I am still unsure if this is a problem worth solving, given the sender should have a strong incentive to specify good values.
I'm ambivalent regarding whether we provide advice on limits. We might try to capture some of this general advice. Or not. I'm mostly concerned that we remove this pointless clause.
That's there for a specific reason. If you store the threshold as a uint_8, and the peer sends you something larger we want clear and consistent guidance on what to do with it(ie: set the value to 255). If that's unclear, editorial suggestions are welcome.
FWIW, when I read this the first time, the thought that an implementation would limit the value to 8 or 16 bit came immediately to mind, like NAME describes.
This text is about choice of internal representation at the receiver. I do not recall what caused this to get added, but as I think about this more now, I am sympathetic to a part of NAME point of view. However, this isn't about performance (as NAME notes) -- we don't want the receiver second guessing the sender's intent. So the text currently says that a receiver can choose to limit the size of this threshold. However, we provide no guidance as to why a receiver might choose a lower value, leaving this question open to implementations. I think we have two choices: the text should be clear about the tradeoffs and how a receiver might make this choice, or simply not say anything here, with the expectation that receivers ought to respect the sender's request (use uint64). In the spirit of the rest of this draft, I propose that: we follow (2), and if there is a compelling reason to allow a receiver to use something smaller, introduce text to say that the receiver "SHOULD respect sender's request" and explain the compelling exceptional case. Do we have a compelling exception?
I think that (2) alone is fine. I don't have a compelling exception. Note that we also have a time limit. This value might be deliberately set high in order to engage the timer rather than the counter. BOTH might be set ridiculously high, in which case the connection might then faceplant spectacularly, but we don't have to make the protocol completely foolproof.
The original issue was . Microsoft is using a uint8_t for this field, not a uint64, so they can't represent the full value. We can remove the sentence and assume implementations will do the right thing, but we can't assume everyone will use a uint64.
I think the main trouble here is with the normative MUST when an endpoint chooses to go off-spec. As NAME noted, we folks do want to use a < 64-bit value, so it might still be useful to note something here. So here's what I propose: "An endpoint might use a local representation for this value that is smaller than 64 bits for optimization purposes. If it does so, and if it receives an ACK-Eliciting Threshold value that is larger than the maximum value it can represent locally, the endpoint can use the largest representable value instead."
I would prefer that people go off-spec rather than make allowances like this in the spec. Honestly though, msquic is probably safe with clamping to 255 maximum. No one can say they aren't compliant if they acknowledge earlier.
I'm fine with leaving the text out. I am sympathetic to NAME view, despite the fact that we've had this text in the spec. I'm ok with removing it. NAME NAME -- thoughts?
I added this text to address NAME issue, so I'll let him decide.
I don't care enough to die on this hill, but I do think it's better to have text indicating the peer might have limits and acknowledge earlier as a result, than to have to text at all.
Ok, I've reworded to simply say that a receiver can ack earlier. PTAL NAME NAME NAME NAME
While reading the latest ack-frequency draft, I am confused with the wording that was added in URL If I have ack-eliciting threshold = 1, to me it means that I can receive 1 ack-eliciting packet before sending an ACK, i.e. I have to send ACK after receiving 1 packet. But looking at the example of 0, this interpretation looks wrong. NAME suggested, maybe the "before" needs to be "without" and an additional sentence added: "An immediate acknowledgement is sent when more than this number of packets have been received." I agree with his suggestion.
Thanks for the issues, this is a bit unclear. PTAL at
A good change, not sure about the other part.

receivers to ignore obsolete frames, see multiple-frames. A variable-length integer representing the maximum number of ack-  <ins> eliciting packets the recipient of this frame can receive without sending an immediate acknowledgment.  An immediate acknowledgement is sent when more than this number of ack-eliciting packets have been received, so value of 0 results in an immediate acknowledgement.  If an endpoint receives an ACK-Eliciting Threshold value that is larger than the maximum value the endpoint can represent, the endpoint MUST use the largest representable value instead. </ins> A variable-length integer representing the value to which the endpoint requests the peer update its "max_ack_delay"


number of acknowledgments allows connection throughput to scale much further. <del> As discussed in implementation however, there are undesirable </del>  consequences to congestion control and loss recovery if a receiver uniltaerally reduces the acknowledgment frequency.  A sender's constraints on the acknowledgement frequency need to be taken into receivers to ignore obsolete frames, see multiple-frames. A variable-length integer representing the maximum number of ack- <del> eliciting packets the recipient of this frame can receive before sending an immediate acknowledgment.  A value of 0 will result in an immediate acknowledgement whenever an ack-eliciting packet received.  If an endpoint receives an ACK-eliciting threshold value that is larger than the maximum value it can represent, the endpoint MUST use the largest representable value instead. </del>  A variable-length integer representing the value to which the endpoint requests the peer update its "max_ack_delay" invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION. <del> An 8-bit field representing a boolean truth value.  This field is </del>  set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer observes reordering <del> (reordering).  The value of this field MUST be 0x00 (representing "false") or 0x01 (representing "true").  Receipt of any other value MUST be treated as a connection error of type FRAME_ENCODING_ERROR. </del>  ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission if an ACK_FREQUENCY frame with a larger 7.2. <del> As specified in Section 13.2.1 of QUIC-TRANSPORT, an endpoint SHOULD immediately acknowledge packets marked with the ECN Congestion Experienced (CE) codepoint in the IP header.  Doing so reduces the peer's response time to congestion events. </del>  7.3. acknowledgments will cause undesirable bursts of data into the network.  10. TBD.

I think we shouldn't make this change until we understand the use case for this field. IMO, it is harmful to the networks. Happy to discuss this over slack or at
When thinking about , it occurred to me that there may be use cases for not changing the acknowledgement behavior in the face of ECN. The most obvious example is an ECN scheme where the bit means faster/slower, and so at equilibrium, every other packet should be marked. This type of marking would trigger the most pathological ACK behavior of either the current draft(ie: RFC9000) or .
This seems like a good way to manage acknowledgment behaviour when different ECN schemes are in use. It's not necessarily a complete solution, but it should help avoid the worst sorts of reactions.
How to send an ACK on CE marking can be a receiver side logic, one ACK when CE marking changes. I agree if every other packet is marked, then that triggers every other ACK, but I don't understand why this would happen in steady state. Would be good to know a use-case for this. Are there networks that are marking CE incorrectly - for that we should disable ECN as described in RFC 9000. This has similar implications as where the delayed CE marks will result in delayed response to congestion at the sender and hence the network queuing will continue to increase possibly causing more CE marks on other responsive flows (that don't have ignore-ecn set to true) and will cause unfairness.
I had this Netdev conference presentation and paper in mind by Hari Balakrishnan of MIT: URL It's research, but it's a sensible algorithm in certain environments like cell networks.
Thank you NAME for providing the link. I watched the video of the talk and it doesn't seem like he is using ECN along with Accelerate and Brake. One can simply disable ECN if they don't wish to use that method in the data packets. Disabling of ECN can also happen in the middle of the connection. I don't think we need to say, Sender wants to use ECN but he doesn't want to get immediate ACKs when CE marks are received at the receiver - that sounds contradictory to me.
NAME your comment URL seems like a good choice to me. ACK immediately whenever there is transition.
Most endpoints ignore ECN today, so I don't see how this is more harmful than the status quo? Some endpoints negotiate ECN and then ignore it as well. With Hari's scheme, the Ignore CE option is exactly what you'd want, because a transition from non-CE to CE happens so frequently.
How the stacks ignore today would make the other side disable ECN even if it was negotiated successfully. Meaning, the sender will start sending Non-ECT in its packets and the routers will drop instead of marking CE. But if the sender continues to send packets with ECT(0) or ECT(1) after sending ignore-ECN = true, that means this sender is greedy as it doesn't want its packets to be dropped at the bottleneck and also, it doesn't want to respond the congestion marks right away. For senders using classic ECN, this is OK only when the ACKs eventually come within 1RTT ballpark as classic congestion controllers reduce cwnd once per RTT. But, for L4S scalable congestion control, this isn't good enough as the ACK with CE feedback should come immediately (with respect to time). Multiple packets with CE marks can be bundled into one ACK - which is the ACK on transition from CE to non-CE method.
There are many scenarios to use ECN, and today the most valuable one is the datacenter, not the public internet. In a datacenter, this may be an optimal ACK-ing scheme. In terms of classic vs L4S, it seems like both have delays which would be fairly tolerable (ie: 1 RTT and 1/8 or 1/4 RTT depending upon one's loss threshold), but larger delays could reduce their responsiveness to congestion. This is similar to Ignore Order, since both are signals which may indicate congestion, so the same guidance could apply to both to ensure sufficiently fast feedback in the case of possible loss or ECN marking.
Yes, they are the same with regards to delayed response to congestion and we are discussing that at URL One subtle difference between the two is that in case of ECN, the packets are never dropped which could cause slightly more unfairness to other flows. Similar to , we can add a limit to Max Ack Delay when ignore-ecn is used.
Even middleboxes which mark ECN also drop packets, they just mark earlier than they drop. I'm not aware of devices which when ECN is negotiated, actually don't drop packets and instead rely entirely on ECN. This seems like a recipe for unfairness, so I can't imagine someone doing this If you have evidence this really exists in the wild, that'd be interesting, but I think it's a fundamentally unfair design choice.
Classic ECN RFC 3168 says, First, ECN-Capable routers will only mark packets (as opposed to dropping them) when the packet marking rate is reasonably low. During periods where the average queue size exceeds an upper threshold, and therefore the potential packet marking rate would be high, our recommendation is that routers drop packets rather then set the CE codepoint in packet headers. For low to medium congestion, the middle box will mark CE without dropping and only after high congestion which could imply non-compliance from sender, it will revert to dropping packets. So, there is some degree of unfairness when congestion starts to build up. The use of ECN in public Internet is to avoid queue buildup and we would be going against that if we use ECN (to avoid packet drops) but don't respond in timely manner. If the only use case of this is data centers (of which I have very limited knowledge), then should we mention that in the draft? Something like, ?
My view is that implementations largely do not conform to 3168, so even though the text says that, I'd only believe it if someone could show me evidence that is what happens in the wild. Also, I think we should permanently deprecate 3168 and take back the bit, but that's a different conversation.
From Mirja: "For ECN, you don't need to send an immediate ACK for each CE. Immediate ACKs are most important when the codepoint switches to CE, but then, if multiple CEs in a row are received, you can bundle the ACK information. See also URL (Note that we are still working on that section for AccECN but I think any changes are only relevant for specifics of TCP)"
From editors' discussion: This text is the same as that in RFC9000 (URL), and we don't believe we need to depart from existing text for CE-marked packets, since those are true congestion events, and we don't need to optimize for collapsing acks during congestion.
Insisting on feeding back an ACK whenever there's a CE will come back to haunt you. After the hundredth CE in a row, you'll probably start to realize that each CE is not a 'congestion event'. You need to make sure that this doesn't force the receiver to ACK every CE-marked ACK, which could lead to interminable ACK ping-pong. Admittedly, the sender should be slowing down, so perhaps it's not so bad to ram the reverse path full of ACKs under 100% congestion in the forward path. But it's not necessary - you could potentially squash other traffic sharing the reverse path. Particularly not nice in the first RTT before the sender starts responding. Instead, a requirement like under the that Mirja pointed to, where you ACK every 'n' CE marks in a row will be sufficient. And if the peers support ACK frequency control, you might want to say 'n' is the greater of a) whatever 'standard' value is set by the IETF and b) the current ACK frequency requested by the sender. Finally, note that the AccECN section linked above was massively changed between -14 (when Mirja pointed to it) and -15 (now). Nearly all of the changes would be relevant to QUIC too (except the stuff about DupACKs, I think). So it's worth reading it all again carefully.
Thanks for the specific suggestion Bob, that's extremely helpful. I think there are four ways forward: 1) Stick with the current text and deal with the performance hit when using non-classic ECN. 2) Always do something along the lines AccECN is suggesting when ACKFREQUENCY is negotiated, ie: Ack immediately when max(2, Ack-Eliciting Threshold) packets with CE marks are received. 3) Change the ignoreorder 'bool' to be an enum with 4 values that are a mix of Classic(RFC9000) and accurate ECN response to CE marks. 4) Do something completely from RFC9000 or AccECN. I'm not sure what would inform this, but maybe there are aspects of QUIC which means the ACK response should be different from TCP Accurate ECN. I favor 2 and wrote to document that. My personal opinion is that classic ECN is not able to perform as well as newer marking schemes and should be deprecated, so perpetuating it is actively harmful.
The design is not (only) for 'non-classic ECN', it's for handling any severe reverse channel congestion. For the avoidance of doubt, if NAME alternative is meant to be "as Bob (AccECN) suggested", a number of possibly important details are missing: a) Immediately ACK the first transition from a non-CE packet to a CE data packet; and b) ACK every max(2, Ack-Eliciting Threshold) contiguous CE marked packets after that. The choice of 'packet' (twice) or 'data packet' (once) in the above is deliberate I agree that ECN feedback in RFC3168 should be deprecated (which is why I didn't object when it was proposed that AccECN should move from experimental to stds track update to 3168).
The PR includes the 'Immediately ACK the first transition from non-CE to CE. PTAL and see if the text looks correct.
After further thinking, I'm wondering if we should have multiple modes that are indicated by bits in the 'Ignore Order' bytes. Specifically: 0 is rfc9000 behavior. Conveniently, this means clients sending Ignore Order values of 0 or 1 will see no change in behavior. 1 is Ignore CE when choosing how frequently to acknowledge packets. This would likely be the best option for ECN algorithms where close to 50% of packets are marked. 2 is accurate ECN, which is the current PR.
Following up on my last comment, I created a separate issue () and PR () for the question about whether we want an Ignore ECN bool.

number of acknowledgments allows connection throughput to scale much further.  <ins> As discussed in implementation however, there can be undesirable </ins> consequences to congestion control and loss recovery if a receiver uniltaerally reduces the acknowledgment frequency.  A sender's constraints on the acknowledgement frequency need to be taken into receivers to ignore obsolete frames, see multiple-frames. A variable-length integer representing the maximum number of ack-  <ins> eliciting packets the recipient of this frame can receive without sending an immediate acknowledgment.  An immediate acknowledgement is sent when more than this number of ack-eliciting packets have been received, so value of 0 results in an immediate acknowledgement.  If an endpoint receives an ACK-Eliciting Threshold value that is larger than the maximum value the endpoint can represent, the endpoint MUST use the largest representable value instead. </ins> A variable-length integer representing the value to which the endpoint requests the peer update its "max_ack_delay" invalid value MUST be treated as a connection error of type PROTOCOL_VIOLATION.  <ins> This field has no meaning in this version of ACK_FREQUENCY.  The value of this field MUST be 0x00.  Receipt of any other value MUST be treated as a connection error of type FRAME_ENCODING_ERROR. A 1-bit field representing a boolean truth value.  This field is </ins> set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer observes reordering  <ins> (reordering).  0 represents 'false' and 1 represents 'true'. A 1-bit field representing a boolean truth value.  This field is set to "true" by an endpoint that does not wish to receive an immediate acknowledgement when the peer receives CE-marked packets (reordering).  0 represents 'false' and 1 represents 'true'. </ins> ACK_FREQUENCY frames are ack-eliciting.  However, their loss does not require retransmission if an ACK_FREQUENCY frame with a larger 7.2.  <ins> An endpoint SHOULD send an immediate acknowledgement when a packet marked with the ECN Congestion Experienced (CE) codepoint in the IP header is received and the previously received packet was not marked CE. Doing this maintains the peer's response time to congestion events, while also reducing the ACK rate compared to Section 13.2.1 of QUIC- TRANSPORT during extreme congestion or when peers are using DCTCP RFC8257 or other congestion controllers that mark more frequently than classic ECN RFC3168. </ins> 7.3. acknowledgments will cause undesirable bursts of data into the network. <ins> 9.4. To avoid additional delays to connection migration confirmation when using this extension, a client can bundle an IMMEDIATE_ACK frame with the first non-probing frame (Section 9.2 of QUIC-TRANSPORT) it sends or it can simply send an IMMEDIATE_ACK frame, which is a non-probing frame. An endpoint's congestion controller and RTT estimator are reset upon confirmation of migration (Section 9.4 of QUIC-TRANSPORT), which can impact the number of acknowledgements received after migration.  An endpoint that has sent an ACK_FREQUENCY frame earlier in the connection SHOULD update and send a new ACK_FREQUENCY frame immediately upon confirmation of connection migration. 9.5. A sender might use timers to detect loss of PMTUD probe packets.  A sender SHOULD bundle an IMMEDIATE_ACK frame with any PTMUD probes to avoid triggering such timers. </ins> 10. TBD.


number of acknowledgments allows connection throughput to scale much further. <del> As discussed in implementation however, there are undesirable </del>  consequences to congestion control and loss recovery if a receiver uniltaerally reduces the acknowledgment frequency.  A sender's constraints on the acknowledgement frequency need to be taken into

“As discussed in Section 9 however, there are undesirable consequences to congestion control and loss recovery if a receiver uniltaerally reduces the acknowledgment frequency.“ This is a harsh statement. I don’t think it is true for smaller changes. There is much experience of using TCP where stretch ACKs cover more than two packets. This is a matter of scale. For instance: I’m confident that stretched TCP ACKs covering 4 segments (maybe more ... I’d love to say up to IW) would not break TCP CC, so I don’t see why the editors want to suggest it could break QUIC CC! This really does need a less obfuscated explanation.
I'd be happy to reword this to "there can be undesirable consequences ...", but were you looking for a larger change?
Thanks - yes, /can be/ would seem fair, since they are enumerated later.
This seems an improvement, thanks.

number of acknowledgments allows connection throughput to scale much further.  <ins> As discussed in implementation however, there can be undesirable </ins> consequences to congestion control and loss recovery if a receiver uniltaerally reduces the acknowledgment frequency.  A sender's constraints on the acknowledgement frequency need to be taken into


acknowledgments will cause undesirable bursts of data into the network.  10. TBD.

The draft doesn't mention anything about ACK frequency and connection migration. What should we do, for example, when a client has migrated and now sends a PING to force the server to use a new path? Should it send PING+IMMEDIATE_ACK?
That seems sensible. Mind writing a PR?
There's nothing odd about an immediate ACK! Perhaps the text should start by saying that the spec allows a receiver to do this anyway? However, the current text doesn't explain why the Immediate ACK might be useful. I could see three places that might benefit from an IMMEDIATE_ACK: Two cases relevant to SCTP [RFC7053] are: When the rate of transmission was very low and packets might be lost and not detected (something not uncommon in some SCTP uses); as an alternative to detect connectivity failure and trigger path failover. I'm not sure how important these specific cases are for QUIC, because the discussion at the time in TSVWG appeared to centre around use for signalling traffic? There could also be real opportunity to use this to get more precise timing/congestion information. A set of per-packet ACKs might be useful while a CC is probing to confirm available capacity, etc. Have I missed a killer application for the method - it would be helpful to me to understand /some example for how the WG think this could be used. (aside: I am actually uncertain whether this would really be a good change to TCP, especially with the heterogeniety of TCP middleboxes and senders,... that's another story, but I'd think it might be nice to not rely on draft-gomez-tcpm-ack-pull as a reference. I could suggest RFC7053, where people might recall there was quite some discussion in TSVWG about whether SCTP would benefit from being able to trigger an immediate response.)
We use an immediate ACK to support CC by delayed ACKs.
URL adds two use cases that uses IMMEDIATE_ACK.
These changes both look good to me.

acknowledgments will cause undesirable bursts of data into the network. <ins> 9.4. To avoid additional delays to connection migration confirmation when using this extension, a client can bundle an IMMEDIATE_ACK frame with the first non-probing frame (Section 9.2 of QUIC-TRANSPORT) it sends or it can simply send an IMMEDIATE_ACK frame, which is a non-probing frame. An endpoint's congestion controller and RTT estimator are reset upon confirmation of migration (Section 9.4 of QUIC-TRANSPORT), which can impact the number of acknowledgements received after migration.  An endpoint that has sent an ACK_FREQUENCY frame earlier in the connection SHOULD update and send a new ACK_FREQUENCY frame immediately upon confirmation of connection migration. 9.5. A sender might use timers to detect loss of PMTUD probe packets.  A sender SHOULD bundle an IMMEDIATE_ACK frame with any PTMUD probes to avoid triggering such timers. </ins> 10. TBD.


A variable-length integer representing the minimum amount of time in microseconds by which the endpoint can delay an <del> acknowledgement. </del>  An endpoint's min_ack_delay MUST NOT be greater than its max_ack_delay.  Endpoints that support this extension MUST treat

Although I understand what minackdelay represents it's not clear to me what an implementation should do with that value. Is the purpose to make sure that stacks sufficiently wait enough time before triggering PTO? There's no guidance on this in the draft. Perhaps there should be.
represents the minimum time that the receiver can delay for. In the common case, this would be the receiver's clock or timer granularity. This is for the receiver to indicate that the sender cannot reduce below this value. So a receiver implementation can simply use its granularity for this value, and a sender must ensure that its is not below the received . Does this help, NAME We can add some text to clarify this.
Thanks. Yes, I think this should be explained in the draft like you said.

A variable-length integer representing the minimum amount of time in microseconds by which the endpoint can delay an  <ins> acknowledgement.  This limit could be based on the receiver's clock or timer granularity. </ins> An endpoint's min_ack_delay MUST NOT be greater than its max_ack_delay.  Endpoints that support this extension MUST treat