4. <del> A QUIC packet is the content of the UDP datagrams exchanged by QUIC endpoints.  This document describes the contents of those datagrams. </del>  QUIC defines two types of packet header: long and short.  Packets with long headers are identified by the most significant bit of the Aside from the values described here, the payload of QUIC packets is version-specific and of arbitrary length. <del> 4.1. </del>  Long headers take the form described in fig-long. The next byte contains the length in bytes of the Destination Connection ID (see connection-id) field that follows it.  This length is encoded as an 8-bit unsigned integer.  The Destination Connection <del> ID field follows the DCID Len field and is between 0 and 255 bytes in length. </del>  The next byte contains the length in bytes of the Source Connection ID field that follows it.  This length is encoded as a 8-bit unsigned <del> integer.  The Source Connection ID field follows the SCID Len field and is between 0 and 255 bytes in length. </del>  The remainder of the packet contains version-specific content. <del> 4.2. </del>  Short headers take the form described in fig-short. The remainder of the packet has version-specific semantics. <del> 4.3. </del>  A connection ID is an opaque field of arbitrary length. methods.  Packets for the same QUIC connection might use different connection ID values. <del> 4.4. </del>  QUIC versions are identified with a 32-bit integer, encoded in network byte order.  Version 0 is reserved for version negotiation additional properties which apply to a specific QUIC version, or to a range of QUIC versions. <del> 5. </del>  A QUIC endpoint that receives a packet with a long header and a version it either does not understand or does not support might send that supports QUIC version 1 generates and consumes a Version Negotiation packet. <del> 6. </del>  It is possible that middleboxes could use traits of a specific version of QUIC and assume that when other versions of QUIC exhibit by off-path attackers.  QUIC versions MUST define a mechanism that authenticates the values it contains. <del> 7. </del>  This document makes no request of IANA. <del> 8.  References </del>  <del> 8.1.  URIs </del>  [1] mailto:quic@ietf.org

Remove the normative reference to -transport in -invariants. This is an editorial change, but I'll pass it to the list. Includes some consistency updates.
Yes, but I think that - at least in principle - each should be as independent of the other as possible. Sure, -transport is a concrete instantiation of -invariants, but you can implement it without -invariants. Similarly, -invariants exists to support -transport, but it describes something bigger than it. So while we might have some pointers, you can read each without the other, which is nice.
introduced a new bespoke packet format. I don't have issues with the format itself, but as I reviewed I was surprised to see it used there without any explanation as to how to decipher it. I think it would be nicer if each document that uses this format either: References the "Notational Conventions" section from draft-ietf-quic-transport Has a "Notational Conventions" section of its own explaining this format
-- URL There is a lot more that the document leans on than the packet format grammar, so any expansion of this might explode. I tend to think that this is adequate.
I agree that the reference is sufficient, except for one aspect: We do want the invariants drafts to be readable without needing to get into the v1 spec, because the target audience is people who need to handle QUIC without regard to versions. I think it's wrong that the reference to QUIC-TRANSPORT is normative instead of informative, and this reference for conventions and definitions is the only reference in the doc that actually needs to be normative. I'm almost be more inclined to move the notation definition across, just as we ultimately did for the definition of VN, and let -transport refer to -invariants for it.
Looks good. Technically, we could reference this draft for the notation, but we don't need to if it makes transport easier to read.

4.  <ins> Packet diagrams in this document use a format defined in QUIC- TRANSPORT to illustrate the order and size of fields. Complex fields are named and then followed by a list of fields surrounded by a pair of matching braces.  Each field in this list is separated by commas. Individual fields include length information, plus indications about fixed value, optionality, or repetitions.  Individual fields use the following notational conventions, with all lengths in bits: Indicates that x is A bits long Indicates that x can be any length from A to B; A can be omitted to indicate a minimum of zero bits and B can be omitted to indicate no set upper limit; values in this format always end on an octet boundary Indicates that x has a fixed value of C Indicates that x is repeated zero or more times (and that each instance is length E) fig-ex-format shows an example structure: 5. QUIC endpoints exchange UDP datagrams that contain one or more QUIC packets.  This section describes the invariant characteristics of a QUIC packet.  A version of QUIC could permit multiple QUIC packets in a single UDP datagram, but the invariant properties only describe the first packet in a datagram. </ins> QUIC defines two types of packet header: long and short.  Packets with long headers are identified by the most significant bit of the Aside from the values described here, the payload of QUIC packets is version-specific and of arbitrary length.  <ins> 5.1. </ins> Long headers take the form described in fig-long. The next byte contains the length in bytes of the Destination Connection ID (see connection-id) field that follows it.  This length is encoded as an 8-bit unsigned integer.  The Destination Connection  <ins> ID field follows the DCID Length field and is between 0 and 255 bytes in length. </ins> The next byte contains the length in bytes of the Source Connection ID field that follows it.  This length is encoded as a 8-bit unsigned  <ins> integer.  The Source Connection ID field follows the SCID Length field and is between 0 and 255 bytes in length. </ins> The remainder of the packet contains version-specific content.  <ins> 5.2. </ins> Short headers take the form described in fig-short. The remainder of the packet has version-specific semantics.  <ins> 5.3. </ins> A connection ID is an opaque field of arbitrary length. methods.  Packets for the same QUIC connection might use different connection ID values.  <ins> 5.4. </ins> QUIC versions are identified with a 32-bit integer, encoded in network byte order.  Version 0 is reserved for version negotiation additional properties which apply to a specific QUIC version, or to a range of QUIC versions.  <ins> 6. </ins> A QUIC endpoint that receives a packet with a long header and a version it either does not understand or does not support might send that supports QUIC version 1 generates and consumes a Version Negotiation packet.  <ins> 7. </ins> It is possible that middleboxes could use traits of a specific version of QUIC and assume that when other versions of QUIC exhibit by off-path attackers.  QUIC versions MUST define a mechanism that authenticates the values it contains.  <ins> 8. </ins> This document makes no request of IANA.  <ins> 9.  References </ins>  <ins> 9.1.  URIs </ins> [1] mailto:quic@ietf.org


attacker is able to successfully forge a packet; see AEBounds and ROBUST.  For AEAD_AES_128_GCM, AEAD_AES_256_GCM, and AEAD_CHACHA20_POLY1305, the limit on the number of packets that fail authentication is 2^36. Note that the analysis in AEBounds supports a higher limit for the

This is an update to , so check where it is being merged to carefully.
I'm going to merge this into the other PR.
NAME pointed out to me that when you receive junk (i.e., unauthenticated) packets, the odds are that these will be distributed evenly between the "current" packet protection keys and the "next" packet protection keys, as the key phase bit will be nearly random if they are true forgeries. The result is that an endpoint that approaches the integrity limit on key usage will update to the next packet protection keys and find that those keys will have similar amounts of authentication failures. That might lead to needing two updates in quick succession. That isn't fatal, but if the timing is tight, then more time might be needed for an update. We might say that endpoints shouldn't be calculating the forgery rate and trying to delay updates as late as possible, nor should they be allowing that many forgeries without doing anything but react with key updates. However, as a matter of good hygiene, it's worth saying something about this. Taking this as an editorial follow-on to .
Integrated into .

attacker is able to successfully forge a packet; see AEBounds and ROBUST. <ins> Due to the way that header protection protects the Key Phase, packets that are discarded are likely to have an even distribution of both Key Phase values.  This means that packets that fail authentication will often use the packet protection keys from the next key phase.  It is therefore necessary to also track the number of packets that fail authentication with the next set of packet protection keys.  To avoid exhaustion of both sets of keys, it might be necessary to initiate two key updates in succession. </ins> For AEAD_AES_128_GCM, AEAD_AES_256_GCM, and AEAD_CHACHA20_POLY1305, the limit on the number of packets that fail authentication is 2^36. Note that the analysis in AEBounds supports a higher limit for the


connection ID values in transport parameters and verifying them ensures that that an attacker cannot influence the choice of connection ID for a successful connection by injecting packets <del> carrying attacker-chosen connection IDs during the handshake.  An endpoint MUST treat any of the following as a connection error of type PROTOCOL_VIOLATION: </del>  <del> absence of the initial_source_connection_id transport parameter from either endpoint, </del>  <del> absence of the original_destination_connection_id transport parameter from the server, </del>  absence of the retry_source_connection_id transport parameter from the server after receiving a Retry packet,

As discussed in URL, most of the errors that can occur when authenticating connection IDs should be TRANSPORTPARAMETERERRORs, not PROTOCOL_VIOLATIONs.
This changes the error codes that we require, so it is technically a design change. I think we'll need an issue to track this.
NAME Created .
The guiding principle here (as we apply it for FRAMEENCODINGERRORs) is that if an error is detectable without accessing connection state, we use the more specific error. So for example, since and are required transport parameters, their absence should be responded to with a TRANSPORTPARAMETERERROR. A similar argument applies to the absence of the TP, although I can see how one might use PROTOCOLVIOLATION here (note that this is still valid behavior with this PR, as every more specific error can be replaced with a PROTOCOL_VIOLATION, if an implementation wishes to do so).
Thanks. I think that this is fine. I will make a few editorial comments on the PR, but I think that the application of the agreed principles is correct.
If you take the suggestions here, I will mark this for inclusion in the next consensus call. (Otherwise, we should continue discussion.)

connection ID values in transport parameters and verifying them ensures that that an attacker cannot influence the choice of connection ID for a successful connection by injecting packets  <ins> carrying attacker-chosen connection IDs during the handshake. </ins>  <ins> An endpoint MUST treat absence of the initial_source_connection_id transport parameter from either endpoint or absence of the original_destination_connection_id transport parameter from the server as a connection error of type TRANSPORT_PARAMETER_ERROR. </ins>  <ins> An endpoint MUST treat the following as a connection error of type TRANSPORT_PARAMETER_ERROR or PROTOCOL_VIOLATION: </ins> absence of the retry_source_connection_id transport parameter from the server after receiving a Retry packet,


specification, the server proceeds with the handshake (handshake). This commits the server to the version that the client selected. <del> If a server isn't currently accepting any new connections, it SHOULD send an Initial packet containing a CONNECTION_CLOSE frame with error code SERVER_BUSY. </del>  If the packet is a 0-RTT packet, the server MAY buffer a limited number of these packets in anticipation of a late-arriving Initial The endpoint encountered an internal error and cannot continue with the connection. <del> The server is currently busy and does not accept any new connections. </del>  An endpoint received more data than it permitted in its advertised data limits; see flow-control.

Please see the discussion on for the rationale, but in short, SERVERBUSY is renamed as CONNECTIONREFUSED. A server might refuse a connection for various reasons. SERVER_BUSY is one specific reason. Having an error code for that specific case while not having an error code for generic case is awkward. Hence the rename.
I'm not sure if this was an editorial PR. The previous text talked about a situation where a server was too busy to accept a new connection. The new text generalizes this to a connection refusal for any reason. That's a design change. I also strongly disagree that this issue should There's still on ongoing discussion if an additional error code should be introduced. I can't see how this is not a design issue.
There was previously no requirement to send this error code under any particular circumstances nor any requirement to take action upon receipt of this error code, and therefore those requirements cannot have been changed. This modifies the prose around an error code the server could use to reject a connection attempt to make the recommendation more narrow.
This is spawned from and . The discussion there revealed that SERVER_BUSY has implied semantics that aren't helpful and it would be better to have a more generic message instead. This tracks that. The resolution might be , depending on how we resolve this.
My personal preference is to not merge . A generic error code is more useful. There is a small loss here in that a server has to use a non-intuitive code for the case that it finds it can't continue with a connection due to load, but I am comfortable with that.
was already merged, and there's some question about whether it was actually editorial. This issue tracks the question retroactively. The proposal here is to leave merged / The alternative is to merge , reverting the other PR.
I prefer the generic message, so I would like to see the current state of affairs retained. That is, use CONNECTIONREFUSED going forward (instead of SERVERBUSY), and do not merge . If there is a need for other more specific error codes, we can deal with that separately.
+1 to leaving the current state. CONNECTION_REFUSED is generic enough to be useful in the situation NAME described, and while it's nice to have more specific error codes, I don't think there's an abundant need for a BUSY-like signal, without also getting into the business of prescribing things like time-based retry for QUIC clients.
+1 to retaining the current state. I'm repeating my comment on , but when QUIC is used as the only transport protocol (rather than with happy-eyeballing), it has to be resilient to MITM attacks as TLS over TCP is. TLS over TCP does not have an unauthenticated error code that tells the client to stop trying to connecting to the server. We cannot have such thing in QUIC, because if we do, QUIC becomes more prone to MITM attacker than TLS over TCP is. CONNECTIONREFUSED is fine in this respect, because it does not indicate to the client what the reason was. But error codes like SERVERBUSY or TRY_LATER would have the implications that we cannot have.
Closing as proposed.
Assume that a server has a blacklist of IP addresses, that it doesn't accept connections from. One way of implementing this would be to drop all packets from those IP addresses. But this would lead to clients time out, which is sad. Let's say the server wants to be more helpful and allow the client to quickly recover from this situation. Sending a CONNECTIONCLOSE would seem like the right thing to do in that case. However, we don't have an error code that fits here. SERVERBUSY is used to reject connection attempts, but it's a bad fit here. Retrying later is not likely to succeed in this case. APPLICATIONERROR also seems too broad, as it's used for any application-level error (in Initial packets). Do we need a new error code to cover this use case? Something like CONNECTIONREFUSED?
If my reading of our code is correct, we're using PROTOCOLVIOLATION for that case, but that's not a very informative choice. In gQUIC, we used public reset for this(as you probably remember), but stateless reset is post-handshake only. CONNECTIONREFUSED sounds better, but I think an important question is wether we expect clients to treat this error code differently than other error codes which occur during the handshake. If we're not going to treat it differently, then it matters less. That being said, I'm a general fan of more granular error codes for debugging.
I guess a reasonable policy regarding errors received during the handshake would be: INVALIDTOKEN: try again immediately SERVERBUSY: try again after a few minutes CONNECTIONREFUSED: don't try again for a day / week or so other errors: I can see TRANSPORTPARAMETERERROR and FRAMEENCODINGERROR (and their generic subsitute, PROTOCOLVIOLATION) being reasonably sent in response to a ClientHello. So that means that either your or your peer's implementation is broken, so maybe you shouldn't speak QUIC at all?
I tend to think that adding an error code that affects client behavior is not a good idea, when that error code is intended to indicate a handshake failure. The signal carried by an Initial packet or a Handshake packet is not authenticated, and we should be wary of using that to convey something that affects more than just that connection. In case of this example, CONNECTIONREFUSED would become an attractive tool for MITM attackers, because it can be used for preventing the client for reconnecting for as long as a day or a week. If we need these signal, I think that they have to be carried at the application-level after the QUIC handshake concludes. In case of HTTP, we already have 503 + Retry-After header field. To summarize, I think my position is: We should not add error codes that affect the client behavior after that connection gets closed. We might want to get rid of SERVERBUSY, or at least state that a client cannot really tell if the server was really busy, as the signal is not authenticated.
We accept this risk in QUIC v1. Removing error codes to mitigate the MITM vulnerability seems like a piecemeal "solution" to a problem that can only be solved by decent cryptography. For the MITM the easiest way is probably injection a Version Negotiation packet that indicates that there are no compatible versions between the client and the server. I'd expect most client implementations to give up a dial attempt in this case, which in effect would be very similar to the actions taken in response to a CONNECTION_REFUSED.
I disagree with that view. Handshake is never going to be disruption resistant, unless there is a shared secret. There are ways of raising the probability of endpoints having shared secret (that we discussed and postponed to v2), but there would always be a case where there is no shared secret. We always need to limit the impact of attacker injecting signals to an unauthenticated channel. I'd also reiterate that in HTTP we already have a way of asking clients to revisit the server after certain amount of time. TLS does not have a way of signaling that. Assuming that HTTP developers are fine with what we have in HTTP + TLS, I do not think there is good justification to add a new feature that has negative impact on security. Giving up one attempt is fine. Having a signal that asks to give up future connection attempts is the problem.
HTTP has 503 to reject request when a server is busy, and still we have SERVER_BUSY at the QUIC layer. There's a value in rejecting connection attempts before spending resources (and round trips) on a handshake. Furthermore, there are other application protocols than HTTP. In my use case, nodes in a p2p network want to enforce a local blacklist of IP addresses. In TCP, they can easily do that by resetting the connection, and before spending any resources on the TLS handshake. Having to go through the whole QUIC handshake with a blacklisted peer before being able to reject them would be a serious regression compared to TCP. I'm not sure how browsers handle a Version Negotiation packet that indicates that there are no matching versions. I doubt that they'd try QUIC again for the next request just a few seconds later. Intuitively, I'd say that a longer period combined with an exponential backoff would be a reasonable thing to implement. Maybe NAME and NAME can shed some light on this?
Yes, TCP or TLS can reject one attempt. QUIC is currently aligned to that, and that's fine. I'd point out that how aggressively browsers try connecting in QUIC depends on if they do happy-eyeballing. When browsers do happy-eyeballing, no matter how less you do in QUIC, the probability of establishing a connection always stays higher (or as high as) just doing TLS. The question is when browsers start doing just QUIC. Would they prefer to cache previous connection failures, as a way to not even try to reconnect when a user requests doing so?
As a way of moving forward, I'd propose renaming SERVERBUSY to CONNECTIONREFUSED, with no recommendation on how the client should react to that. The rationale behind providing no recommendation is as stated above. QUIC needs to be strong against MITM attacks, at least as TLS over TCP is. TLS over TCP does not have an error code that tells the client to refrain from retrying for some amount of time. To achieve parity, we cannot have that. AFAIK, our experience with HTTPS tells us that there is no need for such an error code. The rationale behind the renaming is that at the moment we do not have a generic error code for refusing connections, even though we have SERVERBUSY, which indicates a specific reason of refusing a connection. Therefore, we need CONNECTIONREFUSED. We can rename SERVERBUSY to CONNECTIONREFUSED. SERVERBUSY does not need to be preserved, since, as stated, the client behavior cannot vary depending on an unauthenticated error code. For debugging purposes, endpoints are free to use the reasonphrase field. WDYT?
I don't think that renaming an existing error will solve my problem. Otherwise, I would've just used that error... The point I was trying to make here is that clients will already make decisions about future connection attempts based on unauthenticated information, both sent in CONNECTION_CLOSE error codes as well as in Version Negotiation packets. Adding one more error doesn't seem to exacerbate that situation, while at the same time enabling a way of blacklisting IP addresses that was possible with TCP and is currently not possible using QUIC.
As stated, my view is that your problem (of sending a signal that affects future connections) should only be solved by sending something after the handshake succeeds. And for HTTP, we already have that signal (i.e. 503 + Retry-After). Therefore, defining such signal does not have to be done in QUICv1. I disagree with that view. If the specification can be read as such, then I think we need to make changes. A client should refrain from using an unauthenticated signal for determining what to do with future connection attempts. It does not matter what that unauthenticated signal is (e.g., an error code in Initial packet, failed version negotiation). Any unsuccessful handshake attempt should be considered as an error of that particular attempt. Otherwise, QUIC handshake becomes prone to MITM attacks than TLS over TCP. If we are to agree that clients should not use an unauthenticated error code in determining what to do in the future, there is no point in having multiple error codes indicating that the server has deliberately refused a handshake. It is sufficient to have just one error code that let's the server say, hey, nothing is wrong with us, but I refuse to accept a connection.
NAME I believe it's up to the client as to how unauthenticated signals are used. There are a lot of ways to interrupt a QUIC handshake early on, and this is one of many. Chrome does exponential backoff with a 5 minute time period anytime the handshake fails, whether it's due to VN, connection close, handshake timeout, etc. In practice, VN and Alt-Svc typically agree, so VN is very rare and connection closes are more common. I can't argue Chrome's algorithm is optimal, but if handshakes failed in the recent past, it seems sub-optimal to try QUIC again immediately.
NAME I think that is a fine thing to do as long as Chrome is happy-eyeballing with TLS-over-TCP. But as a specification, it is my understanding that happy-eyeballing is not a prerequisite. I'd assume that we'd want applications that use QUIC only to be as ignorant against unauthenticated signals as those using TLS-over-TCP is.
If only QUIC is used, then I agree the behavior should match that of TLS over TCP.
I like , but it is a semantic change. It makes the existing code more general, which is appropriate. We would need to make this issue a design issue to do that though.
This is practically impossible. To function reliably on the internet a client has to use heuristics and unauthenticated information in order to provide a good user experience and not overwhelm server resources. As a trivial example, if I wrote a client which completely ignores things like VN and immediately retries a QUIC connection, I am locked into supporting that version on my server indefinitely, since my client will now effectively DOS me if I remove support for this version from the server. This is unacceptable for internet scale. Additionally, rejecting a connection post-handshake makes fallback to TCP more complicated -- it is much simpler to use a happy eyeballs mechanism and "race" TCP and QUIC, stopping the TCP connection once the handshake is successful. I am highly sympathetic to NAME point here. As has been said already, we already have 100% reliable and trivially implementable way to interrupt the handshake in v1 with version negotiation. Having an additional signal which is more explicit can only help clients making new connection decisions. We don't necessarily have to prescribe what the client does, but we can say why a server may issue certain errors during the handshake.
NAME I'd emphasize that the specification does not talk about happy eyeballing. We should concentrate on discussing what to do when only QUIC is being used. Having that said, I'd hope that we can agree on one principle: QUIC cannot respect unauthenticated information any more than TLS over TCP does. Otherwise, QUIC becomes more prone to MITM attacks that TLS over TCP is - a disobedience to our charter. That might be true, but our experience with TLS over TCP is that we do not need an error code for the purpose. FWIW, you do not need a version negotiation packet. Per my understanding, any server can send a private error code. Or a reason phrase explaining things. Am I correct to understand that the reason you are advocating for having more error codes is to allow a server to control the behavior of client for future connection attempts? Assuming that is the case, I'd oppose to having such error codes regardless of how we would describe them. Because IMO that is effectively about making QUIC more prone to attacks than TLS over TCP is.
I think that NAME is entirely right here. The renaming of the error code is useful as it removes the implication of a semantic that would be destructive. That implication is that the code is like a 503, which comes with the further implication that holding back for some time is desirable (due to the common use of Retry-After with 503). QUIC has no Retry-After, so any implied time delay is bogus. As mentioned, that browsers might sit and wait is a policy decision that makes sense in the current state of the world. I'm going to merge the PR. But I get the sense that this is not directly addressing the issue. For the issue itself, I think that the only real resolution is: that's tough, but too bad. As numerous people have said, this phase of connection establishment is ripe for DoS and we have chosen to accept that risk in QUIC version 1.
Based on a request on . I have also opened .
I can see why this issue being classified as a design change, though at the same time, I think that was a good editorial change. made things betterWe now have an error code for server indicating that it rejected the connection even though nothing seemed to be wrong with the handshake. Without having CONNECTIONREFUSEDERROR, a client had to guess if PROTOCOLVIOLATION meant a problem in the handshake, or if it was simply a refusal. From now on, on this issue, I hope that we can concentrate on discussing if we want to have more than one error code indicating the specifics of connection refusals (e.g., SERVERBUSY). Unsurprisingly, my position is that we cannot have specific codes due to the reasons laid out above.
I agree with NAME in that was good (and I don't want to revert it). I also don't think we need anything more than an appropriately generic error code for the stated problem, so I believe that that PR was adequate.
I agree with NAME I think we should and I would argue that we should do a new issue if anyone wants more specific error codes.
I love more specific error codes, but many in the WG do not, so I still like given the constraints we're working with.
NAME No introduction of error codes can possibly make QUIC more prone to this than TCP + TLS. A TCP connection can be completely interrupted at any time with an unauthenticated RST. In that sense I don't think it's possible to make QUIC more prone to MITM attacks than TCP + TLS, since at the very least we only have a small window for the similar MITM interruption. It's not a fair comparison, as TCP does not have an opportunity to fallback to anything. You posit that we have to view the QUIC specification in isolation from the existence TCP or any "happy eyeballs" mechanism, and I don't totally disagree but I don't think it follows we should make decision ignoring the realities of deploying QUIC on the Internet. That reality is, today, that QUIC will not work 100%, whereas we expect TCP to function. If we can add things to the specification which aid in a seamless coexistence of QUIC and alternatives, why would we not? I don't think I fully understand your point here. You asserted that the client "should refrain from using an unauthenticated signal for determining what to do with future connection attempts". I maintain that this is fantastical. The client has to do something with future connection attempts whether we prescribe a behavior or not, and as such it is abundantly useful to be able to influence, if not dictate, the behavior of clients with more explicit signals from the server. As it is I am fine with leaving this discussion with the generic CONNECTION_REFUSED and calling it a day, but I think there's value in discussing the real-world considerations here, which was NAME original motivation.
Is anyone arguing for leaving this issue open? If folks still want to continue discussing, I propose that we open a new issue to do it.
No response to NAME question in ten days; it looks like this issue is closed.
Closing then.
Spawned from . NAME typed: "Probably not for this change, but do we really want to encourage people to be estimating delays in the kernel? Because it seems like the point of some of the other changes is to include that sort of delay in the RTT estimate. What matters here is end-to-end delays - the time between when the packet was consumed and when the packet containing the ACK was sent out. Yes, it's true that you get a bigger RTT estimate if you don't correct for delays in the kernel or NIC, but those are just another piece of the plumbing now. If we fail to cover things like scheduling delays in the RTT estimate, we're just deluding ourselves about where the delays are." NAME typed: "Yeah, I was thinking about this too, but it's text that we already had. FWIW, I think it's useful to separate out these delays, not because they shouldn't get used for PTO estimates, but because knowing component delays might be helpful. That said, I don't have a good use case for these component delays yet. OTOH, I've started that separation between stack-controlled delays and other delays in the text, it might be worth using that distinction here, and arguing that these delays ought to be part of the path RTT since they're not controlled delays."
:+1: I share the view with NAME and NAME that ack-delay should represent only the delay introduced intentionally by the QUIC stack. "stack-controlled delay" sounds like a good way of explaining them.
+1 to the idea that ack-delay should be intentionally introduced delay, since almost anything on path(including the local host NIC/OS/Firmware) can delay things without the receiver's control.
From London: Platform delays SHOULD be included in RTT, but ack_delay should only include intentionally introduced delays, not platform delays.

specification, the server proceeds with the handshake (handshake). This commits the server to the version that the client selected.  <ins> If a server refuses to accept a new connection, it SHOULD send an Initial packet containing a CONNECTION_CLOSE frame with error code CONNECTION_REFUSED. </ins> If the packet is a 0-RTT packet, the server MAY buffer a limited number of these packets in anticipation of a late-arriving Initial The endpoint encountered an internal error and cannot continue with the connection.  <ins> The server refused to accept a new connection. </ins> An endpoint received more data than it permitted in its advertised data limits; see flow-control.


certain that it has not processed another packet with the same packet number from the same packet number space.  Duplicate suppression MUST happen after removing packet protection for the reasons described in <del> Section 9.3 of QUIC-TLS.  An efficient algorithm for duplicate suppression can be found in Section 3.4.3 of RFC4303. </del>  Packet number encoding at a sender and decoding at a receiver are described in packet-encoding.

Including mention of a minimum packet number and the risks in setting that value.
requires deduplication, delegating the details to RFC 4303. I am not sure if that is enough. IIUC, the difference between RFC 4303 and QUIC is that in case of QUIC, packets are sent using multiple paths. Consider the case of a client connected through Wi-fi, probing a 3G network. The latency of Wi-fi could be much shorter than that of 3G, and the bandwidth could be high. In such case, the requirement on the deduplication window becomes high: the window needs to cover packet numbers across multiple PTOs (where PTO is calculated from the RTT of the Wi-fi). A server can respond reliably to a path probing packet on the 3G network, only if the server's deduplication window is fairly larger than the bandwidth of the Wi-fi multiplied by the latency of the 3G network. I do not think that this case is covered by RFC 4303. IMO, we need to at least clarify this corner case exists, otherwise, we might end up seeing servers not responding to probes sent on cellular networks, and clients failing to migrate due to that failure.
As a fix, I think we should remove reference to RFC 4303, and state that an endpoint SHOULD be capable of deduplicating packets that are received across more than half the largest RTT that the endpoint intends to support.
I agree that RFC 4303 is showing its age here. 32 packets is probably woefully inadequate here. But there is an additional insight that is important: you need to drop all packets older than the oldest packet you are individually tracking or you are vulnerable to duplication attack. It's a little awkward restating that. It's not obvious that you need to allow for an increase of half of the difference between the current min_rtt and some maximum value either, so it is worth saying that. Noting that the collective packet rate on all paths affects any limit is similarly non-intuitive. The nice thing about this is that the most obvious packet-tracking structures (aside perhaps from those that are subject to DoS) are not susceptible to this problem. For instance, many people are tracking ranges and limiting the total number of ranges. If you do that, you only run into problems after the number of slow probes sent within one of these extra-long round trip times exceeds the number of ranges the receiver is tracking. Most implementations are tracking more ranges than there are packets in an initial congestion window, so you won't get issues.
Regarding the insight, I agree that we need to state that. We can site RFC 4303 as part of that attempt, even though we need to refrain from suggesting that the design documented in RFC 4303 works for QUIC. As you state, the practical advice we'd better provide is that an endpoint can retain ranges of packets. The number of ranges that an endpoint should retain is questionable, but hopefully, endpoints would make a sensible choice.
The RTT estimator currently does not use RTT samples if the largestacked is not ack-eliciting. This can lead to the endpoint missing RTT samples unnecessarily, in two ways. First, checking only for the largestacked to be ack-eliciting means that we ignore other newly acked packets that were ack-eliciting. The peer is expected to send an ack within maxackdelay if it has received any ack-eliciting packets, not just the largest acked one. This is probematic if the connection has a pattern that results in more non-ack-eliciting packets being the largestacked. I propose that we change the condition for a valid RTT sample to be if any newly acked is ack-eliciting (in addition to the ACK frame carrying a new largestacked). Second, non-ack-eliciting packets are a perfectly fine source of RTT samples, with the caveat that ackdelay might be larger than maxackdelay; a receiver that has only received non-ack-eliciting packets might delay acknowledging these packets much longer than otherwise. I propose that we don't drop all RTT samples associated with non-ack-eliciting packets, but only those with unusable ackdelay. Basically, I propose that we use the RTT sample if ackdelay < maxack_delay.
Why only accept the RTT measurement if ackdelay is less than (or equal I assume) maxack_delay? So the peer held on to the packet a little longer than normal? What's the big deal? If they report to you exactly how long they held on to it, you should still get an accurate RTT measurement, right?
NAME you make some good points, but the sender needs to know which packet to use to create an RTT sample. The obvious answer is the largest ack-eliciting packet number. I can think of some cases involving ACK loss and/or reordering that would be slightly incorrect, but I can't think of any cases that would be disastrous. However, it seems important that ackdelay is from the largestacked ack-eliciting packet in that case, otherwise the RTT sample will be incorrect. I'm not feeling great about the fact we're not explicitly specifying the largest ack eliciting packet an instead letting the sender figure it out, but besides that this seems ok. I will note that we could also advise not including non-ack-eliciting packets as the largest acked and only ACK them when they're less than an ack-eliciting packet. I wonder if that might be simpler, given this is somewhat of an edge case? NAME The reason for limiting ackdelay to maxackdelay is to incentivize correct reporting of maxackdelay. Otherwise, one could report a maxackdelay of 0ms and consistently report a larger ackdelay value, allowing the receiver to decrease the sender's SRTT. I'm not sure how large a problem this is in practice, but it's nice if the mechanisms incentivize accurate reporting. The algorithms should devolve into something very similar to TCP if maxackdelay is 0ms, so the worst case doesn't seem that bad.
It just seems like a bad idea to always ignore RTT measurements if ackdelay is larger than maxackdelay. For instance, what about one of the following scenarios: Sheer CPU load on the peer prevents it from ACK'ing a packet within maxackdelay. An implementation that literally sets a maxackdelay timer on receipt of a packet, then when the timer fires, queues a send packet work item. When that work item is processed, the ackdelay is calculated and it ends up being just over maxackdelay.
NAME In both cases, I believe the current algorithm of limiting ackdelay to maxackdelay makes perfect sense. In the case of CPU load, RTT should show up in SRTT, RTTVar, or both. In the second case, the text also advises including any expected timer delay in your advertised maxack_delay. This issue is about what happens if the largest acked is an ACK-only packet(not ack-eliciting). In this case, we do ignore the RTT sample, and I do think that could turn out to be problematic, particularly in very asymmetric flows.
NAME my comments were in reference to NAME last sentence: I am arguing that we shouldn't throw away an RTT sample all together if ackdelay is greater than maxack_delay.
Even if it is not ack-eliciting? Doing that could subject calculations to huge and unpredictable RTT spikes.
No. I thought Jana's comment was even for ack-eliciting packets. It wasn't clear.
Apologies for any confusion. NAME I was referring to non-ack-eliciting packets in that second half of my description, where I made that proposal.
I've been doing more thinking, and here's where I've landed. (Thanks NAME and NAME for conversations.) The higher order principle is this: maxackdelay suggests a contract between this endpoint and its peer. The peer is supposed to send ACKs for ack-eliciting frames within maxackdelay amount of time. Effectively, the peer wants to wait for no more than maxackdelay amount of time, but if the actual received ackdelay is larger (due to ack loss or scheduler latency), the RTT should be increased to compensate for this additional delay beyond maxackdelay, because it's not within the peer's control and we might consider it part of "path" delay. With this principle in place, I think it still makes sense to make the first change I proposed: that an ACK should be used for RTT measurement if any packet it newly acks is ack-eliciting. The RTT measurement is still on the largestacked (since ack-delay is reported for the largestacked). It is difficult to reason about ACKs that only ack non-ack-eliciting packets, since the maxackdelay contract does not hold. It is specifically difficult to reason about the reported ackdelays, since the peer can wait arbitrarily long and report arbitrarily long ackdelays. It's incorrect to add any additional delays (beyond maxackdelay) into the RTT since the increased delay was not due to uncontrollable circumstances. Simply using the ackdelay, however large it may be, allows for the peer to simply game the RTT. Using only the samples that have ackdelay < maxack_delay might introduce a bias. It gets complicated quickly. So, I conclude that we should not use ACKs of non-ack-eliciting packets for RTT measurement. This is what we do right now, so that's good. I'll add some text explaining why. We should also recommend that an endpoint that is only sending non-ack-eliciting packets should piggyback a PING frame along with an ACK roughly once an RTT so that its RTT estimate is reasonably current.
The sounds like a complication - and extra timer check and keeping track of whether ACK eliciting packets were sent. Also, it could lead to a lot of unnecessary PONG traffic. Is it really that important to update RTT that often, and if so, wouldn't it be better to revisit the idea of measuring RTT for non-ack eliciting packets dispite the potential downfalls?
+1 to NAME 's last comment. I look forward to a PR. In regards to the last paragraph, we already have some text in transport about this, and I'm not sure how specific I want to be, given it's only really important as a mechanism for the sender to drop state if they're waiting for acks of acks. "Because ACK frames are not sent in response to ACK-only packets, a receiver that is only sending ACK frames will only receive acknowledgements for its packets if the sender includes them in packets with non-ACK frames. "
My +1 as well. I think it's reasonable to recommend endpoints to send PINGs, if it wants to calculate RTT when it is not sending data. First of all, this is not a departure from the design we have in TCP; in TCP, an endpoint that is not sending data has no way of inferring RTT from ACKs. Second, requiring the peer to send ACKs for ACK-only packets promptly makes the specification complex. Because such a mechanism cannot be solely based on a timer (because ACK-of-ACK cannot be sent without piggybacking), even though it needs to be based on a timer (otherwise the endpoint cannot receive an ACK in a timely manner). Considering these two aspects, I think using PINGs is simpler. Also, endpoints can choose different strategies on how frequent they would send PING; some might send a PING every one second (or 1 RTT), or a download-only application might not send PINGs at all.
The problem with this change is that it requires a sender to keep state for non-ack-eliciting. Currently, in my implementation, I only save a sent packet (including its metadata like the time sent and the packet size) if that packet is ack-eliciting.
NAME That seems like a sensible point. Possibly this should be a SHOULD, not a MUST when the largest acked is not ack-eliciting?
We could also require (or at least recommend?) the peer to not send acknowledgements for non-ack-eliciting packets.
I disagree that we shouldn't be acknowledging non-ack-eliciting packets as the largest_ack. We should always be sending the most up to date information available? NAME why don't you track the non-ack-eliciting packets? They shouldn't be very high in number or really structurally different from other packets? It seems like an unnecessary optimization that would just cause you more problems in this scenario.
I agree with Nick that we should keep sending the most up-to-date information.
It's been a while since I worked actively on a QUIC impl., but in my implementation, buffers would be retained until all pending packets in a block have been acked or lost, then that block is flushed. Having an unnecessary complication to this system would be painful. If it is only the leading block, i.e. one of a few recently sent non ack eliciting packets, it won't impact the overall design as much.
NAME I agree with NAME that tracking non-ack-eliciting packets shouldn't be much additional load, but I don't think that's necessary. If an ACK acknowledges packets that you don't know about, they've been acked or they're non-ack-eliciting. Either ways, this ACK should not generate a valid RTT sample. I'll also note that you are not required to use every ACK that acks any ack-eliciting packets for an RTT sample. If you're not tracking a packet, you can't use it's ack for an RTT measurement. Ignoring these ACKs is the only way forward, and the text will allow that.
It might not be a lot of additional load, but a lot of additional complexity. For example, what happens if the peer never acks an ack-eliciting packet (either because it was actually lost, or as part of an attack)? You then have to somehow clear that packet from your history. If you use the normal loss detection mechanism for it, you have to be careful not to take any congestion controller action upon loss of that packet. You also don't want to set any timers based on non-ack-eliciting packets, etc. I never suggested sending out-of-date information, that would indeed be a bad idea. There's just no need to send an ACK at all if largest_acked is not ack-eliciting, and I'm suggesting that we recommend (mandate?) that peers don't do it. They should just wait until they receive the next ack-eliciting packet.
NAME : Agreed, which is why I said that it's not necessary to track non-ack-eliciting packets. We already say in the transport draft that an endpoint not send an ACK if all it's received is ACKs. Is that what you meant? The problem here is that a peer might decide to piggyback an ACK frame with some other stuff it is about to send (MAXSTREAMDATA perhaps). That could result in an ACK frame that acks only non-ack-eliciting packets. It seems unnecessary to put the burden on a peer to not send this ack. It's just as easy to ignore such acks when received.
NAME : FWIW, I did suggest a way forward -- that an endpoint ignore ACK frames (for RTT sampling) when the ACK frame doesn't newly acknowledge ack-eliciting packets that the endpoint is tracking. Does this work for you?
I just realized that there's a pathological traffic pattern which would prevent a peer from ever getting a valid RTT sample (e.g. if all ACKs are sent on the maxackdelay timer, and if after every ack-eliciting packet a non-ack-eliciting packet is received). It might be safe to ignore this edge case and assume that it won't happen a lot in practice.
I don't want to have a design that has a certain scenario where things break down, but just ignores it. Also, I don't want to have a design where the receiver has to decide if I can send an ACK with my current state or not. Generally, I expect receivers to only keep track of the time they received the largest packet number; and just use that when they frame an ACK. NAME as to your issues with keeping track of ACK-only packets I'd argue they are almost entirely the same as normal packets. The ONLY difference is congestion control. As far as the peer trying to attack you by explicitly not ACK'ing the packet, that's no different for any packet. The same logic applies to all packets you send. Eventually, you age out lost packets. IMO, the simplest and most straight forward design is to always send ACK frames for your complete, current state, independent of packet contents. If the receiver of the ACK frame decides the RTT measurement shouldn't be used, that seems to be trivial enough logic to add in that specific ACK processing code. But I still argue that so long as the ACK wasn't delayed too long (~maxackdelay) then the RTT measurement should be used. You don't want the worst case traffic pattern of each flight/burst of packets ends with an ACK-only packet to cause you never to accept RTT measurements from the correspond ACK you get back.
NAME : This pathological case is exactly why I'm proposing a change to "any acked packet was ack-eliciting". NAME I wasn't proposing new text. To be clear, if you do not have any state for packets that are being acked, you have no choice but to ignore the ack. This is not a new rule, it just falls out of standard ack processing as we have it now.
There is the interesting derived effect that if you do not ACK non-ack-eliciting packets, this could be used as optimistic ack attack mitigation. This means that you can avoid gaps in the packet number space. You could periodically coalesce non-ack-eliciting packets with other packets as necessary. The problem is of course that the receiver needs to track those gaps, but I'm not sure that is much different than what it already needs to track.
I believe this was NAME I'll let you confirm that and close this.
Thank you! Looks good to me.

certain that it has not processed another packet with the same packet number from the same packet number space.  Duplicate suppression MUST happen after removing packet protection for the reasons described in  <ins> Section 9.3 of QUIC-TLS. Endpoints that track all individual packets for the purposes of detecting duplicates are at risk of accumulating excessive state. The data required for detecting duplicates can be limited by maintaining a minimum packet number below which all packets are immediately dropped.  Any minimum needs to account for large variations in round trip time, which includes the possibility that a peer might probe network paths with a much larger round trip times; see migration. </ins> Packet number encoding at a sender and decoding at a receiver are described in packet-encoding.


This document defines QUIC version 1, which conforms to the protocol invariants in QUIC-INVARIANTS.  1.2. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",

I don't see much point in doing the same for invariants, but I'm willing to take input on that choice.
The chairs just had a question from someone who wanted to cite "QUIC" on what document they should be citing. We responded that they should cite -transport, since we see this as the central document. Should we add brief notes to the documents saying "if you want to cite QUIC, cite -transport"?
That seems reasonable, since we say in there that accompanying documents describe TLS and recovery. Where are you thinking of adding that note?
As early as possible?
If you want to cite QUIC version 1, transport. If you want to cite QUIC all-up, invariants.
, or immediately after it, seems like as good a place as any.
I probably would have made it part of the preceding paragraph, but I don't feel strongly.

This document defines QUIC version 1, which conforms to the protocol invariants in QUIC-INVARIANTS. <ins> To refer to QUIC version 1, cite this document.  References to the limited set of version-independent properties of QUIC can cite QUIC- INVARIANTS. </ins> 1.2. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",


or MAY take no action. When a server sends CANCEL_PUSH, it is indicating that it will not be <del> fulfilling a promise and has not created a push stream.  The client should not expect the corresponding promise to be fulfilled. </del>  Sending CANCEL_PUSH has no direct effect on the state of existing <del> push streams.  A server SHOULD NOT send a CANCEL_PUSH when it has already created a corresponding push stream, and a client SHOULD NOT send a CANCEL_PUSH when it has already received a corresponding push stream.  If a push stream arrives after a client has sent CANCEL_PUSH, this MAY be treated as a stream error of type H3_STREAM_CREATION_ERROR. </del>  A CANCEL_PUSH frame is sent on the control stream.  Receiving a CANCEL_PUSH frame on a stream other than the control stream MUST be

by reversing the polarity of a SHOULD. Doesn't break interop, but it's always safer to send CANCEL_PUSH than to omit it, so let's not recommend omitting it under any circumstances.
The spec suggest that a server should send a CANCELPUSH to notify a client that it will not fulfill a PUSHPROMISE. The server should send CANCELPUSH before a push stream has been open. The server still can cancel a push stream after it has been created and that will have a similar effect as a CANCELPUSH frame. There is a small problem with that. If the server resets a push stream before client starts reading, the client may have not read the pushid from the stream and that pushid will never be fulfill or cancel for the client point of view. Should there be a sentence in the spec explaining this? Probably suggesting not to cancel a push stream until server is certain that client has read it, which is difficult. Should the spec suggest ot send CANCEL_PUSH even after a push stream has been created?
The server doesn't need to ensure that the push ID is read if CANCELPUSH is used, mostly. If the stream type is read, but not the push ID (which is entirely possible), then the client will know that there was a push and that it didn't happen, but can't connect that with the promise or cancellation. In that case, clients need to handle that. In fact, this highlights the value of CANCELPUSH in the case where the stream is reset.
This will not break interop either way. We can change the recommendation easily.
This issue and its fix has been pretty well discussed already. The proposal is borderline on a design change, so I've marked this as design to get some visibility during the next consensus call.
lgtm

or MAY take no action. When a server sends CANCEL_PUSH, it is indicating that it will not be  <ins> fulfilling a promise.  The client cannot expect the corresponding promise to be fulfilled, unless it has already received and processed the promised response.  A server SHOULD send a CANCEL_PUSH even if it has opened the corresponding stream. </ins> Sending CANCEL_PUSH has no direct effect on the state of existing  <ins> push streams.  A client SHOULD NOT send a CANCEL_PUSH when it has already received a corresponding push stream.  A push stream could arrive after a client has sent CANCEL_PUSH, because a server might not have processed the CANCEL_PUSH.  The client SHOULD abort reading the stream with an error code of H3_REQUEST_CANCELLED. </ins> A CANCEL_PUSH frame is sent on the control stream.  Receiving a CANCEL_PUSH frame on a stream other than the control stream MUST be


14. The QUIC packet size includes the QUIC header and protected payload, <del> but not the UDP or IP header. </del>  <del> A client MUST expand the payload of all UDP datagrams carrying Initial packets to at least 1200 bytes, by adding PADDING frames to the Initial packet or by coalescing the Initial packet; see packet- coalesce.  Sending a UDP datagram of this size ensures that the network path from the client to the server supports a reasonable Maximum Transmission Unit (MTU).  Padding datagrams also helps reduce the amplitude of amplification attacks caused by server responses toward an unverified client address; see address-validation. </del>  Enforcement of the max_udp_payload_size transport parameter (transport-parameter-definitions) might act as an additional limit on <del> packet size.  Exceeding this limit can be avoided once the value is known.  However, prior to learning the value of the transport parameter, endpoints risk datagrams being lost if they send packets larger than 1200 bytes. </del>  Datagrams containing Initial packets MAY exceed 1200 bytes if the client believes that the network path and peer both support the size the DF bit MUST be set to prevent fragmentation on the path. A server MUST discard an Initial packet that is carried in a UDP <del> datagram with a payload that is less than 1200 bytes.  A server MAY also immediately close the connection by sending a CONNECTION_CLOSE frame with an error code of PROTOCOL_VIOLATION; see immediate-close- hs. </del>  The server MUST also limit the number of bytes it sends before validating the address of the client; see address-validation. payload an endpoint sends at any given time is referred to as the endpoint's maximum packet size. <del> QUIC depends on a PMTU of at least 1280 bytes.  This is the IPv6 minimum size RFC8200 and is also supported by most modern IPv4 networks.  All QUIC packets (except for PMTU probe packets) SHOULD be sized to fit within the maximum packet size to avoid the packet being fragmented or dropped RFC8085. An endpoint SHOULD use Datagram Packetization Layer PMTU Discovery (DPLPMTUD) or implement Path MTU Discovery (PMTUD) RFC1191 RFC8201 to </del>  determine whether the path to a destination will support a desired <del> message size without fragmentation. </del>  <del> In the absence of these mechanisms, QUIC endpoints SHOULD NOT send IP packets larger than 1280 bytes.  Assuming the minimum IP header size, this results in a QUIC maximum packet size of 1232 bytes for IPv6 and 1252 bytes for IPv4.  A QUIC implementation MAY be more conservative in computing the QUIC maximum packet size to allow for unknown tunnel overheads or IP header options/extensions. </del>  Each pair of local and remote addresses could have a different PMTU. QUIC implementations that implement any kind of PMTU discovery therefore SHOULD maintain a maximum packet size for each combination of local and remote IP addresses. <del> If a QUIC endpoint determines that the PMTU between any pair of local and remote IP addresses has fallen below the size needed to support the smallest allowed maximum packet size, it MUST immediately cease sending QUIC packets, except for PMTU probe packets, on the affected path.  An endpoint MAY terminate the connection if an alternative path cannot be found. </del>  <del> 14.2. </del>  <del> PMTU discovery RFC1191 RFC8201 relies on reception of ICMP messages (e.g., IPv6 Packet Too Big messages) that indicate when a packet is dropped because it is larger than the local router MTU.  DPLPMTUD can also optionally use these messages.  This use of ICMP messages is potentially vulnerable to off-path attacks that successfully guess the addresses used on the path and reduce the PMTU to a bandwidth- inefficient value. </del>  An endpoint MUST ignore an ICMP message that claims the PMTU has <del> decreased below 1280 bytes. </del>  The requirements for generating ICMP (RFC1812, RFC4443) state that the quoted packet should contain as much of the original packet as size of the quoted packet can actually be smaller, or the information unintelligible, as described in Section 1.1 of DPLPMTUD. <del> QUIC endpoints SHOULD validate ICMP messages to protect from off-path injection as specified in RFC8201 and Section 5.2 of RFC8085.  This validation SHOULD use the quoted packet supplied in the payload of an ICMP message to associate the message with a corresponding transport connection DPLPMTUD. </del>  ICMP message validation MUST include matching IP addresses and UDP ports RFC8085 and, when possible, connection IDs to an active QUIC <del> session. The endpoint SHOULD ignore all ICMP messages that fail validation. </del>  An endpoint MUST NOT increase PMTU based on ICMP messages; see Section 3, clause 6 of DPLPMTUD.  Any reduction in the QUIC maximum QUIC's loss detection algorithm determines that the quoted packet has actually been lost. <del> 14.3. </del>  <del> Section 6.3 of DPLPMTUD provides considerations for implementing Datagram Packetization Layer PMTUD (DPLPMTUD) with QUIC. </del>  <del> When implementing the algorithm in Section 5 of DPLPMTUD, the initial value of BASE_PMTU SHOULD be consistent with the minimum QUIC packet size (1232 bytes for IPv6 and 1252 bytes for IPv4). </del>  <del> PING and PADDING frames can be used to generate PMTU probe packets. These frames might not be retransmitted if a probe packet containing them is lost.  However, these frames do consume congestion window, which could delay the transmission of subsequent application data. </del>  <del> A PING frame can be included in a PMTU probe to ensure that a valid probe is acknowledged. </del>  <del> The considerations for processing ICMP messages in the previous section also apply if these messages are used by DPLPMTUD. </del>  14.3.1. Endpoints that rely on the destination connection ID for routing incoming QUIC packets are likely to require that the connection ID be <del> included in PMTU probe packets to route any resulting ICMP messages (icmp-pmtud) back to the correct endpoint.  However, only long header packets (long-header) contain source connection IDs, and long header </del>  packets are not decrypted or acknowledged by the peer once the handshake is complete. <del> One way to construct a probe for the path MTU is to coalesce (see packet-coalesce) a Handshake packet (packet-handshake) with a short header packet in a single UDP datagram.  If the UDP datagram reaches the endpoint, the Handshake packet will be ignored, but the short header packet will be acknowledged.  If the UDP datagram causes an ICMP message to be sent, the first part of the datagram will be quoted in that message.  If the source connection ID is within the quoted portion of the UDP datagram, that could be used for routing. </del>  15. interfere with existing connections. Once a connection is established QUIC endpoints might accept some <del> unauthenticated ICMP packets (see icmp-pmtud), but the use of these </del>  packets is extremely limited.  The only other type of packet that an endpoint might accept is a stateless reset (stateless-reset) which relies on the token being kept secret until it is used.

This is with some minor additional changes: Reflow. Move the odd text about using Handshake back where it belongs. Make the language about how to construct a probe less prescriptive. Mention 1200 when referring to the "smallest allowed maximum packet size" so it is clear what that means.
NAME I went with "PMTU probe packet", which is an IP packet. Let me know if that makes sense.
OK - There are already "IP packets" and "QUIC packets" - so some other term works fro me.
A top-level comment on the term "probing": In Section: Probing a New Path: A packet containing only probing frames is a "probing packet", and a packet containing any other frame is a "non-probing packet". Please be sure that "probe" can not be confused with "probing" in PLPMTUD.
In the subsection on “Packet Size {#packet-size}”, I think the blocks of text describe two things. There is some general guidance on size, and there are specific instructions for the size of the initial packet. I’d suggest we do try to separate these two, it would make it much easier for someone to understand the size constraints for forming the initial packet.
Regarding the sizing of Initial packets, see . I checked the text for probing and we have "PMTUD probe" in one case and "probing packet" elsewhere. It's not ideal, but I didn't see any cases that were clearly busted. I don't have a good idea for alternative labels, but would welcome suggestions.
It turns out that coordinating the publication of all the documents is a real challenge. Right now, DPLPMTUD depends on QUIC, which is fine, but it turns out that this is a problem for other things that depend on DPLPMTUD who now transitively depend on QUIC. QUIC could adopt the DPLPMTUD text that is specific to QUIC and loosen this dependency chain. The text changes are relatively modest, see . While this could be editorial, I think that this will require a consensus call. The nature of the changes is such that it should be treated as design. Also, I think that there are substantive changes required.
So from an AD perspective, draft-ietf-tsvwg-datagram-plpmtud is currently blocking another document (URL) from being published that has a missref on it. Except for this question of QUIC I am ready to approve draft-ietf-tsvwg-datagram-plpmtud. To me it is also clear that integration of DPLPMTUD done in the QUIC WG will result in less editorial clarity issues that potentially could make an future implementer uncertain. As can already been seen in the PR there a few question that potentially can improve the operation and not unnecessary restrict the operation of DPLPMTUD. From my perspective it looks like the core is a set of minor editorial issues to resolve that will make the DPLPMTUD be clearer and resolve the missref issue and function as well as keeping it in the TSVWG document would. If a little bit more effort is invested after the first step then there is potential for additional improvements.

14. The QUIC packet size includes the QUIC header and protected payload,  <ins> but not the UDP or IP headers. </ins>  <ins> QUIC depends upon a minimum IP packet size of at least 1280 bytes. This is the IPv6 minimum size RFC8200 and is also supported by most modern IPv4 networks.  Assuming the minimum IP header size, this results in a QUIC maximum packet size of 1232 bytes for IPv6 and 1252 bytes for IPv4. The QUIC maximum packet size is the largest size of QUIC packet that can be sent across a network path using a single packet.  Any maximum packet size larger than 1200 bytes can be discovered using Path Maximum Transmission Unit Discovery (PMTUD; see pmtud) or Datagram Packetization Layer PMTU Discovery (DPLPMTUD; see dplpmtud). </ins> Enforcement of the max_udp_payload_size transport parameter (transport-parameter-definitions) might act as an additional limit on  <ins> the maximum packet size.  A sender can avoid exceeding this limit, once the value is known.  However, prior to learning the value of the transport parameter, endpoints risk datagrams being lost if they send packets larger than the smallest allowed maximum packet size of 1200 bytes. A client MUST expand the payload of all UDP datagrams carrying Initial packets to at least the smallest allowed maximum packet size (1200 bytes) by adding PADDING frames to the Initial packet or by coalescing the Initial packet; see packet-coalesce.  Sending a UDP datagram of this size ensures that the network path from the client to the server supports a reasonable Path Maximum Transmission Unit (PMTU).  This also helps reduce the amplitude of amplification attacks caused by server responses toward an unverified client address; see address-validation. </ins> Datagrams containing Initial packets MAY exceed 1200 bytes if the client believes that the network path and peer both support the size the DF bit MUST be set to prevent fragmentation on the path. A server MUST discard an Initial packet that is carried in a UDP  <ins> datagram with a payload that is less than the smallest allowed maximum packet size of 1200 bytes.  A server MAY also immediately close the connection by sending a CONNECTION_CLOSE frame with an error code of PROTOCOL_VIOLATION; see immediate-close-hs. </ins> The server MUST also limit the number of bytes it sends before validating the address of the client; see address-validation. payload an endpoint sends at any given time is referred to as the endpoint's maximum packet size.  <ins> An endpoint SHOULD use DPLPMTUD (dplpmtud) or PMTUD (pmtud) to </ins> determine whether the path to a destination will support a desired  <ins> maximum packet size without fragmentation.  In the absence of these mechanisms, QUIC endpoints SHOULD NOT send IP packets larger than the smallest allowed maximum packet size. </ins>  <ins> Both DPLPMTUD and PMTUD send IP packets that are larger than the current maximum packet size.  We refer to these as PMTU probes.  All QUIC packets that are not sent in a PMTU probe SHOULD be sized to fit within the maximum packet size to avoid the packet being fragmented or dropped RFC8085. If a QUIC endpoint determines that the PMTU between any pair of local and remote IP addresses has fallen below the smallest allowed maximum packet size of 1200 bytes, it MUST immediately cease sending QUIC packets, except for those in PMTU probes or those containing CONNECTION_CLOSE frames, on the affected path.  An endpoint MAY terminate the connection if an alternative path cannot be found. </ins> Each pair of local and remote addresses could have a different PMTU. QUIC implementations that implement any kind of PMTU discovery therefore SHOULD maintain a maximum packet size for each combination of local and remote IP addresses.  <ins> A QUIC implementation MAY be more conservative in computing the maximum packet size to allow for unknown tunnel overheads or IP header options/extensions. </ins>  <ins> 14.1.1. </ins>  <ins> Path Maximum Transmission Unit Discovery (PMTUD; RFC1191, RFC8201) relies on reception of ICMP messages (e.g., IPv6 Packet Too Big messages) that indicate when a packet is dropped because it is larger than the local router MTU.  DPLPMTUD can also optionally use these messages.  This use of ICMP messages is potentially vulnerable to off-path attacks that successfully guess the addresses used on the path and reduce the PMTU to a bandwidth-inefficient value. </ins> An endpoint MUST ignore an ICMP message that claims the PMTU has  <ins> decreased below the minimum QUIC packet size. </ins> The requirements for generating ICMP (RFC1812, RFC4443) state that the quoted packet should contain as much of the original packet as size of the quoted packet can actually be smaller, or the information unintelligible, as described in Section 1.1 of DPLPMTUD.  <ins> QUIC endpoints using PMTUD SHOULD validate ICMP messages to protect from off-path injection as specified in RFC8201 and Section 5.2 of RFC8085.  This validation SHOULD use the quoted packet supplied in the payload of an ICMP message to associate the message with a corresponding transport connection (see Section 4.6.1 of DPLPMTUD). </ins> ICMP message validation MUST include matching IP addresses and UDP ports RFC8085 and, when possible, connection IDs to an active QUIC  <ins> session.  The endpoint SHOULD ignore all ICMP messages that fail validation. </ins> An endpoint MUST NOT increase PMTU based on ICMP messages; see Section 3, clause 6 of DPLPMTUD.  Any reduction in the QUIC maximum QUIC's loss detection algorithm determines that the quoted packet has actually been lost.  <ins> 14.2. Datagram Packetization Layer PMTU Discovery (DPLPMTUD; DPLPMTUD) relies on tracking loss or acknowledgment of QUIC packets that are carried in PMTU probes.  PMTU probes for DPLPMTUD that use the PADDING frame implement "Probing using padding data", as defined in Section 4.1 of DPLPMTUD. Endpoints SHOULD set the initial value of BASE_PMTU (see Section 5.1 of DPLPMTUD) to be consistent with the minimum QUIC packet size.  The MIN_PLPMTU is the same as the BASE_PMTU. QUIC endpoints implementing DPLPMTUD maintain a maximum packet size (DPLPMTUD MPS) for each combination of local and remote IP addresses. 14.2.1. </ins>  <ins> From the perspective of DPLPMTUD, QUIC transport is an acknowledged packetization layer (PL).  A sender can therefore enter the DPLPMTUD BASE state when the QUIC connection handshake has been completed. </ins>  <ins> 14.2.2. </ins>  <ins> QUIC provides an acknowledged PL, therefore a sender does not implement the DPLPMTUD CONFIRMATION_TIMER while in the SEARCH_COMPLETE state; see Section 5.2 of DPLPMTUD. 14.2.3. An endpoint using DPLPMTUD requires the validation of any received ICMP Packet Too Big (PTB) message before using the PTB information, as defined in Section 4.6 of DPLPMTUD.  In addition to UDP port validation, QUIC validates an ICMP message by using other PL information (e.g., validation of connection IDs in the quoted packet of any received ICMP message). The considerations for processing ICMP messages described in pmtud also apply if these messages are used by DPLPMTUD. 14.3. </ins>  <ins> PMTU probes are ack-eliciting packets. </ins>  <ins> Endpoints could limit the content of PMTU probes to PING and PADDING frames as packets that are larger than the current maximum packet size are more likely to be dropped by the network.  Loss of a QUIC packet that is carried in a PMTU probe is therefore not a reliable indication of congestion and SHOULD NOT trigger a congestion control reaction; see Section 3, Bullet 7 of DPLPMTUD.  However, PMTU probes consume congestion window, which could delay subsequent transmission by an application. </ins> 14.3.1. Endpoints that rely on the destination connection ID for routing incoming QUIC packets are likely to require that the connection ID be  <ins> included in PMTU probes to route any resulting ICMP messages (pmtud) back to the correct endpoint.  However, only long header packets (long-header) contain the Source Connection ID field, and long header </ins> packets are not decrypted or acknowledged by the peer once the handshake is complete.  <ins> One way to construct a PMTU probe is to coalesce (see packet- coalesce) a packet with a long header, such as a Handshake or 0-RTT packet (long-header), with a short header packet in a single UDP datagram.  If the resulting PMTU probe reaches the endpoint, the packet with the long header will be ignored, but the short header packet will be acknowledged.  If the PMTU probe causes an ICMP message to be sent, the first part of the probe will be quoted in that message.  If the Source Connection ID field is within the quoted portion of the probe, that could be used for routing or validation of the ICMP message. The purpose of using a packet with a long header is only to ensure that the quoted packet contained in the ICMP message contains a Source Connection ID field.  This packet does not need to be a valid packet and it can be sent even if there is no current use for packets of that type. </ins> 15. interfere with existing connections. Once a connection is established QUIC endpoints might accept some  <ins> unauthenticated ICMP packets (see pmtud), but the use of these </ins> packets is extremely limited.  The only other type of packet that an endpoint might accept is a stateless reset (stateless-reset) which relies on the token being kept secret until it is used.


RFC3629. The application-specific variant of CONNECTION_CLOSE (type 0x1d) can <del> only be sent using 0-RTT or 1-RTT packets (QUIC-TLS, Section 4). </del>  When an application wishes to abandon a connection during the handshake, an endpoint can send a CONNECTION_CLOSE frame (type 0x1c) with an error code of APPLICATION_ERROR in an Initial or a Handshake

The longer the draft sits in my edit buffer, the more opportunities I have to see this sort of thing.

RFC3629. The application-specific variant of CONNECTION_CLOSE (type 0x1d) can  <ins> only be sent using 0-RTT or 1-RTT packets; see Section 4 of QUIC-TLS. </ins> When an application wishes to abandon a connection during the handshake, an endpoint can send a CONNECTION_CLOSE frame (type 0x1c) with an error code of APPLICATION_ERROR in an Initial or a Handshake


5.1.1. Each Connection ID has an associated sequence number to assist in <del> deduplicating messages.  The initial connection ID issued by an endpoint is sent in the Source Connection ID field of the long packet header (long-header) during the handshake.  The sequence number of the initial connection ID is 0.  If the preferred_address transport </del>  parameter is sent, the sequence number of the supplied connection ID is 1. connection (transport-parameters). An application protocol can also operate in a limited fashion during <del> the handshake phase.  0-RTT allows application messages to be sent by a client before receiving any messages from the server.  However, 0-RTT lacks certain key security guarantees.  In particular, there is no protection against replay attacks in 0-RTT; see QUIC-TLS. </del>  Separately, a server can also send application data to a client before it receives the final cryptographic handshake messages that allow it to confirm the identity and liveness of the client.  These 21.7. <del> QUIC and TLS both contain messages that have legitimate uses in some contexts, but that can be abused to cause a peer to expend processing resources without having any observable impact on the state of the connection. </del>  Messages can also be used to change and revert state in small or inconsequential ways, such as by sending small increments to flow

Some of these uses were a little too vague. The PMTUD one I will add to . The others I don't plan to fix.
I reviewed , and it is not always clear what a "message" is supposed to mean. It needs clarification in the text to disambiguate between different meanings, and could also benefit from definitions. Here is a review of the different meanings of "message" I found in the draft: TLS message (referred to as cryptographic message or cryptographic handshake message) ICMP message application message, which seems to refer to messages sent by applications with their own internal framing (not visible to QUIC, since each stream provides a simple bytestream with no framing) QUIC "message", which seems to roughly mean "any QUIC control frame" but this is not defined. In any case, there is no "QUIC message" as such in the base protocol. The first three meanings are understandable, but the last one should be clarified or removed altogether. Additionally, any usage of "message" not clearly falling into one of these four categories should be clarified. Here is a breakdown of usages that need clarification: [2.] An alternative view of QUIC unidirectional streams is a "message" abstraction of practically unlimited length. -> this is not exactly the same as an application message. That being said, the quotes make it clear that it is used in an approximate sense. [5.1.1.] Each Connection ID has an associated sequence number to assist in deduplicating messages -> this seems to refer to and frames specifically [5.3.] 0-RTT allows application messages to be sent by a client before receiving any messages from the server -> the second occurrence is unclear: does it refer to application messages sent by the server application? Or does it also include TLS messages or other "QUIC messages"? [10.3.] The application protocol exchanges whatever messages that are needed to cause both endpoints to agree to close the connection -> this is quite clear from the context, but it would be more consistent to use "application messages" [14.1.] An endpoint SHOULD use Datagram Packetization Layer PMTU Discovery (...) to determine whether the path to a destination will support a desired message size without fragmentation -> this should probably be replaced with "QUIC maximum packet size" as defined just above in the text. [21.7.] QUIC and TLS both contain messages that have legitimate uses in some contexts, but that can be abused (..) -> this section contains several other occurrences of "message". This is well-defined for TLS, but there is no such thing as a "QUIC message" currently.
I did not fix the ones in the following: 2 - this is clearly intended in the abstract, as you say. this is also in the abstract and I believe clear enough without further qualification
Thanks for the fix!

5.1.1. Each Connection ID has an associated sequence number to assist in  <ins> detecting when NEW_CONNECTION_ID or RETIRE_CONNECTION_ID frames refer to the same value.  The initial connection ID issued by an endpoint is sent in the Source Connection ID field of the long packet header (long-header) during the handshake.  The sequence number of the initial connection ID is 0.  If the preferred_address transport </ins> parameter is sent, the sequence number of the supplied connection ID is 1. connection (transport-parameters). An application protocol can also operate in a limited fashion during  <ins> the handshake phase.  0-RTT allows application data to be sent by a client before receiving any response from the server.  However, 0-RTT lacks certain key security guarantees.  In particular, there is no protection against replay attacks in 0-RTT; see QUIC-TLS. </ins> Separately, a server can also send application data to a client before it receives the final cryptographic handshake messages that allow it to confirm the identity and liveness of the client.  These 21.7.  <ins> QUIC and TLS both contain frames or messages that have legitimate uses in some contexts, but that can be abused to cause a peer to expend processing resources without having any observable impact on the state of the connection. </ins> Messages can also be used to change and revert state in small or inconsequential ways, such as by sending small increments to flow


(ECN) in the first packets it sends, as described in ecn-validation. The CRYPTO frame can be sent in different packet number spaces <del> (packet-numbers).  The sequence numbers used by CRYPTO frames to ensure ordered delivery of cryptographic handshake data start from zero in each packet number space. </del>  Endpoints MUST explicitly negotiate an application protocol.  This avoids situations where there is a disagreement about the protocol Handshake packets. This enforces cryptographic separation between the data sent in the <del> different packet sequence number spaces.  Packet numbers in each space start at packet number 0.  Subsequent packets sent in the same packet number space MUST increase the packet number by at least one. </del>  0-RTT and 1-RTT data exist in the same packet number space to make loss recovery algorithms easier to implement between the two packet

There were two cases where we used "sequence number" when we really meant something else.
Reading version -27 of the transport draft, I was surprised to see something called "sequence numbers", until I understood that it had nothing to do with either TCP sequence numbers or QUIC packet numbers. This is an unfortunate choice of words for two reasons: 1) people familiar with TCP may initially interpret "sequence numbers" in the TCP sense, causing confusion; 2) "packet numbers" and "sequence numbers" are too easily confused. For instance, I found a single instance in the draft where "sequence numbers" is used to actually mean "packet numbers": And another instance which uses a mix of both expressions: These instances can be easily fixed, but it would be beneficial to eliminate any source of confusion by renaming "sequence numbers" to something else. A simple suggestion: "connection ID counter". Babel (RFC6126) has a similar notion of "sequence number", and it uses the unambiguous abbreviation "Seqno" throughout the text: this may be an alternative solution.
"Packet sequence number" is the full name, but "packet number" is the shorter term used throughout the documents. I think we need to fix section 7 or anywhere else we use "sequence number". I'm ok with us using "packet sequence number", but I think "packet number" is preferable. I really do not want to rename packet number to something else.
NAME I agree that "packet number" should not be renamed, it's a central concept of the protocol. I was suggesting renaming "sequence numbers" instead (as used to track connection IDs)
Sorry for my confusion. I agree it'd be nice to rename "sequence numbers" in regards to CIDs as well. Maybe "Connection ID Index"?
So there are three things here: The mention of sequence number in relation to CRYPTO frames is a mistake. This should be "offset". Packet numbers are just packet numbers. Yes, it is a sequence number, but we don't use "sequence" here anywhere else, so that one instance is just a mistake. Sequence numbers for connection IDs. I'm not inclined to change these. Using index/indices would be fine, but the risk involved in making a change isn't worth it in my opinion. As this is sufficiently distinct from the TCP usage, I see no real risk of confusion either.
+1. I agree that they are sufficiently distinct. Also, I do not think we'd want to go down the path of changing the terms that are used in TCP to something else. To give an example, "FIN" in QUIC is different than that in TCP. Do we need to look for such cases and change all the names? I think making such changes could cause more confusion.
Agreed. This is just wrong. trying to submit edits to (D)TLS and QUIC.

(ECN) in the first packets it sends, as described in ecn-validation. The CRYPTO frame can be sent in different packet number spaces  <ins> (packet-numbers).  The offsets used by CRYPTO frames to ensure ordered delivery of cryptographic handshake data start from zero in each packet number space. </ins> Endpoints MUST explicitly negotiate an application protocol.  This avoids situations where there is a disagreement about the protocol Handshake packets. This enforces cryptographic separation between the data sent in the  <ins> different packet number spaces.  Packet numbers in each space start at packet number 0.  Subsequent packets sent in the same packet number space MUST increase the packet number by at least one. </ins> 0-RTT and 1-RTT data exist in the same packet number space to make loss recovery algorithms easier to implement between the two packet


fig-auth-cid-retry shows a similar handshake that includes a Retry packet. <del> For the handshakes in fig-auth-cid and fig-auth-cid-retry the client sets the value of the initial_source_connection_id transport parameter to "C1".  In fig-auth-cid-retry, the server sets original_destination_connection_id to "S1", </del>  retry_source_connection_id to "S2", and initial_source_connection_id <del> to "S3".  In fig-auth-cid, the server sets original_destination_connection_id to "S1", initial_source_connection_id to "S3", and does not include retry_source_connection_id.  Each endpoint validates the transport parameters set by their peer, including the client confirming that retry_source_connection_id is absent if no Retry packet was processed. </del>  7.4.

An attempt to make this slightly easier to read, based on feedback that this was hard to read and follow.
Looks good, but I think the paragraph on lines 1574 to 1582 might be best split too, into sub-paragraphs for each TP and how it's used.

fig-auth-cid-retry shows a similar handshake that includes a Retry packet.  <ins> In both cases (fig-auth-cid and fig-auth-cid-retry), the client sets the value of the initial_source_connection_id transport parameter to "C1". When the handshake does not include a Retry (fig-auth-cid), the server sets original_destination_connection_id to "S1" and initial_source_connection_id to "S3".  In this case, the server does not include a retry_source_connection_id transport parameter. When the handshake includes a Retry (fig-auth-cid-retry), the server sets original_destination_connection_id to "S1", </ins> retry_source_connection_id to "S2", and initial_source_connection_id  <ins> to "S3". Each endpoint validates transport parameters set by the peer.  The client confirms that the retry_source_connection_id transport parameter is absent if it did not process a Retry packet. </ins> 7.4.


generally, this is one higher than the offset of the byte with the largest offset sent on the stream, or zero if no bytes were sent. <del> For a stream that is reset, the final size is carried explicitly in a RESET_STREAM frame.  Otherwise, the final size is the offset plus the length of a STREAM frame marked with a FIN flag, or 0 in the case of incoming unidirectional streams. </del>  An endpoint will know the final size for a stream when the receiving part of the stream enters the "Size Known" or "Reset Recvd" state

Section: Stream Final Size /An endpoint MUST NOT send data on a stream at or beyond the final size./ It seems reasonable not to send after the Final Size , but why is this a “MUST NOT”? The need to require this is not stated, and the reaction of the receiver appears to be ignore. Is this requirement justified?
Because the Final Size is used to synchronize flow control consumption. If the Final Size is not, in fact, the last data ever sent on a stream, the two sides will disagree about how much flow control credit has been consumed. The reaction of the receiver isn't to ignore; in the very next paragraph:
Aha, I knew there must be a reason - could we add something like this sentence? "This is used to synchronize flow control consumption between the endpoints."
:+1: LGTM modulo a nitpick.Some suggestions

generally, this is one higher than the offset of the byte with the largest offset sent on the stream, or zero if no bytes were sent.  <ins> The final size of a stream is always signaled to the recipient.  The final size is the sum of the Offset and Length fields of a STREAM frame with a FIN flag, noting that these fields might be implicit. Alternatively, the Final Size field of a RESET_STREAM frame carries this value.  This ensures that all ways that a stream can be closed result in the number of bytes on the stream being reliably transmitted.  This guarantees that both endpoints agree on how much flow control credit was consumed by the stream. </ins> An endpoint will know the final size for a stream when the receiving part of the stream enters the "Size Known" or "Reset Recvd" state


A sender bundles one or more frames in a QUIC packet; see frames. A sender can minimize per-packet bandwidth and computational costs by <del> bundling as many frames as possible within a QUIC packet.  A sender MAY wait for a short period of time to bundle multiple frames before </del>  sending a packet that is not maximally packed, to avoid sending out large numbers of small packets.  An implementation MAY use knowledge about application sending behavior or heuristics to determine whether across multiple streams.  When a packet loss occurs, only streams with data in that packet are blocked waiting for a retransmission to be received, while other streams can continue making progress.  Note <del> that when data from multiple streams is bundled into a single QUIC </del>  packet, loss of that packet blocks all those streams from making <del> progress.  Implementations are advised to bundle as few streams as </del>  necessary in outgoing packets without losing transmission efficiency to underfilled packets.

We didn't really need to use a new "term" for the concept of including multiple frames in a packet. The use of "bundle" implied more than what was really happening. Using plain words is probably better.
What does “bundling” refer to? I think I might now understand coalescing, but I don’t understand if bundling is subtly different?
Several instances I see in the draft refer to the situation where an endpoint is required to send a packet containing certain frames. The endpoint might opportunistically add other frames, not mandated to be sent, as long as it's doing the work of generating and sending a packet anyway. These opportunistically added frames are "bundled" with the mandatory frames. More generally, the term refers to selecting a collection of frames waiting to be sent when generating an outgoing packet.
(Versus coalescing, which is putting more than one packet in a single UDP datagram.)
tbh, I think the text reads a lot better with bundling.Thanks! Clearer is better.

A sender bundles one or more frames in a QUIC packet; see frames. A sender can minimize per-packet bandwidth and computational costs by  <ins> including as many frames as possible in each QUIC packet.  A sender MAY wait for a short period of time to collect multiple frames before </ins> sending a packet that is not maximally packed, to avoid sending out large numbers of small packets.  An implementation MAY use knowledge about application sending behavior or heuristics to determine whether across multiple streams.  When a packet loss occurs, only streams with data in that packet are blocked waiting for a retransmission to be received, while other streams can continue making progress.  Note  <ins> that when data from multiple streams is included in a single QUIC </ins> packet, loss of that packet blocks all those streams from making  <ins> progress.  Implementations are advised to include as few streams as </ins> necessary in outgoing packets without losing transmission efficiency to underfilled packets.


token, it MUST attempt to validate the token, unless it has already completed address validation.  If the token is invalid then the server SHOULD proceed as if the client did not have a validated <del> address, including potentially sending a Retry.  If the validation succeeds, the server SHOULD then allow the handshake to proceed. </del>  The rationale for treating the client as unvalidated rather than discarding the packet is that the client might have received the token in a previous connection using the NEW_TOKEN frame, and if the server has lost state, it might be unable to validate the token at all, leading to connection failure if the packet is <del> discarded.  A server SHOULD encode tokens provided with NEW_TOKEN frames and Retry packets differently, and validate the latter more strictly. </del>  In a stateless design, a server can use encrypted and authenticated tokens to pass information to clients that the server can later

I think that this can be this simple.
section: Address Validation for Future Connections The following RFC2119 keyword appears within a note, which seems wrong, could the RFC2119 recommendation be moved outside of the note?: /A server SHOULD encode tokens provided with NEW_TOKEN frames and Retry packets differently, and validate the latter more strictly./

token, it MUST attempt to validate the token, unless it has already completed address validation.  If the token is invalid then the server SHOULD proceed as if the client did not have a validated  <ins> address, including potentially sending a Retry.  A server SHOULD encode tokens provided with NEW_TOKEN frames and Retry packets differently, and validate the latter more strictly.  If the validation succeeds, the server SHOULD then allow the handshake to proceed. </ins> The rationale for treating the client as unvalidated rather than discarding the packet is that the client might have received the token in a previous connection using the NEW_TOKEN frame, and if the server has lost state, it might be unable to validate the token at all, leading to connection failure if the packet is  <ins> discarded. </ins> In a stateless design, a server can use encrypted and authenticated tokens to pass information to clients that the server can later


This limit does act as an additional constraint on datagram size in the same way as the path MTU, but it is a property of the <del> endpoint and not the path.  It is expected that this is the space an endpoint dedicates to holding incoming packets. </del>  The initial maximum data parameter is an integer value that contains the initial value for the maximum amount of data that can

I did not add a citation for MAX_PLPMTU as that is subtly different and that seems like it would be more of a liability than a help in this case.
Text looks OK.
In maxudppayloadsize (0x03) the following text seems good: /This limit does act as an additional constraint on datagram size in the same way as the path MTU, but it is a property of the endpoint and not the path./ could it refer to the appropriate section in the QUIC spec? In DPLPMTUD the same value is known as “MAXPLPMTU”, Section 5.3.1 - in case you wish to cite that.
MAX_PLPMTU is slightly different as it refers to the IP layer, but you are right that it constrains the same underlying variable. I'll add a cross-reference though.

This limit does act as an additional constraint on datagram size in the same way as the path MTU, but it is a property of the  <ins> endpoint and not the path; see packet-size.  It is expected that this is the space an endpoint dedicates to holding incoming packets. </ins> The initial maximum data parameter is an integer value that contains the initial value for the maximum amount of data that can