The QUIC packet size includes the QUIC header and protected payload, but not the UDP or IP header. <del> Clients MUST ensure they send the first Initial packet in a single IP packet.  Similarly, the first Initial packet sent after receiving a Retry packet MUST be sent in a single IP packet. The payload of a UDP datagram carrying the first Initial packet MUST be expanded to at least 1200 bytes, by adding PADDING frames to the Initial packet and/or by coalescing the Initial packet (see packet- </del>  coalesce).  Sending a UDP datagram of this size ensures that the <del> network path supports a reasonable Maximum Transmission Unit (MTU), and helps reduce the amplitude of amplification attacks caused by server responses toward an unverified client address; see address- validation. </del>  <del> The datagram containing the first Initial packet from a client MAY exceed 1200 bytes if the client believes that the Path Maximum Transmission Unit (PMTU) supports the size that it chooses. </del>  A server MAY send a CONNECTION_CLOSE frame with error code <del> PROTOCOL_VIOLATION in response to the first Initial packet it receives from a client if the UDP datagram is smaller than 1200 bytes.  It MUST NOT send any other frame type in response, or otherwise behave as if any part of the offending packet was processed as valid. </del>  The server MUST also limit the number of bytes it sends before validating the address of the client; see address-validation.

This section had a bunch of text that wasn't updated as the result of other changes elsewhere. There was a suggestion that this could reference other sections, but as this is in some ways the authoritative section on the subject, I decided to keep the requirements. That's duplicative, but I couldn't see a simple way to deal with that.
The draft requires that if an Initial packet is not padded, the server sends a CONNECTIONCLOSE. I think that the goal was to have the server drop those packets and optionally send a signal. But that only applies if the server hasn't established state. We should reword that. NAME says: Originally posted by NAME in URL
Note that our principles here mean that we aren't obligated to defend against this style of attack, but rewording here is probably a good idea.
I agree that we've decided not to try to protect against attacks in the first flight, but this seems open up an extremely trivial attack vector that we can easily shutdown. I believe we should prevent it.
Why would a server want to send a CONNECTION_CLOSE in the first place? I think dropping the packet is a perfectly valid thing to do, and in fact is preferable at such an early state in the connection.
I'm going to request that this go for a consensus call. It's borderline, but I think that it is worth running the process here. And it isn't critical that this get fixed right now.
Following a discussion on Slack (cc NAME NAME NAME ), I feel a clarification might be needed on how/which INITIAL packets should be padded to 1200 bytes. E.g., URL states: have UDP payloads of at least 1200 bytes, adding padding to packets in the datagram as necessary. Some people (cc NAME interpret this to mean ALL packets containing initials should be padded (e.g., also an initial ACK from the client to the server) and not just the first initials with crypto data from the client. However, URL somewhat contradicts/restates this: be expanded to at least 1200 bytes, by adding PADDING frames to the Initial packet and/or by coalescing the Initial packet (see Section 12.2). NAME said he feels this should be removed/restated. I personally don't know why ALL initials should be padded outside of the first flight... NAME mentioned the stateless LB use case, where they could drop all initials < 1200 bytes, though I'm not sure why a LB would do this... NAME mentioned that padding the client's initial ACKs gives additional allowance towards the 3X amplification factor limit, but as those intitial ACKS will arrive with/close to the client's first handshake packet (which validates the path, IIUC, URL). Either way, I feel the text should be tightened to clarify the intended use. Full disclosure: we're working on some dirty stuff that spoofs/abuses QUIC initials to carry in-band metadata, so we'd really not like intermediaries dropping < 1200 byte initials by default.
"Some people (cc NAME interpret this to mean ALL packets containing initials should be padded" This matches my interpretation and sounds simple and straightforward.
Yes. We have the consensus to require every datagram containing an Initial packet to be padded. See .
To clarify, section 8.1 talks about client Initials. NAME at NAME are you saying all datagrams containing client Initial packets or all datagrams containing Initial packets? I think there are some uses for non-padded server to client Initials, such as a CONNECTION_CLOSE.
I also meant client-sent initials. Though, I assume for asymmetric PMTU reasons, the server's initials should also be 1200+ bytes? IIUC correctly from the comments above: requiring client initial ACKs to also be 1200+ is mainly out of convenience/consistency ()? In that case, the text in section 14 should indeed be updated to make that more clear? I can submit a PR for that.
NAME That is my understanding, and I think that a PR would be very welcome. To clarify, the following statement in section 8.1 is correct as I understand: Clients MUST ensure that UDP datagrams containing Initial packets have UDP payloads of at least 1200 bytes, adding padding to packets in the datagram as necessary.. The text in section 14 seems to be incorrect (likely out-of-date).
I'm pretty sure that the text in Section 14 is outdated, and should probably be removed, with a short reference to section 8.1.
That's right, that's just cruft.
Thank you for the PR Martin. Just for my understanding: the requirement to pad client initials from the second flight onward (i.e., after receiving server packets, ACKing server initials) is purely for convenience. It is not necessary for PMTU or anti-amplification-allowance. As such, the limit should also not be enforced at intermediaries wholesale?
For the most part, yes, the second flight is generally sent with some evidence that the return address is live, however we can't guarantee that the second flight carries sufficient entropy. For instance, HelloRetryRequest causes a second flight that still requires anti-amplification. Initial packets with acknowledgments in the second flight might still require padding in the case that the client isn't able to send Handshake packets (which can happen when you have a partial server flight). In the end, this is just simpler. I would be sad if intermediaries were to "enforce" MUSTs from QUIC.
In thinking about this a bit more, I wonder if 8.1 says everything we want it to say: Clients MUST ensure that UDP datagrams containing Initial packets have UDP payloads of at least 1200 bytes, adding padding to packets in the datagram as necessary. What about INITIAL packets which contain only a CONNECTION_CLOSE frame? Should those be padded? It doesn't seem terribly necessary to do this...
For the client, I think requiring padding even a CONNECTIONCLOSE is the right thing to do, since it simplifies processing on the server side. On the server, I definitely do not want to require padding Initials that only contain CONNECTIONCLOSE.
Yeah, it should be super uncommon for a client to send a CONNECTION_CLOSE in Initial, and I don't think the cost of padding it is terribly high in that uncommon case. I think keeping it simple is probably best here.
The far most likely case of a server receiving a CONNECTION_CLOSE is a middlebox trying to disrupt the handshake (i.e. RST injection). I do not think we should help them by relaxing the full-size padding requirement.
some editorializing

The QUIC packet size includes the QUIC header and protected payload, but not the UDP or IP header.  <ins> A client MUST expand the payload of all UDP datagrams carrying Initial packets to at least 1200 bytes, by adding PADDING frames to the Initial packet or by coalescing the Initial packet (see packet- </ins> coalesce).  Sending a UDP datagram of this size ensures that the  <ins> network path from the client to the server supports a reasonable Maximum Transmission Unit (MTU).  Padding datagrams also helps reduce the amplitude of amplification attacks caused by server responses toward an unverified client address; see address-validation. </ins>  <ins> Datagrams containing Initial packets MAY exceed 1200 bytes if the client believes that the Path Maximum Transmission Unit (PMTU) supports the size that it chooses. </ins> A server MAY send a CONNECTION_CLOSE frame with error code  <ins> PROTOCOL_VIOLATION in response to an Initial packet it receives from a client if the UDP datagram is smaller than 1200 bytes.  It MUST NOT send any other frame type in response, or otherwise behave as if any part of the offending packet was processed as valid. </ins> The server MUST also limit the number of bytes it sends before validating the address of the client; see address-validation.


If a sender runs out of flow control credit, it will be unable to send new data and is considered blocked.  A sender SHOULD send a STREAM_DATA_BLOCKED or DATA_BLOCKED frame to indicate it has data to <del> write but is blocked by flow control limits.  These frames are expected to be sent infrequently in common cases, but they are considered useful for debugging and monitoring purposes. A sender SHOULD NOT send multiple STREAM_DATA_BLOCKED or DATA_BLOCKED frames for the same data limit, unless the original frame is determined to be lost.  Another STREAM_DATA_BLOCKED or DATA_BLOCKED frame can be sent after the data limit is increased. </del>  4.2. when sending an ack-eliciting packet (see QUIC-RECOVERY), but only if no other ack-eliciting packets have been sent since last receiving a packet.  Restarting when sending packets ensures that connections do <del> not prematurely time out when initiating new activity. </del>  The value for an idle timeout can be asymmetric.  The value advertised by an endpoint is only used to determine whether the

To prevent a connection from going idle while it has application data to send, this PR recommends periodically sending the blocked frame as a keep alive.
Hmm, this is a design change.
NAME I'd argue this whole section should have similar text for STREAMS_BLOCKED, not just the text this PR is adding.
An issue to correspond to PR NAME and I believe this is likely a design issue.
Oops, this was unnecessary. Sorry for the churn.
If a connection gets send blocked for long enough (i.e. the peer application isn't draining the data right now, and thus not giving more flow control credit) the idle timer will fire. This is because no data is being sent over the wire while the sender is blocked. To me, it seems that we shouldn't consider a connection as idle if the application is actively trying to send. While in this blocked state, should the transport periodically send some ACK eliciting packet to keep the connection alive?
It is not much different from being blocked because there is currently nothing to send. The timeout will clean up connections that do not make progress. If the application is happy with waiting a very long time to proceed, it could send pings, but it might as well want to call it a day. So I'm not sure the spec needs to say anything about that. We are talking about 10 minutes of no progress by default.
It seems reasonable to use a keep-alive. If you have something you want to use the connection for and you aren't able to maintain activity on the connection, then either poke it or - more likely - give up. I don't think that we need a design change for this, but if you want to write some text, I wouldn't be opposed to taking a look.
Thinking about this, another solution would be to continue to send a *blocked frame, while in the blocked state. Even if the peer acknowledged the first blocked frame, if you're still blocked, you send it again. Personally, for my implementation, this would be the easiest thing to implement. I don't think the spec should make any requirements in this area, but I do think it should call this problem out and possibly make a recommendation.
Sending a BLOCKED frame repeatedly, maybe on exponential backoff, makes sense to me. NAME any interest in writing a PR for this one?
I could definitely write a PR. We implement exactly this. I would argue though that it's not exponential. It's immediate and will go forever. The app has data to send. The connection is not idle, and should never idle out in this scenario. It's the app's responsibility to time out, if their protocol dictates as such (which it might not, if they expect to be blocked while a long running operation is processed).
This should be a design issue. The proposal to send multiple BLOCKED frames is a design change.
NAME Corresponding text already exists in section 4.5. My point is that we might want to update this too, as the sender should behave the same way when it is blocked by data flow control vs. when blocked by stream-level concurrency.Still LGTM

If a sender runs out of flow control credit, it will be unable to send new data and is considered blocked.  A sender SHOULD send a STREAM_DATA_BLOCKED or DATA_BLOCKED frame to indicate it has data to  <ins> write but is blocked by flow control limits.  If a sender is blocked for a period longer than the idle timeout (idle-timeout), the connection might be closed even when data is available for transmission.  To keep the connection from closing, a sender that is flow control limited SHOULD periodically send a STREAM_DATA_BLOCKED or DATA_BLOCKED frame when it has no ack-eliciting packets in flight. </ins> 4.2. when sending an ack-eliciting packet (see QUIC-RECOVERY), but only if no other ack-eliciting packets have been sent since last receiving a packet.  Restarting when sending packets ensures that connections do  <ins> not prematurely time out when initiating new activity.  An endpoint might need to send packets to avoid an idle timeout if it is unable to send application data due to being blocked on flow control limits; see flow-control. </ins> The value for an idle timeout can be asymmetric.  The value advertised by an endpoint is only used to determine whether the


client believes that the Path Maximum Transmission Unit (PMTU) supports the size that it chooses.  A server MAY send a CONNECTION_CLOSE frame with error code PROTOCOL_VIOLATION in response to an Initial packet it receives from a client if the UDP datagram is smaller than 1200 bytes.  It MUST NOT

This won't necessarily prevent fragmentation from happening, but it's the best we can do. I didn't REQUIRE dropping of fragments, because some systems do that for you, but a SHOULD seems appropriate.
Gorry notes correctly that fragmentation can play poorly with using IW10. If a sender uses a large maxdatagramsize, and if the network fragments, then IW10 can result in substantially larger amplification than 10x. A simple fix here is to say that if the endpoint uses maxdatagramsize larger than 1280 bytes (this is what the transport spec limits it to in the absence of DPLPMTUD), then the endpoint MUST set DF on outgoing packets. This is a problem with RFC 6928 as well, but we should be clear about it for QUIC.
Shouldn't endpoint always set DF for all QUIC packets? TCP does so.
I tend to agree with Praveen here. I think this is something that's under-specified in transport? I believe it should be in section 14: URL
Yeah, this is probably an issue in transport. DF seems like the obvious answer.
That was my first answer, but I thought that might be too intrusive. If we all agree that is a fine response, I'm happy with that. I can write up a PR.
From memory, I think there was once a discussion on DF bit not being accessible in some language interfaces. As such, the formulation should perhaps be that the DF bit SHOULD be set whenever possible. Having a MUST whenever possible could be problematic because possible could involve a lot of complexity.
-- URL
NAME yes, that is exactly the text I am referring to. I'm questioning if that requirement is practically enforceable.

client believes that the Path Maximum Transmission Unit (PMTU) supports the size that it chooses. <ins> UDP datagrams MUST NOT be fragmented at the IP layer.  In IPv4 IPv4, the DF bit MUST be set to prevent fragmentation on the path. </ins> A server MAY send a CONNECTION_CLOSE frame with error code PROTOCOL_VIOLATION in response to an Initial packet it receives from a client if the UDP datagram is smaller than 1200 bytes.  It MUST NOT


allow values to be linked by an on-path observer to the connection on which it was issued, unless the values are encrypted.  For example, it cannot include the previous connection ID or addressing <del> information.  Information that allows the server to distinguish between tokens from Retry and NEW_TOKEN MAY be accessible to entities other than the server. </del>  It is unlikely that the client port number is the same on two different connections; validating the port is therefore unlikely to connection error of type FRAME_ENCODING_ERROR. An endpoint might receive multiple NEW_TOKEN frames that contain the <del> same token value.  Endpoints are responsible for discarding duplicate values, which might be used to link connection attempts; see validate-future. </del>  Clients MUST NOT send NEW_TOKEN frames.  Servers MUST treat receipt of a NEW_TOKEN frame as a connection error of type

This is already implied by the existing text, but it isn't said directly. That is, we say that the values can't be linkable, which is a stronger requirement. Saying that you can't send the same value on two connections is much clearer.
This text does not forbid the server from sending the same token on multiple connections. Consequently the client has a hard, or impossible, task of excluding duplicates. At least if you only read that section. Later the text says: >The token MUST NOT include information that would allow it to be linked by an on-path observer to the connection on which it was issued. This suggests that the token MUST indeed be unique, but in a roundabout manner. It would be helpful to make the initial paragraph clearer by stating the that the duplicate token can only happen on the same connection and that the server MUST ensure different connections use different tokens.
A nit, but looks good, thanks for working patiently!

allow values to be linked by an on-path observer to the connection on which it was issued, unless the values are encrypted.  For example, it cannot include the previous connection ID or addressing  <ins> information.  A server MUST ensure that every NEW_TOKEN frame it sends is unique across all clients, with the exception of those sent to repair losses of previously sent NEW_TOKEN frames.  Information that allows the server to distinguish between tokens from Retry and NEW_TOKEN MAY be accessible to entities other than the server. </ins> It is unlikely that the client port number is the same on two different connections; validating the port is therefore unlikely to connection error of type FRAME_ENCODING_ERROR. An endpoint might receive multiple NEW_TOKEN frames that contain the  <ins> same token value if packets containing the frame are incorrectly determined to be lost.  Endpoints are responsible for discarding duplicate values, which might be used to link connection attempts; see validate-future. </ins> Clients MUST NOT send NEW_TOKEN frames.  Servers MUST treat receipt of a NEW_TOKEN frame as a connection error of type


disabled for a connection.  Implementations MUST allow administrators of clients and servers to disable the spin bit either globally or on a per-connection basis.  Even when the spin bit is not disabled by <del> the administrator, implementations MUST disable the spin bit for a given connection with a certain likelihood.  The random selection process SHOULD be designed such that on average the spin bit is disabled for at least one eighth of network paths.  The selection process performed at the beginning of the connection SHOULD be applied for all paths used by the connection. </del>  When the spin bit is disabled, endpoints MAY set the spin bit to any value, and MUST ignore any incoming value.  It is RECOMMENDED that

This builds on to address the linkability issues that it added. It recommends 1/16 of network paths are disabled randomly as suggested by NAME It also allows the determination to be made per connection ID rather than per-path (which is more granular, but I expect it will have a nearly identical effect). The net effect is that 1/8 paths will not spin as originally intended.
The draft says: Given that each endpoint needs to decide for itself whether it is enabling spin bit, it is not exactly clear whether each endpoint SHOULD disable spin bit on at least 1/8 (=12.5%) of connections, or it can assume that the other endpoint is also making a random decision to disable and disable only on 1-sqrt(7/8) (=6.5%) of connections.
as the author of this passage, I intended the latter (assume the other side is disabling as well, so 1-sqrt(7/8)), but the "at least" is in there since just disabling 1/8 is probably what people will do, and the result (about 1/4) is "at least 1/8".
NAME could you review ?
One suggestionLooks good technically. It may be slightly confusing, though, that the last sentence only talks about paths, while a sentence before talks about "paths or connection IDs". In any case, the behavior ensures a sufficient amount of traffic without the spin bit to prevent ossification of the bit.

disabled for a connection.  Implementations MUST allow administrators of clients and servers to disable the spin bit either globally or on a per-connection basis.  Even when the spin bit is not disabled by  <ins> the administrator, endpoints MUST disable their use of the spin bit for a random selection of at least one in every 16 network paths, or for one in every 16 connection IDs.  As each endpoint disables the spin bit independently, this ensures that the spin bit signal is disabled on approximately one in eight network paths. </ins> When the spin bit is disabled, endpoints MAY set the spin bit to any value, and MUST ignore any incoming value.  It is RECOMMENDED that


A client SHOULD NOT reuse a NEW_TOKEN token for different connection attempts.  Reusing a token allows connections to be linked by <del> entities on the network path; see migration-linkability.  A client MUST NOT reuse a token if it believes that its point of network attachment has changed since the token was last used; that is, if there is a change in its local IP address or network interface.  A client needs to start the connection process over if there is any change in its local address prior to completing the handshake. </del>  Clients might receive multiple tokens on a single connection.  Aside from preventing linkability, any token can be used in any connection

states: This is an unnecessary constraint and there’s value in not having it. A server can, for example, include multiple IP addresses for a client in a token, enabling client validation from multiple attachment points.
Agreed. Just to give one example, it'd make sense for a server to issue a NEWTOKEN token containing two addresses, if the client is using one path to communicate while having another path being probed and validated. Note also that it is difficult to implement the MUST NOT that exists now. Consider the case where a client receives a NEWTOKEN frame during or after path migration. Is the client expected to bind the token to the old IP address? Or the new address? It is hard to tell the intention of the server.
I agree this seems unnecessary, given tokens cannot be used multiple times. The worst case scenario is including the token increases the number of Initial packets the client needs to send to two packets from one and the token turns out not to be useful for address validation.
As is clear from the PR, this should be marked design.

A client SHOULD NOT reuse a NEW_TOKEN token for different connection attempts.  Reusing a token allows connections to be linked by  <ins> entities on the network path; see migration-linkability. </ins> Clients might receive multiple tokens on a single connection.  Aside from preventing linkability, any token can be used in any connection


packets, and the more out-of-order the packets are, the more important it is to send an updated ACK frame quickly, to prevent the peer from declaring a packet as lost and spuriously retransmitting <del> the frames it contains. </del>  ack-tracking and ack-limiting describe an exemplary approach for determining what packets to acknowledge in each ACK frame.

If so, don't serialize the oldest ranges, as previously specified.
This isn't the right section; Section 13.2.4 already has context for this. I'll make a small PR.
NAME In terms of which is the right section, I'll explain my thinking. In my mind, I think of the subsequent two sections as examples of how one could limit ACK frame size, but not really required reading, because I think that's how they were originally written? However, there's quite a bit of normative text in there, so maybe I have an obsolete mental model. An easy alternative to doing what's in those sections is to set a max number of ACK ranges(ie: 3), which I think should be considered acceptable, even if it might not be optimal.
Let's land this and handle any redundancies in an editorial pass along with .
In trying to do a simple fix for , I found other irrelevant or poorly-written recommendations in how a receiver could limit ACK ranges it sends. This section needs to be rewritten. Specifically, the sentence below is a poor recommendation and contradicts earlier recommendation on acking until the ACK frame is acknowledged. The recommendation below is fine, but the rationale does not make sense: This should be a design change.
I'm not in love with this (entire) section either, but I'd like to better understand this issue. I agree that sentence is mis-written, though I think the intent was fine. On the rationale, can you clarify why it does not make sense?
The of the Transport Draft should explain what to do when generated ACK is too large to fit in a packet: Generate a shorter ACK frame (instead of aborting connection or erroring out). To generate the shorter frame, don't include older (smaller) packet numbers. (As opposed to not including newer packet numbers.)
There is a good amount of existing text about what ranges to send, ie "ACK frames SHOULD always acknowledge the most recently received packets," I think this can be solved with some editorial additions to 13.2.2. URL
The text we already have in 13.2.4 has the right context:
Ok, this became a bigger fix. See and .
Seems fine.A few suggestions

packets, and the more out-of-order the packets are, the more important it is to send an updated ACK frame quickly, to prevent the peer from declaring a packet as lost and spuriously retransmitting  <ins> the frames it contains.  An ACK frame is expected to fit within a single QUIC packet.  If it does not, then older ranges (those with the smallest packet numbers) are omitted. </ins> ack-tracking and ack-limiting describe an exemplary approach for determining what packets to acknowledge in each ACK frame.


remains acknowledged, even if it does not appear in a future ACK frame.  This is unlike TCP SACKs (RFC2018). <del> It is expected that a sender will reuse the same packet number across different packet number spaces.  ACK frames only acknowledge the packet numbers that were transmitted by the sender in the same packet number space of the packet that the ACK was received in. </del>  Version Negotiation and Retry packets cannot be acknowledged because they do not contain a packet number.  Rather than relying on ACK

by taking NAME suggestion and adding a sentence about how the same PN may be used in each space.
NAME -- PTAL at .
Thanks for the suggestion NAME , accepted.
I have an editorial disagreement here, but as with all such disagreements, I'm happy to go with whatever more people think reads better. I find this use of "numeric value" to be unnecessary. Making it implicit (which is what this sentence is doing) doesn't really change the way a reader consumes this sentence. As a reader, I will replace "numeric value" with "packet number"; there isn't a different definition for packet number that is anything more than the numeric value. At a minimum, I would swap the order of the two sentences.
The contains the following paragraph: I see a few problems with the first sentence: As written, it does not make sense. There is no mention of packet number reuse expectations across packet number spaces in . The connection between it and the sentence that follows is not clear. I propose that the first sentence simply be dropped.
The second sentence can stand some improvement as well: There is no need for the word "only." It is not necessary to say that the ACK frame was received. Packets, not packet numbers, are acknowledged. How about something like this:
Looks good.

remains acknowledged, even if it does not appear in a future ACK frame.  This is unlike TCP SACKs (RFC2018).  <ins> Packets from different packet number spaces can be identified using the same numeric value.  An acknowledgment for a packet needs to indicate both a packet number and a packet number space.  This is accomplished by having each ACK frame only acknowledge packet numbers in the same space as the packet in which the ACK frame is contained. </ins> Version Negotiation and Retry packets cannot be acknowledged because they do not contain a packet number.  Rather than relying on ACK


13.2.4. <del> To limit ACK Ranges (see ack-ranges) to those that have not yet been received by the sender, the receiver SHOULD track which ACK frames have been acknowledged by its peer.  The receiver SHOULD exclude already acknowledged packets from future ACK frames whenever these packets would unnecessarily contribute to the ACK frame size.  When the receiver is only sending non-ack-eliciting packets, it can bundle a PING or other small ack-eliciting frame with a fraction of them, such as once per round trip, to enable dropping unnecessary ACK ranges and any state for previously sent packets.  The receiver MUST NOT bundle an ack-eliciting frame, such as a PING, with all packets that would otherwise be non-ack-eliciting, in order to avoid an infinite feedback loop of acknowledgements. To limit receiver state or the size of ACK frames, a receiver MAY limit the number of ACK Ranges it sends.  A receiver can do this even without receiving acknowledgment of its ACK frames, with the knowledge this could cause the sender to unnecessarily retransmit some data.  Standard QUIC algorithms (QUIC-RECOVERY) declare packets lost after sufficiently newer packets are acknowledged.  Therefore, the receiver SHOULD repeatedly acknowledge newly received packets in preference to packets received in the past. </del>  13.2.5.

This PR does not substantively change the recommendations, but it does remove one SHOULD that did not make any sense. I would prefer to go through with this as a design change.
NAME might want to scrub this up a little and land it.
Ok, comments incorporated, PTAL.
Woo!
In , NAME notes that added this "MUST": My understanding is that this prevents the largest acknowledged packet from going backwards. We use the monotonic increase of the Largest Acknowledged field in two ways: In ECN validation, this is used to filter out ACK frames that might have arrived out of order, which might have lower counts legitimately. In key updates, where the value of the largest acknowledged is used to drive the change to new keys and to prevent use of old keys after an update. In my view, having ECN validation fail is quite unfortunate, but likely workable. On the other hand, uncertainty about largest acknowledged could introduce instability in the key update process. If an endpoint's conception of what the value is changes, I doubt that we'll see extra key updates (I think that the AEAD would protect us from that), but we might leave open the possibility of interleaving of packets from two different key phases. I would suggest that we retain this "MUST".
For out of order ACK frames, you can use the packet number the ACK arrives in. I realize that we have tried to allow frames to be retransmitted, but I think retransmitting ACK frames without modifying the ACK delay/etc is so harmful we should probably prohibit that one frame from being retransmitted, since it creates a bunch of edge cases. Or we could let ECN validation could fail, but that'd be a bit sad. I'm confused about what the issue with key updates is? The largest acked you've ever seen doesn't change, just the largest acked from the last ACK frame. When determining the size of the packet number, we say: "The sender MUST use a packet number size able to represent more than twice as large a range than the difference between the largest acknowledged packet and packet number being sent." I'd rather not make this a MUST unless we have a good reason. There are valid use cases that become more complex with this constraint.
I really want to understand the use cases that you think that this might break. Because key updates very much depend on tracking largest acknowledged. The key updates case has a fairly simple example. Say you receive packet 10 in key phase 1. Then 11 in key phase 2. That's valid. But if you ACK those and then only remember up to packet 8, packet 9 can use key phase 2 and you won't know that it is invalid.
I'd like to suggest tracking largest acknowledged from forcing all ACK frames to contain the largest acknowledged and have that value never decrease. Given you need to track largest acknowledged locally(ie: not only in the in-flight ACK frame), I don't think there are any problems there. I have two cases in mind where the largest acked could validly decrease: 1) A receiver that tracks a limited number of ACK blocks and the oldest block happens to be the one containing largest acked. 2) If an application was reordering tolerant(ie: from URL) then two different senders on a single 'connection' may be sending packets with disparate packet numbers, causing what appears to be very large scale reordering. In this case, it'd be valuable to discard the older ACK ranges even if they contained the largest acked. Maybe the first is an edge case we don't care about in a use case we aren't optimizing for and the latter is solvable with careful issuing of packet number ranges. Even so, I'm not sure what the motivation is for this change? I guess the ECN case could be, but using largest acked as a proxy for retransmitted ACKs is a very bad design in my opinion.
I don't think we really need to cater for retransmitted ACK frames in any way. What I'm concerned about is reordering. Yes, using the increase of largest acknowledged as a proxy for strict ordering is a little weird, but the assumption is that frame handling is distinct from packet handling and packet numbers are not available when frames are handled. (I think that's the principle we agreed on; that's why it's OK to use packet numbers in key updates, for example, something that I can confirm is borne out by implementation experience.) The case you describe here has the appearance of exactly that type of reordering if the two senders also generate their own ACK frames (though this seems very strange). But I think that the consequences of that sort of design are not our responsibility to deal with.
I don't recall on us agreeing that packet numbers weren't available when frames are handled, can you clarify why that property is important?
I agree with NAME on the principle of not requiring the enclosing packet's packet number during frame handling, and we use sequence numbers within frames to help with reordering. This was at least true in an earlier version of NCID frames, and we had discussed this principle then. This is also why the ACK_FREQUENCY frame in URL includes a sequence number. That said, I don't understand the issue with key updates here. Largest acked so far is a connection-level state variable that can be used for key updates. I suspect that I'm not understanding the example scenario. NAME can you elaborate on your key update example? NAME Is your example motivated by a receiver that does not remember the largest acked or one that does not wish to ack anything but the packet that was just received?
I was under the impression that the argument was that largest acknowledged was not being tracked, but Ian made it clear that the concern was about not signaling it. That removes key updates as a concern. If remains unclear, then I can't really do better.
NAME my example was motivated by either case, but primarily not wanting to ACK anything besides what was just received.
Perhaps it's a naive model, but I had envisioned an implementation remembering a chain of ranges of packet numbers it had received or not. The implementation would save space by forgetting the earliest packet numbers when the chain got too long, and would construct an ACK frame out of the latest set of these ranges it had room for. In that mental model, at least, there's not really room for the idea that you might drop the largest-acked frame number from your memory because it's too old -- it's always at the most-recent end of your chain, even if you currently happen to be receiving packets that fill in the middle for some implementation-specific reason. (Implementation-specific to the peer, which you obviously can't know about or code for!)
That matches my model for this. It is also likely the case that the next packet you receive is largest+1, so there is no real advantage to forgetting the largest acknowledged. Not sending largest acknowledged is slightly different though. I can understand how an implementation might choose to send only ranges that have recently changed. My contention is that this is not a good choice and we should recommend against that.
NAME to produce some text here that separates retention policy (MUST) from signaling policy (SHOULD).
In trying to do a simple fix for , I found other irrelevant or poorly-written recommendations in how a receiver could limit ACK ranges it sends. This section needs to be rewritten. Specifically, the sentence below is a poor recommendation and contradicts earlier recommendation on acking until the ACK frame is acknowledged. The recommendation below is fine, but the rationale does not make sense: This should be a design change.
I'm not in love with this (entire) section either, but I'd like to better understand this issue. I agree that sentence is mis-written, though I think the intent was fine. On the rationale, can you clarify why it does not make sense?
The of the Transport Draft should explain what to do when generated ACK is too large to fit in a packet: Generate a shorter ACK frame (instead of aborting connection or erroring out). To generate the shorter frame, don't include older (smaller) packet numbers. (As opposed to not including newer packet numbers.)
There is a good amount of existing text about what ranges to send, ie "ACK frames SHOULD always acknowledge the most recently received packets," I think this can be solved with some editorial additions to 13.2.2. URL
The text we already have in 13.2.4 has the right context:
Ok, this became a bigger fix. See and .
LGTM modulo one paragraph.

13.2.4.  <ins> A receiver limits the number of ACK Ranges (ack-ranges) it remembers and sends in ACK frames, both to limit the size of ACK frames and to avoid resource exhaustion.  After receiving acknowledgments for an ACK frame, the receiver SHOULD stop tracking those acknowledged ACK Ranges. It is possible that retaining many ACK Ranges could cause an ACK frame to become too large.  A receiver can discard unacknowledged ACK Ranges to limit ACK frame size, at the cost of increased retransmissions from the sender.  This is necessary if an ACK frame would be too large to fit in a packet, however receivers MAY also limit ACK frame size further to preserve space for other frames. When discarding unacknowledged ACK Ranges, a receiver MUST retain the largest received packet number.  A receiver SHOULD retain ACK Ranges containing newly received packets or higher-numbered packets. A receiver that sends only non-ack-eliciting packets, such as ACK frames, might not receive an acknowledgement for a long period of time.  This could cause the receiver to maintain state for a large number of ACK frames for a long period of time, and ACK frames it sends could be unnecessarily large.  In such a case, a receiver could bundle a PING or other small ack-eliciting frame occasionally, such as once per round trip, to elicit an ACK from the peer. A receiver MUST NOT bundle an ack-eliciting frame with all packets that would otherwise be non-ack-eliciting, to avoid an infinite feedback loop of acknowledgements. </ins> 13.2.5.


accordingly.  Note that more than one connection ID can be associated with a connection; see connection-id. <del> If the Destination Connection ID is zero length and the packet matches the local address and port of a connection where the endpoint used zero-length connection IDs, QUIC processes the packet as part of that connection. </del>  Endpoints can send a Stateless Reset (stateless-reset) for any packets that cannot be attributed to an existing connection.  A

Make the rules on matching packets to connections consistent with our existing advice on use of zero-length connection IDs. cc NAME for review
: : If only the local address is used, the described multiplexing is impossible. Multiplexing zero-length connection IDs is useful for reducing overhead when remote clients are known to not use the described features, so I think this language should be updated to refer to the 4-tuple rather than only the local connection.
I don't think that these are completely inconsistent, though there is an unstated section in those quotes from Section 5.2: that the implementation has followed the advice and provided a unique local address where it uses a zero-length connection ID. This needs a little care, but we could just remove "local" and this would be OK, but it's probably best to be a little less crude.
I think literally "the packet matches the address and port of a connection" would be much too vague, particularly due to the singular forms used.
Thanks!

accordingly.  Note that more than one connection ID can be associated with a connection; see connection-id.  <ins> If the Destination Connection ID is zero length and the addressing information in the packet matches the addressing information the endpoint uses to identify a connection with a zero-length connection ID, QUIC processes the packet as part of that connection.  An endpoint can use just destination IP and port or both source and destination addresses for identification, though this makes connections fragile as described in connection-id. </ins> Endpoints can send a Stateless Reset (stateless-reset) for any packets that cannot be attributed to an existing connection.  A


21.10. <del> This document defines QUIC Version Negotiation packets version- </del>  negotiation, which can be used to negotiate the QUIC version used between two endpoints.  However, this document does not specify how this negotiation will be performed between this version and

Typically we use "in {{}}" or "(see {{}})", so this lines up with that.

21.10.  <ins> This document defines QUIC Version Negotiation packets in version- </ins> negotiation, which can be used to negotiate the QUIC version used between two endpoints.  However, this document does not specify how this negotiation will be performed between this version and


example, draft-ietf-quic-http-01 is identified using the string "h3-01".  Non-compatible experiments that are based on these draft versions MUST append the string "-" and an experiment name to the identifier. For example, an experimental implementation based on draft-ietf-quic- 3.3. <del> HTTP/3 relies on QUIC as the underlying transport.  The QUIC version being used MUST use TLS version 1.3 or greater as its handshake </del>  protocol.  HTTP/3 clients MUST indicate the target domain name during the TLS handshake.  This may be done using the Server Name Indication (SNI) RFC6066 extension to TLS or using some other mechanism.

As discussed in Singapore, this ties HTTP/3 to QUIC version 1 until someone writes down how another QUIC version is used.
We've removed the text saying that application protocols can constrain which QUIC versions they can run over. We need to discuss where the correlation between ALPN tokens and version numbers gets considered. Some proposals discussed in Cupertino: ALPN tokens imply one specific QUIC version. This leads to a possible error case, where an ALPN token is offered/accepted over another QUIC version. Application protocols can restrict which QUIC versions they run over. This implies that the QUIC version being spoken must be considered when selecting an ALPN token.
Discussed with NAME this is fine in the TLS draft, but the HTTP/3 draft needs to have a firmer opinion. Right now it says "anything with TLS 1.3".
The discussion in Singapore seemed to end with the conclusion that HTTP/3 would be defined to refer to QUIC version 1 exclusively. But we decided not to answer the broader question of what ALPN actually refers to. That was considered too broad a topic for the QUIC WG to address without involving TLS, HTTP, and others.

example, draft-ietf-quic-http-01 is identified using the string "h3-01". <ins> Draft versions MUST use the corresponding draft transport version as their transport.  For example, the application protocol defined in draft-ietf-quic-http-25 uses the transport defined in draft-ietf- quic-transport-25. </ins> Non-compatible experiments that are based on these draft versions MUST append the string "-" and an experiment name to the identifier. For example, an experimental implementation based on draft-ietf-quic- 3.3.  <ins> HTTP/3 relies on QUIC version 1 as the underlying transport.  The use of other QUIC transport versions with HTTP/3 MAY be defined by future specifications. QUIC version 1 uses TLS version 1.3 or greater as its handshake </ins> protocol.  HTTP/3 clients MUST indicate the target domain name during the TLS handshake.  This may be done using the Server Name Indication (SNI) RFC6066 extension to TLS or using some other mechanism.


specifications. QUIC version 1 uses TLS version 1.3 or greater as its handshake <del> protocol.  HTTP/3 clients MUST indicate the target domain name during the TLS handshake.  This may be done using the Server Name Indication (SNI) RFC6066 extension to TLS or using some other mechanism. </del>  QUIC connections are established as described in QUIC-TRANSPORT. During connection establishment, HTTP/3 support is indicated by

To avoid conflicts, this includes what appears to be the generally-agreed .
Something seems to have gone wrong with here.
Oh, yes. I missed which document this was in.
In reading PR, I noticed that we have the text (unchanged by that PR): HTTP/3 clients MUST indicate the target domain name during the TLS handshake. This may be done using the Server Name Indication (SNI) {{!RFC6066}} extension to TLS or using some other mechanism. I read this as saying that in the absence of "some other mechanism" the SNI extension must be present. But as I read 6066, it prohibits IP literals: Literal IPv4 and IPv6 addresses are not permitted in "HostName". As such, I'm not quite sure how to interpret the HTTP/3 MUST when the browser is requesting a URL with an IP literal as the host of the URL's. I could imagine two interpretations: When HTTP/3 says "target domain name" that implicitly excludes IP literals, so no SN extension is expected. When HTTP/3 says "target domain name" that implicitly includes IP literals, so an SNI extension is expected. (Or probably more likely, 3. I'm misunderstanding something more deeply) If 1 is the intended behavior it might be worth some text clarifying that. However, it seems like it would be "good" if QUIC handshakes always included the client's understanding of the hostname so I'd love the interpretation to be 2 (though that seems to conflict with 6066 so perhaps that's hard.) Thoughts?
For HTTP/1.1 and HTTP/2 over TLS, we do not send the IP literal in SNI. Chrome accidentally did it once, which broke Apache httpd. It got confused because IPv6 literals have colons. Of course, the compatibility issue does not apply to a new protocol like QUIC. It would be nice for QUIC to mandate SNI, and, all else equal, avoiding random TLS inconsistencies is good. But this rule is odd and "you MUST send SNI whenever you are supposed to send SNI" is tautological. TLS 1.3 says something woefully noncommittal: I dunno, everything is a mess. :-( If we were doing things anew, perhaps we'd have stuck the port number in there too. (As it stands, TLS does not authenticate the port, yet the port is part of the origin, and I doubt HTTP servers reliably check the port in the Host header.)
Maybe the rule can be concretely: if you have something else/better use that, otherwise if you have a domain name, include SNI, and finally if you don't have a domain name (including IP literals), do whatever. The HTTP scheme permits a broader range of "host" identifiers than we usually contemplate, so the 3rd option has to catch more than just IP literals. TLS was always necessarily lax because different applications have different constraints, but the idea was to ensure that for cases where you have the ability to use SNI that you used it.
That sounds reasonable. How about some text like: HTTP/3 clients MUST support a mechanism to indicate the target host to the server during the TLS handshake. This may be done using the Server Name Indication (SNI) {{!RFC6066}} extension to TLS or using some other mechanism. If the target host is valid for this mechanism (in the case of SNI this means the host is a domain name) the client MUST indicate the host using this mechanism.
I didn't check before, but I should point out that RFC6066 is very clear about IP literals: PR opened. NAME do you think that this is OK as editorial?
Right. This is what I said in my first post on this issue :) Thanks. Look great!
I think it's fine -- we're clarifying the requirement we already have. Nothing we previously allowed is being prohibited, and nothing previously prohibited is now allowed. However, you've based the PR on another one that's clearly not editorial, so.... Maybe we just do a consensus call on the combined change anyway.
I like NAME wording, but the content looks sound.

specifications. QUIC version 1 uses TLS version 1.3 or greater as its handshake  <ins> protocol.  HTTP/3 clients MUST support a mechanism to indicate the target host to the server during the TLS handshake.  If the server is identified by a DNS name, clients MUST send the Server Name Indication (SNI) RFC6066 TLS extension unless an alternative mechanism for indicate the target host is used. </ins> QUIC connections are established as described in QUIC-TRANSPORT. During connection establishment, HTTP/3 support is indicated by


to treat a stream error as a connection error. Because new error codes can be defined without negotiation (see <del> extensions), receipt of an unknown error code or use of an error code in an unexpected context MUST NOT be treated as an error.  However, closing a stream can constitute an error regardless of the error code (see request-response). </del>  This section describes HTTP/3-specific error codes which can be used to express the cause of a connection or stream error.

This is not quite editorial. We previously cautioned implementations not to freak out about unexpected uses of error codes; this text requires them to treat unknown/unexpected codes as equivalent to H3NOERROR.
Because this changes a MUST, this might deserve the full process, even if it's borderline.
Creating an issue on behalf of NAME that reflects . On that PR they quote a section of the document: Section 9), receipt of an unknown error code or use of an error code in an unexpected context MUST NOT be treated as an error. However, closing a stream can constitute an error regardless of the error code (see Section 4.1). and say: Maybe [replacing "an error" with "a further error"] is what's meant? (If not, please explain it better.)
I don't think the suggested change is aligned with the intent but there might be something we can do to improve the readability. "error codes" are used for normal operations when it comes to closing streams. So one interpretation of the current text is to say that if you get an unknown error code in a RESET_STREAM, don't treat it as any kind of error. The suggested text "further error" confuses this interpretation.
Maybe something like "MUST NOT be interpreted as an indication of a problem"?
The paragraph in question (see ) talks about different things: Let's break it up to make it easier to talk about: Treating an unexpected error code as an error is not allowed, in particular: a. Receiving unknown error is not an error in itself (a "further" error); and b. Receiving an unexpected error code (known or unknown) is not an error. A stream close may be an error independent of error code. Item 1(a) follows from Section 9. This does not need to be stated here. Item 1(b) seems to suggest that error codes are not to be used to make decisions, for they could be anything, notwithstanding all the "MUST treat as error type" admonitions elsewhere in the code. (This is what the text implies, which may not be what was intended.) Item 2 says that some error occur due to changes in stream state and are not communicated via an error code from peer. I would drop Items 1(a) and 1(b); Item 2 can be rephrased. P.S. Given the loose treatment given to error codes by their recipient, it is logical to allow their sender to keep on treating specified errors as errors ("MUST treat as error..."), but allow setting the error codes to anything. Stating that "MUST treat as error of type X and SHOULD set error code to X" sounds awkward; perhaps this guidance could be added to Section 8 as a new item.
1a/1b are two sides of the same coin. Because error codes are extensible, a future document might define a new error code, or might prescribe the use of an existing error code in a new circumstance. Therefore, the takeaway should be: An error code the recipient knows in an situation where they expect it has understood semantics that the recipient might be able to act on. An unknown error code, or an error code in an unexpected situation, has semantics which the recipient does not understand and cannot act on. The statement here is that the second case is not a violation of the protocol by the peer; you know the stream (or connection) is closing even if you don't understand why. The admonition is to respond to the closure itself, not the unknown semantics of why it was closed. While I agree that both versions of the second case follow from Section 9, I think it's useful to mention them in the context of error handling as well. We've seen recently with HTTP/2 what happens when the possibility of extension values isn't mentioned/considered in the description of how the base-protocol pieces are to be handled.
NAME points out that promotion from stream to connection error is something you shouldn't do absent an understanding of the error semantics.
The only case where this matters is for stream errors (RESETSTREAM / STOPSENDING). The scenario we want to avoid is closing the connection in response to an unexpected error code in a stream closure. So the right guidance here might be to say that you MUST NOT close the connection based on the unexpected error code.
Correction: We also care about closing one side of bidirectional streams. You shouldn't close the other direction because the first direction closed. One possibility is to treat unknown error codes as equivalent to NO_ERROR.
If you receive a STOPSENDING, aren't you supposed to RESETSTREAM if you haven't sent a FIN yet? I guess the other direction is weirder (abrupt termination of response, should you terminate the request too. etc?) but probably tearing everything down is the right move most of the time? At least it's not totally unexpected or catastrophic. Preventing connection closure still seems like the main objective. I think the general guidance of 'treat unknown error codes as you would treat NO_ERROR' is good, though.
Right, but if you receive a RESET_STREAM, that doesn't tell you anything about what to do with the other direction.
NAME / NAME this is borderline, but the PR does touch the specifics of the "MUST." Shall we run the process on this?
I don't think it changes the intent; let's keep it editorial unless someone explicitly objects to that.
SOLD, as editorial, to the chairs.
LGTM. I like the new text. It's super clear!

to treat a stream error as a connection error. Because new error codes can be defined without negotiation (see  <ins> extensions), use of an error code in an unexpected context or receipt of an unknown error code MUST be treated as equivalent to H3_NO_ERROR.  However, closing a stream can have other effects regardless of the error code (see request-response). </ins> This section describes HTTP/3-specific error codes which can be used to express the cause of a connection or stream error.


A client has more flexibility in the value it chooses for the Push ID in a GOAWAY that it sends.  A value of 2^62 - 1 indicates that the <del> server can continue fulfilling pushes promised while processing outstanding requests, and the client can continue granting push credit as needed (see frame-max-push-id).  A smaller value indicates the client will reject pushes with Push IDs greater than or equal to this value.  Like the server, the client MAY send subsequent GOAWAY frames so long as the specified Push ID is strictly smaller than all previously sent values. </del>  Once all accepted requests and pushes have been processed, the endpoint can permit the connection to become idle, or MAY initiate an

Proposed update to to get better H2 parity. Note: This is a PR on Alan's PR, rather than a replacement PR.
The current spec only allows GOAWAY to be sent from the server to the client and contains only the last processed bidirectional stream ID. This makes sense for the vanilla HTTP use case, but may not play well with extensions. Unidirectional streams are a flexible extension point, and it may be useful for each endpoint to communicate to the peer the last processed unidirectional stream id. An HTTP extension may also define a use for server initiated bidirectional streams, and may want the GOAWAY signal for these also. True if it's an extension then the extension author can also extend GOAWAY to cover these cases or introduce another mechanism, but it would be a shame if many extensions had to cover the same ground. I suggest we allow GOAWAY to be sent by either endpoint, and that it should contain both the last processed peer initiated bidirectional and unidirectional stream.
I was compelled by Alan's discussion about this. It sounds like you are suggesting having two fields in the GOAWAY frame, one for each type. My thought was to mirror the way MAX_STREAMS works and have independent GOAWAY frames.
I think this argument makes sense, but this feels increasingly like a transport feature and not an HTTP feature. If you add a unidirectional limit, I'd prefer a single frame, unless NAME has a reason he thinks separate frames are likely to be useful in practice.
I don't have a strong opinion here. For reference, the original issue about moving GOAWAY out of transport is URL For the separate frames topic: HTTP/3 currently lacks explanation about how GOAWAY works with MAXSTREAMS (see ). So my suggestion pre-empts that without much thought put into the final fix. Therefore it might be wrong. Some weak arguments: Before sending a GOAWAY, there is no explicit limit set by the server. The stream ID limit is implicit and real but more importantly, you don't invoke the requirement . Imagination required on how we fix this language up for MAXSTREAMS. From Alan's problem statement description, I imagined GOAWAY to operate more like a drain state trigger. So having separate frames would allow the draining of unidirectional traffic and bidirectional traffic independently. E.g. a client declares "I'm intending to leave soon, and I don't want to accept new things from you (pushes) but I want the ability to create new requests, so please continue to give me bidi stream credits". On a more practical level, a frame that includes both fields when only one type wants to be drained will likely end up with a set to max_int. So we'd waste bytes to express "don't do anything". Modelling this by making fields optional adds complexity for little gain. Finally, we a separate GOAWAY for unidirectional streams opens the door for it to be an extension frame. However, I think this would be the wrong thing to do as it would make managing extensions very difficult.
The salient comment from for me is "I think that the GOAWAY frame needs to identify requests, not streams" Making GOAWAY bidirectional and cover unidirectional and bidirectional streams seems to run counter to that statement, which is why it was starting to feel more transport-y. But, I don't feel that strongly about this, unless it becomes an implementation hassle of some sort.
I think the use case I'm not sure is compelling is the one where we only want to drain one type of stream, but no the other. My model of GOAWAY is that it's used for graceful shutdown, and I agree that could be useful on either side(moreso for non-HTTP applications, but maybe with WebTransport?). I have a hard time thinking of an application where graceful shutdown requires continued creation of one of the stream types, but they may exist.
I think this is the pragmatic argument for making a single frame. You don't want to make it easy for implementations to get into semi-draining states unintentionally. Sending GOAWAY with a limit of both forces that decision, which is good.
I had thought the H2 requirement was that after sending you will not accept or generate new streams, but I can't actually find that text in 7540. If we're concerned about being semi-draining, this seems like a restriction I'd want to have as well.
I don't think that we need to make special accommodations for possible future extensions, where the requirements for graceful shutdown are unpredictable. Strange. There's an example in h3. In order to answer a request, a server might create multiple push streams after the point that shutdown is initiated. The model for GOAWAY in h3 is based on the realization that the primary purpose of HTTP is to provide answers to requests. Stop the requests and you eventually run out of work. For the client, no signaling is required. For the server, it just has to have a way to block incoming requests. To Ian's point about identifying requests, I believe that it does (if the text doesn't support that, that's just a bug). Extensions can change anything. But we can't really say what the graceful shutdown model would be without understanding the extension. Assuming that once shutdown is initiated there will no longer be any need for new streams is not safe. I can easily think of cases where new streams are needed to fulfill existing commitments. Any protocol that treats streams as messages with an 1:N or N:M relationship between the message that initiates a transaction and the messages required to complete that transaction is open to this. Take a SQL or similar exchange where a transaction is opened, some queries and updates performed, and the transaction is committed or rolled back. A graceful shutdown might trigger rollback or it needs to allow for existing transactions to complete. If you want to allow transactions to complete, then you might have multiple streams created after the shutdown order is issued. And the SQL case is still simple, because it is still fundamentally transactional. If the protocol is truly bidirectional, with either peer initiating activity that might take time, then you need a more sophisticated shutdown process. The quanta used to measure what is accepted or rejected during shutdown might not be a discrete linear sequence as it is for simple transactional protocols; it could be multi-dimensional. Indeed, the reason for this issue is the possibility that h3 with extensions could end up being multi-dimensional. The stuff people are doing with bidirectional transport use could end up like that, especially where it is intended to be multiplexed with h3. Expecting things to line up around streams is probably reasonable, but when streams are used for multiple purposes, things get messy. You need to be identifying the unit. We get to use streams for GOAWAY because client-initiated bidirectional streams are exclusively used for requests, but we use stream identifiers only as means of identifying requests there (again, if the text implies otherwise, we need to fix that). So my view is that GOAWAY is good enough for shutdown of requests. But that is all we can realistically provide for at this layer. If an extension needs to be shutdown gracefully, then the extension that adds that thing will need to provide signaling in support of that shutdown. Practically, that means multiple frames involved in shutdown, but that's not terrible, just wait for all shutdown-y things to complete shutdown before tearing down the connection.
I think I'm fine with GOAWAY as is, but PUSH is never a required feature, it's a server-side optimization/pessimization.
This wasn't making a value-judgment of server push, just a note that there are cases that don't fit the model proposed.
To clarify, I was trying to note push is optional, and not required. In practice, I don't think PUSH would work in this case, at least in browsers. If the client receives a GOAWAY, then I wouldn't expect any new requests to go to that connection, including those which may be matched with results from the push cache.
That's true, but URLs may have already been promised (not yet pushed) and there may already be requests waiting to rendezvous with the pushed stream. WDYT?
That's also a detail of one particular push implementation, not an inherent way that push works. I think the Push implementation in WinInet would still work, because the push cache is at the level of a request group rather than an individual connection. (Somewhat moot for deployment use, but existence proof that there's more than one possible implementation of that.)
I currently see 4 options: 1) Do nothing and assume there are some implementation specific edge cases that are going to go fairly sideways, but they're unlikely to cause user visible issues. 2) Move GOAWAY to transport and have it include uni and bidi. 3) Add a max push id to the existing GOAWAY 4) Add a new frame for push. I'm ok with 1 and 2, and would be hesitant to pursue 3 or 4.
I prefer option 1. I dont see what 3 provides. I think we could achieve the same by adding text that says something like "the server commits to resolving all existing promises sent on streams up to and including the stream ID indicated in GOAWAY. Pushes may be cancelled or reset as normal". My 2p. GOAWAY is starting to sound like more of a STOP_CREATING (you have credit but it's a waste to use it in the current state) and PROCESSED (endpoint processed all entities up to ID, it is safe to retry higher IDs on a different connection).
NAME besides the theoretical interest of how GOAWAY works with extensions, do you have any HTTP/3 extension in mind where the current GOAWAY breaks badly? And if so, would it be feasible to add another GOAWAY_EXTENSION frame for that use case? ie: Can we just not fix this, or make some editorial tweaks?
The extension we are developing allows complete bidirectionality to HTTP, so a server can initiate a request/response to the client. The H2 mapping of this extension uses a client-sent GOAWAY to clue the server in to stop doing this. I don't know that missing this will break-badly; it probably depends on what application developers do with the bidirectionality. It's asymmetric with H2, seemingly because we think we got it wrong in H2 and are trying to correct it? I'm trying to say there is some value in keeping it symmetric. I think Lucas correctly identified the semantics of what GOAWAY is - STOP_CREATING and PROCESSED. As mentioned in the original issue, anyone writing an extension can also extend the GOAWAY frame under that extension that does what I proposed, so doing nothing is of course viable.
Discussed in London; we should mimic whatever H2 does, and there's some uncertainty in the room about what a client-sent GOAWAY is intended to mean. H2 GOAWAY says: So HTTP/2 does define it in terms of streams, and a server upon receipt of a GOAWAY frame MUST NOT open more streams, which would include push. The other side probably can't police you, however, because they can't reliably know that the opening of the stream wasn't already in flight; they'll just ignore it.
It was mentioned in Montreal that the HTTP WG was asked about this. Was the response reflected back in this ticket? apologies, this one completely popped off my stack
To summarize the discussion on the HTTP list: H2 GOAWAY is symmetric A GOAWAY sent by the client will prevent any pushes higher than the last-stream-id in the GOAWAY Martin feels like this behavior is wrong, and it shouldn't be carried over into H3. Do we need to build consensus from the HTTP working group that it is OK to deviate from H2 semantics in H3, like we are now planning to do for priorities?
While this is true, we have - as a working group - previously discussed the principles underlying this and agreed that the unit we should care about is the request. However, I agree that this is something we should discuss at the HTTP working group. Where we can each put on a different set of hats.
What's ill defined is whether the pushes that come on a request are part of the request or not. I think a reasonable argument can be made for either interpretation.
A different use case came up: Consider streaming a (potentially infinite) live video as a sequence of pushes. How does a client signal its intent to close this connection, and stop the server from pushing more video segments? The client can send a STOPSENDING frame on the request stream which is receiving the PUSHPROMISEs, but this may result in less than graceful termination of that stream.
In this case what do you advertise MAXPUSHID as? By limiting the value and not incrementing it, you can cause the server to be unable to drive the connection, and allow the client to hit its idle timeout.
Yeah, limiting credit via MAXPUSHID is one way, but I still prefer the explicit signal from GOAWAY - it's like immediately zeroing out the credit.
Agree. This is like writing a cheque but asking the other side not to cash it (out of band) to avoid it bouncing and causing pain all round.
Even with a client sent GOAWAY, you'd receive a few more pushes within the next RTT. If you managed MAXPUSHID like flow control and only gave out enough extra for the time window you wanted to shut down within(ie: 1-2RTTs), I believe this achieves the desired behavior? If that time window is <1RTT, then your only real option is CONNECTION_CLOSE?
I think it's more about challenging the assumption that stopping sending requests (or push credits) is the same as explicitly signaling your intent to close the connection. If the endpoint knows it will shutdown, it signals the peer, and the peer can take actions in response to ensure the close is graceful.
NAME Can you clarify what your current proposal is? Is it to move GOAWAY to transport or keep it in H3 and make it bidi?
Discussed in Cupertino: Clients can send GOAWAY. Add transition considerations to talk about relaying GOAWAY from an H2 to H3 connection / vice versa. We would like the client-sent GOAWAY to carry a Push ID. There is still disagreement over what the precise semantics of that are. NAME to write a PR to provoke argument.
The semantics of an identifier in the server-to-client GOAWAY are twofold. The identifier clearly identifies requests that were not "processed", which allows the client to retry those requests. The identifier also identifies which of those requests can be cleaned up expeditiously. My view is that the former is significantly more important. Enabling clean retry is one of the best features of h2. For pushes, the need to signal whether the data was "processed" is moot, because only safe requests can be pushed. So most of this comes down to what optimizations are enabled by the explicit Push ID. In h2, requests are automatically "cancelled", though the bits are still retransmitted by TCP if the connection is kept alive. A client isn't required to reset or finalize any associated streams if the requests were not complete, but the transport will automatically repair lost segments. In h3, the same applies, though there is an optimization that might be enabled. The transport layer can be told not to retransmit data for "cancelled" streams if packets are lost. This doesn't require explicit RESETSTREAM or STOPSENDING frames, reprioritization might be enough. Depending on the transport implementations, endpoints might need to use RESET_STREAM to ensure that transport stacks are aware of the need to suppress retransmissions. From that perspective, an explicit Push ID would enable new capabilities in terms of suppression of unnecessary retransmission. For example, say that there is an open request that is receiving associated pushes for data that arrives on a fixed cadence. Say that a client wants to move to a different connection (because it was told with Alt-Svc, for instance). The client might initiate a request starting at a particular time on a new connection. When the client cancels this connection, there might be a number of pushes in flight. If the client can precisely identify the push that immediately precedes the request it initiates on the new connection, the server can identify which of these pushes need to be fully delivered, and which can be abandoned. However, this optimization really only works if the client can make this identification precisely. If there are multiple requests in flight on the connection, the client might not be able to identify or predict the Push ID that will be used for the next push in the sequence. The server might avoid this by providing promises well ahead (>1RTT) of the actual pushes so that the client doesn't need to guess, but that might not be possible in the general case. This also requires that Push IDs are used strictly sequentially with respect to the global clock. This doesn't guarantee clean separation even if Push IDs monotonically increase for each resource. For instance, if two resources were providing the same sort of pushes, response 0 might promise Push ID 10 for time=44, but response 4 might promise Push ID 9 that corresponds to time=45. That makes it difficult for a client to make a clean cut with a single value. This also doesn't interact well with a model where the answering of a request depends on receiving a set of pushes. If you have a resource that produces a continuous set of pushes on the same connection as a resource that (ideally) produces a fixed number of pushes (see also Prefer-Push), then the cut for the continuous resource might be clean, but it might split the pushes for the resource with a fixed set of associated pushes. Of course, all of these arguments could easily be countered by saying that this is just an optimization, but given that it's an optimization we don't have for h2 (because TCP), I think that I'd prefer not to provide the capability. Finer-grained signals might be valuable: such as the idea that individual requests might have separate budgets for push that can be progressively expanded by clients as they are ready for more. That seems like something worth considering for future versions of this protocol.
makes a subtle but major change to the semantics, and I think we need to carefully consider whether that's something we want. We were conflating these two models during interim discussion, and I don't think we're clear on our direction yet. Instead of "Clients MUST NOT send new requests on the connection after receiving GOAWAY", we now have "Endpoints MUST NOT initiate new requests or pushes on the connection with an identifier greater than or equal to the smallest identifier received in a GOAWAY frame." In other words, GOAWAY() has changed from meaning "I'll service your in-flight requests, but don't make any new ones" to a no-op, since the client couldn't have initiated such a request anyway. I understand that this might be a desirable pattern in the server-to-client direction, where certain use patterns consider the set of pushes part of the fulfillment of the client's request, and the client actively wants the server to continue initiating pushes until the request is complete. This doesn't seem a wise semantic to bring to the client-to-server direction. In HTTP/2, the recipient of the GOAWAY was prohibited from initiating any more activity (requests or pushes) regardless of the value in the frame. My first preference is to preserve that, and actually have the same semantics as HTTP/2.
I agree with NAME that's a useful semantic we should preserve in HTTP/3.
I think Mike articulated a point I was trying to make on Slack but in a much better way. +1.
Ah, I was trying to get the behavior NAME was after for pushes (eg: let the server keep pushing if it wants to), and didn't intend to allow the client to send more requests after receiving a GOAWAY -- that doesn't work. But it sounds like we'd prefer to keep the push behavior symmetric with h2 (no more pushes after GOAWAY receipt) than introduce a new push semantic where the client can allow pushes to continue even after signaling its intent to close. This is more along the lines that NAME was advocating for - where the intent to close and how pushes are wound down are completely decoupled frames. The discussion on slack I thought was different - it was about what the sender of a GOAWAY should do with new incoming streams. I intended the PR to reflect that the sender of a GOAWAY should continue allowing requests or pushes with ID's smaller than the sent GOAWAY value. Do we agree this is the right behavior?
The sender of GOAWAY needs to continue accepting incoming streams up to the indicated ID, yes. If we decide the other semantic is useful, it's easy enough to define another frame (SOFTGOAWAY, INCONCLUSION, etc.) in-spec or as an extension that simply hints to the other side that they should start cleaning up, but doesn't fully prohibit new streams.
It sounds like LAST_ORDERS would be the classic optional-to-understand extension -- "I'd like you to wrap up gracefully now, so it doesn't hurt when I forcibly close in a few seconds." Can we agree on (and update the PR to) keeping H2 semantics of "no new requests/pushes" after receipt of GOAWAY in either direction, and leave that for another day? I've created for this direction.

A client has more flexibility in the value it chooses for the Push ID in a GOAWAY that it sends.  A value of 2^62 - 1 indicates that the  <ins> server can continue fulfilling pushes which have already been promised, and the client can continue granting push credit as needed (see frame-max-push-id).  A smaller value indicates the client will reject pushes with Push IDs greater than or equal to this value. Like the server, the client MAY send subsequent GOAWAY frames so long as the specified Push ID is strictly smaller than all previously sent values. </ins> Once all accepted requests and pushes have been processed, the endpoint can permit the connection to become idle, or MAY initiate an


A sender can make exceptions for probe packets so that their loss detection is independent and does not unduly cause the congestion controller to reduce its sending rate.  An endpoint might set a <del> separate timer when a PATH_CHALLENGE is sent, which is cancelled when </del>  the corresponding PATH_RESPONSE is received.  If the timer fires before the PATH_RESPONSE is received, the endpoint might send a new PATH_CHALLENGE, and restart the timer for a longer period of time.  9.5.

Section 9.4 of draft-ietf-quic-transport: An endpoint might set a separate timer when a PATHCHALLENGE is sent, which is cancelled when the corresponding PATHRESPONSE is received. If the timer fires before the PATHRESPONSE is received, the endpoint might send a new PATHCHALLENGE, and restart the timer for a longer period of time. GF: Possibly so, but what value has this timeout? To avoid potential congestion collapse, I think this needs to be conservative enough to deal with paths with significant RTT variance, or where there is a change of base RTT. 1 sec, then 3 seconds, and then exponentially backed-off would be expected for TCP SYNs, and I suggest could be appropriate.
The suggestion is to move some of this text to recovery(ie: Handshakes and New Paths) and reference that text from transport.
As Jana pointed out, this specifies a separate timer, which fits fine with the model that there's one timer per path.
This can be simpler -- just add the new sentence. I've made suggestions.

A sender can make exceptions for probe packets so that their loss detection is independent and does not unduly cause the congestion controller to reduce its sending rate.  An endpoint might set a  <ins> separate timer when a PATH_CHALLENGE is sent, which is cancelled if </ins> the corresponding PATH_RESPONSE is received.  If the timer fires before the PATH_RESPONSE is received, the endpoint might send a new PATH_CHALLENGE, and restart the timer for a longer period of time. <ins> This timer SHOULD be set as described in Section 5.3 of QUIC-RECOVERY and MUST NOT be more aggressive. </ins> 9.5.


5.5. <del> When packet protection keys are discarded (see Section 4.9 of QUIC- </del>  TLS), all packets that were sent with those keys can no longer be acknowledged because their acknowledgements cannot be processed anymore.  The sender MUST discard all recovery state associated with It is expected that keys are discarded after packets encrypted with them would be acknowledged or declared lost.  Initial secrets however might be destroyed sooner, as soon as handshake keys are available <del> (see Section 4.9.1 of QUIC-TLS). </del>  6. packet.  Though packets might still be in flight or awaiting acknowledgment, no further Initial packets need to be exchanged beyond this point.  Initial packet protection keys are discarded (see <del> Section 4.9.1 of QUIC-TLS) along with any loss recovery and </del>  congestion control state (see Section 6.5 of QUIC-RECOVERY). Any data in CRYPTO frames is discarded - and no longer retransmitted

and 4.9.1 are now 4.10 and 4.10.1, I believe.

5.5.  <ins> When packet protection keys are discarded (see Section 4.10 of QUIC- </ins> TLS), all packets that were sent with those keys can no longer be acknowledged because their acknowledgements cannot be processed anymore.  The sender MUST discard all recovery state associated with It is expected that keys are discarded after packets encrypted with them would be acknowledged or declared lost.  Initial secrets however might be destroyed sooner, as soon as handshake keys are available  <ins> (see Section 4.10.1 of QUIC-TLS). </ins> 6. packet.  Though packets might still be in flight or awaiting acknowledgment, no further Initial packets need to be exchanged beyond this point.  Initial packet protection keys are discarded (see  <ins> Section 4.10.1 of QUIC-TLS) along with any loss recovery and </ins> congestion control state (see Section 6.5 of QUIC-RECOVERY). Any data in CRYPTO frames is discarded - and no longer retransmitted