A client need not store a transport parameter it cannot process. A client MUST NOT use remembered values for the following parameters: <del> original_connection_id, preferred_address, stateless_reset_token, and ack_delay_exponent.  The client MUST use the server's new values in the handshake instead, and absent new values from the server, the default value. </del>  A client that attempts to send 0-RTT data MUST remember all other transport parameters used by the server.  The server can remember

We've added new transport parameters since we wrote section 7.3.1. The following TPs are neither MUST NOT remember nor MUST remember: maxidletimeout maxpacketsize maxackdelay disableactivemigration
In my opinion, the only one that the client SHOULD remember is maxpacketsize. This is an interesting special case since the server doesn’t need to remember this one (nor store it in the session ticket). I don’t see any motivation for remembering the other three TPs.
Remember ing maxackdelay is useless I believe, because Initial and Handshake packets use an maxackdelay of kAlarmGranularity. I would not want to remember disableactivemigration or maxidletimeout if at all possible.
As maxidletimeout and disableactivemigration can only ever apply after the handshake is complete, remembrance is unnecessary. maxackdelay is annoying because it implies a particular acknowledgment schedule for 0-RTT that might not be compatible with the path, but setting it to the absolute minimum is problematic also.
See also .
I don't read the text this way. In the fourth paragraph it says that "A client that attempts to send 0-RTT data MUST remember all other transport parameters used by the server." [i.e. all but a short list of obviously insane TPs listed in the previous paragraph. I suggest we close with no action.
NAME that's a good point that the list doesn't have to be exhaustive. The text says clients MUST NOT remember certain parameters and MUST remember all others. So perhaps the more apt question is whether any of these unmentioned TPs belong in the MUST-NOT list.
So I'm reviewing this and the only case I'm concerned about is maxackdelay for 0-RTT. The server's values might be remembered by the client and applied to its logic. But I can't see any reason why this might cause actual problems as long as the client adjusts its value when it can. If the server increases this value, then any increase might accrue to the RTT estimates that are received before the client installs the new value. But the client won't be generating RTT estimates based on application data, so this is a wash. And the effect on the PTO duration has no effect because you don't arm PTO because the recovery draft . So, I see three valid choices: Say nothing. Let people work out that maxackdelay has no effect prior to the handshake completing on their own. Explain this. Probably just by saying "Note that maxackdelay has no effect during the handshake therefore cannot be violated by a client sending 0-RTT data." OR something along those lines. Explicitly forbid remembering this parameter, by adding it to the list but don't explain it. I'm inclined toward number 3 here, if only to save people the effort of working through this one.
I agree with your analysis and your recommendation(3) that clients MUST NOT remember it. We're pretty clear that the maxackdelay is 0 until you receive the value in TPs, and this fits with that model.
The proposed resolution in PR 3545, although short and sweet, categorises a TP as a MUST NOT. In the interest of process, I'm marking this issue as design. Since there is a proposal things should make the next round of consensus call, unless some objections are raised in the meantime.
Accidental close, sorry.

A client need not store a transport parameter it cannot process. A client MUST NOT use remembered values for the following parameters:  <ins> ack_delay_exponent, max_ack_delay, original_connection_id, preferred_address, and stateless_reset_token.  The client MUST use the server's new values in the handshake instead, and absent new values from the server, the default value. </ins> A client that attempts to send 0-RTT data MUST remember all other transport parameters used by the server.  The server can remember


The PTO value MUST be set to at least kGranularity, to avoid the timer expiring immediately. <del> A sender computes its PTO timer every time an ack-eliciting packet is sent.  When ack-eliciting packets are in-flight in multiple packet number spaces, the timer MUST be set for the packet number space with the earliest timeout, except for ApplicationData, which MUST be ignored until the handshake completes; see Section 4.1.1 of QUIC-TLS. Not arming the PTO for ApplicationData prevents a client from retransmitting a 0-RTT packet on a PTO expiration before confirming that the server is able to decrypt 0-RTT packets, and prevents a server from sending a 1-RTT packet on a PTO expiration before it has the keys to process an acknowledgement. When a PTO timer expires, the PTO period MUST be set to twice its current value.  This exponential reduction in the sender's rate is important because consecutive PTOs might be caused by loss of packets or acknowledgements due to severe congestion.  Even when there are ack-eliciting packets in-flight in multiple packet number spaces, the </del>  exponential increase in probe timeout occurs across all spaces to prevent excess load on the network.  For example, a timeout in the Initial packet number space doubles the length of the timeout in the pto-handshake) for a new path, but the delay SHOULD NOT be considered an RTT sample. <del> Until the server has validated the client's address on the path, the amount of data it can send is limited to three times the amount of data received, as specified in Section 8.1 of QUIC-TRANSPORT.  If no data can be sent, then the PTO alarm MUST NOT be armed until datagrams have been received from the client. Since the server could be blocked until more packets are received from the client, it is the client's responsibility to send packets to unblock the server until it is certain that the server has finished its address validation (see Section 8 of QUIC-TRANSPORT).  That is, the client MUST set the probe timer if the client has not received an acknowledgement for one of its Handshake or 1-RTT packets, and has not received a HANDSHAKE_DONE frame. </del> Prior to handshake completion, when few to none RTT samples have been generated, it is possible that the probe timer expiration is due to an incorrect RTT estimate at the client.  To allow the client to keys indicates forward progress and the loss detection timer might have been set for a now discarded packet number space.  5.2.3. When a server receives an Initial packet containing duplicate CRYPTO is no data available to send.  An endpoint MAY send up to two full- sized datagrams containing ack-eliciting packets, to avoid an expensive consecutive PTO expiration due to a single lost datagram or <del> transmit data from multiple packet number spaces. </del>  In addition to sending data in the packet number space for which the timer expired, the sender SHOULD send ack-eliciting packets from skip a packet number to eliminate the ack delay. When the PTO timer expires, and there is new or previously sent <del> unacknowledged data, it MUST be sent. </del>  It is possible the sender has no new or previously-sent data to send. As an example, consider the following sequence of events: new expiration increases resilience to packet drops, thus reducing the probability of consecutive PTO events. <del> Probe packets sent on a PTO MUST be ack-eliciting.  A probe packet SHOULD carry new data when possible.  A probe packet MAY carry retransmitted unacknowledged data when new data is unavailable, when flow control does not permit new data to be sent, or to opportunistically reduce loss recovery delay.  Implementations MAY use alternative strategies for determining the content of probe packets, including sending new or retransmitted data based on the application's priorities. </del> When the PTO timer expires multiple times and new data cannot be sent, implementations must choose between sending the same payload every time or sending different payloads.  Sending the same payload

In order to avoid sending anti-deadlock packets too quickly when the server doesn't yet have anything to send. Also sets ptocount to 0 in pseudocode when Initial or Handshake keys are dropped, lining it up with existing normative text.(this could be split into a separate non-design PR if people prefer) Adds the term 'backoff' which is usually reset upon receiving an acknowledgement. The lines up more closely with 'ptocount' in the pseudocode.
It looks like under the current PTO algorithm PTO probes are sent too frequently under certain condition. To create such a condition, suppose that client never send Handshake packets. According to the -27 spec, client sends ack-eliciting anti-deadlock packet on ptocount=1. Then it gets ACK from server for it, which resets ptocount to 0. After next pto timeout, it sends anti-deadlock packet again. This process repeats indefinitely until either endpoint gives up handshake. The problem is that depending RTT, PTO probes are sent too frequently. With RTT=6ms, it sends PTO probe once per ~33ms. And both sides do this. ACK+PADDING packets are also sent without delay in this period if remote endpoint does PTO. I'm not sure TCP does the similar things, but it sounds like PTO probe is too aggressive. If both endpoints are implemented properly, this probably does not happen, but in practice, we don't know the remote endpoint is broken in this case, and have no means to check that out.
If the server keeps quickly ACKing packets but not sending handshake data, then yes, the client will keep sending full-sized packets, believing the server is blocked by the anti-amplification limit. At first I thought this couldn't happen in a real-world scenario, but possibly if one had async TLS handshaking so the ServerHello could not be sent immediately upon receipt of the ClientHello and the server and client were very close, a packet could be exchanged as quickly as 1 packet each direction per kGranularity time until the async process returned? TCP doesn't have an anti-amplification limit and it doesn't require Handshake packets to provide address validation. I would characterize this as an issue with the client's anti-deadlock response, rather than a problem with PTO.
I think the most intellectually clean thing to do here would be to not reset PTO count when an ACK is received and the PTO is only armed for anti-amplification purposes. This is because even though at a transport level, everything is working(ACKs are quickly received), the client is arming the PTO for a very different reason(anti-deadlock). And the PTO packets aren't allowing the server to make any more progress, because it's not blocked. But I don't think that suggestion is very easy to implement, so I'm not sure it's what we want. I'd suggest restricting the client to one anti-deadlock packet per second or some arbitrary period, which would be easier, but means someone has to pick a magic number. I mention 1 second because it's a magic number already in QUIC recovery(default handshake timeout) and in TCP. This basically amounts to anti-anti-amplification attack prevention I think :(
Does it make simpler if we reset ptocount only when ack-eliciting not pure probe packet (which includes lost data or new data, not just PING) is received without special casing for anti-deadlock packet? This ensures that ptocount is reset only when endpoint has made a progress and delaying consecutive PTO if it only receives ACK for pure probe packet. Server might be in the same situation if client somehow only sends ACK frame in Initial and server keeps sending Initial packet in PTO
This is an interesting issue and I think NAME is correct in pointing out that this could be a real problem for slow devices doing public key operations in an asynchronous manner. I like NAME suggestion to not reset the PTO timer. Maybe we can always increment it, regardless of the packet received from peer being ACK-eliciting or not.
There are other cases when we send ack-eliciting packets including only PINGs or other non-application data and we do want to set ptocount back to 0 when those are acknowledged. We could specifically recommend sending non-ack-eliciting packets for the anti-deadlock packet, but then you lose an opportunity for an RTT measurement early in the connection, which is unfortunate, though not a dealbreaker. For full-sized Initial packets, it's likely they'd never be acknowledged if they weren't ack-eliciting, because no ack-eliciting Initial packets would be sent after them. NAME I believe always incrementing ptocount would be detrimental to performance, unless it's under some very narrow circumstances.
Sorry if I was unclear, I meant not resetting PTO timer when the client receives an Initial packet. Assuming that the server sends only a few (most likely one), I think not resetting is fine. Note that in case the server is going to send a fair number of Initial packets (e.g., post quantum) it should be sending a retry regardless of what we decide here, as relying on CC wold be better than relying on 3x limit.
We only reset pto_count upon receiving an Initial ACK, not any Initial packet, but I think your suggestion could work well. It amounts to continuing with exponential backoff until the client has Handshake keys. It's fairly simple to implement, and I wouldn't expect it to cause a noticeable regression, so unless someone comes up with something better, let's do it.
So the operating principle here is that we are expecting progress to Handshake to be made, which means that we need to align with our existing logic that enables PTO in that case. For reference, the check we use to enable PTO during the anti-amplification phase is the inverse of the following: Which follows the "server at anti-amplification limit" check. So maybe is the check you want rather than .
I updated the PR to only do this for the client, since I think that's the right thing to do. I did the Initial check because I thought it was slightly simpler than the check, which I believe is equivalent, since receiving an Initial ACK means you still have Initial keys, so the server is awaiting address validation. As soon as a Handshake ACK is received by the client, it knows the server has completed address validation. But I could also use the PeerNotAwaitingAddressValidation() method that's already defined(and mistakenly not used, see ) Then I think the check would be:
Looks good to me.
Thanks, Ian, a couple of comments.Looks great!

The PTO value MUST be set to at least kGranularity, to avoid the timer expiring immediately.  <ins> A sender recomputes and may need to reset its PTO timer every time an ack-eliciting packet is sent.  When ack-eliciting packets are in- flight in multiple packet number spaces, the timer MUST be set for the packet number space with the earliest timeout, except for ApplicationData, which MUST be ignored until the handshake completes; see Section 4.1.1 of QUIC-TLS.  Not arming the PTO for ApplicationData prevents a client from retransmitting a 0-RTT packet on a PTO expiration before confirming that the server is able to decrypt 0-RTT packets, and prevents a server from sending a 1-RTT packet on a PTO expiration before it has the keys to process an acknowledgement. When a PTO timer expires, the PTO backoff MUST be increased, resulting in the PTO period being set to twice its current value. The PTO period is set based on the latest RTT information after receiving an acknowledgement.  The PTO backoff is reset upon receiving an acknowledgement unless it's a client unsure if the the server has validated the client's address.  Not resetting the backoff during peer address validation ensures the client's anti-deadlock timer is not set too aggressively when the server is slow in responding with handshake data. This exponential reduction in the sender's rate is important because consecutive PTOs might be caused by loss of packets or acknowledgements due to severe congestion.  Even when there are ack- eliciting packets in-flight in multiple packet number spaces, the </ins> exponential increase in probe timeout occurs across all spaces to prevent excess load on the network.  For example, a timeout in the Initial packet number space doubles the length of the timeout in the pto-handshake) for a new path, but the delay SHOULD NOT be considered an RTT sample.  Prior to handshake completion, when few to none RTT samples have been generated, it is possible that the probe timer expiration is due to an incorrect RTT estimate at the client.  To allow the client to keys indicates forward progress and the loss detection timer might have been set for a now discarded packet number space. <ins> 5.2.2.1. Until the server has validated the client's address on the path, the amount of data it can send is limited to three times the amount of data received, as specified in Section 8.1 of QUIC-TRANSPORT.  If no additional data can be sent, the server's PTO alarm MUST NOT be armed until datagrams have been received from the client, because packets sent on PTO count against the anti-amplification limit.  Note that the server could fail to validate the client's address even if 0-RTT is accepted. Since the server could be blocked until more packets are received from the client, it is the client's responsibility to send packets to unblock the server until it is certain that the server has finished its address validation (see Section 8 of QUIC-TRANSPORT).  That is, the client MUST set the probe timer if the client has not received an acknowledgement for one of its Handshake or 1-RTT packets, and has not received a HANDSHAKE_DONE frame. </ins> 5.2.3. When a server receives an Initial packet containing duplicate CRYPTO is no data available to send.  An endpoint MAY send up to two full- sized datagrams containing ack-eliciting packets, to avoid an expensive consecutive PTO expiration due to a single lost datagram or  <ins> transmit data from multiple packet number spaces.  All probe packets sent on a PTO MUST be ack-eliciting. </ins> In addition to sending data in the packet number space for which the timer expired, the sender SHOULD send ack-eliciting packets from skip a packet number to eliminate the ack delay. When the PTO timer expires, and there is new or previously sent  <ins> unacknowledged data, it MUST be sent.  A probe packet SHOULD carry new data when possible.  A probe packet MAY carry retransmitted unacknowledged data when new data is unavailable, when flow control does not permit new data to be sent, or to opportunistically reduce loss recovery delay.  Implementations MAY use alternative strategies for determining the content of probe packets, including sending new or retransmitted data based on the application's priorities. </ins> It is possible the sender has no new or previously-sent data to send. As an example, consider the following sequence of events: new expiration increases resilience to packet drops, thus reducing the probability of consecutive PTO events.  When the PTO timer expires multiple times and new data cannot be sent, implementations must choose between sending the same payload every time or sending different payloads.  Sending the same payload


The PTO value MUST be set to at least kGranularity, to avoid the timer expiring immediately. <del> A sender computes its PTO timer every time an ack-eliciting packet is sent.  When ack-eliciting packets are in-flight in multiple packet number spaces, the timer MUST be set for the packet number space with the earliest timeout, except for ApplicationData, which MUST be ignored until the handshake completes; see Section 4.1.1 of QUIC-TLS. Not arming the PTO for ApplicationData prevents a client from retransmitting a 0-RTT packet on a PTO expiration before confirming that the server is able to decrypt 0-RTT packets, and prevents a server from sending a 1-RTT packet on a PTO expiration before it has the keys to process an acknowledgement. </del>  When a PTO timer expires, the PTO period MUST be set to twice its current value.  This exponential reduction in the sender's rate is is no data available to send.  An endpoint MAY send up to two full- sized datagrams containing ack-eliciting packets, to avoid an expensive consecutive PTO expiration due to a single lost datagram or <del> transmit data from multiple packet number spaces. </del>  In addition to sending data in the packet number space for which the timer expired, the sender SHOULD send ack-eliciting packets from skip a packet number to eliminate the ack delay. When the PTO timer expires, and there is new or previously sent <del> unacknowledged data, it MUST be sent. </del>  It is possible the sender has no new or previously-sent data to send. As an example, consider the following sequence of events: new expiration increases resilience to packet drops, thus reducing the probability of consecutive PTO events. <del> Probe packets sent on a PTO MUST be ack-eliciting.  A probe packet SHOULD carry new data when possible.  A probe packet MAY carry retransmitted unacknowledged data when new data is unavailable, when flow control does not permit new data to be sent, or to opportunistically reduce loss recovery delay.  Implementations MAY use alternative strategies for determining the content of probe packets, including sending new or retransmitted data based on the application's priorities. </del> When the PTO timer expires multiple times and new data cannot be sent, implementations must choose between sending the same payload every time or sending different payloads.  Sending the same payload

When writing , I found some of the text organization to be odd, so this moves it some.

The PTO value MUST be set to at least kGranularity, to avoid the timer expiring immediately.  <ins> A sender recomputes and may need to reset its PTO timer every time an ack-eliciting packet is sent.  When ack-eliciting packets are in- flight in multiple packet number spaces, the timer MUST be set for the packet number space with the earliest timeout, except for ApplicationData, which MUST be ignored until the handshake completes; see Section 4.1.1 of QUIC-TLS.  Not arming the PTO for ApplicationData prevents a client from retransmitting a 0-RTT packet on a PTO expiration before confirming that the server is able to decrypt 0-RTT packets, and prevents a server from sending a 1-RTT packet on a PTO expiration before it has the keys to process an acknowledgement. </ins> When a PTO timer expires, the PTO period MUST be set to twice its current value.  This exponential reduction in the sender's rate is is no data available to send.  An endpoint MAY send up to two full- sized datagrams containing ack-eliciting packets, to avoid an expensive consecutive PTO expiration due to a single lost datagram or  <ins> transmit data from multiple packet number spaces.  All probe packets sent on a PTO MUST be ack-eliciting. </ins> In addition to sending data in the packet number space for which the timer expired, the sender SHOULD send ack-eliciting packets from skip a packet number to eliminate the ack delay. When the PTO timer expires, and there is new or previously sent  <ins> unacknowledged data, it MUST be sent.  A probe packet SHOULD carry new data when possible.  A probe packet MAY carry retransmitted unacknowledged data when new data is unavailable, when flow control does not permit new data to be sent, or to opportunistically reduce loss recovery delay.  Implementations MAY use alternative strategies for determining the content of probe packets, including sending new or retransmitted data based on the application's priorities. </ins> It is possible the sender has no new or previously-sent data to send. As an example, consider the following sequence of events: new expiration increases resilience to packet drops, thus reducing the probability of consecutive PTO events.  When the PTO timer expires multiple times and new data cannot be sent, implementations must choose between sending the same payload every time or sending different payloads.  Sending the same payload


when it receives a packet with a previously unused connection ID.  An endpoint MAY limit the frequency or the total number of connection IDs issued for each connection to avoid the risk of running out of <del> connection IDs; see reset-token. </del>  An endpoint that initiates migration and requires non-zero-length connection IDs SHOULD ensure that the pool of connection IDs

Add another note when cautioning endpoints about how many CIDs to issue: in this case to note that if you issue a CID you need to be willing for the other endpoint to use 100% of the issued CIDs, which comes with an implicit willingness to maintain state for each path on which that CID could be used. (That's one per path, but # paths == # CIDs in the worst case.)
This adds a MAY and the issue is already marked design, so I'm inclined to mark this design and do a consensus call on the issue with the next batch.
We could also leave this as choose, which seems completely reasonable as well. The only reason I went for was because the previous sentence is the same level of advisory and we also used there.
NAME I'd prefer you keep it as is. The process isn't going to cause an appreciable delay in this case.
Update: I thought simultaneous path validation wasn't necessary, but it looks like I misunderstood. Changing this to an editorial issue so I can re-read this text and hopefully make it clearer. I believe this is already implied, but I couldn't find it stated anywhere. I think it'd be ideal to set expectations that servers are not expected to validate multiple paths at the same time. Validating multiple paths seems like a plausible feature for multipath, but adds undue complexity for connection migration.
Does that imply that a client can only probe a single path at a time? What's the time it has to wait after apparent unsuccessful probing of one path until it can probe the next?
NAME I'm not sure if I agree. It has been my assumption that a server is expected to respond to clients probing multiple paths concurrently. Note that in , we added the following text:
Hmm, I didn't think this was already implied. You're not expected (or allowed) to send non-probing packets on multiple paths, but you can certainly get probes in from multiple different remote endpoints. This is somewhat necessary if we're to be resilient to the various packet racing attacks -- if an attacker can cause you to be upset about a packet coming in from a different destination then they can really mess you up.
I'm a bit confused about why a client needs to probe multiple paths simultaneously, given we have connection migration, not multipath. To me, it just adds complexity. I guess I can just give out a single CID to mitigate the issue, but do we really expect servers to correctly implement probing on multiple paths? What if they all succeed? A client can still only use one. NAME can you clarify the packet racing issue a bit more or point me to text that already does? I'd like to draw a state transition diagram for path validation, but if we can have multiple paths being validated at once, I'm not sure that's practical.
NAME So the scenario is that a client is using one path and has 2 or more new ones available? I've heard future hardware could make that possible, though I've never seen that case myself. The short of this is now that we're trying to move to IETF QUIC style connection migration and path validation, and I'm finding it fairly confusing.
I think there's a pretty big distinction between probing a path to establish the potential for connectivity and migrating the connection to that path. For example, I would not be at all surprised if a client were to probe all available paths and choose the "best" one to migrate to when the need arises (often, although not always, there is some warning that conditions are worsening on the current path which provides the opportunity to evaluate alternatives). That said, I think a large part of the reason to allow probing on multiple paths is to avoid requiring synchronization of a shared understanding as to the state of current probes. We don't want to add a bunch of machinery to establish failure of a probe on a different path. So even if we don't think that a client would ever intentionally probe multiple paths, if a client probes one path and hears no response, it seems completely reasonable to think that it would switch and probe a different path if it had one available. If it turns out that there was just some delay for that initial path, say it had a 1.5s RTT, now the server is going to think the client probed two paths at the same time, when in reality the client thinks it's just talking to one. It also doesn't seem particularly harmful to allow this? As NAME noted, there's some good text in there about limiting state required on the server, and of course path validation can happen at any time, so it seems completely reasonable to "forget" that any given path was validated if a client probes and then doesn't end up using the path. (Note that you'd likely want to retire the CID used on that path, so as to replace it with a new one when you did that forgetting, which also serves as a nice heuristic for the other endpoint to say "oh, hmm, they don't think they're talking to me over there", but we can dig into that later.) The packet racing thing is a reference to and the previous section. This led us to require endpoints to validate both the current and the new path when it changes, which conveniently provokes some traffic to reconfirm the active path and makes it more difficult to disrupt a connection that's gone idle. While technically this is only discussing one "new" path and then the old path being validated at the same time, it seems as though it would be trivial for such an attacker to convince an endpoint that it was seeing "probes" (since we've discussed that it's not that difficult to identify a probing packet as an observer) from multiple paths and therefore be quite upset if such behavior were restricted. I'm not saying this happens all the time, but it seems pretty reasonable to have a Wi-Fi interface and then a dual SIM device with multiple cellular interfaces. Overall, it seems as though (even if this isn't crazy common) it's not unreasonable to expect that this behavior might happen, and it doesn't seem like there's a big benefit to restricting it, nor harm in allowing it, beyond the existing text reminding endpoints to be careful with the amount of state they're willing to allocate to any individual client (which is a thing outside of any probing/migration concept anyways).
+1 to what NAME says. The resolution of (which ended up as PR ) was that a server has to remember state of N paths where N is the number of CIDs active at one time. As NAME says, a client cannot tell when a server thinks that it received two probes concurrently. Therefore, it is very tricky to have this kind of restriction within a specification that allows clients to retain multiple backup paths.
I agree with NAME and NAME here. If a client sends a probe on interface A, and then interface A disconnects and then interface B appears immediately, the client should be able to probe on interface B even though to the server will receive two independent probes at roughly the same time. Servers need to be able to handle multiple concurrent probes in order for migration to be reliable.
Note that while the server needs to respond to the probes (so the client can validate the paths), I don't think there's any requirement that the server continue to regard all those paths as validated. When the client actually migrates, if the server wants to treat the migration as an unvalidated path and initiate validation again, that seems fine.
Well, the paths aren't validated until the server explicitly validates them, so it seems like the cost is pretty minimal until the server decides to engage resources for that path. The server can also respond on the old path if the client is just probing.
NAME do you want to think about how you reached your conclusion here and see if there is something we can do to avoid that? It seems to me that the cost to a server is limited by how many paths it wants to probe, not how many the client probes, so the state is well managed. It's a little less neat if the client actively migrates multiple times. But I would say that the server can abandon probes to keep the number of active probes down to at most 2 (one for the most recent validated path, one for the most recent active path). Is that something we need to explain more thoroughly?
I have read this text quite a bit, but not end-to-end in quite a long time, so I believe that's the cause of my confusion. I'm planning to re-read all the path validation and connection migration text and I'll send out a PR if there are points that I think I are unclear.
NAME have you had a chance to do this?
Unfortunately, not yet.
Note that man-on-the-side attacker can take a valid packet and resend it from a different address. The crypto does not protect against that. If the attacker somehow isolates probe packets, the server will end up receiving probes from a variety of endpoints. Better not do something silly in that condition.
Would people please take a final look at the PR?
At the moment, it is my understanding that we do not give any advice in particular. But to me it seems that there would be issues if the number of probed paths that the server remembers is less than the number of spare connection IDs it issues. Consider the case of a server that advertises three spare connection IDs, but retains information of one probing path (i.e. remember one set of 5-tuple that has been probed, and the connection ID being associated to that tuple). If a client connecting to this server probes three paths concurrently, and if it keeps on sending PATHCHALLENGE packets periodically on each of those paths (in order to keep the holes in NAT open), two issues arise: The connection ID pool of the server would run out quickly, as the server would use a new connection ID for each probing packet it sends in response (because each probing packet sent by the client would have a different source address than the previous packet). This is an issue, as we state that an endpoint MAY limit the frequency or the total number of connection IDs issued for each connection to avoid the risk of running out of connection IDs_ (). The purpose of path probing is to prepare an alternate path beforehand. That benefit diminishes if the server remembers only one (or few) of the paths being probed. Instead of recommending servers to remember paths as much as the number of spare connection IDs that they issue, we can introduce a transport parameter that caps the number paths that a client may probe. But IMO having a new TP seems unnecessarily complex, because I think that servers can simply associate path information to each active connection ID it maintains.
Given QUIC v1 is chartered to support connection migration and not multipath, we could also say clients shouldn't probe more than one extra path at once. I do not want to add another TP for this use case, because I don't think it's sufficiently valuable.
I'm fine with having a hard-coded limitation, though if we are to set the maximum to one, then we need to decide whether or not migration due to SPA is counted. It would be simpler to include it, but the potential downside would be that probing using a different network interface would be forbidden until migration to the preferred address concludes.
So a server doesn't need to probe in response to a client that probes a path, so it doesn't really have any state to attribute to probed paths. A server can just respond to probes. The cost to it, if it doesn't, is that it has to probe after a migration and that could have performance implications. I would be happy saying that an endpoint ideally both probes and remembers the status of those probes for as many paths as it has issued, active connection IDs. Probing paths and remembering the status of probes partly mitigates the reduction in CWND that follows migration, so there is a potential performance gain from keeping this state. But the window is still likely reset, so the performance hit is essentially unavoidable either way. The total number of paths a server might have to remember is limited by the activeconnectionid_limit of the client, but it can be less (as you already noted), so the server has the ability to limit its exposure. Thus, a server can probe and remember the status of as many paths as it likes. I'd say that this is a fine piece of advice, but that's all it needs to be. I agree that yet another transport parameter is not a great reaction to this relatively minor issue.
NAME I think I disagree with the observation. The problem of responding to a probe without retaining state for that path (in addition to this performance implication) is that the server burns a CID for each probing packet. As stated in the , the worst case scenario is that CIDs will be consumed at a high rate when a client probes multiple paths simultaneously.
Personally, I don't think this worst case scenario is very bad at all. But I'd support adding some cautionary/explanatory text.
I don't think migration vs. multipath is currently the distinction for how many paths a client probes, it's the restriction on how many paths the client uses at any one time, and I think that's appropriate. There are many cases where I'd want to be able to probe more than one path, and assuming that a device has a single network attachment (where we need to include IPv4 and IPv6 as more than one given that you need a new CID for each local address) doesn't make sense either. The number of paths that a client can probe simultaneously is limited by how many CIDs the server is willing to issue to that client. To the point of the original issue: If adding that TP is unnecessarily complex (and I agree that it probably is), what's wrong with recommending servers to remember paths up to as many as the number of spare CIDs that they issue? The whole point of issuing more than one spare CID is that the other endpoint may need to probe additional paths or those probes may be lost, but if you aren't willing to deal with things coming in from 100 places, don't issue 100 CIDs. Obviously that's an exaggeration, but 2/4/6/8 for each client puts a nice bound and is roughly what I'd expect to be sending in a TP anyways. There's already a mechanism where the person doing the probing can "retire" that CID and signal to you that you no longer need to have that state around, and that's tied directly into their ability to generate new state that you'd need to keep around. In essence, I think I agree with NAME statement above, if a server needs to reduce its exposure further it can do so, at the potential cost of performance or burning CIDs faster (as NAME points out). Enhancing the text around the existing sentence in Section 5.1.1 (that cautions against issuing too many CIDs) to also say that a server needs to maintain this state and should consider that when issuing new CIDs could be reasonable, but I wouldn't consider that time for a new TP?
I tend to agree with Eric, but I think there are some aspects to consider: You don't want to limit yourself to one extra path as Ian suggests because you might have 3 or 4 paths forward, e.g. sitting on WLAN 2.5GHz with optional 5GHz if the first becomes congested, along with a 4G and 5G modem path. You need to monitor these and choose the best path forward once the preferred path fails. Maybe this is a bit extreme, but just to illustrate path migration isn't linear. At the same time, the peer needs to issue enough IDs to not stall progress in addition to accommodating the number of paths forward a typical client might need. I'd say having double the number of CIDs in spare is probably OK, but maybe there isn't a need for a spare CID for every path. So it could be number of paths + 2 or something. Thus, a recommendation could be that a client SHOULD not track paths beying having 2 spare CIDs and a server SHOULD be prepared to track the number of issued CIDs minus two, as a general rule. Exceeding that may require delayed probing which could lead to temporary delays.
BTW: a related migration issue: a client might have to track separate root certificates for separate paths because one path could be on a corporate WLAN with substituted root certificates for traffic monitoring while the other path happens over 4G or 5G. This is probably not something that can be supported on a single connection neither in QUIC v1 or v2, but it is actually a concern in a real use case I'm aware of.
I tend to agree with what NAME says. If I haven't been clear, I share the view that probably the best solution to this issue is to recommend servers to remember at most N paths, where N is the maximum number of active connection IDs it provides to a peer at any given moment. I think NAME is correct in pointing out that the client might not use all the spare CIDs for probing, but I would argue that that can be a local decision of each client implementation. In terms of operational complexity on the server-side, the recommendation being N or N - 1 or N / 2 does not matter at all. Let's simply recommend N for the servers, and be silent about what clients should do.
I think it makes sense to simply advise that servers should remember as many paths as the additional CIDs they hand out, as NAME suggests. NAME can you write up a PR for this?
Sure, will do! I'm thinking this will live right next to the issuing CIDs bit: as we note that you may want to rate limit, we can say it would be a good idea to recognize the state that may be associated with each CID, in this case, the path validation status (which is needed for the validating paths section anyways, since we say you might have remembered it from previously) and any other per-path state that you might keep.

when it receives a packet with a previously unused connection ID.  An endpoint MAY limit the frequency or the total number of connection IDs issued for each connection to avoid the risk of running out of  <ins> connection IDs; see reset-token.  An endpoint MAY also limit the issuance of connection IDs to reduce the amount of per-path state it maintains, such as path validation status, as its peer might interact with it over as many paths as there are issued connection IDs. </ins> An endpoint that initiates migration and requires non-zero-length connection IDs SHOULD ensure that the pool of connection IDs


Once a client knows that an HTTP/3 server exists at a certain endpoint, it opens a QUIC connection.  QUIC provides protocol <del> negotiation, stream-based multiplexing, and flow control.  An HTTP/3 endpoint can be discovered using HTTP Alternative Services; this process is described in greater detail in discovery. </del>  Within each stream, the basic unit of HTTP/3 communication is a frame (frames).  Each frame type serves a different purpose.  For example, 3.2.  An HTTP origin advertises the availability of an equivalent HTTP/3 endpoint via the Alt-Svc HTTP response header field or the HTTP/2 ALTSVC frame (ALTSVC), using the ALPN token defined in connection- port and, if successful, send HTTP requests using the mapping described in this document. <del> Connectivity problems (e.g. firewall blocking UDP) can result in QUIC connection establishment failure, in which case the client SHOULD continue using the existing connection or try another alternative endpoint offered by the origin. </del>  <del> Servers MAY serve HTTP/3 on any UDP port, since an alternative always includes an explicit port. </del>  3.3. Once a connection exists to a server endpoint, this connection MAY be reused for requests with multiple different URI authority components. <del> The client MAY send any requests for which the client considers the server authoritative. An authoritative HTTP/3 endpoint is typically discovered because the client has received an Alt-Svc record from the request's origin which nominates the endpoint as a valid HTTP Alternative Service for that origin.  As required by RFC7838, clients MUST check that the nominated server can present a valid certificate for the origin before considering it authoritative.  Clients MUST NOT assume that an HTTP/3 endpoint is authoritative for other origins without an explicit signal. </del>  Clients SHOULD NOT open more than one HTTP/3 connection to a given host and port pair, where the host is derived from a URI, a selected different transport or TLS configurations but SHOULD avoid creating multiple connections with the same configuration. <del> Prior to making requests for an origin whose scheme is not "https," the client MUST ensure the server is willing to serve that scheme. If the client intends to make requests for an origin whose scheme is "http", this means that it MUST obtain a valid "http-opportunistic" response for the origin as described in RFC8164 prior to making any such requests.  Other schemes might define other mechanisms. </del> Servers are encouraged to maintain open connections for as long as possible but are permitted to terminate idle connections if necessary.  When either endpoint chooses to close the HTTP/3 session,

Clients MAY establish QUIC connections using the host and port indicated in the URL for "https" URLs only Non-"https" origins are able to be delegated using Alt-Svc Cribs a lot of text from .
Are we waiting on a consensus call for this?
The call has been issued and is running until Saturday UTC. If you have comments, please make them now!
Excellent! Can't wait for this to land. (Looks like I need to finish reading my WG email :>)
NAME I'll get your comments as a separate PR.
S 2.4. This text seems quite a bit more restrictive than S 9.1.1. of RFC 7540, which just allows reuse as long as the SAN is present in the cert. Is that intentional?
I think that the intent was to present equivalent text. Though the thinking about coalescing has evolved some in the meantime. We should discuss where we think the state of this discussion is at and see if we can capture something sensible.
Yes, it'd be good to capture something more up-to-date if we can come to consensus on that. Is explicit signal meant to indicate DNS and/or ORIGIN frame? If so, maybe call those two out as examples?
NAME I think the dilemma is that some of those things don't exist yet for H3. For example, Mike did some earlier work to redefine ALTSVC frame for H3 but there is yet to be equivalent for ORIGIN or secondary certs.
Yes, it's intentional. RFC7540 allows coalescing based on the cert and DNS, because you're connected to the authoritative origin. However, HTTP/3 is never connected on the authoritative endpoint for an http(s?):// URL, so you need something else to tell you that's okay. This could be an Alt-Svc record for that origin delegating to the same UDP port. This could be an ORIGIN-for-H3 frame telling you to throw caution to the wind and trust the cert, if the client thinks that's sensible.
Discussed in London; also blocked on URL defining how a client decides which connection / server to ask for a given resource.
Discussed in ZRH. Waiting for HTTP changes to materialize.

Once a client knows that an HTTP/3 server exists at a certain endpoint, it opens a QUIC connection.  QUIC provides protocol  <ins> negotiation, stream-based multiplexing, and flow control.  Discovery of an HTTP/3 endpoint is described in discovery. </ins> Within each stream, the basic unit of HTTP/3 communication is a frame (frames).  Each frame type serves a different purpose.  For example, 3.2. <ins> HTTP relies on the notion of an authoritative response: a response that has been determined to be the most appropriate response for that request given the state of the target resource at the time of response message origination by (or at the direction of) the origin server identified within the target URI.  Locating an authoritative server for an HTTP URL is discussed in Section 5.4 of SEMANTICS. The "https" scheme associates authority with possession of a certificate that the client considers to be trustworthy for the host identified by the authority component of the URL.  If a server presents a certificate and proof that it controls the corresponding private key, then a client will accept a secured connection to that server as being authoritative for all origins with the "https" scheme and a host identified in the certificate. A client MAY attempt access to a resource with an "https" URI by resolving the host identifier to an IP address, establishing a QUIC connection to that address on the indicated port, and sending an HTTP/3 request message targeting the URI to the server over that secured connection. Connectivity problems (e.g., blocking UDP) can result in QUIC connection establishment failure; clients SHOULD attempt to use TCP- based versions of HTTP in this case. Servers MAY serve HTTP/3 on any UDP port; an alternative service advertisement always includes an explicit port, and URLs contain either an explicit port or a default port associated with the scheme. 3.2.1. </ins> An HTTP origin advertises the availability of an equivalent HTTP/3 endpoint via the Alt-Svc HTTP response header field or the HTTP/2 ALTSVC frame (ALTSVC), using the ALPN token defined in connection- port and, if successful, send HTTP requests using the mapping described in this document.  <ins> 3.2.2. Although HTTP is independent of the transport protocol, the "http" scheme associates authority with the ability to receive TCP connections on the indicated port of whatever host is identified within the authority component.  Because HTTP/3 does not use TCP, HTTP/3 cannot be used for direct access to the authoritative server for a resource identified by an "http" URI.  However, protocol extensions such as ALTSVC permit the authoritative server to identify other services which are also authoritative and which might be reachable over HTTP/3. </ins>  <ins> Prior to making requests for an origin whose scheme is not "https", the client MUST ensure the server is willing to serve that scheme. If the client intends to make requests for an origin whose scheme is "http", this means that it MUST obtain a valid "http-opportunistic" response for the origin as described in RFC8164 prior to making any such requests.  Other schemes might define other mechanisms. </ins> 3.3. Once a connection exists to a server endpoint, this connection MAY be reused for requests with multiple different URI authority components.  <ins> In general, a server is considered authoritative for all URIs with the "https" scheme for which the hostname in the URI is present in the authenticated certificate provided by the server, either as the CN field of the certificate subject or as a dNSName in the subjectAltName field of the certificate (see RFC6125).  For a host that is an IP address, the client MUST verify that the address appears as an iPAddress in the subjectAltName field of the certificate. </ins> Clients SHOULD NOT open more than one HTTP/3 connection to a given host and port pair, where the host is derived from a URI, a selected different transport or TLS configurations but SHOULD avoid creating multiple connections with the same configuration.  Servers are encouraged to maintain open connections for as long as possible but are permitted to terminate idle connections if necessary.  When either endpoint chooses to close the HTTP/3 session,


packets, such as timing and size, from being used to correlate activity.  Unintentional changes in path without a change in connection ID are possible.  For example, after a period of network inactivity, NAT rebinding might cause packets to be sent on a new path when the

Not making a design/editorial call here. There is new normative language, so maybe it is design. But this is really just documenting an emergent property of the existing system. There are no concrete changes other than a stronger recommendation against migration in cases where it either doesn't work or would not provide the privacy properties we have attempted to guarantee.
I think we already knew this, but it's also not that relevant to most migration scenarios when the client is migrating and has a 0-len CID, so I think it's fine.
NAME the issue is that, while the server might be confident it will work (and therefore not set the TP), the only way it works is by being highly linkable. If we think linkability is a bad thing, advising clients not to do it isn't unreasonable. Of course, servers have other ways of making a connection linkable by observers if they choose to do so, and clients can't detect all of them. I'm not sure this warrants normative force; I'd simply mention that clients who intend to minimize linkability might want to consider that they're definitely linkable in this situation and could choose to avoid it.
We don't prohibit the use of a zero-length connection ID in the preferredaddress transport parameter. So a server can use a connection ID during connection establishment, then remove privacy protections by moving to a unique address (see ). Of course, this is largely moot, as the privacy protection offered through the use of connection IDs toward a unique address is nil. In either case, we should clarify whether preferredaddress is exactly like NEWCONNECTIONID, or whether it has its own rules. Right now, it looks like it has its own rules and that is likely bad.
Zero-length connection IDs don't have a stateless reset token associated with them, but the preferred_address TP requires you to set a token. That's an awkward corner case.
That is just one of the strange things, yeah. We should clarify that.
So I see a number of ways forward here: Do nothing. Leave this ambiguous and potentially special. Make the fields in preferredaddress exactly equivalent to a NEWCONNECTIONID with a sequence number of 1. But allow the connection ID to be zero-length. A zero-length connection ID provides a new stateless reset token that is bound to the new address. (This is a weird option that would go against previous decisions to eliminate special handling for this connection ID, but I include it for the sake of completeness.) As 2, except that the new stateless reset token has connection-wide scope such that either of the provided tokens can be used. This is more consistent with established principles. Prohibit the use of zero-length connection IDs in preferredaddress. That is, you can't use preferredaddress without a connection ID. That is, preferredaddress effectively is a new server address plus a NEWCONNECTIONID frame. Something else. From my perspective, I rank these as 4 > 3 >> 1 > 2. But that is largely based on my bias against zero-length connection IDs (as much as it seems like that might be good for privacy, I think that privacy is better served by the use of connection IDs). If people intend to use preferred_address to migrate to a connection-specific address, then I'm OK with 3; we already have to cover the privacy hole that arises from that.
My preference goes to option 4 too. I do not think servers would use both 0-byte CID and preferredaddress TP at the same time. The only case a server using 0-byte CID can also use preferredaddress is: when the server is never accepting more than one connection at a time, or when the server knows that the client's address tuple does not change when the client migrates to a new address IMO that's a niche. And when these cases can be met, I do not see why the client cannot know the preferred address of the server beforehand.
I forgot to add: There is (maybe) a case for using a connection ID during setup and a zero-length connection ID after migrating to a preferred address. That isn't entirely crazy, but I personally would be comfortable losing that option. But I don't operate a server, so I want to give those who do a chance to let me know that this is preemptive.
NAME My personal take would be that people would not do that. When a client migrates to the preferred address, there's fair chance that the client's address tuple will change. That means that the server should be capable of receiving short header packets with 0-byte CID from any address, and determine that the packet belongs to the connection that is migrating to the preferred address. That's possible theoretically, but in practice ... I agree that a server might want to switch to zero-byte CID after migration to SPA completes. But to do that, we need a NEWCONNECTIONID that can carry a zero-byte CID, which is forbidden at the moment. I do not think we'd want to change the spec now to allow that.
That's an excellent point, and a good reason not to allow changing to zero-length connection IDs in combination with preferred_address. The nasty part of that is that in most cases that design would work because clients won't use the "new" connection ID until after the handshake is complete. Of course, it would fail spectacularly if a client did decide to switch earlier - and the client might attract the blame, where it is clearly the fault of the server.
I'll note that prohibiting a 0 length CID doesn't actually ensure the server's IP isn't unique, though it does remove an incentive to make it unique. I'd like to leave the option to switch to a zero-length CID later in the connection, and I think I'd be fine with 4 if we allowed that. An easy example is a large upload. I mostly don't care about the CID, but if I'm uploading a 10GB file, maybe saving some bytes matters. Also, if you're constantly filling the pipe, NAT rebinds are extremely unlikely. Another example is an environment like a datacenter where there are no NATs and routes are very stable, unless a peer intentionally changes it, so the 5-tuple is sufficient. In that case, the ideal operating condition is to typically use a 0 byte CID and switch to a non-0 length anytime a peer changes IP or port. Changing port has some clear benefits in some cases, so it's in no way a threoretical use case. NAME may have thoughts on this range of use case as well.
In our implementation, we plan to do all routing based purely on the CID. IMO, the couple of byte overhead in a 1500 MTU (at least; in datacenter) packet isn't going to make much difference, so it's just simpler to have a common CID routing scheme. That being said, I personally am not against allowing for others to do something different. I've never understood why we make any restrictions on changing between 0-length and non-0-length CIDs at all. IMO, the complexity is that it can change length at all, not that it's zero or not.
NAME we already prohibit changing from non-zero-length to zero-length using NEWCONNECTIONID. I don't want to relitigate that. I want to concentrate on this specific issue.
If we're not going to re-litigate that, then I'd suggest we continue allowing a 0-length CID for the preferred address case.
NAME While I would not argue strongly against continue allowing that, I would appreciate it if you could clarify why allowing a zero-byte CID would be useful. As pointed out in URL, when you provide a zero-byte CID in SPA, you need to be prepared for receiving a short header packet using zero-length CID from any IP address, and be capable of associating that to an existing connection. This is because the client (or the network) might change the client's address tuple when it sends packets to the preferred address. I would anticipate that you wouldn't want to do that in your file upload case (or in any deployment that might accept multiple connections concurrently).
NAME If I know 5-tuples are stable, then using a zero-length CID is always fine, just like it's typically fine in the server to client direction. But in that case, hopefully I can avoid needing to use SPA, making the decision on this issue moot. After thinking about this more, we're probably better off with option 4 for the sake of consistency and if we want to allow 0-length CIDs in more cases, I can write an extension for how that would work, since I don't think it's only a matter of relaxing some constraints.
Thanks all. I think that we reached the same conclusion in the end, via different paths. The proposed fix here is firmly design chairs. A bugfix only, but still worth careful review.
Apologies for getting to this late, but I'm not sure I agree that we should disallow this. Here's a use-case that I think makes sense: the client wants to upload a very large file to the server over HTTP/3 and it has a small MTU, making it particularly sensitive to per-packet overheads. Let's also assume that we are in the future and the server supports IPv6 and has its own /64 (shocking!). In this scenario, a good way to reduce per-packet overhead is to operate the server on a given address, and then switch each client to their own unique server IP using preferred address, and then use zero-length CIDs. I've always found this use-case a good use of preferred address, and I would rather we didn't ban it without a good reason (and making the design cleaner doesn't necessarily count as a good reason at this stage in the process).
That is an example we discussed. As Kazuho noted, that doesn't work as we have already agreed that any connection ID (in this case, a zero-length one) can be used prior to handshake completion. That's why we ended up on 4. As Ian noted, this isn't ideal, but this is an area that is ripe for extensions that might allow this sort of thing.
I don't understand what you meant by the following: Can you clarify? I was under the impression that the client switches to the preferred address after the handshake is done.
Yes, you can't migrate (act on the address information) before the handshake is confirmed. However, the connection ID is not special; it can be treated just like any other connection ID you learn, so you can act on the connection ID information before you confirm the handshake. So that means that a server that opts to include a zero-length connection ID in the preferred_address might receive a packet with the zero-length connection ID at the old address. That might not work out.
What's the point of including a connection ID (and stateless reset token) in the transport parameter if that CID isn't tied to the preferred address in any way? At that point might as well remove that CID entirely and rely on NEWCONNECTIONID frames. My understanding was that the reason for there being a CID in the TP was specifically to make that CID special. Would it make sense to just say that this CID MUST NOT be used when sending to the handshake server IP?
the reason for having a connection ID in the transport parameter is only so that the client has a connection ID to use for the new path
NAME I agree with your intuition, but I think we discussed this in Zurich and ended up concluding the preferredaddress CIDs weren't special. This was in issue and also came up in (which I realize hasn't gone out for a consensus call yet, that PR seems stalled?). In the upload example, the server needs to know that the client is going to send a lot of data, which it might know via SNI, or it might not. I tend to think the more annoying restriction is that you can't change from a non-0 length CID to a 0-length one and vice versa. By supplying a 0-length CID later in the connection along with other CIDs, the client could choose whether it wanted to use a 0-length CID at any time. Thinking out loud, an easier solution may be a TP like 'acceptemptyconnectionid' that indicates at anytime, the peer can switch to a 0-length CID if it wants. It could either think the 5-tuple is stable or its happy re-establishing the path with a non 0-length CID after a NAT rebind. In this world, the Connection ID is more of a path re-establishment token.
Thanks for the details NAME and NAME I get a sense I might be trying to re-litigate past consensus without providing new information, so I'll back down. I do think that the ghost of hypothetical linkability has caused us to make some very odd design choices, but this isn't new to this issue/PR. In the end, this is the kind of feature that can be enabled by extensions so I can live with us further restricting the base spec for consistency.
One way in which a server might use preferred_address is to enable routing based on the server address alone. Just as a client might benefit from a zero-length connection ID by selecting a unique address from which to initiate a connection (i.e., an ephemeral source port), this enables the server to pick an address that is unique to the connection. Such a server can then establish a connection by routing based on the source tuple, then move to a preferred address that is unique to the connection. This assumes that clients aren't using connection IDs to multiplex multiple connections to the same server, or that they only open one connection at a time, or that servers are willing to do trial decryption during handshakes (that last one being unlikely. because in the general case it is more than just trial decryption). As a result, the server does not have to request that the client not migrate. The server can use this unique address at its end to identify which connection packets belong to. That's all great, but it creates linkability for the client. When it migrates, the destination address is unique to that connection, for at least as long as that connection is live (which includes the idle timeout, which we have to assume is known to an attacker). It also potentially creates address exhaustion problems at the server as the server has to keep address allocations for a fairly long time to allow for idle connections to time out. Stateless resets are possible, but challenging, because the server has to rely on state in order to generate one. Stateless resets can't be static unless a port is only ever used exactly once (which seems ludicrous). I might conclude here that a zero-length connection ID for a server is just a bad idea, as the server address is stable across migrations and links flows on different paths. There is some uncertainty about linkability when it comes to port reuse, but this does undermine guarantees that clients might assume they have. I see two choices: Prohibit (active) migration when there is a zero-length connection ID at the server end. This would be on the basis that there is a privacy leak that is not obvious. Acknowledge that this happens and leave this to client discretion.
A server that wants to defeat linkability always can. Using , a server could easily enough issue each client its own IPv6 address or, even with IPv4, a unique ephemeral port. Even without it, a server could issue trivially linkable CIDs, disclose its CID key, etc. So the goal is to reduce the chances that this happens accidentally, not to prevent it entirely. We already decided not to prohibit active migration in the presence of zero-length CIDs, IIRC. Given that, I think this is just a textual warning to be cautious in this circumstance.
This is a maybe-design issue with an editorial-shaped proposed resolution. I think we'll start to see a lot more of these. I think we should run the design issue process still, but I leave that to chair discretion.
NAME ^
As I recall and unless something changed recently, active migration is not permitted for 0 length CID, but it is accepted that a NAT migrates outside the peers control.
I read this as a clarification of existing protocol design. I'd like to get eyes on it to make sure there is sufficient opportunity for discussion. Marking as design.
Transport section 9.5 says "An endpoint MUST use a new connection ID if it initiates connection migration as described in Section 9.2 or probes a new network path as described in Section 9.1. ". Section 19.15 (NEWCONNECTIONID) says: "An endpoint MUST NOT send this frame if it currently requires that its peer send packets with a zero-length Destination Connection ID. " I take this means active migration cannot happen with 0-length ID. BTW: in a few places "zero length" used over the more common "zero-length".
I really think we shouldn't disallow connection migration when the server uses zero-length CIDs. As stated above, linkability can happen for a variety a reasons, and we should leave it up to the client to decide whether it wants its bytes to go through or not. Perhaps an editorial paragraph stating this in the migration section or security considerations would do the job.
Thanks for incorporating my suggestions!Latest changes look good!

packets, such as timing and size, from being used to correlate activity. <ins> An endpoint SHOULD NOT initiate migration with a peer that has requested a zero-length connection ID, because traffic over the new path might be trivially linkable to traffic over the old one.  If the server is able to route packets with a zero-length connection ID to the right connection, it means that the server is using other information to demultiplex packets.  For example, a server might provide a unique address to every client, for instance using HTTP alternative services ALTSVC.  Information that might allow correct routing of packets across multiple network paths will also allow activity on those paths to be linked by entities other than the peer. </ins> Unintentional changes in path without a change in connection ID are possible.  For example, after a period of network inactivity, NAT rebinding might cause packets to be sent on a new path when the


See the Retry Packet Integrity section of QUIC-TLS.  The server populates the Destination Connection ID with the connection ID that the client included in the Source Connection ID of the Initial packet. Initial or 0-RTT packets.  A server MUST NOT send more than one Retry packet in response to a single UDP datagram.  A client MUST accept and process at most one Retry packet for each connection attempt.  After the client has received and processed an Initial or Retry packet from the server, it MUST discard any connection ID as part of its token validation logic (see token- integrity).  The next Initial packet from the client uses the connection ID and token values from the Retry packet; see negotiating-connection-ids. Aside from this, the Initial packet sent by the client is subject to absent.  A client MUST treat a failed validation as a connection error of type TRANSPORT_PARAMETER_ERROR. <del> A Retry packet does not include a packet number and cannot be explicitly acknowledged by a client. </del> 17.3. This version of QUIC defines a single packet type which uses the

This was getting a little long. You could get lost in there. The text was fine, it was mostly well structured. All it needed was one small move and some headers.

See the Retry Packet Integrity section of QUIC-TLS. <ins> 17.2.5.1. </ins> The server populates the Destination Connection ID with the connection ID that the client included in the Source Connection ID of the Initial packet. Initial or 0-RTT packets.  A server MUST NOT send more than one Retry packet in response to a single UDP datagram. <ins> 17.2.5.2. </ins> A client MUST accept and process at most one Retry packet for each connection attempt.  After the client has received and processed an Initial or Retry packet from the server, it MUST discard any connection ID as part of its token validation logic (see token- integrity). <ins> A Retry packet does not include a packet number and cannot be explicitly acknowledged by a client. 17.2.5.3. </ins> The next Initial packet from the client uses the connection ID and token values from the Retry packet; see negotiating-connection-ids. Aside from this, the Initial packet sent by the client is subject to absent.  A client MUST treat a failed validation as a connection error of type TRANSPORT_PARAMETER_ERROR.  17.3. This version of QUIC defines a single packet type which uses the


threshold less than 3, to keep in line with TCP RFC5681. Some networks may exhibit higher degrees of reordering, causing a <del> sender to detect spurious losses.  Implementers MAY use algorithms developed for TCP, such as TCP-NCR RFC4653, to improve QUIC's reordering resilience. </del>  5.1.2.

by tweaking the text, but doesn't add a new SHOULD.
Since RFC4653 doesn't seem like it would map to QUIC as readily as the stuff in the RACK draft about reorder-threshold tuning (it seems to be all about extending the Limited Transmit algorithm), maybe RACK is a better reference? URL c.f. "Update RACK reordering window"
I would prefer to reference RACK, but it's not an RFC yet and I'm not sure it will be prior to QUIC shipping. What's the best way to handle that?
The QUIC recovery spec already references the RACK draft in multiple places.
Fair point, I replaced the old reference with the RACK reference. PTAL.
LG!
I don't see any conversation in this PR about it, but I see a commit titled simply "non-normative" reverting the reference from RACK back to NCR. It's not a big deal, but I'm just curious about the reasoning.
There's conversation on the issue(URL) and Jana was concerned about adding another reference to RACK as well as recommending something that we don't have a detailed algorithm(and pseudocode) for. In general, anything we have a SHOULD for has more text(and typically pseudocode) than this, so I accepted that point and adjusted the PR. For example, the pseudocode doesn't detail how to detect spurious losses, since it's expected that could be fairly complex and implementation dependent.
I see, thanks for pointing me to that. But Jana's ask seems to me to be all about SHOULD versus MAY, and orthogonal to referencing NCR vs RACK; if it was a SHOULD, then you'd still have the same incompletely-specified-algorithm problem with NCR. It seems to me that now that SHOULD is removed, Jana would agree that RACK is a better reference since it maps more cleanly to QUIC.
Actually, I'm not so confident that RACK maps all that cleanly either, and Jana made the point that RACK's ReoWnd adjustment algorithm hasn't been closely examined, so I'm going to bow out here- I'm happy with either NCR or RACK.
The RACK draft has this as a SHOULD, and this was worthwhile in our deployment experience, so I think we should change our MAY to a SHOULD. From Section 4 of URL "3. The RACK reordering window SHOULD leverage Duplicate Selective Acknowledgement (DSACK) information [RFC3708] to adaptively estimate the duration of reordering events."
Another normative dependency on an experimental RFC. And one that isn't in the . Great. This seems fine, but it's a design change.
I would like to not see a dependency on the RACK draft here. If we want this to become a SHOULD, we really need to spell out what an implementation SHOULD do more precisely and not just point to RACK. That said, I'm not convinced that we should do a SHOULD here. I understand that you see an improvement (that might be worth noting in the spec), which is a perfectly good reason encourage others to implement and experiment, but why is a MAY not adequate, given that we have limited experience? I also am not convinced that this is important enough to warrant a design change at this late stage. Do you think this performance impact is significant enough to argue for this change?
As Matt Olson pointed out, we already reference RACK informationally. We could remove all the references to RACK, but I think that's a separate question? We can more clearly explain what this means or re-instate the reference to RFC4653. TCP has been doing this for a very long time, so I don't think the fact we have limited experience in QUIC is as important. In terms of 'new information', the performance improvement from adaptive packet threshold was measurable for both application latency and by reducing the rate of spurious retransmits. And I finally read the RACK draft and noticed it was a SHOULD there. The original intent was to change a MAY to a SHOULD, which is a very small design change, but then I realized the existing text wasn't that clear, so I tried to improve it. We have to do another round of consensus calls in the next few weeks, so though this change is very small, it seems like the right thing to do to me.
I understand that this might have performance benefit that is worth exploring further. I question the need for doing it at this late stage, given that we will find more and more things that help with performance as we gain further experience with the protocol. RACK is currently an informational reference, meaning that it is not required reading for an implementer. This change would make it a normative reference, and we don't have such dependencies on any TCP document at the moment. (I see that F-RTO is listed as normative, but it isn't really, I'll send a PR to fix that.) This isn't a small change because just changing a MAY to a SHOULD isn't enough. We cannot change a "MAY adapt" to a "SHOULD adapt" without saying exactly what "SHOULD adapt" means for an implementer. At a minimum, this requires changes to the text and to the pseudo-code that include a precise description and an algorithm for adapting. (The RACK draft suggests one, but again, it's not clear to me whether that one is what we should be recommending. I want to try it out for sure though.) On TCP's experience with this: While I understand that RACK has been deployed for a few years, I don't believe we know how much improvement this specific part of it brings, especially given the DSACK dependency in TCP (Praveen can perhaps shed some light on DSACK deployment at clients, but ... this isn't my central argument.). If there is public information about this, I would love to look at it to have a better understanding.
I still intend RACK to be an informational reference(PR uses a ?, not a !), not normative. The intent was to update from an older informational reference(TCP-NCR) to a newer one. This mechanism was deployed in Linux long before RACK. It seems your point is that by changing from MAY to SHOULD, that puts a requirement upon us to fully describe the algorithm in text and pseudocode? I don't see that as a hard requirement, and as you can see in my PR, I didn't intent to do that.
Does it need to be normative at all? What about simply noting that adaptive strategies such as X, Y, and Z have proven to be useful in TCP, and would apply equally to QUIC?
NAME Sure, that could work.
That works for me too, thanks!
My thinking is that "adaptive strategies" would not only "apply equally to QUIC" but be "significantly more important for QUIC". Operators are aware of the havoc that excessive reordering in their networks causes to TCP and have deployed middle boxes for restoring TCP ordering in such networks. Such network "optimizations" are not possible with QUIC, so QUIC will see more TCP reordering. We've seen such extreme reordering of QUIC in production.
NAME that's a good point. Do you think it warrants adding a statement like "Re-ordering could be more common with QUIC than TCP, because network elements cannot observe and fix the order of out-of-order packets."?
NAME I like this.
NAME do you have a reference of those data? It'd be good to understand how the reordering pattern looks like in the production network.
NAME The public data on this can be found in . Take a look at red "below the line" red dots on the graphs. At the time we thought the red dots represented loss in our observer infrastructure, but additional analysis after the IETF meeting confirmed that the signal was mostly due to very high reordering (more than 31 subsequent packets arrive before an older packet has arrived).

30 packet reordering certainly happens, though it is quite rare. Some other MAPRG I'll update the PR to point out that UDP reordering may be more prevalent, but not add any new normative statements.
Thanks, NAME We were deliberately focusing data collection on both "boring" and "interesting" network locations to get a wide range of scenarios.

threshold less than 3, to keep in line with TCP RFC5681. Some networks may exhibit higher degrees of reordering, causing a  <ins> sender to detect spurious losses.  Algorithms that increase the reordering threshold after spuriously detecting losses, such as TCP- NCR RFC4653, have proven to be useful in TCP and are expected to at least as useful in QUIC.  Re-ordering could be more common with QUIC than TCP, because network elements cannot observe and fix the order of out-of-order packets. </ins> 5.1.2.


1.3. <del> Packet and frame diagrams in this document use the format described in Section 3.1 of RFC2360, with the following additional conventions: </del>  <del> Indicates that x is optional </del>  <del> Indicates that x is A bits long </del>  <del> Indicates that x is one of A, B, or C bits long </del>  Indicates that x uses the variable-length encoding in <del> Indicates that x is variable-length </del>  2. packet protection keys for Initial packets. The client populates the Source Connection ID field with a value of <del> its choosing and sets the SCID Len field to indicate the length. </del>  The first flight of 0-RTT packets use the same Destination Connection ID and Source Connection ID values as the client's first Initial SHOULD be able to read longer connection IDs from other QUIC versions in order to properly form a version negotiation packet. <del> The Destination Connection ID field follows the DCID Len and is between 0 and 20 bytes in length. negotiating-connection-ids describes the use of this field in more detail. </del>  The byte following the Destination Connection ID contains the length in bytes of the Source Connection ID field that follows it. IDs from other QUIC versions in order to properly form a version negotiation packet. <del> The Source Connection ID field follows the SCID Len and is between 0 and 20 bytes in length. negotiating-connection-ids describes the use of this field in more detail. </del>  In this version of QUIC, the following packet types with the long header are defined: A Retry packet (shown in retry-format) does not contain any protected fields.  The value in the Unused field is selected randomly by the <del> server.  In addition to the long header, it contains these additional fields: </del>  An opaque token that the server can use to validate the client's address. in the same way as the path MTU, but it is a property of the endpoint and not the path.  It is expected that this is the space an endpoint dedicates to holding incoming packets. <del> initial_max_data (0x04): </del> The initial maximum data parameter is an integer value that contains the initial value for the maximum amount of data that can 19.3.1. <del> The ACK Ranges field consists of alternating Gap and ACK Range values in descending packet number order.  The number of Gap and ACK Range values is determined by the ACK Range Count field; one of each value is present for each value in the ACK Range Count field. </del>  ACK Ranges are structured as shown in ack-range-format. <del> The fields that form the ACK Ranges are: </del>  A variable-length integer indicating the number of contiguous unacknowledged packets preceding the packet number one lower than MAX_STREAMS frames contain the following fields: A count of the cumulative number of streams of the corresponding <del> type that can be opened over the lifetime of the connection. Stream IDs cannot exceed 2^62-1, as it is not possible to encode stream IDs larger than this value.  Receipt of a frame that permits opening of a stream larger than this limit MUST be treated as a FRAME_ENCODING_ERROR. </del>  Loss or reordering can cause a MAX_STREAMS frame to be received which states a lower stream limit than an endpoint has previously received. STREAMS_BLOCKED frames contain the following fields: <del> A variable-length integer indicating the stream limit at the time the frame was sent.  Stream IDs cannot exceed 2^62-1, as it is not possible to encode stream IDs larger than this value.  Receipt of a frame that encodes a larger stream ID MUST be treated as a STREAM_LIMIT_ERROR or a FRAME_ENCODING_ERROR. </del>  19.15. 19.18. The PATH_RESPONSE frame (type=0x1b) is sent in response to a <del> PATH_CHALLENGE frame.  Its format is identical to the PATH_CHALLENGE frame (frame-path-challenge). </del>  If the content of a PATH_RESPONSE frame does not match the content of a PATH_CHALLENGE frame previously sent by the endpoint, the endpoint

This is not an editorial decision, this is a request for feedback on whether this form of presentation makes more sense. So far we have mixed feedback.
Did you consider TLS presentation language? It would be nice to not have n+1 formats.
NAME TLS presentation language implies length prefixes in a lot of places. Plus we already ripped out TLS presentation language for definition of the transport parameters. Bringing it back for definition of packet layouts would be... a bit weird.
As long as it was considered and found to be unsuitable, that's fine with me.
I definitely like this general structure better. As far as the question of whether we should import the TLS language, you would need some extensions, but I suspect that you could get pretty far with the following three: uintX == unsigned integer of size X uintv == varint opaque x[URL] == opaque vector without a header (TLS uses <> to mean that it has a header) and opaque x[y..] means "to end of the container and minimum size y] I believe we've already done uintv for MLS and NAME has been thinking about a generalized an a "TLS spec language"
Orthogonal to this PR, but perhaps we should align the "Maximum Data" and "Data Limit" terms which reference the same limit in different frames? Originally posted by NAME in URL
SGTM. Maximum Data(or Max Data) seems like the preferable option to me.
NAME The link to NAME post seems to be broken.
Odd, that was automatically generated by GitHub. That comment was on a review of .
But...! But...! Muh ASCII art! More seriously, this is probably a net improvement for readability. There's value in conforming to existing paradigms where possible, but we'd already had to define custom extensions to the notation from RFC2360 and from RFC7540, so this is probably no worse. What are your thoughts on applying this to QPACK as well? I'm really not a fan of the trailing comma in the lists, but that's neither here nor there.I really like this format.

1.3.  <ins> Packet and frame diagrams in this document use a bespoke format.  The purpose of this format is to summarize, not define, protocol elements.  Prose defines the complete semantics and details of structures. </ins>  <ins> Complex fields are named and then followed by a list of fields surrounded by a pair of matching braces.  Each field in this list is separated by commas. </ins>  <ins> Individual fields include length information, plus indications about fixed value, optionality, or repetitions.  Individual fields use the following notational conventions, with all lengths in bits: </ins>  <ins> Indicates that x is A bits long </ins> Indicates that x uses the variable-length encoding in  <ins> Indicates that x can be any length from A to B; A can be omitted to indicate a mimumum of zero bits and B can be omitted to indicate no set upper limit; values in this format always end on an octet boundary Indicates that x has a fixed value of C Indicates that x has a value in the range from C to D, inclusive Indicates that x is optional (and has length of E) Indicates that x is repeated zero or more times (and that each instance is length E) By convention, individual fields reference a complex field by using the name of the complex field. For example: </ins> 2. packet protection keys for Initial packets. The client populates the Source Connection ID field with a value of  <ins> its choosing and sets the SCID Length field to indicate the length. </ins> The first flight of 0-RTT packets use the same Destination Connection ID and Source Connection ID values as the client's first Initial SHOULD be able to read longer connection IDs from other QUIC versions in order to properly form a version negotiation packet.  <ins> The Destination Connection ID field follows the DCID Length field and is between 0 and 20 bytes in length. negotiating-connection- ids describes the use of this field in more detail. </ins> The byte following the Destination Connection ID contains the length in bytes of the Source Connection ID field that follows it. IDs from other QUIC versions in order to properly form a version negotiation packet.  <ins> The Source Connection ID field follows the SCID Length field and is between 0 and 20 bytes in length. negotiating-connection-ids describes the use of this field in more detail. </ins> In this version of QUIC, the following packet types with the long header are defined: A Retry packet (shown in retry-format) does not contain any protected fields.  The value in the Unused field is selected randomly by the  <ins> server.  In addition to the fields from the long header, it contains these additional fields: </ins> An opaque token that the server can use to validate the client's address. in the same way as the path MTU, but it is a property of the endpoint and not the path.  It is expected that this is the space an endpoint dedicates to holding incoming packets.  The initial maximum data parameter is an integer value that contains the initial value for the maximum amount of data that can 19.3.1.  <ins> Each ACK Range consists of alternating Gap and ACK Range values in descending packet number order.  ACK Ranges can be repeated.  The number of Gap and ACK Range values is determined by the ACK Range Count field; one of each value is present for each value in the ACK Range Count field. </ins> ACK Ranges are structured as shown in ack-range-format.  <ins> The fields that form each ACK Range are: </ins> A variable-length integer indicating the number of contiguous unacknowledged packets preceding the packet number one lower than MAX_STREAMS frames contain the following fields: A count of the cumulative number of streams of the corresponding  <ins> type that can be opened over the lifetime of the connection.  This value cannot exceed 2^60, as it is not possible to encode stream IDs larger than 2^62-1.  Receipt of a frame that permits opening of a stream larger than this limit MUST be treated as a FRAME_ENCODING_ERROR. </ins> Loss or reordering can cause a MAX_STREAMS frame to be received which states a lower stream limit than an endpoint has previously received. STREAMS_BLOCKED frames contain the following fields:  <ins> A variable-length integer indicating the maximum number of streams allowed at the time the frame was sent.  This value cannot exceed 2^60, as it is not possible to encode stream IDs larger than 2^62-1.  Receipt of a frame that encodes a larger stream ID MUST be treated as a STREAM_LIMIT_ERROR or a FRAME_ENCODING_ERROR. </ins> 19.15. 19.18. The PATH_RESPONSE frame (type=0x1b) is sent in response to a  <ins> PATH_CHALLENGE frame.  Its format, shown in fig-path-response is identical to the PATH_CHALLENGE frame (frame-path-challenge). </ins> If the content of a PATH_RESPONSE frame does not match the content of a PATH_CHALLENGE frame previously sent by the endpoint, the endpoint


For example, if the dynamic table is 100 bytes, then the Required Insert Count will be encoded modulo 6.  If a decoder has received 10 <del> inserts, then an encoded value of 3 indicates that the Required </del>  Insert Count is 9 for the header block. 4.5.1.2.

The algorithm was updated at somepoint but the example was lagging.
Insert Count will be encoded modulo 6. If a decoder has received 10 inserts, then an encoded value of 3 indicates that the Required Insert Count is 9 for the header block. should be Insert Count will be encoded modulo 6. If a decoder has received 10 inserts, then an encoded value of 4 indicates that the Required Insert Count is 9 for the header block. Background: I implemented ReqInsertCount for decoder. When I test it with: MaxTableCapacity = 100 EncodedInsertCount = 3 TotalNumberOfInserts = 10 My function returns 8, not 9.I guess that EncodedInsertCount = 4 is right since ReqInsertCount for encoder calculates 4 with ReqInsertCount = 9. NAME pointed out that it looks like the algorithm was replaced, and the example text we are looking at is left as is.

For example, if the dynamic table is 100 bytes, then the Required Insert Count will be encoded modulo 6.  If a decoder has received 10  <ins> inserts, then an encoded value of 4 indicates that the Required </ins> Insert Count is 9 for the header block. 4.5.1.2.


MAX_STREAMS frames contain the following fields: A count of the cumulative number of streams of the corresponding <del> type that can be opened over the lifetime of the connection. Stream IDs cannot exceed 2^62-1, as it is not possible to encode stream IDs larger than this value.  Receipt of a frame that permits opening of a stream larger than this limit MUST be treated as a FRAME_ENCODING_ERROR. </del>  Loss or reordering can cause a MAX_STREAMS frame to be received which states a lower stream limit than an endpoint has previously received. STREAMS_BLOCKED frames contain the following fields: <del> A variable-length integer indicating the stream limit at the time the frame was sent.  Stream IDs cannot exceed 2^62-1, as it is not possible to encode stream IDs larger than this value.  Receipt of a frame that encodes a larger stream ID MUST be treated as a STREAM_LIMIT_ERROR or a FRAME_ENCODING_ERROR. </del>  19.15.

This is based on and is targeted for that. Careful. I did a bit of language tweaking to the 260 limits on the MAX_STREAMS frame to make it a little more useful. The text there was a little old.

MAX_STREAMS frames contain the following fields: A count of the cumulative number of streams of the corresponding  <ins> type that can be opened over the lifetime of the connection.  This value cannot exceed 2^60, as it is not possible to encode stream IDs larger than 2^62-1.  Receipt of a frame that permits opening of a stream larger than this limit MUST be treated as a FRAME_ENCODING_ERROR. </ins> Loss or reordering can cause a MAX_STREAMS frame to be received which states a lower stream limit than an endpoint has previously received. STREAMS_BLOCKED frames contain the following fields:  <ins> A variable-length integer indicating the maximum number of streams allowed at the time the frame was sent.  This value cannot exceed 2^60, as it is not possible to encode stream IDs larger than 2^62-1.  Receipt of a frame that encodes a larger stream ID MUST be treated as a STREAM_LIMIT_ERROR or a FRAME_ENCODING_ERROR. </ins> 19.15.


or more packets are sent prior to a dramatic decrease in congestion window; see persistent-congestion.  4. At a high level, an endpoint measures the time from when a packet was

Feel free to suggest better text or other reasons, but this is what I came up with.
QUIC spec uses kMinimumWindow of 2 MSS compared to TCP's 1 MSS. Let's state the deviation and add justifiaiton text.
LGTM, modulo one suggestion

or more packets are sent prior to a dramatic decrease in congestion window; see persistent-congestion. <ins> 3.1.8. TCP uses a minimum congestion window of one packet.  However, loss of that single packet means that the sender needs to waiting for a PTO (pto) to recover, which can be much longer than a round-trip time. Sending a single ack-eliciting packet also increases the chances of incurring additional latency when a receiver delays its acknowledgement. QUIC therefore recommends that the minimum congestion window be two packets.  While this increases network load, it is considered safe, since the sender will still reduce its sending rate exponentially under persistent congestion (pto). </ins> 4. At a high level, an endpoint measures the time from when a packet was


for this connection received on the old IP address, but MAY continue to process delayed packets.  9.6.3. A client might need to perform a connection migration before it has A client that migrates to a new address SHOULD use a preferred address from the same address family for the server.  9.7. Endpoints that send data using IPv6 SHOULD apply an IPv6 flow label

This just writes down what NAME suggested in .
Two things need to be clarified around SPA: The CID is not path-specific; it's there to ensure that the client has a second CID before the handshake completes. If the client opts not to migrate, the client can use that CID on the The IP addresses in the SPA are only for use in the transition post-handshake; the client should not cache the IP addresses for any other use.
(Just filling in the text here) on the original address from the server, which is the existing connection, not-migrated. +1 to the second point
If the server is telling the client to migrate to a SPA, should this be changed to "the client MUST select one of the two server's preferred addresses ..."? If the text remains a SHOULD, and the client stays on the old server address, what can the server do if it is only willing to continue with the new connection on the SPA? The server could close the initial connection using the old server address but there isn't a transport error code (e.g. MIGRATION_ERROR) that would explicitly tell the client why the connection is being closed.
The "SHOULD" is intentional. A server cannot rely on clients moving.
Then perhaps a cautionary note such as "If the client opts not to migrate to a preferred address, the server may [lower case] decide to close the connection (e.g. SERVER_BUSY)".
If the server isn't willing at all to handle connections on its published IP address, then (naively) it shouldn't accept connections on that IP address and should publish the other one instead. The scenario this targets is that it's somehow better for the client to use a different address, but that other address can't necessarily be known in advance.
Migrating to a unicast address avoids having to perform connection-based routing through a load balancer.

for this connection received on the old IP address, but MAY continue to process delayed packets. <ins> The addresses that a server provides in the preferred_address transport parameter are only valid for the connection in which they are provided.  A client MUST NOT use these for other connections, including connections that are resumed from the current connection. </ins> 9.6.3. A client might need to perform a connection migration before it has A client that migrates to a new address SHOULD use a preferred address from the same address family for the server. <ins> The connection ID provided in the preferred_address transport parameter is not specific to the addresses that are provided.  This connection ID is provided to ensure that the client has a connection ID available for migration, but the client MAY use this connection ID on any path. </ins> 9.7. Endpoints that send data using IPv6 SHOULD apply an IPv6 flow label