An endpoint that wants to close a path SHOULD use explicit request to terminate the path by sending the PATH_ABANDON frame (see path- <del> abandon-close).  Note that while abandoning a path implies Connection ID retirement, only retiring the associated Connection ID does not necessarily advertise path abandon (see retire-cid-close).  However, implicit signals such as idle time or packet losses might be the only way for an endhost to detect path closure (see idle-time-close). </del>  Note that other explicit closing mechanisms of QUIC-TRANSPORT still apply on the whole connection.  In particular, the reception of

Minor tweak. The text says that a path abandon "implies" retirement of the CID. Readers could be mislead and assume that they don't need to send a retire frame after sending abandon, when in fact they do.
NAME , NAME NAME please review.
This Github setting is pretty annoying. NAME I understand why only the selected authors would be able to merge a pull request, but as it stands I am also blocked from requesting reviews of a PR that I just created.
LGTMThe proposed text is indeed clearer.

An endpoint that wants to close a path SHOULD use explicit request to terminate the path by sending the PATH_ABANDON frame (see path-  <ins> abandon-close).  Note that while abandoning a path will cause Connection ID retirement, only retiring the associated Connection ID does not necessarily advertise path abandon (see retire-cid-close). However, implicit signals such as idle time or packet losses might be the only way for an endhost to detect path closure (see idle-time- close). </ins> Note that other explicit closing mechanisms of QUIC-TRANSPORT still apply on the whole connection.  In particular, the reception of


ID of the 1-RTT packets which are acknowledged by the ACK_MP frame. If the endpoint receives 1-RTT packets with zero-length Connection ID, it SHOULD use Packet Number Space Identifier 0 in ACK_MP frames. <del> If an endpoint receives an ACK_MP frame with a non-existing packet number space ID, it MUST treat this as a connection error of type MP_PROTOCOL_VIOLATION and close the connection. </del>  When using a single packet number space, endhosts MUST NOT send ACK_MP frames.  If an endhost receives an ACK_MP frame while a single

Fix issue
When using Identifier Types 0 or 1, the current text does not discuss how we should handle related path Identifier that the receiver cannot map to a path. When using Identifier Type 1, the Path Identifier corresponds to the sequence number of the Destination CID used by the sender to send packets on the related path, hence a sequence number of a source Connection ID of the receiver. When using Identifier Type 0, the Path Identifier corresponds to the sequence number of the Source CID of the sender, hence the sequence number of the Destination CID used by the receiver to send packets on the related path. While there are obvious values that could be considered as bad and may lead to, e.g., a MPPROTOCOLVIOLATION (e.g., a sequence number larger than any sequence number advertised), I don't think we should be as radical with regular values as they might have been retired meanwhile (also possible to face this with packet reordering). The worst edge cases I see are the followings. With type 1, the sender might send a PATHABANDON with the seq of the CID in one packet, and then the RETIRECID retiring that CID in a subsequent one, but the RETIRECID reaches the receiver before the PATHABANDON one. If the receiver drops state upon RETIRECID reception, it won't be able to map the PATHABANDON to the path. With type 0, the sender might send a PATHABANDON with the seq of the CID, but before it reaches the peer, the peer sends a RETIRECID retiring that CID. If the receiver drops the CID state upon RETIRECID sending, it won't be able to map the PATHABANDON to the path. Handling these edge cases with connection closure is clearly too strong. Still, ignoring or even acknowledging these PATHABANDON may have bad consequences. Silently dropping packets (i.e., not acknowledge the packet having PATHABANDON) will make the peer retransmit the frame (hoping it updates its path identifier) but could cause the loss/non-acknowledgment of accompanying frames (e.g., STREAM ones), and processing packet content without acknowledging it sounds odd. Acknowledging PATHABANDON that cannot be mapped to a path would make some desynchronisation about the status of the path between the peers (the sender now assumes the path is closed, but the receiver not). To address the type 1 issue, we could write some text stating that a sender cannot retire a DCID if a PATHABANDON containing the related sequence number is still in flight. But for the type 0, the fix is not obvious (I don't think we could have the same solution as in )...
One solution is to put the retired cid into a cidhistorylist and keep it for a while before its final deletion. In doing so, one can still find the cid-to-path mapping if the PATHSTATUS frame uses a retired cid. In this case, as long as the path that the peer wants to retire has no ambiguity, the current path close process can still work. If the receiver can no longer find the cid in both the active list and the history list, then it should treat this as a connection error of MPPROTOCOL_VIOLATION and close the connection.
Another solution would to to not change the path ID (see issue ). In this case all unknown path IDs could be handled as a connection error. Or is it anyway better to just ignore those frames (e.g. if completely outdated packet arrives and the path was already closed)?
I think the simplest solution is to just ignore Path Abandon that refer to abandoned connection ID sequence numbers. The problem is self correcting. If the connection ID is not used anymore, it is either because the path was actually abandoned (no problem), or because packets on the same 5-tuple are now sent with a different CID. In that case, the peer will see these packets arriving on the path that it wants to abandon, and can simply send a new Path Abandon.
Given that we agree that PATHABANDON relates to CID sequence numbers, we can probably apply the same approach to all multipath-specific frames (including ACKMP): if you receive such a frame with a CID sequence number you cannot map anymore, just ignore it.
This is somewhat related to . Section 12.2 states that The notion of "non-existing" packet number space ID is somewhat blurry and maybe too aggressive. If for some reason an ACKMP frame referencing an old, retired DCID sequence number arrives at the endhost, probably we just want to ignore it? And we could instead consider ACKMP related to a packet number space ID larger than the greatest advertised one being MPPROTOCOLVIOLATION?

ID of the 1-RTT packets which are acknowledged by the ACK_MP frame. If the endpoint receives 1-RTT packets with zero-length Connection ID, it SHOULD use Packet Number Space Identifier 0 in ACK_MP frames.  <ins> If an endpoint receives an ACK_MP frame with a packet number space ID which was never issued by endpoints (i.e., with a sequence number larger than the largest one advertised), it MUST treat this as a connection error of type MP_PROTOCOL_VIOLATION and close the connection.  If an endpoint receives an ACK_MP frame with a packet number space ID which is no more active (e.g., retired by a RETIRE_CONNECTION_ID frame or belonging to closed paths), it MUST ignore the ACK_MP frame without causing a connection error. </ins> When using a single packet number space, endhosts MUST NOT send ACK_MP frames.  If an endhost receives an ACK_MP frame while a single


as the content of lost packets initially sent on one path can be sent on another path without further frame scheduling adaptations.  12. All the new frames MUST only be sent in 1-RTT packet, and MUST NOT

Currently we don't mention PING frames in the draft, however, I guess there are use cases where an app wants to send a ping over one specific or both paths. We should probably say something about handling of PING frames if multiple path are available.
We already have the path challenge frame, which provides a way to test continuity of a specific path. I think it is better to leave ping "connection wide", parallel to data frames.
That okay as well but we still would need more guidance how to send ping frames when multiple path are available. E.g. try one and if it fails, try the next. or just send a ping on each path...?
It depends whether you want to test continuity on the forward path, the connection, or the return path. The forward path test is used form example when doing PMTUD or probe time-out. In that case, the Ping frame is sent on the path that the implementation wants to test, and the corresponding ACK will indicate that the packet sent on the specified path has been received. This is exactly as expected, and I don't think it requires clarification. The connection test is used for example in keep-alive mechanisms. There are many possibilities, because this is only used when the connection is almost idle. Should such connections try to keep many paths alive or just one? I suppose different applications will have different constraints. In any case, once they have decided that, sending Ping frames on the selected path or paths is straightforward. The return path is different. I don't think it can be tested using Ping, because there is no way to force ACK frames to use a specific return path. If an application want to do that, they have to use Path Challenge.
I tend to agree with Christian. How the application wants to send PING frames on available paths to probe forward path looks more like an API issue than a specification one. And I agree that testing the return path is not easy (beyond PATHCHALLENGE/PATHRESPONSE), but in general "forcing" scheduling policies on the other end is not possible (you can only suggest, as with PATH_STATUS frames).
Yes, it mainly an API issue. However, I think there is one point we need to address. This is what RFC9000 says: If you send a PING only one path and don't get a response, you won't reset the timer and might close the connection even though another path might still be alive. I think that would be wrong. So I think the two options are you either send PINGs on all paths; or you try one path first and if you don't get a reply, reset the timer and try the next one.
NAME you are describing the "keep alive" scenario. Yes, keep alive becomes a bit tricky in multipath, because the client may want to select how many paths it wants to keep alive, and which ones. For example, a client with Wi-Fi and cellular may want to keep the cellular path alive, because it will survive a mobility event. It may want to do that even if it is sending traffic over Wi-Fi. That's a local implementation decision. How they do that, however, is straightforward: send a Ping on the path that you want to keep alive. I think we should let implementers decide what they want to do. If we do add some text, is should be about keep alive, not about ping. Something like:
I think we should recommend sending out ping frames periodically on all paths for keep alive purpose. Sometimes, if you don't send data on a path for some time, then the path will become a "black hole" for UDP packets, probably due to NAT black hole. But I think we already address it in 4.3.4. Idle Timeout part of the draft with the following text: "To avoid idle timeout of a path, endpoints can send ack-eliciting packets such as packets containing PING frames ( of ]) on that path to keep it alive. Sending periodic PING frames also helps prevent middlebox timeout, as discussed in of ]."
Sounds like we can add some guidance in the implementation section!
I agree with Quentin and Christian that the problem is application dependent. We should avoid over specifying things in this section. For example, sending PING frames on all paths does not seem a good suggestion given that a connection may use more than two paths. We could have something like The QUIC specification defines an optional keep alive process, see {{Section 5.3 of RFC9000}}. Implementations of the multipath extension should map this keep alive process to a number of paths. Some applications may wish to ensure that one path remains active, while others could prefer to have two or more active paths during the connection lifetime. Different applications will likely require different strategies. Once the implementation has decided which paths to keep alive, it can do so by sending Ping frames on each of these paths before the idle timeout expires. Implementations should jitter their timers to avoid sending Ping frames at the same time over different paths.
Right now, we have some texts like sending ack-eliciting packets such as PING frames to keep a path alive in Section 4.3.4. But I think we can merge the above text to better clarify this implementation guidance.
So for keep alive purposes if you decide to send pings only over one or at least not all open paths, that would mean that you may not get a ping reply on those paths and close the connection even though another path, that you just decide for what every reason to not ping, is still working. That does seem quite right do me? Also not sure about the recommendation to jitter pings. Ideally paths are distinct (to some extend at least) but even if not a ping frame is quite low load. I guess if you send too many ping frames at ones this could lead to a burst that overflows the short buffers but I guess that would need in most cases at least 10 or more pings. Not sure we need to give this recommendation...?
I create a new PR () with the proposed text as a new subsection in the implementation consideration part. However, not sure if this is ready to merge or needs further discussion...
I actually have similar questions. I think to keep it simple, if you want to keep paths alive, just send pings on every path. Sending pings at the same time over different paths should not be problematic as long as you have the frequency control.
We already have some text about how the PING frames influence the paths in Section . It mostly depends on implementations. We could simply add:
I'm fine with the proposed addition.

as the content of lost packets initially sent on one path can be sent on another path without further frame scheduling adaptations. <ins> 11.2. The QUIC specification defines an optional keep alive process, see Section 5.3 of QUIC-TRANSPORT.  Implementations of the multipath extension should map this keep alive process to a number of paths. Some applications may wish to ensure that one path remains active, while others could prefer to have two or more active paths during the connection lifetime.  Different applications will likely require different strategies.  Once the implementation has decided which paths to keep alive, it can do so by sending Ping frames on each of these paths before the idle timeout expires. </ins> 12. All the new frames MUST only be sent in 1-RTT packet, and MUST NOT


1-RTT packets use DCID S3, which has a sequence number of 3.  Note that the paths use different packet number spaces.  In this case, the client is going to close the first path.  It identifies the path by <del> the sequence number of the received packet's DCID over that path (path identifier type 0x00), hence using the path_id 1.  Optionally, the server confirms the path closure by sending an PATH_ABANDON frame using the sequence number of the received packet's DCID over that path (path identifier type 0x00) as path identifier, which </del>  corresponds to the path_id 2.  Both the client and the server can close the path after receiving the RETIRE_CONNECTION_ID frame for that path. PATH_ABANDON frames contain the following fields: <del> Path Identifier: An identifier of the path, which is formatted as shown in fig-path-identifier-format. Identifier Type: Identifier Type field is set to indicate the type of path identifier. Type 0: Refer to the connection identifier issued by the sender of the control frame.  Note that this is the connection identifier used by the peer when sending packets on the to-be- closed path.  This method SHOULD be used by default. Type 1: Refer to the connection identifier issued by the receiver of the control frame.  Note that this is the connection identifier used by the sender when sending packets on the to-be-closed path. Type 2: Refer to the path over which the control frame is sent or received. Path Identifier Content: A variable-length integer specifying the path identifier.  If Identifier Type is 2, the Path Identifier Content MUST be empty. </del>  A variable-length integer that indicates the reason for abandoning this path. PATH_ABANDON frames SHOULD be acknowledged.  If a packet containing a PATH_ABANDON frame is considered lost, the peer SHOULD repeat it. <del> If the Identifier Type is 0x00 or 0x01, PATH_ABANDON frames MAY be sent on any path, not only the path identified by the Path Identifier Content field.  If the Identifier Type if 0x02, the PATH_ABANDON frame MUST only be sent on the path that is intended to be abandoned. </del>  12.2. PATH_STATUS Frames contain the following fields: <del> Path Identifier: An identifier of the path, which is formatted as shown in fig-path-identifier-format.  Exactly the same as the definition of Path Identifier in {#path-abandon-frame}. </del>  <del> Path Status sequence number: A variable-length integer specifying the sequence number assigned for this PATH_STATUS frame.  The sequence number MUST be monotonically increasing generated by the sender of the Path Status frame in the same connection.  The receiver of the Path Status frame needs to use and compare the sequence numbers separately for each Path Identifier. </del>  Available values of Path Status field are: Compared to the ACK frame specified in QUIC-TRANSPORT, the following field is added. <del> Packet Number Space Identifier: An identifier of the path packet number space, which is the sequence number of Destination Connection ID of the 1-RTT packets which are acknowledged by the ACK_MP frame. </del>  If an endpoint receives an ACK_MP frame with a packet number space ID which was never issued by endpoints (i.e., with a sequence number larger than the largest one advertised), it MUST treat this as a

Since , we now define the Path ID as (Section 1.1) And the definition of the as the field of PATHABANDON and PATHSTATUS frames directly maps to that identifier path (i.e., the sequence number of the destination Connection ID used for sending packets on that particular path, Sections 8.1 and 8.2). But in Section 6.2, the example states that I believe the current text is unclear and should be fixed. Yet, I see two possible ways of doing so. We should either state that the field of these frames refer to the Path ID used by the receiver of the frame, or add some information to indicate that it refers to the Path ID viewed from the sender's/receiver's viewpoint (that could be encoded as a bit in the frame's type)
Is there any use case for either using the sender's or receiver's ID? If not, I'd prefer we simply choose one and define it clearly.
(P.S.: I think the text in the example is just wrong. I think I messed this up in .)
IMHO it makes more sense to me to use the sender's ID by default, especially if we consider only client-initiated paths.
From PR by NAME
Looks good to me

1-RTT packets use DCID S3, which has a sequence number of 3.  Note that the paths use different packet number spaces.  In this case, the client is going to close the first path.  It identifies the path by  <ins> the sequence number of the received packet's DCID over that path, hence using the path_id 1.  Optionally, the server confirms the path closure by sending an PATH_ABANDON frame using the sequence number of the received packet's DCID over that path as path identifier, which </ins> corresponds to the path_id 2.  Both the client and the server can close the path after receiving the RETIRE_CONNECTION_ID frame for that path. PATH_ABANDON frames contain the following fields:  <ins> The identifier of the path that should be abandoned. </ins> A variable-length integer that indicates the reason for abandoning this path. PATH_ABANDON frames SHOULD be acknowledged.  If a packet containing a PATH_ABANDON frame is considered lost, the peer SHOULD repeat it.  <ins> PATH_ABANDON frames MAY be sent on any path, not only the path identified by the Packet Number Space Identifier. </ins> 12.2. PATH_STATUS Frames contain the following fields:  <ins> The identifier of the path that the status update correspondes to. </ins>  <ins> A variable-length integer specifying the sequence number assigned for this PATH_STATUS frame.  The sequence number MUST be monotonically increasing generated by the sender of the Path Status frame in the same connection.  The receiver of the Path Status frame needs to use and compare the sequence numbers separately for each Path Identifier. </ins> Available values of Path Status field are: Compared to the ACK frame specified in QUIC-TRANSPORT, the following field is added.  <ins> The identifier of the path packet number space of the 1-RTT packets which are acknowledged by the ACK_MP frame. </ins> If an endpoint receives an ACK_MP frame with a packet number space ID which was never issued by endpoints (i.e., with a sequence number larger than the largest one advertised), it MUST treat this as a


frames to signal acknowledgement of 1-RTT packets on path number 0, and also 0-RTT packets as specified in handling-of-0-rtt-packets. <del> 1-RTT packets sent to different paths SHOULD carry different connection identifiers, if a non-zero connection ID is used. </del>  6.1.

There is a SHOULD here, which I believe should be a MUST as we taking about the receiver behaviour here.
NAME NAME is this ready to merge?
Sorry, I am working on several projects in parallel, missed this. Text is fine.

frames to signal acknowledgement of 1-RTT packets on path number 0, and also 0-RTT packets as specified in handling-of-0-rtt-packets.  <ins> An endpoint sending 1-RTT packets MUST use different Connection IDs on different paths, if non-zero-length Connection IDs are used. Still, the receiver may observe the same Connection ID used on different 4-tuples due to, e.g., NAT rebinding.  In such case, the receiver reacts as specified in Section 9.3 of QUIC-TRANSPORT. </ins> 6.1.


1-RTT packets use DCID S3, which has a sequence number of 3.  Note that the paths use different packet number spaces.  In this case, the client is going to close the first path.  It identifies the path by <del> the sequence number of the received packet's DCID over that path, hence using the path_id 1.  Optionally, the server confirms the path closure by sending an PATH_ABANDON frame using the sequence number of the received packet's DCID over that path as path identifier, which corresponds to the path_id 2.  Both the client and the server can close the path after receiving the RETIRE_CONNECTION_ID frame for </del>  that path. 7.

Since , we now define the Path ID as (Section 1.1) And the definition of the as the field of PATHABANDON and PATHSTATUS frames directly maps to that identifier path (i.e., the sequence number of the destination Connection ID used for sending packets on that particular path, Sections 8.1 and 8.2). But in Section 6.2, the example states that I believe the current text is unclear and should be fixed. Yet, I see two possible ways of doing so. We should either state that the field of these frames refer to the Path ID used by the receiver of the frame, or add some information to indicate that it refers to the Path ID viewed from the sender's/receiver's viewpoint (that could be encoded as a bit in the frame's type)
Is there any use case for either using the sender's or receiver's ID? If not, I'd prefer we simply choose one and define it clearly.
(P.S.: I think the text in the example is just wrong. I think I messed this up in .)
IMHO it makes more sense to me to use the sender's ID by default, especially if we consider only client-initiated paths.
I would also suggest to slightly update the text related to the field of the PATHABANDON and PATHSTATUS frames to indicate it refers to the sender's Path ID.LGTM

1-RTT packets use DCID S3, which has a sequence number of 3.  Note that the paths use different packet number spaces.  In this case, the client is going to close the first path.  It identifies the path by  <ins> the sequence number of the DCID it uses for sending packets over that path, hence using the path_id 2.  Optionally, the server confirms the path closure by sending an PATH_ABANDON frame using the sequence number of the DCID it uses to send over that path as path identifier, which corresponds to the path_id 1.  Both the client and the server can close the path after receiving the RETIRE_CONNECTION_ID frame for </ins> that path. 7.


connection IDs for distinguishing packets sent to that node over different paths.  This proposal does not cover address discovery and management. Addresses and the actual decision process to setup or tear down paths are assumed to be handled by the application that is using the QUIC <del> multipath extension.  Further, this proposal only specifies a simple basic packet scheduling algorithm, in order to provide some basic </del>  implementation guidance.  However, more advanced algorithms as well as potential extensions to enhance signaling of the current path state are expected as future work.

This draft only enables clients to open a new path, so the most common scenario is that the client has another interface and opens a new path from a new IP address. However, if the client knows another IP that the server is listing on to accept new paths for an existing quic connection (e.g. provided on the higher layer or out of band), that can be used as well. It would be useful to discussion this case more in the draft to ensure that it is clear that this case is supported.
I think as people who implement multipath will mostly refer to this draft, we may want to clarify all scenarios supported by this draft: (1) client has multiple IPs while server listens on single IP, (2) client uses single IP while server listens on multiple IP, and (3) client uses multiple IPs and the server also uses multiple IPs.
LGTM

connection IDs for distinguishing packets sent to that node over different paths. <ins> Each endhost may use several IP addresses to serve the connection. In particular, the multipath extension supports the following scenarios. The client uses multiple IP addresses and the server listens on only one. The client uses only one IP address and the server listens on several ones. The client uses multiple IP addresses and the server listens on several ones. </ins> This proposal does not cover address discovery and management. Addresses and the actual decision process to setup or tear down paths are assumed to be handled by the application that is using the QUIC  <ins> multipath extension.  This is sufficient to address the first aforementioned scenario.  However, this document does not prevent future extensions from defining mechanisms to address the remaining scenarios.  Further, this proposal only specifies a simple basic packet scheduling algorithm, in order to provide some basic </ins> implementation guidance.  However, more advanced algorithms as well as potential extensions to enhance signaling of the current path state are expected as future work.


5. <del> The simultaneous usage of several sending uniflows introduces new </del>  algorithms (packet scheduling, path management) whose specifications are out of scope of this document.  Nevertheless, these algorithms are actually present in any multipath-enabled transport protocol like

This is really a minor issue, but the word "uniflow" appears just once in the draft, in section 5, packet scheduling. I think the packet scheduling text could simply use the word "path".

5.  <ins> The simultaneous usage of several sending paths introduces new </ins> algorithms (packet scheduling, path management) whose specifications are out of scope of this document.  Nevertheless, these algorithms are actually present in any multipath-enabled transport protocol like


4.3.3. <del> Receiving a RETIRE_CONNECTION_ID frame causes the endpoint to discard the resources associated with that connection ID.  If the connection ID was used by the peer to identify a path from the peer to this endpoint, the resources include the list of received packets used to send acknowledgements.  The peer MAY decide to keep sending data </del>  using the same IP addresses and UDP ports previously associated with <del> the connection ID, but MUST use a different connection ID when doing so. Note that if the sender retires a Connection ID that is still used by in-flight packets, it may receive ACK_MP frames referencing the retired Connection ID.  If the sender stops tracking sent packets with retired Connection ID, these would be spuriously marked as lost. To avoid such performance issue without keeping retired Connection ID </del>  state, an endpoint should first stop sending packets with the to-be- retired Connection ID, then wait for all in-flight packets to be either acknowledged or marked as lost, and finally retire the

After author discussion: we need to also add a sentence about control frames
This seems ready to merge. Plan is to merge end of tomorrow (Friday).
Currently it seems that the assumption is that the Path ID changes if the CID changes, however, this makes handling of MP frames that contain the path ID more complicated because you have to remember the old path ID to process these frames even when the CID is retired. Is there any good reason why we don't keep the Path ID constant over the life time of a path?
Also note if you retire an CID (e.g. because it's requested by the peer in the NEWCONNECTIONID Frame Retire Prior To field) and you don't have a new CID available anymore (yes, this should usually not happen), you cannot send a PATH_ABANDON frame anymore because you don't have valid Path ID (and we don't have the type field anymore).
The sequence number is tied to the CID, because of encryption. If we tie it to something else, we are just moving the problem, because then we have to worry about mapping the CID to a path ID. Yes, this has a couple of constraints. If a CID is actually retired, there is not way to acknowledge the packets that were sent using that CID. Effectively, they are treated as lost. The obvious solution is to write "don't do that" guidelines. Basically, wait a couple RTO before retiring a CID, so the ACK have time to arrive. The other constraint is that ACK-MP carry the ID used when sending the packets. This is not a huge constraint, but it is an exception to the proposed "same path for ACK" policy. But the main argument for that same path policy is about computing timers. That argument does not really apply when renewing the CID. Same issue for the path abandon. If packets are sent on the same four tuple with a new CID, just abandon that.
In addition to the encryption consideration as mentioned above. I remember there was another reason why the sequence number of CID was chosen to identify a path. It brings us back to the definition of what is a path and what is a path identifier. By definition of the draft and RFC9000, a path is defined by a 4-tuple {source IP address, source port number, destination IP address, destination port number}, and a path ID is an identifier used to identify a path. That means we need to construct a mapping between a path ID and the associated 4-tuple. However, there are couple of issues. First, the client's view and the server's view of the 4-tuple are not the same due to NAT. Second, the 4-tuple can change due to (1) network path migration (think about the case where our phone's wifi is connected to a wifi router that has two outbound networks to the internet, one wireline and one satellite, and the router can switch between the two), and (2) NAT rebinding. If we use a constant path ID, then the path ID does not uniquely identify a 4-tuple, but instead, there will be a one-to-many mappings from a path ID to 4-tuples. In , Quentin proposed to use entry point IDs to identify a path, I think this identification is stable in the case of NAT rebinding, where the 4-tuple changes but the physical path is actually unchanged. However, the problem is that when the underlying path has changed (in the above case, a router may switch packet from the wireline to the satellite to perform failover) while the entry point IDs on the end-hosts remain unchanged, then we end up using the same path identifier to identify different paths. These problems can be solved when using QUIC's CID sequence number to identify a path. First, it allows the client and server to identify a path separately as they see different 4-tuples. Second, when 4-tuple changes, we eventually use new CIDs to communicate and thus, the mapping between a path ID and the 4-tuple remains as a one-to-one mapping. Third, we are logically correct in the case when the physical path indeed has migrated while the endhost interfaces are not changed.
I don't think that "this makes handling of MP frames that contain the path ID more complicated". You have a code object holding the list of packets to acknowledge (receiver side), or an object containing the list of packets not acknowledged yet (sender side). These objects are tied to the connection ID. Nothing particularly complicated there: the sender knows exactly which connection ID it uses for a given packet; the receiver knows exactly what connection ID was carried in the packets that it received. The issue that you mention comes from the ambiguity of "Abandon Path", "Retire connection ID", and "retire previous connection ID". We have to be clear about what that means: Abandon Path means "please do not send any more packets on that path". No ambiguity there. It also implies: "you can free the resource of that path at your convenience". We need to specify how and when, there is some leeway. Retire Connection ID" means "I have deleted all resource tied to the connection ID on my side", which implies "I will not use this connection ID to send any new packet" and also "I will not accept any new MP-ACK sent using the sequence number of that CID". "retire previous CID" means "please retire these CID quickly". RFC 9000 says "The endpoint SHOULD continue to accept the previously issued connection IDs until they are retired by the peer", so there is some leeway. So I guess we have the following actions: A node decides to stop using a CID, i.e., not send any more packet using that CID. The node can do that at any time on its own will, and SHOULD do that immediately after receiving an AbandonPath frame, or after receiving a "retire before" indication. The node should wait 3RTO after that decision, so pending acknowledgements have enough time to arrive. After 3RTO, the node deletes the local resource, including the list of packets not acknowledged yet. Those packets should be considered lost. The node then sends a Retire Connection ID frame. The node receiving the Retire Connection ID frame deletes the resource associated with the CID, i.e., the list of packets to acknowledge. If the peer did wait long enough before sending Retire Connection ID frame, that list should be empty. If it is not, too bad, it can still be deleted, these packets will never be acknowledged, but the sender of the CID does not care. After that, a node can still received an MP-ACK or an AbandonPath mentioning an old CID, because we do have race conditions. Such frames should be silently ignored. Does that remove the ambiguities?
I think the current specification (i.e., rely on sequence number of Destination CID to identify a path in PATHSTATUS/PATHABANDON) works fine as long as 1) the perceived 4-tuple of a network path remains stable, or 2) the DCID used on a network path remains stable. Also, if the peer has no timing restrictions about when retiring a CID, this is fine as well. However, when receiving a NEWCONNECTIONID frame with a "Retire Prior To" field forcing the receiver to remove DCIDs, RFC 9000 states that "Upon receipt of an increased Retire Prior To field, the peer MUST stop using the corresponding connection IDs and retire them with RETIRECONNECTIONID frames before adding the newly provided connection ID to the set of active connection IDs.". Not sure the receiver of the NEWCONNECTIONID may delay the retirement of DCIDs for a long period. Once retired, the host cannot reference the paths identified by the retired CIDs in, i.e., PATHABANDON frames, which may be an issue if the peer does not provide enough new CIDs. But the current path identification is fragile when one host (the client) changes the DCID used over a same perceived 4-tuple, but that the other (the server) perceives a different 4-tuple (typical case of NAT rebinding). The client will keep a single view of that path, but the server will have two different views having different 4-tuples and Path IDs for an actual same path, hence bringing path identification ambiguity here. NAME As long as the 4-tuple does not change, there is no "path change" from the QUIC viewpoint. Having "backbone router path migration" will be transparent to the endmost (although the path characteristic conditions such as the latency and bandwidth may change). The EntryPoint ID proposal is mainly to keep stable path identification in PATHABANDON/PATH_STATUS frames when 1) CIDs change and the sender's perceived 4-tuple is stable but the receiver's perceived 4-tuple is not, and 2) when there is no more usable CID over the (network) path to reference.
So the scenario that you worry about is "what if the client sleeps for a while, then decides to switch the connection ID before sending new packets". This is actually a recommended behavior, for privacy reasons, so we have better get it right. I think that the current proposal mostly works. After the migration, the new packets sent by the client will be tied to the new CID on both sides -- see previous points about tying number space to connection IDs. There is no ambiguity about packet number space, packets will be correctly acknowledged, etc. The AbandonPath or MPACK frames will not be ambiguous: they refer to the number space, not to the abstract concept of Path. The main problem is that the server will probably NOT treat the incoming packet with a new CID as a "nat rebinding". The algorithm pretty much defines NAT rebinding as "same CID, different addresses or ports". The conforming server will treat that as a new path, perform address validation, etc. If the server tries to use the old "path", the packets will be dropped by the NAT. The connection will recover eventually, but after some packet losses, and yes, that's not desirable. I think the simplest solution is for the client to somehow tell the server what it is doing. "These packets carry CID sequence number 7. For me, this is the same path as when previously using CID sequence number 4." My gut feeling is that this can be achieved by sending an "Abandon_Path(id=4, errorCode=CidRenewal)" when starting sending with CID 7. Specify something like that in the spec, document the error code, etc.
Please note that PR is related here. In this PR I propose that a new 4-tuple is always treated as a new path and therefore always triggers path validation which then ensures to create a new path on both sides. I thought a while about this scenario when working on this PR and believe that's the easiest, non ambiguous solution which I think is inline with the multipath idea. Further, I think that en-/decryption and the path ID do not need to be entangled. Of course we need to use the CID to decrypt the packet, however, I don't think that changing the CID (without changes of the 4-tuple) creates a new packet number space. This is not the case in RFC9000 and should not be the case here.
It is not necessarily true that the router path migration is always transparent. In the case when a wifi router switches between a terrestrial ISP and a satellite backbone, the outbound packet may have different source IP addresses since the satellite network and the terrestrial network are operated by different ISPs. In this case, the path switching will cause 4-tuple to change.
The difference between "new path" and "rebinding" is probably less than it sounds. In the behavior suggested by RFC 9000, arrival of packets with the same CID and a new 4-tuple triggers both validation of the new path using Path Challenge, and verification that the old path is actually gone by a parallel Path Challenge on the old path. The verification is there to deal with a potential attack in which an on path attacker (e.g., somebody on the same Wi-Fi network) can capture a 1RTT packet sent by the client, and then resent it from a spoofed IP address. I am a bit concerned that "treating NAT rebinding as a new path" will keep the Path Challenge on the new path, but not the verification attempt of the old path, and thus expose us to the attack that the verification is meant to mitigate.
NAME when you say "a wifi router switches between a terrestrial ISP and a satellite backbone" is that you want the concept of path in QUIC to follow the topology changes in the network. But we should recognize that this will not always be possible. For example, routing changes can always cause a single 4-tuple to be suddenly routed in a very different way.
I think if the 4-tuple does not change, we do not pursue the goal of following the topology change as it is basically not possible. However, my point is when there is a topology change and a 4-tuple change (now we have a detectable signal), we probably do not want to treat it as a NAT rebinding. As we are actually on a different path, retaining the congestion control state and round-trip estimate does not seem to be right. A more reasonable action is to treat it as a new path and reset the congestion controller, right?
Exactly. I think the proposed approach here could keep changes minimal, although this is not yet perfect. If the client sends two packets with the new CID, but the first one containing such information is lost (but not the second one, which experiences NAT rebinding), then we still face the issue. Of course, we will eventually recover the situation, but there will be a transient state here. Maybe, but I'm not convinced we should take this assumption as granted. More generally, I start thinking about possible security issues we may encounter with this, where the client may frequently rotate the DCID it uses over a path for legitimate purposes, but an on-path attacker may tweak the 4-tuple, hoping it will make it create a lot of (invalid) paths' states at server side. Not sure this will be a strong issue, but we will need to document that at some point. I think in case we notice a path "migrated" (same CID, different 4-tuple), we can apply the "RFC9000 connection migration" feature on a per-path level. In such case, we can follow Section 9.4 of RFC9000, stating that congestion control/RTT must be reset, unless it experiences a port-only change (reset is not mandatory in such cases).
Let's take a step back. Using the current path identifier mechanism, we are fine if (1) CID changes but tuple does not change and (2) tuple changes but CID does not change. The only difficult case is when the client uses a new CID and the tuple changes (NAT rebinding) at the same time. Note that a NAT rebinding is improbable if packets were recently received on the old path (also see section ). A NAT rebinding mostly happens if a path is idle for some time, but that is already managed by the : "Hosts SHOULD stop sending traffic on a path if for at least the period of the idle timeout as specified in of ] (a) no non-probing packet was received or (b) no non-probing packet sent over this path was acknowledged, but MAY ignore that rule if it would disqualify all available paths." In other word, if a path is idle long enough such that the 4-tuple is changed by NAT, it should have already been closed except there is only one path left. Therefore, the chance that the server sees (1) >=2 active paths and (2) a packet that has a new CID and a new 4-tuple at the same time is low. When the server sees (1) >=2 active paths and (2) a packet that has a new CID and a new 4-tuple, the issue is that we don't know which path the packet belongs to. Hence, we just treat it as an attempt for a new path as NAME suggested. However, in the current draft, when the client initiates a new path, the packet should contain a PATHCHALLENGE frame (see ). If the packet does not contain PATHCHALLENGE, we should just ignore the packet, which eventually leads to path closure as specified by the current draft (see ). Note that in doing so, we are also resilient to the attacks mentioned by NAME that "the client may frequently rotate the DCID it uses over a path for legitimate purposes, but an on-path attacker may tweak the 4-tuple, hoping it will make it create a lot of (invalid) paths' states at server side". When the server sees (1) only 1 active path and (2) a packet of a new CID and a new 4-tuple but that packet does not have PATH_CHALLENGE, it knows which path the packet belongs to, and we just follow what single path QUIC does and the behavior of multipath quic and single path quic converge in this case. Therefore, I think the current draft can already address the issue, and we probably just want to stick to what we have with some minor modifications.
If the server sees a packet with a new CID and new 4-tuple but without path challenges it should ignore it as it can't associate it with an existing path. If the client switches to a new path (new 4-tuple) it has to use a new CID and send a path challenges frame. The problem I'm trying to address are cases where the CID (and path ID) changes for whatever reason and we received some valid frames with the old path ID that we can't associate to anything anymore and therefore (unnecessarily) need to drop. Yes, this should not happened that often, so dropping is not the worst option, however, I think having a stable path ID would also make the whole approach logically easier, but maybe that's something we need to discuss and agree on. Note that if we change the path ID, that automatically means that we have to silently ignore any unknown path ID as we never know for sure if it was an old path ID or really a completely invalid one. Otherwise we could create an error if we receive an invalid path ID. This could help to avoid that the end get out of sync. However, not sure if that is actual a problem that needs solving. In both cases I think we definitely need to clarify some things in the draft and make sure the taken approach is crystal clear to the reader.
When to reset the congestion controller? That's a very generic problem, and the IETF is creating a WG to study this kind of issues. I think we should not try to invent a multipath specific solution, and just stick with whatever RFC 9000 says now, or what new RFCs will say later.
The more I read this discussion, the more I think we should stick to the path handling specification in RFC 9000. Do a in the archive of QUIC GitHub issues, and you find a big list of messages. Lots of history there, with discussion of congestion control, security issues, DOS attacks, etc. Of course, we could have these discussions again in the multipath context, but we would converge much faster if we did not!
We should not change the guidance on congestion control in RFC9000. However, otherwise I don't really understand what you mean by "stick to path handling in RFC9000". RFC9000 only has one path at a time, however, this is exactly the part that this extension is changing, so I don't see it as a deviation from the migration handling in RFC9000. Again we have to make a decision and define it clear but wouldn't one or the other solution is more inline with RFC9000 or not.
Looking back at Mirja's original question, "Currently it seems that the assumption is that the Path ID changes if the CID changes, however, this makes handling of MP frames that contain the path ID more complicated because you have to remember the old path ID to process these frames even when the CID is retired. Is there any good reason why we don't keep the Path ID constant over the life time of a path?" I think we should just bite the bullet, and acknowledge that retiring a CID makes it unusable in the future -- MP frames mentioning the corresponding ID will just be silently ignored. Yes, this can cause for example some spurious packet loss detection, but that can be minimized with strict rules about when to retire a CID -- the various RTO guidelines that we discussed in other threads. Better expose the consequence clearly, so nodes that think of retiring a CID think twice.
As for "same as RFC 9000", yes we will of course have one difference, since we handle multiple parallel paths. I am really looking at the reactions to NAT rebinding, how we define what is a NAT rebinding and what is not, etc.
I think we can use path ID mechanism of the current draft with the addition of the following rules to address the issues discussed in this thread: (1) Addressing the corner case when new CID and NAT rebinding happen at the same time. When an endpoint receives a packet that has (new CID, new tuple), check if the packet is trying to initialize a new path (i.e., whether it contains a PATHCHALLENGE). If it has PATHCHALLENGE, try creating a new path. If it has no PATHCHALLENGE and we don't know which path the packet is associated with (num. of paths>=2), discard the packet. If there is only one path, we perform path validation as single path QUIC does. (2) Addressing the issue that you cannot send PATHABANDON when you don't have CIDs available to use on a path. >Also note if you retire an CID (e.g. because it's requested by the peer in the NEWCONNECTIONID Frame Retire Prior To field) and you don't have a new CID available anymore (yes, this should usually not happen), you cannot send a PATHABANDON frame anymore because you don't have valid Path ID (and we don't have the type field anymore)." When an endpoint receives a NEWCONNECTIONID frame that has retirepriorfield set and it finds that the retirepriorto field will cause it to have no more CIDs to send packets on a particular path, before sending RETIRECONNECTIONID frame, it first sends PATHABANDON frame to signal path closure to the peer.
NAME I think I can live with your proposed rules. Adding some text stating that a client SHOULD include a PATHCHALLENGE frame when it changes the DCID it uses (even if it does not change the 4-tuple used) is reasonable to address the aforementioned corner cases. I wonder though if it would make sense to define a small informative frame to indicate to the client that the new DCID seen has been rejected as it cannot map to an existing path, to suggest the client it should retire the CID and retry with a new one while bundling PATHCHALLENGE, but not a big concern though.
So, yes we can make it work with changing the path ID. But I wonder if things would be simpler if we keep it stable. So asking my initial question again: what are the drawbacks when we try to keep it stable?
I think it would be nice to have stable Path IDs. However, all the challenge resides in how we could define such stable "path identifiers", as the sending/receiving paths may use different CIDs over their lifetime and may perceive different 4-tuples.
That is a good point. The fundamental issue is that in reality a path is only precisely defined by the sequence of every possible hops between two endpoints, but that can not be observed by the endpoint, so we have to use the 4-tuple definition of a path as a simplified and compromised notation that unfortunately does not capture the full picture. QUIC goes around this issue with the use of connection ID, so that we are not bothered by unstable path identifiers and can enable path migration. Therefore, I feel in multipath QUIC, we may want to inherit what QUICv1 does and reuse the CID mechanism to identify paths.
Created to address the new CID and NAT rebinding case.
RFC 9000 defines a way to request a peer to retire old CIDs, but only in-sequence (i.e., all CIDs with a sequence number less than n). This doesn't work when multiple CIDs are in use over different paths and you only want to retire one of them. (issue originally raised by NAME
I think you actually mean the Retire Prior To field in the NEWCONNECTIONID frame? In the RETIRECONNECTIONID frame, the sequence number field is the connection ID being retired, so it should be fine, but yes, in the context of multipath, we should apply Retire Prior To field in NEWCONNECTIONID with caution. I am wondering shall we say that when multipath is in use, an endpoint SHOULD NOT use Retire Prior To when issuing a new connection ID, but instead, retire CID one by one with the use of RETIRECONNECTIONID frame?
Yes, I mean that field. Note that the Retire Prior To field is a request to the peer to use RETIRECONNECTIONID frames to retire CIDs issued by the first party. This means that you can not replace the Retire Prior To field with RETIRECONNECTIONID frames as you suggest, since they work in "different directions". URL
Yes, there is no direct relation between the ordering in which you receive the CIDs and the order of removing them in a multipath context. I'm also not sure that an endhost is required to use available DCIDs "in-sequence" (i.e., it may use the CID with seq 2 before using the CID with seq 1). Basically, let's consider the following example. Assume a two path scenario where the client uses DCID seq 0 on a path A and DCID seq 2 on a path B. At some point (e.g., the server might notice a NAT rebinding on path B), the server would like the client to retire the DCID with seq 2. The only way the server could force this behaviour is to send NEWCONNECTIONID frames with Retire Prior To set to 3, which would force the client to send RETIRE_CID frames. However, doing so would also force the client to retire DCID seq 0, which might not be required/needed/wanted. FWIW, proposed to let the CID issuer split the CIDs into "uniflow spaces", i.e., pre-assign CIDs to specific paths. Such a design enables retiring a specific CID using the Retire Prior To without affecting the other paths, as the CID seq is path-specific. However, such a solution adds more complexity than the current draft, which is likely not desirable. In the end, I'm not sure with the current proposal the aforementioned issue is a big one, but it would indeed deserve some documentation/discussion in the draft.
Actually if the other peer asks you to retire a CID that is currently in use, you can simply use a new CID on the path (this is true for both the uni- as well as multi-path case). As such this is also related to issue and we should clarify these things.
I created PR to clarify what happens if no new CID is available. Maybe this helps here as well?
Depending on the CID sequence numbers, it may be necessary for the peer to ask you to retire the CIDs also on some or all other paths (and thus also generate, install and signal a number of new CIDs) even if it just wants you to retire it on one of them. This is clearly suboptimal.
I don't think that's problem. The question why would the CID issuer ask you to retire a CID. Usually this is probably some kind of clean up and in this cases it's actually the purpose that all old CID are replaces with newer ones. I believe this simply how it is intended. However, sure how this is actually implemented and used at the moment.
Fixed by that got merged.
When using Identifier Types 0 or 1, the current text does not discuss how we should handle related path Identifier that the receiver cannot map to a path. When using Identifier Type 1, the Path Identifier corresponds to the sequence number of the Destination CID used by the sender to send packets on the related path, hence a sequence number of a source Connection ID of the receiver. When using Identifier Type 0, the Path Identifier corresponds to the sequence number of the Source CID of the sender, hence the sequence number of the Destination CID used by the receiver to send packets on the related path. While there are obvious values that could be considered as bad and may lead to, e.g., a MPPROTOCOLVIOLATION (e.g., a sequence number larger than any sequence number advertised), I don't think we should be as radical with regular values as they might have been retired meanwhile (also possible to face this with packet reordering). The worst edge cases I see are the followings. With type 1, the sender might send a PATHABANDON with the seq of the CID in one packet, and then the RETIRECID retiring that CID in a subsequent one, but the RETIRECID reaches the receiver before the PATHABANDON one. If the receiver drops state upon RETIRECID reception, it won't be able to map the PATHABANDON to the path. With type 0, the sender might send a PATHABANDON with the seq of the CID, but before it reaches the peer, the peer sends a RETIRECID retiring that CID. If the receiver drops the CID state upon RETIRECID sending, it won't be able to map the PATHABANDON to the path. Handling these edge cases with connection closure is clearly too strong. Still, ignoring or even acknowledging these PATHABANDON may have bad consequences. Silently dropping packets (i.e., not acknowledge the packet having PATHABANDON) will make the peer retransmit the frame (hoping it updates its path identifier) but could cause the loss/non-acknowledgment of accompanying frames (e.g., STREAM ones), and processing packet content without acknowledging it sounds odd. Acknowledging PATHABANDON that cannot be mapped to a path would make some desynchronisation about the status of the path between the peers (the sender now assumes the path is closed, but the receiver not). To address the type 1 issue, we could write some text stating that a sender cannot retire a DCID if a PATHABANDON containing the related sequence number is still in flight. But for the type 0, the fix is not obvious (I don't think we could have the same solution as in )...
One solution is to put the retired cid into a cidhistorylist and keep it for a while before its final deletion. In doing so, one can still find the cid-to-path mapping if the PATHSTATUS frame uses a retired cid. In this case, as long as the path that the peer wants to retire has no ambiguity, the current path close process can still work. If the receiver can no longer find the cid in both the active list and the history list, then it should treat this as a connection error of MPPROTOCOL_VIOLATION and close the connection.
Another solution would to to not change the path ID (see issue ). In this case all unknown path IDs could be handled as a connection error. Or is it anyway better to just ignore those frames (e.g. if completely outdated packet arrives and the path was already closed)?
I think the simplest solution is to just ignore Path Abandon that refer to abandoned connection ID sequence numbers. The problem is self correcting. If the connection ID is not used anymore, it is either because the path was actually abandoned (no problem), or because packets on the same 5-tuple are now sent with a different CID. In that case, the peer will see these packets arriving on the path that it wants to abandon, and can simply send a new Path Abandon.
Given that we agree that PATHABANDON relates to CID sequence numbers, we can probably apply the same approach to all multipath-specific frames (including ACKMP): if you receive such a frame with a CID sequence number you cannot map anymore, just ignore it.
Yes, this looks good.

4.3.3.  <ins> Receiving a RETIRE_CONNECTION_ID frame causes an endpoint to discard the resources associated with that Connection ID.  Specifically, the endpoint should not use the sequence number of the retired Connection ID anymore in any control frames, as the peer will not be able to associate those frames to a path and will therefore ignore them. This means an endpoint is also not required to acknowledge any late packets carrying that Connection ID and, hence, it can remove the list of received packets used to send acknowledgements after receiving the RETIRE_CONNECTION_ID frame. The peer, that sent RETIRE_CONNECTION_ID frame, can keep sending data </ins> using the same IP addresses and UDP ports previously associated with  <ins> that Connection ID, but MUST use a different connection ID when doing so.  If no new connection ID is available anymore, the endpoint cannot send on this path.  This can happen if, e.g., the Connection ID issuer requests retirement of a Connection ID using the Retire Prior To field in the NEW_CONNECTION_ID frame but does provide sufficient new CIDs. Note that even if a peer cannot send on a path anymore because it does not have a valid Connection ID to use, it can still acknowledge packets received on the path, by sending ACK_MP frames on another path, if available.  But also note that, as there is no valid CID associated with the path, the other end cannot send multipath control frames that contain the sequence number of a Connection ID, such as PATH_ABANDON or PATH_STATUS. If the peer cannot send on a path and no data is received on the path, the idle time-out will close the path.  If, before the idle timer expires, a new Connection ID gets issued by its peer, the endpoint can re-activate the path by sending a packet with a new Connection ID on that path. If the sender retires a Connection ID that is still used by in-flight packets, it may receive ACK_MP frames referencing the retired Connection ID.  If the sender stops tracking sent packets with retired Connection ID, these would be spuriously marked as lost.  To avoid such performance issue without keeping retired Connection ID </ins> state, an endpoint should first stop sending packets with the to-be- retired Connection ID, then wait for all in-flight packets to be either acknowledged or marked as lost, and finally retire the


If the 4-tuple changes without the use of a new connection ID (e.g.  due to a NAT rebinding), this is considered as a migration <del> event and not as a new path. </del>  The path management specified in Section 9 of QUIC-TRANSPORT fulfills multiple goals: it directs a peer to switch sending through a new paths to keep alive, it can do so by sending Ping frames on each of these paths before the idle timeout expires.  8. All the new frames MUST only be sent in 1-RTT packet, and MUST NOT

As I added this text into the implementation consideration section, I on purpose did not use normative language. However, we might want to double check that the normative language we used elsewhere in the document is covering this appropriately.
The current text is ready to merge. There is a remaining question about proposed additional text. If we don't conclude until end of tomorrow (Friday), we will merge as is (without the addition) and create a new PR/issue for that addition instead.
Currently it seems that the assumption is that the Path ID changes if the CID changes, however, this makes handling of MP frames that contain the path ID more complicated because you have to remember the old path ID to process these frames even when the CID is retired. Is there any good reason why we don't keep the Path ID constant over the life time of a path?
Also note if you retire an CID (e.g. because it's requested by the peer in the NEWCONNECTIONID Frame Retire Prior To field) and you don't have a new CID available anymore (yes, this should usually not happen), you cannot send a PATH_ABANDON frame anymore because you don't have valid Path ID (and we don't have the type field anymore).
The sequence number is tied to the CID, because of encryption. If we tie it to something else, we are just moving the problem, because then we have to worry about mapping the CID to a path ID. Yes, this has a couple of constraints. If a CID is actually retired, there is not way to acknowledge the packets that were sent using that CID. Effectively, they are treated as lost. The obvious solution is to write "don't do that" guidelines. Basically, wait a couple RTO before retiring a CID, so the ACK have time to arrive. The other constraint is that ACK-MP carry the ID used when sending the packets. This is not a huge constraint, but it is an exception to the proposed "same path for ACK" policy. But the main argument for that same path policy is about computing timers. That argument does not really apply when renewing the CID. Same issue for the path abandon. If packets are sent on the same four tuple with a new CID, just abandon that.
In addition to the encryption consideration as mentioned above. I remember there was another reason why the sequence number of CID was chosen to identify a path. It brings us back to the definition of what is a path and what is a path identifier. By definition of the draft and RFC9000, a path is defined by a 4-tuple {source IP address, source port number, destination IP address, destination port number}, and a path ID is an identifier used to identify a path. That means we need to construct a mapping between a path ID and the associated 4-tuple. However, there are couple of issues. First, the client's view and the server's view of the 4-tuple are not the same due to NAT. Second, the 4-tuple can change due to (1) network path migration (think about the case where our phone's wifi is connected to a wifi router that has two outbound networks to the internet, one wireline and one satellite, and the router can switch between the two), and (2) NAT rebinding. If we use a constant path ID, then the path ID does not uniquely identify a 4-tuple, but instead, there will be a one-to-many mappings from a path ID to 4-tuples. In , Quentin proposed to use entry point IDs to identify a path, I think this identification is stable in the case of NAT rebinding, where the 4-tuple changes but the physical path is actually unchanged. However, the problem is that when the underlying path has changed (in the above case, a router may switch packet from the wireline to the satellite to perform failover) while the entry point IDs on the end-hosts remain unchanged, then we end up using the same path identifier to identify different paths. These problems can be solved when using QUIC's CID sequence number to identify a path. First, it allows the client and server to identify a path separately as they see different 4-tuples. Second, when 4-tuple changes, we eventually use new CIDs to communicate and thus, the mapping between a path ID and the 4-tuple remains as a one-to-one mapping. Third, we are logically correct in the case when the physical path indeed has migrated while the endhost interfaces are not changed.
I don't think that "this makes handling of MP frames that contain the path ID more complicated". You have a code object holding the list of packets to acknowledge (receiver side), or an object containing the list of packets not acknowledged yet (sender side). These objects are tied to the connection ID. Nothing particularly complicated there: the sender knows exactly which connection ID it uses for a given packet; the receiver knows exactly what connection ID was carried in the packets that it received. The issue that you mention comes from the ambiguity of "Abandon Path", "Retire connection ID", and "retire previous connection ID". We have to be clear about what that means: Abandon Path means "please do not send any more packets on that path". No ambiguity there. It also implies: "you can free the resource of that path at your convenience". We need to specify how and when, there is some leeway. Retire Connection ID" means "I have deleted all resource tied to the connection ID on my side", which implies "I will not use this connection ID to send any new packet" and also "I will not accept any new MP-ACK sent using the sequence number of that CID". "retire previous CID" means "please retire these CID quickly". RFC 9000 says "The endpoint SHOULD continue to accept the previously issued connection IDs until they are retired by the peer", so there is some leeway. So I guess we have the following actions: A node decides to stop using a CID, i.e., not send any more packet using that CID. The node can do that at any time on its own will, and SHOULD do that immediately after receiving an AbandonPath frame, or after receiving a "retire before" indication. The node should wait 3RTO after that decision, so pending acknowledgements have enough time to arrive. After 3RTO, the node deletes the local resource, including the list of packets not acknowledged yet. Those packets should be considered lost. The node then sends a Retire Connection ID frame. The node receiving the Retire Connection ID frame deletes the resource associated with the CID, i.e., the list of packets to acknowledge. If the peer did wait long enough before sending Retire Connection ID frame, that list should be empty. If it is not, too bad, it can still be deleted, these packets will never be acknowledged, but the sender of the CID does not care. After that, a node can still received an MP-ACK or an AbandonPath mentioning an old CID, because we do have race conditions. Such frames should be silently ignored. Does that remove the ambiguities?
I think the current specification (i.e., rely on sequence number of Destination CID to identify a path in PATHSTATUS/PATHABANDON) works fine as long as 1) the perceived 4-tuple of a network path remains stable, or 2) the DCID used on a network path remains stable. Also, if the peer has no timing restrictions about when retiring a CID, this is fine as well. However, when receiving a NEWCONNECTIONID frame with a "Retire Prior To" field forcing the receiver to remove DCIDs, RFC 9000 states that "Upon receipt of an increased Retire Prior To field, the peer MUST stop using the corresponding connection IDs and retire them with RETIRECONNECTIONID frames before adding the newly provided connection ID to the set of active connection IDs.". Not sure the receiver of the NEWCONNECTIONID may delay the retirement of DCIDs for a long period. Once retired, the host cannot reference the paths identified by the retired CIDs in, i.e., PATHABANDON frames, which may be an issue if the peer does not provide enough new CIDs. But the current path identification is fragile when one host (the client) changes the DCID used over a same perceived 4-tuple, but that the other (the server) perceives a different 4-tuple (typical case of NAT rebinding). The client will keep a single view of that path, but the server will have two different views having different 4-tuples and Path IDs for an actual same path, hence bringing path identification ambiguity here. NAME As long as the 4-tuple does not change, there is no "path change" from the QUIC viewpoint. Having "backbone router path migration" will be transparent to the endmost (although the path characteristic conditions such as the latency and bandwidth may change). The EntryPoint ID proposal is mainly to keep stable path identification in PATHABANDON/PATH_STATUS frames when 1) CIDs change and the sender's perceived 4-tuple is stable but the receiver's perceived 4-tuple is not, and 2) when there is no more usable CID over the (network) path to reference.
So the scenario that you worry about is "what if the client sleeps for a while, then decides to switch the connection ID before sending new packets". This is actually a recommended behavior, for privacy reasons, so we have better get it right. I think that the current proposal mostly works. After the migration, the new packets sent by the client will be tied to the new CID on both sides -- see previous points about tying number space to connection IDs. There is no ambiguity about packet number space, packets will be correctly acknowledged, etc. The AbandonPath or MPACK frames will not be ambiguous: they refer to the number space, not to the abstract concept of Path. The main problem is that the server will probably NOT treat the incoming packet with a new CID as a "nat rebinding". The algorithm pretty much defines NAT rebinding as "same CID, different addresses or ports". The conforming server will treat that as a new path, perform address validation, etc. If the server tries to use the old "path", the packets will be dropped by the NAT. The connection will recover eventually, but after some packet losses, and yes, that's not desirable. I think the simplest solution is for the client to somehow tell the server what it is doing. "These packets carry CID sequence number 7. For me, this is the same path as when previously using CID sequence number 4." My gut feeling is that this can be achieved by sending an "Abandon_Path(id=4, errorCode=CidRenewal)" when starting sending with CID 7. Specify something like that in the spec, document the error code, etc.
Please note that PR is related here. In this PR I propose that a new 4-tuple is always treated as a new path and therefore always triggers path validation which then ensures to create a new path on both sides. I thought a while about this scenario when working on this PR and believe that's the easiest, non ambiguous solution which I think is inline with the multipath idea. Further, I think that en-/decryption and the path ID do not need to be entangled. Of course we need to use the CID to decrypt the packet, however, I don't think that changing the CID (without changes of the 4-tuple) creates a new packet number space. This is not the case in RFC9000 and should not be the case here.
It is not necessarily true that the router path migration is always transparent. In the case when a wifi router switches between a terrestrial ISP and a satellite backbone, the outbound packet may have different source IP addresses since the satellite network and the terrestrial network are operated by different ISPs. In this case, the path switching will cause 4-tuple to change.
The difference between "new path" and "rebinding" is probably less than it sounds. In the behavior suggested by RFC 9000, arrival of packets with the same CID and a new 4-tuple triggers both validation of the new path using Path Challenge, and verification that the old path is actually gone by a parallel Path Challenge on the old path. The verification is there to deal with a potential attack in which an on path attacker (e.g., somebody on the same Wi-Fi network) can capture a 1RTT packet sent by the client, and then resent it from a spoofed IP address. I am a bit concerned that "treating NAT rebinding as a new path" will keep the Path Challenge on the new path, but not the verification attempt of the old path, and thus expose us to the attack that the verification is meant to mitigate.
NAME when you say "a wifi router switches between a terrestrial ISP and a satellite backbone" is that you want the concept of path in QUIC to follow the topology changes in the network. But we should recognize that this will not always be possible. For example, routing changes can always cause a single 4-tuple to be suddenly routed in a very different way.
I think if the 4-tuple does not change, we do not pursue the goal of following the topology change as it is basically not possible. However, my point is when there is a topology change and a 4-tuple change (now we have a detectable signal), we probably do not want to treat it as a NAT rebinding. As we are actually on a different path, retaining the congestion control state and round-trip estimate does not seem to be right. A more reasonable action is to treat it as a new path and reset the congestion controller, right?
Exactly. I think the proposed approach here could keep changes minimal, although this is not yet perfect. If the client sends two packets with the new CID, but the first one containing such information is lost (but not the second one, which experiences NAT rebinding), then we still face the issue. Of course, we will eventually recover the situation, but there will be a transient state here. Maybe, but I'm not convinced we should take this assumption as granted. More generally, I start thinking about possible security issues we may encounter with this, where the client may frequently rotate the DCID it uses over a path for legitimate purposes, but an on-path attacker may tweak the 4-tuple, hoping it will make it create a lot of (invalid) paths' states at server side. Not sure this will be a strong issue, but we will need to document that at some point. I think in case we notice a path "migrated" (same CID, different 4-tuple), we can apply the "RFC9000 connection migration" feature on a per-path level. In such case, we can follow Section 9.4 of RFC9000, stating that congestion control/RTT must be reset, unless it experiences a port-only change (reset is not mandatory in such cases).
Let's take a step back. Using the current path identifier mechanism, we are fine if (1) CID changes but tuple does not change and (2) tuple changes but CID does not change. The only difficult case is when the client uses a new CID and the tuple changes (NAT rebinding) at the same time. Note that a NAT rebinding is improbable if packets were recently received on the old path (also see section ). A NAT rebinding mostly happens if a path is idle for some time, but that is already managed by the : "Hosts SHOULD stop sending traffic on a path if for at least the period of the idle timeout as specified in of ] (a) no non-probing packet was received or (b) no non-probing packet sent over this path was acknowledged, but MAY ignore that rule if it would disqualify all available paths." In other word, if a path is idle long enough such that the 4-tuple is changed by NAT, it should have already been closed except there is only one path left. Therefore, the chance that the server sees (1) >=2 active paths and (2) a packet that has a new CID and a new 4-tuple at the same time is low. When the server sees (1) >=2 active paths and (2) a packet that has a new CID and a new 4-tuple, the issue is that we don't know which path the packet belongs to. Hence, we just treat it as an attempt for a new path as NAME suggested. However, in the current draft, when the client initiates a new path, the packet should contain a PATHCHALLENGE frame (see ). If the packet does not contain PATHCHALLENGE, we should just ignore the packet, which eventually leads to path closure as specified by the current draft (see ). Note that in doing so, we are also resilient to the attacks mentioned by NAME that "the client may frequently rotate the DCID it uses over a path for legitimate purposes, but an on-path attacker may tweak the 4-tuple, hoping it will make it create a lot of (invalid) paths' states at server side". When the server sees (1) only 1 active path and (2) a packet of a new CID and a new 4-tuple but that packet does not have PATH_CHALLENGE, it knows which path the packet belongs to, and we just follow what single path QUIC does and the behavior of multipath quic and single path quic converge in this case. Therefore, I think the current draft can already address the issue, and we probably just want to stick to what we have with some minor modifications.
If the server sees a packet with a new CID and new 4-tuple but without path challenges it should ignore it as it can't associate it with an existing path. If the client switches to a new path (new 4-tuple) it has to use a new CID and send a path challenges frame. The problem I'm trying to address are cases where the CID (and path ID) changes for whatever reason and we received some valid frames with the old path ID that we can't associate to anything anymore and therefore (unnecessarily) need to drop. Yes, this should not happened that often, so dropping is not the worst option, however, I think having a stable path ID would also make the whole approach logically easier, but maybe that's something we need to discuss and agree on. Note that if we change the path ID, that automatically means that we have to silently ignore any unknown path ID as we never know for sure if it was an old path ID or really a completely invalid one. Otherwise we could create an error if we receive an invalid path ID. This could help to avoid that the end get out of sync. However, not sure if that is actual a problem that needs solving. In both cases I think we definitely need to clarify some things in the draft and make sure the taken approach is crystal clear to the reader.
When to reset the congestion controller? That's a very generic problem, and the IETF is creating a WG to study this kind of issues. I think we should not try to invent a multipath specific solution, and just stick with whatever RFC 9000 says now, or what new RFCs will say later.
The more I read this discussion, the more I think we should stick to the path handling specification in RFC 9000. Do a in the archive of QUIC GitHub issues, and you find a big list of messages. Lots of history there, with discussion of congestion control, security issues, DOS attacks, etc. Of course, we could have these discussions again in the multipath context, but we would converge much faster if we did not!
We should not change the guidance on congestion control in RFC9000. However, otherwise I don't really understand what you mean by "stick to path handling in RFC9000". RFC9000 only has one path at a time, however, this is exactly the part that this extension is changing, so I don't see it as a deviation from the migration handling in RFC9000. Again we have to make a decision and define it clear but wouldn't one or the other solution is more inline with RFC9000 or not.
Looking back at Mirja's original question, "Currently it seems that the assumption is that the Path ID changes if the CID changes, however, this makes handling of MP frames that contain the path ID more complicated because you have to remember the old path ID to process these frames even when the CID is retired. Is there any good reason why we don't keep the Path ID constant over the life time of a path?" I think we should just bite the bullet, and acknowledge that retiring a CID makes it unusable in the future -- MP frames mentioning the corresponding ID will just be silently ignored. Yes, this can cause for example some spurious packet loss detection, but that can be minimized with strict rules about when to retire a CID -- the various RTO guidelines that we discussed in other threads. Better expose the consequence clearly, so nodes that think of retiring a CID think twice.
As for "same as RFC 9000", yes we will of course have one difference, since we handle multiple parallel paths. I am really looking at the reactions to NAT rebinding, how we define what is a NAT rebinding and what is not, etc.
I think we can use path ID mechanism of the current draft with the addition of the following rules to address the issues discussed in this thread: (1) Addressing the corner case when new CID and NAT rebinding happen at the same time. When an endpoint receives a packet that has (new CID, new tuple), check if the packet is trying to initialize a new path (i.e., whether it contains a PATHCHALLENGE). If it has PATHCHALLENGE, try creating a new path. If it has no PATHCHALLENGE and we don't know which path the packet is associated with (num. of paths>=2), discard the packet. If there is only one path, we perform path validation as single path QUIC does. (2) Addressing the issue that you cannot send PATHABANDON when you don't have CIDs available to use on a path. >Also note if you retire an CID (e.g. because it's requested by the peer in the NEWCONNECTIONID Frame Retire Prior To field) and you don't have a new CID available anymore (yes, this should usually not happen), you cannot send a PATHABANDON frame anymore because you don't have valid Path ID (and we don't have the type field anymore)." When an endpoint receives a NEWCONNECTIONID frame that has retirepriorfield set and it finds that the retirepriorto field will cause it to have no more CIDs to send packets on a particular path, before sending RETIRECONNECTIONID frame, it first sends PATHABANDON frame to signal path closure to the peer.
NAME I think I can live with your proposed rules. Adding some text stating that a client SHOULD include a PATHCHALLENGE frame when it changes the DCID it uses (even if it does not change the 4-tuple used) is reasonable to address the aforementioned corner cases. I wonder though if it would make sense to define a small informative frame to indicate to the client that the new DCID seen has been rejected as it cannot map to an existing path, to suggest the client it should retire the CID and retry with a new one while bundling PATHCHALLENGE, but not a big concern though.
So, yes we can make it work with changing the path ID. But I wonder if things would be simpler if we keep it stable. So asking my initial question again: what are the drawbacks when we try to keep it stable?
I think it would be nice to have stable Path IDs. However, all the challenge resides in how we could define such stable "path identifiers", as the sending/receiving paths may use different CIDs over their lifetime and may perceive different 4-tuples.
That is a good point. The fundamental issue is that in reality a path is only precisely defined by the sequence of every possible hops between two endpoints, but that can not be observed by the endpoint, so we have to use the 4-tuple definition of a path as a simplified and compromised notation that unfortunately does not capture the full picture. QUIC goes around this issue with the use of connection ID, so that we are not bothered by unstable path identifiers and can enable path migration. Therefore, I feel in multipath QUIC, we may want to inherit what QUICv1 does and reuse the CID mechanism to identify paths.
Created to address the new CID and NAT rebinding case.
RFC9000 allows endpoints to change the CID any time on a path. So it's not automatically a new path if a new CID is seen. This should be noted explicitly in the draft. Further, also migration can happened without endpoint control (NAT rebinding). In this case the server will see a new 4-tuple but no CID change. In line with RFC9000 the server should start path validation and then the client should use a new CID. In case of multipath support we need to clarify if and when this is treated as a new path. If seen as a new path, we would probably need to close the old one (which however is not functional anymore). However, if e.g. only the port number changes, RFC9000 also would also to not reset the congestion control. In this case it would make sense to not treat it as a new path.
I agree. I think we might add sth like: New CID ---yes---> new 4-tuple ---yes---> new path | No No | | ---> use new CID on same path | ----->4-tuple change----yes----> only port number change ----yes----> NAT rebinding ( do not rest CC) | No No | | -----> connection migration & reset CC | -------> no change
NAME I think your decision tree above is too brittle. In particular, if an implementation has a time-based CID change policy, it's very easy that both the CID and the NAT binding change at the same time after an idle period. As I have indicated elsewhere, I think that there should be a PATHSETUP (or maybe PATHMANAGEMENT) frame, where path setup, CID change etc can be explicitly signaled. Implicit path setups and CID changes are just too vague and error prone...
I created PR that states that any path validation creates a new path (even if only the port changes). I believe this is more clear and avoids inconsistencies on both sides. However, we might discuss this case a bit more.

If the 4-tuple changes without the use of a new connection ID (e.g.  due to a NAT rebinding), this is considered as a migration  <ins> event. </ins> The path management specified in Section 9 of QUIC-TRANSPORT fulfills multiple goals: it directs a peer to switch sending through a new paths to keep alive, it can do so by sending Ping frames on each of these paths before the idle timeout expires. <ins> 7.7. Section 5.1.2 of QUIC-TRANSPORT indicates that an endpoint can change the Connection ID it uses for to another available one at any time during the connection.  As such a sole change of the Connection ID without any change in the address does not indicate a path change and the endpoint can keep the same congestion control and RTT measurement state. While endpoints assign a Connection ID to a specific sending 4-tuple, networks events such as NAT rebinding may make the packet's receiver observe a different 4-tuple.  Servers observing a 4-tuple change will performs path validation (see Section 9 of QUIC-TRANSPORT).  If path validation process succeeds, the endpoints set the path's congestion controller and round-trip time estimator according to Section 9.4 of QUIC-TRANSPORT. Section 9.3 of QUIC-TRANSPORT allows an endpoint to skip validation of a peer address if that address has been seen recently.  However, when the multipath extension is used and an endpoint has multiple addresses that could lead to switching between different paths, it should rather maintain multiple open paths instead. If an endpoint uses a new Connection ID after an idle period and a NAT rebinding leads to a 4-tuple changes on the same packet, the receiving endpoint may not be able to associate the packet to an existing path and will therefore consider this as a new path.  This leads to an inconsistent view of open paths at both peers, however, as the "old" path will not work anymore, it will be silently closed after the idle timeout expires. </ins> 8. All the new frames MUST only be sent in 1-RTT packet, and MUST NOT


This requires changes to the ACK frame as well as packet protection as described in the following subsections.  5.1. The ACK_MP frame, as specified in ack-mp-frame, is used to

A new sentence is added to clarify that when a new CID is used, packet number space's largest_ack should be reset to None for the packet number encoding algorithm specified in RFC9000.
Based on discuss in we need to clarify if the packet number is reset if the CID changes. If only the CID changes but not the 4-tuple, the 4-tuple can be used to map to a path and packet number space. If both changes (the client reset the CID after an idle period and on the same packet a NAT rebinding happens), I guess you can/have to try the exiting packet numbers first... however, this is anyway a case that needs more discussion.
In the presence of NAT, we cannot use the 4 tuple to map to a path and packet number space. Not on the receive side. If we want to not deviate from RFC 9000, then we ought to bring back the single number space design and make it work. I think I should write an experimental draft to do explain how. My implementation is pretty close to parity between single space and multiple space, but I never spent the time to completely explain the ACK sending logic and the loss detection logic, including how to properly estimate per path delays. And I would need to add an MP_ECN frame so we get proper ECN counts per path. I gave up on this in the interest of converging soon on an agreed solution. But instead, we are going in the weeds, revisiting past decisions. I guess I could just as well restart the single number design.
I don't think we revisit past decision (or where was that decided?). This was so far underspecified in the spec and we need to clarify. I don't have a strong option on either approach but I honestly don't know what the best approach is.
This goes back to the decoding argument. When the CID changes, the sender cannot always predict that the packet will be received on the same 4 tuple as packets sent before the change -- a NAT in the way may have a different idea. That means the receiver cannot properly find the max PN used on the receiving path from the 4 tuple alone. Thus you must start a new numbering space, with the highest received PN at 0.
The weed part is because a lots of the comments I see try a different model than either unique number space or CID based number space. They assume what I would call a "strong path ID" model, in which the path has a unique ID defined in a protocol exchange, and then we get a number space per path. But that strong path ID model will not work unless we change the packet header to insert a path ID. It also does not support NAT very well. Yet, the proposals keep coming, and they have to be explained out each time. This is slowing consensus a lot.
I agree that we should start packet number from 0 when we start a new number space. It is more robust and less error prone, and we probably want to clarify this in the draft.
Sorry to have raised this again but I actually didn't recognise that we discussed this in detail. I actually hope this is not slowing consensus but helps to make progress quickly. Again even if we discussed is before, we need to clearly define and explain this in the draft and in order conclude this discussion and avoiding coming it up again. So let's do that! I optimistic we can reach agreement quickly!
A few more thoughts, just for completeness, not trying to push in any direction. I don't think we necessarily need to couple the question of reseting the packet number and using a stable path id. Resetting the packet number with a new CID makes decryption easier, however, after you have decrypted the packet you may still be able to assign to an existing path. Actually it is not a problem neither for decryption nor for path assignment, if only the 4-tuple or the CID changes. The problem is that if the client changes the CID, it never can be sure if not also the 4-tuple changes (which can even be likely after an idea period). Therefore the client need to reset the packet number for easy decryption when it uses a new CID (this is deviation from RFC9000 - just to note but that's probably fine). However, to my understanding just changeing the CID and reseting the packet number without a change in 4-tuple doesn't necessarily mean that this will create a new path, e.g. the server doesn't have to start validation and you don't have to close the old path as it is the same. So while you need to update your mapping from CID to path ID, I don't think the path ID has to change. This is the discussion we should have in .
One addition: decoupling the CID/packet number space from the path ID however means that we need to also have a packet number space id for the MP_ACK frame, while using the path id for the other frames. Actually I think this is not too bad because ACK frames contain information that relate to packets and the other frames contain information that relate to the path.
Discussion at author meeting: just add one sentence to clarify that packet number is changed/reset when a new CID is used (even on the same path)
Add PR .
Please use Quentin's text!

This requires changes to the ACK frame as well as packet protection as described in the following subsections. <ins> When multipath is negotiated, each Destination Connection ID is linked to a separate packet number space.  Each CID-specific packet number space starts at packet number 0.  When following the packet number encoding algorithm described in Section A.2 of QUIC-TRANSPORT, the largest packet number (largest_acked) that has been acknowledged by the peer in this new CID's packet number space is initially set to "None". </ins> 5.1. The ACK_MP frame, as specified in ack-mp-frame, is used to