Path Identifier (Path ID): An identifier that is used to identify a path in a QUIC connection at an endpoint.  Path Identifier is <del> used in multi-path control frames (etc.  PATH_STATUS frame) to </del>  identify a path.  By default, it is defined as the sequence number of the destination Connection ID used for sending packets on that particular path, but alternative definitions can be used if the This proposal adds one multi-path control frame for path management: <del> PATH_STATUS frame for the receiver side to claim the path state and preference </del>  All the new frames are sent in 1-RTT packets QUIC-TRANSPORT. 3.2. <del> An endpoint uses PATH_STATUS frames to inform that the peer should send packets in the preference expressed by these frames.  An </del>  endpoint uses the sequence number of the CID used by the peer for <del> PATH_STATUS frames (describing the sender's path identifier). </del>  <del> PATH_STATUS frame describes 4 kinds of path states: Abandon a path, and release the corresponding resource. Mark a path as "available", i.e., allow the peer to use its own logic to split traffic among available paths. Mark a path as "standby", i.e., suggest that no traffic should be sent on that path if another path is available. Mark the priority of a path, i.e, path 1 is weight 8, path 2 is weight 2, suggest that path 1 has higher priority than path 2, and peer should try to send more data in path 1. PATH_STATUS frame can be sent on any path, not only the path </del>  identified by the Path Identifier field. 3.3. An endpoint that wants to close a path SHOULD NOT rely on implicit signals like idle time or packet losses, but instead SHOULD use <del> explicit request to terminate path by sending the PATH_STATUS frame. </del>  3.3.1. <del> Both client and server can close a path, by sending PATH_STATUS frame which abandons the path with a corresponding Path Identifier.  Once a path is marked as "abandoned", it means that the resources related to the path can be released. </del>  3.3.2. that are not used anymore, and the server is supposed to provide replacements, as specified in QUIC-TRANSPORT. <del> In this example, if the client wants to send a PATH_STATUS frame to tell the server that it prefers the path with CID sequence number 1 (of the server's side), the client should use the identifier of the server (sequence 1) in PATH_STATUS frame. </del> 7.2. fig-example-path-close illustrates an example of path closing.  In number of 2.  For the second path, the server's 1-RTT packets use DCID C2, which has a sequence number of 2; the client's 1-RTT packets use CID S3, which has a sequence number of 3.  Note that two paths <del> use different packet number space.  (For the convience of distinguishing the CID sequence number and PATH_STATUS sequence number, we call the "PATH_STATUS sequence number" as "PSSN".) </del>  In scenarios such as client detects the network environment change (client's 4G/Wi-Fi is turned off, Wi-Fi signal is fading to a 9.1. <del> PATH_STATUS Frame are used by endpoints to inform the peer of the </del>  current status of one path, and the peer should send packets according to the preference expressed in these frames.  Endpoint use <del> the sequence number of the CID used by the peer for PATH_STATUS frames (describing the sender's path identifier).  PATH_STATUS frames are formatted as shown in fig-path-status-format. </del>  <del> PATH_STATUS frames contain the following fields: </del>  Path Identifier: An identifier of the path, which is formatted as shown in fig-path-identifier-format. Path Identifier Content: A variable-length integer specifying the path identifier. <del> Note: If the receiver of the PATH_STATUS frame is using non-zero </del>  length Connection ID on that path, endpoint SHOULD use type 0x00 for path identifier in the control frame.  If the receiver of the <del> PATH_STATUS frame is using 0-length Connection ID, but the peer is </del>  using non-zero length Connection ID on that path, endpoints SHOULD use type 0x01 for path identifier.  If both endpoints are using 0-length Connection IDs on that path, endpoints SHOULD only use type 0x02 for path identifier. <del> Path Status sequence number: A variable-length integer specifying the sequence number assigned for this PATH_STATUS frame.  There is a different path status sequence number space for each path. Available values of Path Status field are: 0: Abandon 1: Standby 2: Available </del>  <del> If the value of Path Status field is 2 ("available"), the receiver side can use the Path Priority field to express the priority weight of a path for the peer. </del>  <del> Frames may be received out of order.  A peer MUST ignore an incoming PATH_STATUS frame if it previously received another PATH_STATUS frame for the same Path Identifier with a sequence number equal to or higher than the sequence number of the incoming frame. </del>  <del> PATH_STATUS frames SHOULD be acknowledged.  If a packet containing a PATH_STATUS frame is considered lost, the peer should only repeat it if it was the last status sent for that path - as indicated by the sequence number. </del>  9.2.

Simplify pathstatus frame with pathabandon frame following the discussion on issue. Error code and Reason Phrase are left TBD.
I actually think we should have keep stand-by. I think this was useful in MPTCP and it's not really related to scheduling. Stand-by only means: I won't use this path but please keep it open.
The current draft says "Servers SHOULD stop sending traffic on paths through which non-probing packet was received in the last 3 path RTTs, but MAY ignore that rule if it would disqualify all available paths". First, I believe there is a typo, where the "non-probing packet" should be "no non-probing packet". Second, 3 RTT is too short for our applications and we prefer a longer timeout. Maybe we want to add a new parameter (e.g. maxpathidle_timeout) to allow the server and client to negotiate?
I fixed the missing no. I took this text from Christian's draft. So maybe he can comment. NAME ?
I would suggest to leave the path closing problem for further work and indicate that the next version of the draft will define how a path can be closed by one of the endpoints.
There are two issues, path closing and packet scheduling. For path closing, the simplest solution is to use an idle timer, either the idle timer negotiated for the connection or a new parameter negotiated as part of multipath setup. For packet scheduling, the first rule is to not send data on paths that do not work, e.g., lots of packet losses, or maybe absence of ACK. This is basically the same issue as determining whether a connection is broken, just applied to a path. But that is not entirely sufficient, because the server needs to also infer whether the client still wants to use that path. The monitoring of non probing packets is just an heuristic. The server observes that if the client is not using the path, and deduces that it should not be using it itself. That's not a bad heuristic, but it requires the client to send some kinds of "keep alive" if it wants the server to continue using a path. We may debate how often that should happen. Maybe reuse the same timeout as for jettisoning paths. Another solution may be for the client to tell something like "please don't use that path anymore", but that requires adding a new frame as part of the core multipath specification. In draft-liu, this was done with a specialization of the "path status", setting the status to "Abandon". Maybe we should just use a simplified for of the PATH_STATUS, leaving out the complexities of managing priorities for the advanced scheduling drafts:
I prefer the second approach that Christian proposed, i.e., sending explicit signals like "abandon". But there is an issue with path id type 0x02 in the current draft, which refers to the path over which the control frame is sent or received. The current draft says " If both endpoints are using 0-length Connection IDs on that path, endpoints SHOULD only use type 0x02 for path identifier", so in this case, if a path is broken, we cannot send an explicit "abandon" signal for that path thru other paths, and we have to rely on implicit signals like an idle timeout. That means we may still want to negotiate the idle timer?
The client can always force the issue by dropping all packets received on an old path. They will not be acknowledged, and the server will automatically deduce that the path does not work. Of course, that is ugly...
Note that the "Abandon Path" would be a replacement or a simplification of the "Path Status" frame in the current draft. In my code, I only use the "Abandon" variant of the Path Status. I wonder whether we need alternative statuses like "standby", and whether we should define priorities here, rather than in a separate draft for advanced scheduling.
Yes, I agree. We should put the alternative statuses and priority in a separate draft on advanced scheduling.
I submitted PR on this issue.
This PR was merged.
Look good to me.I made a typo, see comment on closing this path. Apart from that, looks good.

Path Identifier (Path ID): An identifier that is used to identify a path in a QUIC connection at an endpoint.  Path Identifier is  <ins> used in multi-path control frames (etc.  PATH_ABANDON frame) to </ins> identify a path.  By default, it is defined as the sequence number of the destination Connection ID used for sending packets on that particular path, but alternative definitions can be used if the This proposal adds one multi-path control frame for path management:  <ins> PATH_ABANDON frame for the receiver side to abandon the path </ins> All the new frames are sent in 1-RTT packets QUIC-TRANSPORT. 3.2.  <ins> The current draft defines PATH_ABANDON frame to inform the peer to abandon a path, and release the corresponding resources.  More complex path management can be made possible with additional extensions (e.g., PATH_STATUS frame in I-D.liu-multipath-quic ).  An </ins> endpoint uses the sequence number of the CID used by the peer for  <ins> PATH_ABANDON frames (describing the sender's path identifier). </ins>  <ins> PATH_ABANDON frame can be sent on any path, not only the path </ins> identified by the Path Identifier field. 3.3. An endpoint that wants to close a path SHOULD NOT rely on implicit signals like idle time or packet losses, but instead SHOULD use  <ins> explicit request to terminate path by sending the PATH_ABANDON frame. </ins> 3.3.1.  <ins> Both client and server can close a path, by sending PATH_ABANDON frame which abandons the path with a corresponding Path Identifier. Once a path is marked as "abandoned", it means that the resources related to the path can be released. </ins> 3.3.2. that are not used anymore, and the server is supposed to provide replacements, as specified in QUIC-TRANSPORT.  7.2. fig-example-path-close illustrates an example of path closing.  In number of 2.  For the second path, the server's 1-RTT packets use DCID C2, which has a sequence number of 2; the client's 1-RTT packets use CID S3, which has a sequence number of 3.  Note that two paths  <ins> use different packet number space. </ins> In scenarios such as client detects the network environment change (client's 4G/Wi-Fi is turned off, Wi-Fi signal is fading to a 9.1.  <ins> PATH_ABANDON Frame are used by endpoints to inform the peer of the </ins> current status of one path, and the peer should send packets according to the preference expressed in these frames.  Endpoint use  <ins> the sequence number of the CID used by the peer for PATH_ABANDON frames (describing the sender's path identifier).  PATH_ABANDON frames are formatted as shown in fig-path-abandon-format. </ins>  <ins> PATH_ABANDON frames contain the following fields: </ins> Path Identifier: An identifier of the path, which is formatted as shown in fig-path-identifier-format. Path Identifier Content: A variable-length integer specifying the path identifier.  <ins> Note: If the receiver of the PATH_ABANDON frame is using non-zero </ins> length Connection ID on that path, endpoint SHOULD use type 0x00 for path identifier in the control frame.  If the receiver of the  <ins> PATH_ABANDON frame is using 0-length Connection ID, but the peer is </ins> using non-zero length Connection ID on that path, endpoints SHOULD use type 0x01 for path identifier.  If both endpoints are using 0-length Connection IDs on that path, endpoints SHOULD only use type 0x02 for path identifier.  <ins> A variable-length integer that indicates the reason for abandoning this path. </ins>  <ins> A variable-length integer specifying the length of the reason phrase in bytes.  Because an PATH_ABANDON frame cannot be split between packets, any limits on packet size will also limit the space available for a reason phrase. </ins>  <ins> Additional diagnostic information for the closure.  This can be zero length if the sender chooses not to give details beyond the Error Code value.  This SHOULD be a UTF-8 encoded string RFC3629, though the frame does not carry information, such as language tags, that would aid comprehension by any entity other than the one that created the text. </ins>  <ins> PATH_ABANDON frames SHOULD be acknowledged.  If a packet containing a PATH_ABANDON frame is considered lost, the peer should repeat it. </ins> 9.2.


other encryption levels, it MUST return MP_PROTOCOL_VIOLATION as a connection error and close the connection.  8.1. The PATH_ABANDON frame informs the peer to abandon a path.  More number space of the 1-RTT packets which are acknowledged by the ACK_MP frame. <del> If an endpoint receives an ACK_MP frame with a Connection ID sequence number which was never issued (i.e., with a sequence number larger than the largest one advertised), it MUST treat this as a connection error of type MP_PROTOCOL_VIOLATION and close the connection. </del> 9. Multipath QUIC transport error codes are 62-bit unsigned integers

More specifically, if the receiver cannot process an old DCID seq number, it MUST ignore it (i.e., no error). Also
Updated the PR to also include the case of a DCID sequence number greater than any sent (connection error).
Somewhat related to (that moved text that was ACKMP specific to all the multi path-specific frames), we currently state in the ACKMP section that number which was never issued (i.e., with a sequence number larger than the largest one advertised), it MUST treat this as a connection error of type MPPROTOCOLVIOLATION and close the connection. Should we also react the same way with PATHABANDON and PATHSTATUS frames? I would say yes, but would be great to agree on this.
I think so. It should lead to connection error.
At least we should react the same to all three frames. Not sure if there is a point in ignoring instead but I also think a connection error is right.
We may add a sentence to clarify that we should just ignore Multipath frames related to retired DCIDs, but I would suggest to add such a sentence at the beginning of the section introducing these frames, as this applies to all of them.
When using Identifier Types 0 or 1, the current text does not discuss how we should handle related path Identifier that the receiver cannot map to a path. When using Identifier Type 1, the Path Identifier corresponds to the sequence number of the Destination CID used by the sender to send packets on the related path, hence a sequence number of a source Connection ID of the receiver. When using Identifier Type 0, the Path Identifier corresponds to the sequence number of the Source CID of the sender, hence the sequence number of the Destination CID used by the receiver to send packets on the related path. While there are obvious values that could be considered as bad and may lead to, e.g., a MPPROTOCOLVIOLATION (e.g., a sequence number larger than any sequence number advertised), I don't think we should be as radical with regular values as they might have been retired meanwhile (also possible to face this with packet reordering). The worst edge cases I see are the followings. With type 1, the sender might send a PATHABANDON with the seq of the CID in one packet, and then the RETIRECID retiring that CID in a subsequent one, but the RETIRECID reaches the receiver before the PATHABANDON one. If the receiver drops state upon RETIRECID reception, it won't be able to map the PATHABANDON to the path. With type 0, the sender might send a PATHABANDON with the seq of the CID, but before it reaches the peer, the peer sends a RETIRECID retiring that CID. If the receiver drops the CID state upon RETIRECID sending, it won't be able to map the PATHABANDON to the path. Handling these edge cases with connection closure is clearly too strong. Still, ignoring or even acknowledging these PATHABANDON may have bad consequences. Silently dropping packets (i.e., not acknowledge the packet having PATHABANDON) will make the peer retransmit the frame (hoping it updates its path identifier) but could cause the loss/non-acknowledgment of accompanying frames (e.g., STREAM ones), and processing packet content without acknowledging it sounds odd. Acknowledging PATHABANDON that cannot be mapped to a path would make some desynchronisation about the status of the path between the peers (the sender now assumes the path is closed, but the receiver not). To address the type 1 issue, we could write some text stating that a sender cannot retire a DCID if a PATHABANDON containing the related sequence number is still in flight. But for the type 0, the fix is not obvious (I don't think we could have the same solution as in )...
One solution is to put the retired cid into a cidhistorylist and keep it for a while before its final deletion. In doing so, one can still find the cid-to-path mapping if the PATHSTATUS frame uses a retired cid. In this case, as long as the path that the peer wants to retire has no ambiguity, the current path close process can still work. If the receiver can no longer find the cid in both the active list and the history list, then it should treat this as a connection error of MPPROTOCOL_VIOLATION and close the connection.
Another solution would to to not change the path ID (see issue ). In this case all unknown path IDs could be handled as a connection error. Or is it anyway better to just ignore those frames (e.g. if completely outdated packet arrives and the path was already closed)?
I think the simplest solution is to just ignore Path Abandon that refer to abandoned connection ID sequence numbers. The problem is self correcting. If the connection ID is not used anymore, it is either because the path was actually abandoned (no problem), or because packets on the same 5-tuple are now sent with a different CID. In that case, the peer will see these packets arriving on the path that it wants to abandon, and can simply send a new Path Abandon.
Given that we agree that PATHABANDON relates to CID sequence numbers, we can probably apply the same approach to all multipath-specific frames (including ACKMP): if you receive such a frame with a CID sequence number you cannot map anymore, just ignore it.
Look good to me

other encryption levels, it MUST return MP_PROTOCOL_VIOLATION as a connection error and close the connection. <ins> All multipath-specific frames relate to a Destination Connection ID sequence number.  If an endpoint receives a Destination Connection ID sequence number greater than any previously sent to the peer, it MUST treat this as a connection error of type MP_PROTOCOL_VIOLATION.  If an endpoint receives a multipath-specific frame with a Destination Connection ID sequence number that it cannot process anymore (e.g., because the Connection ID might have been retired), it MUST silently ignore the frame. </ins> 8.1. The PATH_ABANDON frame informs the peer to abandon a path.  More number space of the 1-RTT packets which are acknowledged by the ACK_MP frame.  9. Multipath QUIC transport error codes are 62-bit unsigned integers


8.1. <del> The PATH_ABANDON frame informs the peer to abandon a path.  More complex path management can be made possible with additional extensions (e.g., PATH_STATUS frame in I-D.liu-multipath-quic ). </del>  PATH_ABANDON frames are formatted as shown in fig-path-abandon- format. PATH_ABANDON frames SHOULD be acknowledged.  If a packet containing a PATH_ABANDON frame is considered lost, the peer SHOULD repeat it. <del> PATH_ABANDON frames MAY be sent on any path, not only the path on which the referenced Destination Connection ID is used. 8.2. </del>  PATH_STATUS Frame are used by endpoints to inform the peer of the current status of one path, and the peer should send packets if it was the last status sent for that path - as indicated by the sequence number. <del> 8.3. The ACK_MP frame (types TBD-00 and TBD-01; experiments use 0xbaba00..0xbaba01) is an extension of the ACK frame defined by QUIC- TRANSPORT.  It is used to acknowledge packets that were sent on different paths using multiple packet number spaces.  If the frame type is TBD-01, ACK_MP frames also contain the sum of QUIC packets with associated ECN marks received on the connection up to this point. ACK_MP frame is formatted as shown in fig-ack-mp-format. Compared to the ACK frame specified in QUIC-TRANSPORT, the following field is added. The sequence number of the Connection ID identifying the packet number space of the 1-RTT packets which are acknowledged by the ACK_MP frame. </del> 9. Multipath QUIC transport error codes are 62-bit unsigned integers

The ACKMP text does not change, it just comes before PATHABANDON and PATHSTATUS. The rationale is that 1) the new order follows the encoding numbering (lowest to highest), and 2) the ACKMP is a critical frame, the PATHABANDON is an important one yet still "indicative" (it suggests the peer to abandon this path) and the PATHSTATUS frame is an indicative one.
Just to not forget this, but this is mostly editorial. Probably present first the ACKMP frame, and then PATHABANDON and PATHSTATUS Drop reference to I-D.liu-multipath-quic as the PATHSTATUS frame is now integrated in the draft. Based on and , and If we agree on , we could also drop the text in PATh_ABANDON stating that on any path, not only the path on which the referenced Destination Connection ID is used.
OK. It's reasonable.

8.1.  <ins> The ACK_MP frame (types TBD-00 and TBD-01; experiments use 0xbaba00..0xbaba01) is an extension of the ACK frame defined by QUIC- TRANSPORT.  It is used to acknowledge packets that were sent on different paths using multiple packet number spaces.  If the frame type is TBD-01, ACK_MP frames also contain the sum of QUIC packets with associated ECN marks received on the connection up to this point. ACK_MP frame is formatted as shown in fig-ack-mp-format. Compared to the ACK frame specified in QUIC-TRANSPORT, the following field is added. The sequence number of the Connection ID identifying the packet number space of the 1-RTT packets which are acknowledged by the ACK_MP frame. 8.2. The PATH_ABANDON frame informs the peer to abandon a path. </ins> PATH_ABANDON frames are formatted as shown in fig-path-abandon- format. PATH_ABANDON frames SHOULD be acknowledged.  If a packet containing a PATH_ABANDON frame is considered lost, the peer SHOULD repeat it.  <ins> 8.3. </ins> PATH_STATUS Frame are used by endpoints to inform the peer of the current status of one path, and the peer should send packets if it was the last status sent for that path - as indicated by the sequence number.  9. Multipath QUIC transport error codes are 62-bit unsigned integers


name: enable_multipath (TBD - experiments use 0xbaba) <del> value: 0 (default) for disabled, 1 for multipath support with multiple packet number spaces, 2 for multipath support with one packet number space (this is compatible with I-D.liu-multipath- quic) </del>  If the peer does not carry the enable_multipath transport parameter, which means the peer does not support multipath, endpoint MUST fallback to QUIC-TRANSPORT with single path and MUST NOT use any <del> frame or mechanism defined in this document. </del>  Note that the transport parameter "active_connection_id_limit" QUIC- TRANSPORT limits the number of usable Connection IDs, and also limits

I just want to note that I used value 1 for multiple packet number spaces to bee compatible with what is already used by Alibaba. However, if that change it not a problem for Alibaba, this order is slightly nicer.
I took the following text from draft-liu: This basically makes the one packet number handling mandatory to implement as you also need it if multiple packet number spaces were negotiated but zero length CID is used. The alternative would be to say that you can't used zero length CID if multiple packet number spaces are negotiated. Also I'm currently proposing enablemultipath=1 for multiple number space and enablemultipath=2 for one number space to be compatible with the draft-liu and the alibaba implementation. However if we think one packet number space is mandatory to implement, it would make more sense to have it the other way around and allow fallback to one packet number space, namely if the client ask for two packet number spaces but the server supports only one but would also lead to a successful negotiation. What do people think about this?
As long as inter-op is allowed, I think it might be better to keep both PN space choices optional (Mirja's proposal of using enable_multipath=1 or 2 works for me). The first thing is to allow people more freedom. If some implementors only want to use one option, they just need to implement one code branch. Second, single packet number space requires the modification of loss detection algorithm, the way how rtt is sampled (the overall largest ack may not be the largest acked packet on a given path), and some delayed ACK strategies, which might take additional effort. People may also want to avoid round-robin scheduling in single PN as it creates a lot of holes in the ack range, so certain things regarding performance get coupled. Christian introduced many good practices and we are currently learning from it, so we probably need more experience on that.
Adding the option to negotiate is a very good solution at this point. It would be helpful to hear more people's implementation experience. To avoid failure for interop tests, I suggest that we could negotiate the transport parameter like this: use 2-bits in the value field for negotiating one or more PN spaces. 0x0: don't support multi-path 0x1: only support multiple PN spaces for multi-path 0x2: only support one PN space for multi-path 0x3: support both one PN space and multiple PN space The client could send all these 4 values in the enable_multipath transport parameter which depends on what kind of modes it supports. If the server just support one mode(such as one PN space), it could return 0x2 so that the client knows how to do next. If the server support both, it could choose 0x1 or 0x2 as it prefers, and make sure that the server don't return 0x3 as it need to decide which mode to choose. In 0x2 mode, the endpoints must not use 0-length CID. Then for people who just want one PN space, it would be ok to just support 0x2 mode. For the other guys who don't use 0-length Connection ID and want to get less overhead for ack ranges, they could choose to use 0x1 mode. And for people who want to support both 0-length CID and multiple PN spaces, they could choose 0x3 and of course it will add more modification in the implementation. Maybe this could meet everyone's need for experiments.
NAME any views?
I agree with Yanmei's proposal of 4 options 0, 1, 2 or 3, although I have a slight preference for "the other way around": Client Option Allowed server responses -------------------------------------------- 0x0 0x0 0x1 0x0 or 0x1 0x2 0x0 or 0x2 0x3 0x0, 0x1 or 0x2 Receiving an unexpected value from the server should be a protocol violation. If the transport parameter is not present, the value defaults to zero.
Agree with Christian's design for value definitions. Submitted PR for this issue.
As got merged, I think this is now resolved.
Actually I think this is not fully resolved. The initial question was about use of multiple packet number spaces and zero length connection ID. Do we now say that if multiple packet number spaces are used, zero connection ID cannot be used? If yes, that means we need to remove the text I've cited in the initial issue description from the draft. And also add some text that "zero-length CID MUST not be used with enable_multipath=2". Or if we want to keep that option that basically means that you need to always implement also singe packet number handling even if use of multiple packet number spaces is negotiated, and then we should say that more explicitly in the negotiation section. To me it's not fully clear which option you have agreed to now...?
I opened to track the remaining point.

name: enable_multipath (TBD - experiments use 0xbaba)  <ins> value: 0 (default) for disabled.  Endpoints use 2-bits in the value field for negotiating one or more PN spaces, available option value for client and server are listed in param_value_definition : </ins> If the peer does not carry the enable_multipath transport parameter, which means the peer does not support multipath, endpoint MUST fallback to QUIC-TRANSPORT with single path and MUST NOT use any  <ins> frame or mechanism defined in this document.  If endpoint receives unexpected value for the transport parameter "enable_multipath", it MUST treat this as a connection error of type MP_CONNECTION_ERROR and close the connection. </ins> Note that the transport parameter "active_connection_id_limit" QUIC- TRANSPORT limits the number of usable Connection IDs, and also limits


it SHOULD use Packet Number Space Identifier 0 in ACK_MP frames.  If an endpoint receives a ACK_MP frame with a non-existing packet number space ID, it MUST treat this as a connection error of type <del> MP_CONNECTION_ERROR and close the connection. </del>  10.  This document defines a new transport parameter for the negotiation of enable multiple paths for QUIC, and two new frame types.  The draft defines provisional values for experiments, but we expect IANA The following frame types defined in frame-types should be added to the "QUIC Frame Types" registry under the "QUIC Protocol" heading. <del> 11. </del>  TBD <del> 12. </del>  TBD

If we want an MP specific error type we need to also add this to the IANA section.
Submitted PR for this issue. I just added MPPROTOCOLVIOLATION for connection close. If we have more specific error codes in the future, we can add them in the table.
Looks OK.

it SHOULD use Packet Number Space Identifier 0 in ACK_MP frames.  If an endpoint receives a ACK_MP frame with a non-existing packet number space ID, it MUST treat this as a connection error of type  <ins> MP_PROTOCOL_VIOLATION and close the connection. </ins> 10. <ins> Multi-path QUIC transport error codes are 62-bit unsigned integers following QUIC-TRANSPORT. This section lists the defined Multi-path QUIC transport error codes that can be used in a CONNECTION_CLOSE frame with a type of 0x1c. These errors apply to the entire connection. MP_PROTOCOL_VIOLATION (experiments use 0xba01): An endpoint detected an error with protocol compliance that was not covered by more specific error codes. 11. </ins> This document defines a new transport parameter for the negotiation of enable multiple paths for QUIC, and two new frame types.  The draft defines provisional values for experiments, but we expect IANA The following frame types defined in frame-types should be added to the "QUIC Frame Types" registry under the "QUIC Protocol" heading.  <ins> The following transport error code defined in tab-error-code should be added to the "QUIC Transport Error Codes" registry under the "QUIC Protocol" heading. 12. </ins> TBD  <ins> 13. </ins> TBD


The ACK_MP frame (types TBD-00 and TBD-01; experiments use 0xbaba00..0xbaba01 or 0x42..x43) is an extension of the ACK frame defined by QUIC-TRANSPORT.  It is used to acknowledge packets that <del> were sent on different paths.  If the frame type is TBD-01, ACK_MP frames also contain the sum of QUIC packets with associated ECN marks received on the connection up to this point. </del>  ACK_MP frame is formatted as shown in fig-mp-ack-format. space ID, it MUST treat this as a connection error of type MP_PROTOCOL_VIOLATION and close the connection.  10. Multi-path QUIC transport error codes are 62-bit unsigned integers

When negotiating the use of a single packet number space, it is not clear whether the use of ACKMP frame is acceptable or not. Should we only stick to ACK frames and consider reception of ACKMP frames to be MPPROTOCOLVIOLATION error? Or should we tolerate their reception if the packet number space identifier is 0?
Ah good point. I moved the text about ACKMP this morning because I assumed that you would only implement this frame if you use multiple packet number spaces. If that's the case receiving an ACKMP with enablemultipath=2 should be a PROTOCOLVIOLATION and we should specify that.
Look good to me.

The ACK_MP frame (types TBD-00 and TBD-01; experiments use 0xbaba00..0xbaba01 or 0x42..x43) is an extension of the ACK frame defined by QUIC-TRANSPORT.  It is used to acknowledge packets that  <ins> were sent on different paths when using multiple packet number spaces.  If the frame type is TBD-01, ACK_MP frames also contain the sum of QUIC packets with associated ECN marks received on the connection up to this point. </ins> ACK_MP frame is formatted as shown in fig-mp-ack-format. space ID, it MUST treat this as a connection error of type MP_PROTOCOL_VIOLATION and close the connection. <ins> When using a single packet number space, endhosts MUST NOT send ACK_MP frames.  If an endhost receives an ACK_MP frame while a single packet number space was negotiated, it MUST treat this as a connection error of type MP_PROTOCOL_VIOLATION and close the connection. </ins> 10. Multi-path QUIC transport error codes are 62-bit unsigned integers


for an excessive time or having to promptly reestablish a path after a spurious estimate of path abandonment by the client.  4. Senders MUST manage per-path congestion status, and MUST NOT send

Initial attempt to solve .
One more editorial comment: This section is now somehow in the middle between the cc section and the packet scheduling section. I think it would be better to have this new section as a subsection of section 3 (on Path Setup and Removal).
Indeed, let me fix that.
Given that paths are initiating/validating/active/closed, it looks like we will need at some point to specify some path state (finite state machine?) and what behavior is acceptable in each of them (e.g., we cannot send packets on a closed state,...).
Can you create a PR?

for an excessive time or having to promptly reestablish a path after a spurious estimate of path abandonment by the client. <ins> 3.3. fig-path-states shows the states that an endpoint's path can have. In non-final states, hosts have to track the following information. Associated 4-tuple: The tuple (source IP, source port, destination IP, destination port) used by the endhost to send packets over the path. Associated Destination Connection ID: The Connection ID used to send packets over the path. If multiple packet number spaces are used over the connection, hosts MUST also track the following information. Path Packet Number Space: The endpoint maintains a separate packet number for sending and receiving packets over this path.  Packet number considerations described in QUIC-TRANSPORT apply within the given path. In the "Active" state, hosts MUST also track the following information. Associated Source Connection ID: The Connection ID used to receive packets over the path. A path in the "Validating" state performs path validation as described in Section 8.2 of QUIC-TRANSPORT.  An endhost should not send non-probing frames on a path in "Validating" state, as it has no guarantee that packets will actually reach the peer. The endhost can use all the paths in the "Active" state, provided that the congestion control and flow control currently allow sending of new data on a path. In the "Closing" state, the endhost SHOULD NOT send packets on this path anymore, as there is no guarantee that the peer can still map the packets to the connection.  The endhost SHOULD wait for the acknowledgment of the PATH_ABANDONED frame before moving the path to the "Closed" state to ensure a graceful termination of the path. When a path reaches the "Closed" state, the endhost releases all the path's associated resources.  Consequently, the endhost is not able to send nor receive packets on this path anymore. </ins> 4. Senders MUST manage per-path congestion status, and MUST NOT send


as specified in QUIC-TRANSPORT.  The new transport parameter is defined as follow: <del> name: enable_multipath (TBD) </del>  value: 0 (default) for disabled.  Endpoints use 2-bits in the value field for negotiating one or more PN spaces, available

For issue
Ok too.
Based on the discussed from the side meeting, we need to Clarify that the goal is to only select one approach for the finally published solution in an RFC Add some discussion about the pros and cons to the appendix
Yes, I think we should have these discussions in the current draft.
Should we use a new code point for the transport parameter of the options? For 2 reasons: It's a new joint version As we use a different value definition in our current implementation, it's hard to deploy the new version in a synchronised way, at least we could keep compatible if we use a new transport parameter. What do you think? Or maybe we should keep it unchanged until the WG is ready to adopt the draft?
I think it is fine to define a new value for the transport parameter, the 00 version of this draft is not yet out anyway.
Yes, please use a new parameter.
I created to add one more sentence and a reference to the slides from the side meeting. I believe this is sufficient to address this issue for -00.
NAME please confirm that this is okay and merge!

as specified in QUIC-TRANSPORT.  The new transport parameter is defined as follow:  <ins> name: enable_multipath (TBD - experiments use 0xbabf) </ins> value: 0 (default) for disabled.  Endpoints use 2-bits in the value field for negotiating one or more PN spaces, available


or received. Path Identifier Content: A variable-length integer specifying the <del> path identifier. </del>  Note: If the receiver of the PATH_ABANDON frame is using non-zero length Connection ID on that path, endpoint SHOULD use type 0x00 for

Fix issue .
In the "path abandon" frame, the path identifier is defined as: But in the identifier type 2, the "path identifier content" is empty. We should fix that.
Submitted PR for this issue.
Yes, perfect.

or received. Path Identifier Content: A variable-length integer specifying the  <ins> path identifier.  If Identifier Type is 2, the Path Identifier Content MUST be empty. </ins> Note: If the receiver of the PATH_ABANDON frame is using non-zero length Connection ID on that path, endpoint SHOULD use type 0x00 for


10.2. The ACK_MP frame (types TBD-00 and TBD-01; experiments use <del> 0xbaba00..0xbaba01 or 0x42..x43) is an extension of the ACK frame defined by QUIC-TRANSPORT.  It is used to acknowledge packets that were sent on different paths when using multiple packet number spaces.  If the frame type is TBD-01, ACK_MP frames also contain the sum of QUIC packets with associated ECN marks received on the connection up to this point. </del>  ACK_MP frame is formatted as shown in fig-mp-ack-format.

Given the deadline, let's first have one set of values instead of two. We may want to decide a third value set later.
Ok for me!
The current text on the defintion of ACK MP says: The figure is too wide, and I am fixing that with issue , but really we should pick just one set of identifiers, not 2. I understand that draft-liu and draft-deconinck were using these two values, but we should pick one of them, or a third one.
I submitted PR to fix this issue.
PR merged.
Fixed conflict.NAME please review and merge if you agree!

10.2. The ACK_MP frame (types TBD-00 and TBD-01; experiments use  <ins> 0xbaba00..0xbaba01) is an extension of the ACK frame defined by QUIC- TRANSPORT.  It is used to acknowledge packets that were sent on different paths when using multiple packet number spaces.  If the frame type is TBD-01, ACK_MP frames also contain the sum of QUIC packets with associated ECN marks received on the connection up to this point. </ins> ACK_MP frame is formatted as shown in fig-mp-ack-format.


more data on a given path than congestion control on that path allows.  This is already a requirement of QUIC-TRANSPORT. <del> Multipath TCP uses the LIA congestion control scheme specified in RFC6356.  This scheme can immediately be adapted to Multipath QUIC. Other coupled congestion control schemes have been proposed for Multipath TCP such as OLIA. </del>  5.

Some more text in the congestion control section explaining why coupled congestion control is suggested. We could later add SHOULD or MUST to request the utilization of coupled congestion control schemes for multipath QUIC.

more data on a given path than congestion control on that path allows.  This is already a requirement of QUIC-TRANSPORT.  <ins> When a Multipath QUIC connection uses two or more paths, there is no guarantee that these paths are fully disjoint.  When two (or more paths) share the same bottleneck, using a standard congestion control scheme could result in an unfair distribution of the bandwidth with the multipath connection getting more bandwidth than competing single paths connections.  Multipath TCP uses the LIA congestion control scheme specified in RFC6356 to solve this problem.  This scheme can immediately be adapted to Multipath QUIC.  Other coupled congestion control schemes have been proposed for Multipath TCP such as OLIA. </ins> 5.


Each endpoint manages the set of paths that are available for transmission.  At any time in the connection, each endpoint can decide to abandon one of these paths, following for example changes <del> in local connectivity or changes in local preferences.  After a client abandons a path, the server will not receive any more non- </del>  probing packets on that path. An endpoint that wants to close a path SHOULD NOT rely on implicit Both endpoints, namely the client and the server, can close a path, by sending PATH_ABANDON frame (see path-abandon-frame) which abandons the path with a corresponding Path Identifier.  Once a path is marked <del> as "abandoned", it means that the resources related to the path can be released. </del>  3.2.2. 9.1. <del> The PATH_ABANDON frame informs the peer to abandon a path, and release the corresponding resources.  An endpoint uses the sequence number of the CID used by the peer for PATH_ABANDON frames (describing the sender's path identifier).  More complex path management can be made possible with additional extensions (e.g., PATH_STATUS frame in I-D.liu-multipath-quic ). </del>  PATH_ABANDON frames are formatted as shown in fig-path-abandon- format. one that created the text. PATH_ABANDON frames SHOULD be acknowledged.  If a packet containing a <del> PATH_ABANDON frame is considered lost, the peer should repeat it. </del>  <del> PATH_ABANDON frames can sent on any path, not only the path </del>  identified by the Path Identifier field. 9.2.

Please carefully review the text about abandon frames on the last active path. This is new and maybe not what you intended, however, because of potential crossing of abandon frames from each direction (as mentioned by NAME in the issue), I don't think receiving an abandon frame on the last active path should be a protocol violation.
The text says that both endpoint can send an abandon frame. However, if I understand the current text correctly it only says that the receiving peer of that frame can remove its state as the sender of the abandon frame indicated it will not send any more packets ion that path. However, the sender of the abandon frame can't remove its state until it also receives an abandon frame from the other end. That would mean a path can "half-open"... Do we need to say something that that the receiver of an abandon frame SHOULD also send an abandon frame in response? Or MAY? Or MUST? Or is the path considered completely closed as soon as an ACK for the packet containing the abandon frame is received? In this case we need to say that when an abandon frame is received, the receiver MUST not use that path anymore. I think we need to clarify that! Please provide a PR or let me know what the intention was and I can write something!
For me, ABANDONPATH means "please stop sending on that path, I may soon stop listening". The peer should definitely stop sending on that path. It may want to keep listening for a short time, because some packets may still be sent out of order. It may also want to keep some context to process acknowledgement of packets previously sent on that path. The general guidance is to to wait for something like 3xRTO before dropping path management resource. I don't think that we need symmetry, that both peers have to abandon the path before they can release resource. But we have to solve a potential path suicide problem. Suppose we have two paths, peer 1 abandons path , peer 2 abandon path . Now we have zero path, and the connection is closed -- unless the client promptly opens a new path . My original intention is that only the client ever sends an ABANDONPATH, since only the client can initiate a new path, which dodges the path suicide issue.
Good questions! For me, the issue is that when an endpoint decides to close a path, it is probably due to that the path condition is getting worse, which also means that further communication on that path may fail. When we use non-zero CID, as we should have at least one path that is working (if no path is working, the connection will be closed), and we are not constraint to send PATHABANDON frame on the same path that we want to close, the sender should be able to receive the ACK of PATHABANDON. But when we use null CID, we use pathidtype 0x02, which refers to the path where the packet is coming from, and hence, we are constraint to send PATHABANDON on the same path we want to close. In such a case, it is likely that the ACK of PATHABANDON will not be received, resulting in the "half-open" problem. Therefore, we should say sth like "when the sender moves into the "ABANDONSENT" state, and after 3RTO, it should close the path even if it hasn't seen the PATHABANDON sent from its peer".
Also, regarding when a path is considered completely closed, the current intention is that a path is considered closed when both the sender-side and the receiver-side resources of that path are released. The sender-side resources can be readily released when PATHABANDON is acked (We may need to retransmit the pathabandon). The receiver-side resources are released when the RETIRECID frame is received. However, when zero-CID is used, it is possible that an endpoint is not going to receive any RETIRECID frames, so the receiver-side resources are released 3RTO after sending out the PATH_ABANDON.
I created PR . I think this is the biggest issue we need to address and agree on before submission of -00. So please review NAME NAME NAME !
Sounds good. I think the next feedback will come once we have implemented and tested the feature.

Each endpoint manages the set of paths that are available for transmission.  At any time in the connection, each endpoint can decide to abandon one of these paths, following for example changes  <ins> in local connectivity or changes in local preferences.  After an endpoint abandons a path, the peer will not receive any more non- </ins> probing packets on that path. An endpoint that wants to close a path SHOULD NOT rely on implicit Both endpoints, namely the client and the server, can close a path, by sending PATH_ABANDON frame (see path-abandon-frame) which abandons the path with a corresponding Path Identifier.  Once a path is marked  <ins> as "abandoned", it means that the resources related to the path, such as the used connection IDs, can be released.  However, information related to data delivered over that path SHOULD not be released immediately as acknowledgments can still be received or other frames that also may trigger retransmission of data on another path. The endpoint sending the PATH_ABANDON frame SHOULD consider a path as abandoned when the packet that contained the PATH_ABANDON frame is acknowledged.  When releasing resources of a path, the endpoint SHOULD send a RETIRE_CONNECTION_ID frame for the connection IDs used on the path, if any. The receiver of a PATH_ABANDON frame SHOULD NOT release its resources immediately but SHOULD wait for the receive of the RETIRE_CONNECTION_ID frame for the used connection IDs or 3 RTOs. Usually it is expected that the PATH_ABANDON frame is used by the client to indicate to the server that path conditions have changed such that the path is or will be not usable anymore, e.g. in case of an mobility event.  The PATH_ABANDON frame therefore indicates to the receiving peer that the sender does not intend to send any packets on that path anymore but also recommends to the receiver that no packets should be sent in either direction.  The receiver of an PATH_ABANDON frame MAY also send an PATH_ABANDON frame to signal its own willingness to not send any packet on this path anymore. If connection IDs are used, PATH_ABANDON frames can be sent on any path, not only the path that is intended to be closed.  Thus a connection can be abandoned even if connectivity on that path is already broken.  If no connection IDs are used and the PATH_ABANDON frame has to sent on the path that is intended to be closed, it is possible that the packet containing the PATH_ABANDON frame or the packet containing the ACK for the PATH_ABANDON frame cannot be received anymore and the endpoint might need to rely on an idle time out to close the path, as described in Section idle-time-close. Retransmittable frame, that have previously been send on the abandoned path and are considered lost, SHOULD be retransmitted on a different path. If a PATH_ABANDON frame is received for the only active path of a QUIC connection, the receiving peer SHOULD send a CONNECTION_CLOSE frame and enters the closing state.  If the client received a PATH_ABANDON frame for the last open path, it MAY instead try to open a new path, if available, and only initiate connection closure if path validation fails or a CONNECTION_CLOSE frame is received from the server.  Similarly the server MAY wait for a short, limited time such as one RTO if a path probing packet is received on a new path before sending the CONNECTION_CLOSE frame. </ins> 3.2.2. 9.1.  <ins> The PATH_ABANDON frame informs the peer to abandon a path.  An endpoint uses the sequence number of the CID used by the peer for PATH_ABANDON frames (describing the sender's path identifier).  More complex path management can be made possible with additional extensions (e.g., PATH_STATUS frame in I-D.liu-multipath-quic ). </ins> PATH_ABANDON frames are formatted as shown in fig-path-abandon- format. one that created the text. PATH_ABANDON frames SHOULD be acknowledged.  If a packet containing a  <ins> PATH_ABANDON frame is considered lost, the peer SHOULD repeat it. </ins>  <ins> PATH_ABANDON frames MAY be sent on any path, not only the path </ins> identified by the Path Identifier field. 9.2.


3. After completing the handshake, endpoints have agreed to enable <del> multipath feature and can start using multiple paths. </del>  This proposal adds one multi-path control frame for path management: 6. <del> The simultaneous usage of several sending paths introduces new algorithms (packet scheduling, path management) whose specifications are out of scope of this document.  Nevertheless, these algorithms are actually present in any multipath-enabled transport protocol like Multipath TCP, CMT-SCTP and Multipath DCCP.  A companion draft I- D.bonaventure-iccrg-schedulers provides several general-purpose packet schedulers depending on the application goals.  A similar document can be created to discuss path management considerations. </del>  7.

Some more text for the scheduling section to clarify it.
A suggestion to resolve Mirja's point, also agree on the proposed change in line 248.

3. After completing the handshake, endpoints have agreed to enable  <ins> multipath feature and can start using multiple paths.  This document does not discuss when a client decides to initiate a new path.  We delegate such discussion in separate documents. </ins> This proposal adds one multi-path control frame for path management: 6.  <ins> The transmission of QUIC packets on a regular QUIC connection is regulated by the arrival of data from the application and the congestion control scheme.  QUIC packets can only be sent when the congestion window of at least one path is open. Multipath QUIC implementations also need to include a packet scheduler that decides, among the paths whose congestion window is open, the path over which the next QUIC packet will be sent.  Many factors can influence the definition of these algorithms and their previse definition is outside the scope of this document.  Various packet schedulers have been proposed and implemented, notably for Multipath TCP.  A companion draft I-D.bonaventure-iccrg-schedulers provides several general-purpose packet schedulers depending on the application goals. </ins> 7.


packet number space for all paths or the use of separate packet number spaces per path.  While separate packet number spaces allow for more efficient ACK encoding, especially when paths have highly <del> difference latencies, this approach requires the use of a connection </del>  ID.  Therefore use of a single number space can be beneficial in highly constrained networks that do not benefit from exposing the <del> connection ID in the header.  More evaluation and implementation experience is needed to select one approach before final publlication. </del>  As currently defined in this version of the draft the use of multiple packet number spaces requires the use of connection IDs is both

Based on the discussed from the side meeting, we need to Clarify that the goal is to only select one approach for the finally published solution in an RFC Add some discussion about the pros and cons to the appendix
Yes, I think we should have these discussions in the current draft.
Should we use a new code point for the transport parameter of the options? For 2 reasons: It's a new joint version As we use a different value definition in our current implementation, it's hard to deploy the new version in a synchronised way, at least we could keep compatible if we use a new transport parameter. What do you think? Or maybe we should keep it unchanged until the WG is ready to adopt the draft?
I think it is fine to define a new value for the transport parameter, the 00 version of this draft is not yet out anyway.
Yes, please use a new parameter.
I created to add one more sentence and a reference to the slides from the side meeting. I believe this is sufficient to address this issue for -00.
Ok with the PR, I can make a quick commit to fix the listed typos

packet number space for all paths or the use of separate packet number spaces per path.  While separate packet number spaces allow for more efficient ACK encoding, especially when paths have highly  <ins> different latencies, this approach requires the use of a connection </ins> ID.  Therefore use of a single number space can be beneficial in highly constrained networks that do not benefit from exposing the  <ins> connection ID in the header.  While both approaches are supported by the specification in this version of the document, the intention for the final publication of a multipath extension for QUIC is to choose one option in order to avoid incompatibility.  More evaluation and implementation experience is needed to select one approach before final publication.  Some discussion about pros and cons can be found here: https://github.com/mirjak/draft-lmbdhk-quic- multipath/blob/master/presentations/PacketNumberSpace_s.pdf </ins> As currently defined in this version of the draft the use of multiple packet number spaces requires the use of connection IDs is both


Following QUIC-TRANSPORT, each endpoint uses NEW_CONNECTION_ID frames to issue usable connections IDs to reach it.  Before an endpoint adds <del> a new path, it SHOULD check whether there is at least one unused </del>  available Connection ID for each side. If the transport parameter "active_connection_id_limit" is negotiated

I copied this sentence from draft-liu: Shouldn't this be a MUST instead of a SHOULD?
We can describe it in a more accurate way: Before an endpoint adds a new path, it MUST check whether there is at lease one unused available Connection ID for each side, unless the endpoints use zero-length Connection ID.
Submitted PR for this issue.
LGTM.

Following QUIC-TRANSPORT, each endpoint uses NEW_CONNECTION_ID frames to issue usable connections IDs to reach it.  Before an endpoint adds  <ins> a new path, it MUST check whether there is at least one unused </ins> available Connection ID for each side. If the transport parameter "active_connection_id_limit" is negotiated


If the multipath option is enabled with a value of 2, each path has its own packet number space for transmitting 1-RTT packets and a new <del> ACK frame format is used as specified in mp-ack-frame.  Compared to the QUIC v1 ACK frame, the MP_ACK frames additionally contains a </del>  Packet Number Space Identifier (PN Space ID).  The PN Space ID used to distinguish packet number spaces for different paths and is simply derived from the sequence number of Destination Connection ID. actively using N paths, the limit is reached.  If the client wants to start a new path, it has to retire one of the established paths. <del> ACK_MP frame mp-ack-frame can be returned via either a different </del>  path, or the same path identified by the Path Identifier, based on different strategies of sending ACK_MP frames. packets with associated ECN marks received on the connection up to this point. <del> ACK_MP frame is formatted as shown in fig-mp-ack-format. </del>  Compared to the ACK frame specified in QUIC-TRANSPORT, the following field is added.

This should be a simple fix. There was only one place where MP_ACK appeared.
In the current draft, we are having two names of multi-path ACK. I think we should stick to just one.
I have a slight preference for ACKMP, on the model of ACKECN.
I submitted PR to fix this.
I would be consistent in how the I-D identifies MP additions vs. RFC9000. Given that the document uses MPCONNECTIONERROR and MPPROTOCOLVIOLATION, I have a preference for MP_ACK.
I added changes of the tag "mp-ack-frame" to "ack-mp-frame" for consistency...

If the multipath option is enabled with a value of 2, each path has its own packet number space for transmitting 1-RTT packets and a new  <ins> ACK frame format is used as specified in ack-mp-frame.  Compared to the QUIC v1 ACK frame, the ACK_MP frames additionally contains a </ins> Packet Number Space Identifier (PN Space ID).  The PN Space ID used to distinguish packet number spaces for different paths and is simply derived from the sequence number of Destination Connection ID. actively using N paths, the limit is reached.  If the client wants to start a new path, it has to retire one of the established paths.  <ins> ACK_MP frame ack-mp-frame can be returned via either a different </ins> path, or the same path identified by the Path Identifier, based on different strategies of sending ACK_MP frames. packets with associated ECN marks received on the connection up to this point.  <ins> ACK_MP frame is formatted as shown in fig-ack-mp-format. </ins> Compared to the ACK frame specified in QUIC-TRANSPORT, the following field is added.


name: enable_multipath (TBD - experiments use 0xbabf) <del> value: 0 (default) for disabled.  Endpoints use 2-bits in the value field for negotiating one or more PN spaces, available option value for client and server are listed in param_value_definition : If the peer does not carry the enable_multipath transport parameter, which means the peer does not support multipath, endpoint MUST fallback to QUIC-TRANSPORT with single path and MUST NOT use any frame or mechanism defined in this document.  If endpoint receives unexpected value for the transport parameter "enable_multipath", it MUST treat this as a connection error of type MP_CONNECTION_ERROR and close the connection. </del>  Note that the transport parameter "active_connection_id_limit" QUIC- TRANSPORT limits the number of usable Connection IDs, and also limits

Lack of the reference to Section 7.4/Section 7.4.2 of 9000 where the general handling of (new) transport parameters is discussed. When the new transport parameter is not present, this does not necessarily mean multipath is not supported as it can be supported, but disabled (for this connection as a function of local policy). I would update this text
The absence of the parameter does indeed indicate that the peer will not support multipath for this connection. But yes, the current text does not read too well. I think we should simplify and just say the same as QUIc V1, i.e. "Transport parameters have a default value of 0 if the transport parameter is absent". So something like:
The negotiation only refers to "support for this connection". We cannot make any statement about whether the same server might negotiate the feature in another connection attempt. There is no practical difference between "not supported" and "supported, but disabled". It could very well be for example that some server in a farm support the feature and some don't.
NAME please review PR
Looks good to me. Thanks.

name: enable_multipath (TBD - experiments use 0xbabf)  <ins> value: 0 (default) for disabled. Endpoints use 2-bits in the value field for negotiating one or more PN spaces, available option value for client and server are listed in param_value_definition : If the parameter is absent or set to 0, the endpoints MUST fallback to QUIC-TRANSPORT with single path and MUST NOT use any frame or mechanism defined in this document. If endpoint receives unexpected value for the transport parameter "enable_multipath", it MUST treat this as a connection error of type MP_CONNECTION_ERROR and close the connection. </ins> Note that the transport parameter "active_connection_id_limit" QUIC- TRANSPORT limits the number of usable Connection IDs, and also limits


1. <del> This document specifies an extension to QUIC v1 QUIC-TRANSPORT to enable the simultaneous usage of multiple paths for a single </del>  connection. This proposal is based on several basic design points: <del> Re-use as much as possible mechanisms of QUIC-v1.  In particular this proposal uses path validation as specified for QUIC v1 and aims to re-use as much as possible of QUIC's connection migration. </del>  <del> Use the same packet header formats as QUIC v1 to avoid the risk of packets being dropped by middleboxes (which may only support QUIC v1) </del>  Congestion Control, RTT measurements and PMTU discovery should be per-path (following QUIC-TRANSPORT) If the multipath option is enabled with a value of 2, each path has its own packet number space for transmitting 1-RTT packets and a new ACK frame format is used as specified in ack-mp-frame.  Compared to <del> the QUIC v1 ACK frame, the ACK_MP frames additionally contains a Packet Number Space Identifier (PN Space ID).  The PN Space ID used </del>  to distinguish packet number spaces for different paths and is simply derived from the sequence number of Destination Connection ID. Therefore, the packet number space for 1-RTT packets can be 7.2.1. <del> Packet protection for QUIC v1 is specified is Section 5 of QUIC-TLS. The general principles of packet protection are not changed for QUIC Multipath.  No changes are needed for setting packet protection keys, initial secrets, header protection, use of 0-RTT keys, receiving out- of-order protected packets, receiving protected packets, or retry packet integrity.  However, the use of multiple number spaces for 1-RTT packets requires changes in AEAD usage. </del>  Section 5.3 of QUIC-TLS specifies AEAD usage, and in particular the use of a nonce, N, formed by combining the packet protection IV with 7.2.2. <del> The Key Phase bit update process for QUIC v1 is specified in </del>  Section 6 of QUIC-TLS.  The general principles of key update are not <del> changed in this specification.  Following QUIC v1, the Key Phase bit is used to indicate which packet protection keys are used to protect the packet.  The Key Phase bit is toggled to signal each subsequent key update.  Because of network delays, packets protected with the older key might arrive later than the packets protected with the new key.  Therefore, the endpoint needs to retain old packet keys to allow these delayed packets to be processed and it must distinguish between the new key and the old key.  In QUIC V1, this is done using packet numbers so that the rule is made simple: Use the older key if packet number is lower than any packet number frame the current key phase. </del>  When using multiple packet number spaces on different paths, some care is needed when initiating the Key Update process, as different

Be consistent with RFC9000
In order to be consistent with RFC9000, use "QUIC version 1" instead of "QUIC v1" through the document.

1.  <ins> This document specifies an extension to QUIC version 1 QUIC-TRANSPORT to enable the simultaneous usage of multiple paths for a single </ins> connection. This proposal is based on several basic design points:  <ins> Re-use as much as possible mechanisms of QUIC version 1.  In particular this proposal uses path validation as specified for QUIC version 1 and aims to re-use as much as possible of QUIC's connection migration. </ins>  <ins> Use the same packet header formats as QUIC version 1 to avoid the risk of packets being dropped by middleboxes (which may only support QUIC version 1) </ins> Congestion Control, RTT measurements and PMTU discovery should be per-path (following QUIC-TRANSPORT) If the multipath option is enabled with a value of 2, each path has its own packet number space for transmitting 1-RTT packets and a new ACK frame format is used as specified in ack-mp-frame.  Compared to  <ins> the QUIC version 1 ACK frame, the ACK_MP frames additionally contains a Packet Number Space Identifier (PN Space ID).  The PN Space ID used </ins> to distinguish packet number spaces for different paths and is simply derived from the sequence number of Destination Connection ID. Therefore, the packet number space for 1-RTT packets can be 7.2.1.  <ins> Packet protection for QUIC version 1 is specified is Section 5 of QUIC-TLS.  The general principles of packet protection are not changed for QUIC Multipath.  No changes are needed for setting packet protection keys, initial secrets, header protection, use of 0-RTT keys, receiving out-of-order protected packets, receiving protected packets, or retry packet integrity.  However, the use of multiple number spaces for 1-RTT packets requires changes in AEAD usage. </ins> Section 5.3 of QUIC-TLS specifies AEAD usage, and in particular the use of a nonce, N, formed by combining the packet protection IV with 7.2.2.  <ins> The Key Phase bit update process for QUIC version 1 is specified in </ins> Section 6 of QUIC-TLS.  The general principles of key update are not  <ins> changed in this specification.  Following QUIC version 1, the Key Phase bit is used to indicate which packet protection keys are used to protect the packet.  The Key Phase bit is toggled to signal each subsequent key update.  Because of network delays, packets protected with the older key might arrive later than the packets protected with the new key.  Therefore, the endpoint needs to retain old packet keys to allow these delayed packets to be processed and it must distinguish between the new key and the old key.  In QUIC version 1, this is done using packet numbers so that the rule is made simple: Use the older key if packet number is lower than any packet number frame the current key phase. </ins> When using multiple packet number spaces on different paths, some care is needed when initiating the Key Update process, as different


address as well as source and destination port.  Therefore there can be at most one active paths/connection ID per 4-tuple. <del> The path management specified in section 9 of QUIC-TRANSPORT fulfills </del>  multiple goals: it directs a peer to switch sending through a new preferred path, and it allows the peer to release resources associated with the old path.  Multipath requires several changes to Manage the removal of paths that have been abandoned. As such this extension specifies a departure from the specification <del> of path management in section 9 of QUIC-TRANSPORT and therefore </del>  requires negotiation between the two endpoints using a new transport parameter, as specified in nego. using the same mechanisms. If validation succeed, the client can send non-probing, 1-RTT packets <del> on the new paths.  In contrast with the specification in section 9 of </del>  QUIC-TRANSPORT, the server MUST NOT assume that receiving non-probing packets on a new path indicates an attempt to migrate to that path. Instead, servers SHOULD consider new paths over which non-probing received for a sufficiently long path-idle delay, but SHOULD only release resource for the last available path if no traffic is received for the duration of the idle timeout, as specified in <del> section 10.1 of QUIC-TRANSPORT.  This means if all paths remain idle </del>  for the idle timeout, the connection is implicitly closed. Server implementations need to select the sub-path idle timeout as a

Fix issue
I went ahead and replaced the references to RFC9000 by QUIC-TRANSPORT, and RFC9001 by QUIC-TLS. I have checked that this produced the expected XML V3.
RFC9000 is cited massively in the draft. It would be helpful to indicate to the reader where to look by indicating the section. Some change examples are provided below: and so on.
Side comment -- please see issue about the use of XML2RFC v3 syntax for links. We can do some of that, but not necessarily everywhere. We do expect that readers are familiar with QUIC V1. If we want to say "we are just in line with QUIC V1", then a global reference to RFC 9000 should suffice. If we want to say "please read this text in QUIC V1, we don't want to replicate here", then yes, we should refer to the specific section.
Thanks, Christian. That's the intent.
The draft is using old style references, such as "section 9 of {{RFC9000}}". Version 3 of XML2RFC defines a structure way to do that, such as: , which can be expressed in recent versions of Markdown as . The new style references don't change the text version much, but in the HTML rendering they generate direct references to the target section, which is nice for the readers.
Sorry, that's my fault because of my local build system. Will fix this for the next revision...
With the fixed references, it works now...

address as well as source and destination port.  Therefore there can be at most one active paths/connection ID per 4-tuple.  <ins> The path management specified in Section 9 of QUIC-TRANSPORT fulfills </ins> multiple goals: it directs a peer to switch sending through a new preferred path, and it allows the peer to release resources associated with the old path.  Multipath requires several changes to Manage the removal of paths that have been abandoned. As such this extension specifies a departure from the specification  <ins> of path management in Section 9 of QUIC-TRANSPORT and therefore </ins> requires negotiation between the two endpoints using a new transport parameter, as specified in nego. using the same mechanisms. If validation succeed, the client can send non-probing, 1-RTT packets  <ins> on the new paths.  In contrast with the specification in Section 9 of </ins> QUIC-TRANSPORT, the server MUST NOT assume that receiving non-probing packets on a new path indicates an attempt to migrate to that path. Instead, servers SHOULD consider new paths over which non-probing received for a sufficiently long path-idle delay, but SHOULD only release resource for the last available path if no traffic is received for the duration of the idle timeout, as specified in  <ins> Section 10.1 of QUIC-TRANSPORT.  This means if all paths remain idle </ins> for the idle timeout, the connection is implicitly closed. Server implementations need to select the sub-path idle timeout as a


Path Identifier (Path ID): An identifier that is used to identify a path in a QUIC connection at an endpoint.  Path Identifier is <del> used in multi-path control frames (etc.  PATH_ABANDON frame) to </del>  identify a path.  By default, it is defined as the sequence number of the destination Connection ID used for sending packets on that particular path, but alternative definitions can be used if the does not discuss when a client decides to initiate a new path.  We delegate such discussion in separate documents. <del> This proposal adds one multi-path control frame for path management: </del>  PATH_ABANDON frame for the receiver side to abandon the path 11. <del> Multi-path QUIC transport error codes are 62-bit unsigned integers </del>  following QUIC-TRANSPORT. This section lists the defined multipath QUIC transport error codes

Be consistent with title.
NAME flagged in his PRs that there are many places using hyphenated "multi-path" and many using "multipath". We need to fix that.
This is due to the merge of the different drafts. I thought I fixed most of them, but I guess I could have done a search :-) Will do

Path Identifier (Path ID): An identifier that is used to identify a path in a QUIC connection at an endpoint.  Path Identifier is  <ins> used in multipath control frames (etc.  PATH_ABANDON frame) to </ins> identify a path.  By default, it is defined as the sequence number of the destination Connection ID used for sending packets on that particular path, but alternative definitions can be used if the does not discuss when a client decides to initiate a new path.  We delegate such discussion in separate documents.  <ins> This proposal adds one multipath control frame for path management: </ins> PATH_ABANDON frame for the receiver side to abandon the path 11.  <ins> Multipath QUIC transport error codes are 62-bit unsigned integers </ins> following QUIC-TRANSPORT. This section lists the defined multipath QUIC transport error codes


arrival of non-probing packets on the available paths.  Servers SHOULD stop sending traffic on paths through where no non-probing packet was received in the last 3 path RTTs, but MAY ignore that rule <del> if it would disqualify all available paths.  Server MAY release the resource associated with paths for which no non-probing packet was received for a sufficiently long path-idle delay, but SHOULD only release resource for the last available path if no traffic is received for the duration of the idle timeout, as specified in Section 10.1 of QUIC-TRANSPORT.  This means if all paths remain idle for the idle timeout, the connection is implicitly closed. </del>  Server implementations need to select the sub-path idle timeout as a trade- off between keeping resources, such as connection IDs, in use

To
The current draft says that "Servers SHOULD stop sending traffic on paths through where no non-probing packet was received in the last 3 path RTTs". However, it is possible that only a few data was generated in the past few RTTs due to app-limit conditions, and as a result, only one path was used. So I think, in section 3.2.3, we should suggest the use of some keep-alive mechanisms such as sending ping frames on a particular path if one is interested in keeping that path alive. Let me know what you think?
I fully agree this is an issue to be solved (). However, I think that before trying to find measures at the client side to take into account that server behavior, the text should provide a motivation for such restriction, especially that in typical scenarios the data is flying from the servers and that MPACK may be over any path (BTW, s/ACKMP frame Section 10.2/ACKMP frame (Section 10.2) I'm afraid the use of keepalives may not work in scenarios where the external addresses are not local to the client (and thus has no control on the paths that will be used), e.g., ~~ C/S negotiate multipathing by default. R distributes the load based on a local policy. S/C uses PATHCHALLENGE and PATH_RESPONSE to validate the paths. C does not control the path that will be used by R when forwarding packets.
Med, I am not sure that I understand the scenario that you are describing. There is a router R balancing the load over two paths, and using some NAT like functionality to rewrite the client source address, and presumably the client's port number. This is not supported by either RFC 9000 or by our multipath draft. The server will likely treat arrival from NAME and NAME as NAT rebinding, and the connection will end up using a single path.
R is multi-homed with PA (Provider Assigned) mode . In such mode, R uses the appropriate source IP address when forwarding packets over a given network attachment. With RFC9000, and depending on the distribution logic used by R (per packet, per flow, round robin, maximize bw usage, RTT-based optimization, ...), there will be a path oscillation for this connection as the server will keep proceeding with path migrations each time it sees a new address (NAME NAME NAME and so on). Now, if both the client and server negotiates multipath, we don't have anymore this oscillation and the connection will use more resources. However, prematurely closing the connection nullifies the benefits of using multipath.
Actually, we will still have oscillation even if the endpoints negotiate multipath. There is not difference of behavior with RFC 9000 here, this scenario will be processed as NAT rebinding. This is not a supported scenario. QUIC is an end to end protocol. Intermediate routers cannot really tell whether the endpoints are supporting multipath, and should not depend on that. For that scenario to work, the alternate addresses NAME and NAME should be somehow managed by the client, and the client should be able to steer traffic towards one or the other.
Sure, the connection is end-to-end. The intermediate routers *do not interfere with the connection. However, an endpoint can still discover the availability of multiple paths (there are plenty mechanisms out there to discover the external IP addresses). Even without a discovery mechanism, many service offerings already allow controlling how flows are bound to external links (including bonding). Think about SDWAN, for example. These policies are configured on the CE router. If this scenario is not supported in this version of the protocol, there should be an applicability section where the target deployments are called out. Some text may be considered to discuss concretely when the client can negotiate multipath activation based on that applicability. Back to the initial issue, I think that more motivation is needed to justify withdrawing a path too prematurely.
The middlebox design that you describe closely resembles address spoofing attacks against QUIC that we discussed and tried to mitigate during the design, so it is no surprise that QUIC does not really support it. It would also break TCP connections, which means it is probably not deployed yet. My advice would be to not try deploy that.
It is deployed in (MP)TCP, but requires another proxy to glue the two connections and hide them to the sever. In QUIC, we can have a much more cleaner design as the router does not have to interfere with the connection + we avoid the second proxy. That's said, I'm not pushing to have this included. We just need to be clear about the applicability scope.
By default, the applicability scope is the same as QUIC version 1.
By default, the applicability scope is the same as QUIC version 1.
Maybe, but I think having an architecture figure to illustrate the basic multipath case that is covered would be helpful. BTW, there is nothing in the current design that prevents multipath QUIC to be established over the scenario we discussed. I admit this is a side effect, not a desired goal.
Let's get back to the problem Yunfei mentioned, I just submitted PR for it. By the way, do we need a transport parameter to negotiate max path idle timeout? Like the connection maxidletimeout in RFC9000, to prevent one of the endpoints end up the path too early...
Why would you have different maxidletimeout for different paths?
The maxidletimeout defined in RFC9000 is used to close whole connection, but not the path. In order to close an idle path more precisely, one might need sth like maxpathidletimeout. I think we want all paths to have the same maxpathidletimeout values, which is used to control when a path should be closed when no non-probing packets was received on that path. Another approach is to use PING frame to keep a path alive.

arrival of non-probing packets on the available paths.  Servers SHOULD stop sending traffic on paths through where no non-probing packet was received in the last 3 path RTTs, but MAY ignore that rule  <ins> if it would disqualify all available paths.  To avoid idle timeout of a path, endpoints can send ack-eliciting packets such as packets containing PING frames Section 19.2 of QUIC-TRANSPORT on that path to keep it alive.  Sending periodic PING frames also helps prevent middlebox timeout, as discussed in Section 10.1.2 of QUIC-TRANSPORT. Server MAY release the resource associated with paths for which no non-probing packet was received for a sufficiently long path-idle delay, but SHOULD only release resource for the last available path if no traffic is received for the duration of the idle timeout, as specified in Section 10.1 of QUIC-TRANSPORT.  This means if all paths remain idle for the idle timeout, the connection is implicitly closed. </ins> Server implementations need to select the sub-path idle timeout as a trade- off between keeping resources, such as connection IDs, in use


value: 0 (default) for disabled. Endpoints use 2-bits in the value field for negotiating one or more <del> PN spaces, available option value for client and server are listed in </del>  param_value_definition : <del> If the parameter is absent or set to 0, the endpoints MUST fallback to QUIC-TRANSPORT with single path and MUST NOT use any frame or mechanism defined in this document. </del>  If endpoint receives unexpected value for the transport parameter "enable_multipath", it MUST treat this as a connection error of type

The current text assumes that the server transport parameters are composed after seeing the client's proposal. As pointed out in issue , this is a deviation from standard QUIC practice. The proposed text fixes that and specifies what happens if both endpoints select option 3, "support both". In that case, option 2 "multiple number spaces" is selected.
In Section 2 you define a transport parameter and set of values. These are presented in a table with a column header "Allowed server responses", what does that mean? One way to read this is that it implies servers would tailor their Transport Parameters to the value sent by the client but that's conflicting with our philosophy that TPs are unilateral declarations, not active negotiation. So the question for me is, is this just a wording issue or a design one?
I think this mainly means that server are not allow to send 0x3. However, if the client sends 0x3, the server's reply will impact the client behaviour for 1-RTT packets. Is that a negotiation? However, keep in mind that the intention is not to publish this as it.
unilateral negotiation is ok by me. If that's the intent then maybe just rewording of the table to be clariy it is a matrix of combinations is ok, and state somewhere that server 0x3 is illegal. do you mean not to adopt the draft with this type of TP, or the final RFC? I think better clarity is needed for implementation and experiementation as we try and answer the PN question
NAME that's my fault. I wrote that table based on my implementation, in which the server response does depend on the client proposal. We should replace it by a matrix showing what is expected if the client proposes 0-1-2-3 and the server 0-1-2-3. The only problematic case is "3 and 3", and we should simply state what happens there.
NAME please review PR
NAME thanks for the clarification, the new text makes the intent much more clear IMO.
Just to clarify, we are not intending to have this in the final RFC. We would like to have it adopted as it to enable more experimentation.
thanks NAME that also works for me!
LTGTM just 2 nits

value: 0 (default) for disabled. Endpoints use 2-bits in the value field for negotiating one or more  <ins> PN spaces, available option values are listed in </ins> param_value_definition :  <ins> If for any one of the endpoints the parameter is absent or set to 0, or if the two endpoints select incompatible values, one proposing 0x1 and the other proposing 0x2, the endpoints MUST fallback to QUIC- TRANSPORT with single path and MUST NOT use any frame or mechanism defined in this document. If an endpoint proposes the value 0x3, the value proposed by the other is accepted.  If both endpoints propose the value 0x3, the value 0x2 is negotiated. </ins> If endpoint receives unexpected value for the transport parameter "enable_multipath", it MUST treat this as a connection error of type


the packet number.  If multiple packet number spaces are used, the packet number alone would not guarantee the uniqueness of the nonce. <del> In order to guarantee the uniqueness of the None, the nonce N is </del>  calculated by combining the packet protection IV with the packet number and with the path identifier.

(OK, so it's not the biggest contribution I've ever made, but I DID notice it, while reading the master branch ...
This relates to only one typo, so I merge this PR.
Thanks for correcting this!

the packet number.  If multiple packet number spaces are used, the packet number alone would not guarantee the uniqueness of the nonce.  <ins> In order to guarantee the uniqueness of the nonce, the nonce N is </ins> calculated by combining the packet protection IV with the packet number and with the path identifier.


We assume that the reader is familiar with the terminology used in QUIC-TRANSPORT.  In addition, we define the following terms:  Path Identifier (Path ID): An identifier that is used to identify a path in a QUIC connection at an endpoint.  Path Identifier is used in multipath control frames (etc.  PATH_ABANDON frame) to

fixes issue
Good point but I guess this definition is inline with what's already specified for QUIC given NATs...
Consider adding this entry to the terminology section: This also clarifies the link with "network path" in RFC9000.
PR merged
Note that this definition is different from RFC6182 that required the utilization of at least one different addresses for different subflows. In practice, there were deployments where this requirement was not respected and it was overly conservative.

We assume that the reader is familiar with the terminology used in QUIC-TRANSPORT.  In addition, we define the following terms: <ins> Path: refers to the 4-tuple {source IP address, source port number, destination IP address, destination port number}. A path refers to "network path" used in QUIC-TRANSPORT. </ins> Path Identifier (Path ID): An identifier that is used to identify a path in a QUIC connection at an endpoint.  Path Identifier is used in multipath control frames (etc.  PATH_ABANDON frame) to


uses a series of consecutive sequence numbers without creating holes.  7.2. If the multipath option is enabled with a value of 2, each path has

To estimate path delay accurately, loss recovery of QUIC utilizes the ACK Delay field of ACK frame (see ). However, when single packet number space is being used, it is difficult for the sender to obtain ACK Delay for all the paths being used, because ACK Delay is reported only for the packet carrying the largest packet number. It is the only the ACK delay of the path with shortest path-delay that can be reported. I think it might be a good idea to acknowledge this shortcoming in the documentation, so that we do not miss the downside when we compare the approaches (or when we try to address the shortcomings of the single packet number space approach).
I agree that this issue should be addressed in the current draft. Regarding the problem itself, I think using the one-way timestamp can solve this problem for single PN.
I believe this is discussed in section 5: URL Can you propose a PR if you think more text is useful?!
I think Section 5 should relate to both packet number space designs, while this issue is specific to one. I summarised the discussion in .
We move section 5 from the single packet number section into an own section, as we agreed that it is applicable to both cases as we don't require ack to be send on the same path and the packet was received. I also don't understand the point about being only able to measure the shortest path with a single PN space. It kind of randomly depends on with packet you ack as discussed in section 5. If we want to add some additional consideration we should probably add them to section 5.
Yes, there is an issue with single space here. Suppose that the sender sends packets 1 and later 3 on path long delay path A, packets 2 and later 4 on short delay path B. The receiver will get the packets in the order 2, 1, 4, 2. The ACK will be: ACK, highest = 2. Ack delay = time ACK sent minus time 2 received. ACK, highest = 2, also ACK 1. Ack delay = time ACK sent minus time 2 received. ACK, highest = 4, also ACK 2 and 1. Ack delay = time ACK sent minus time 4 received. ACK, highest = 4, also ACK 3, 2 and 1. Ack delay = time ACK sent minus time 4 received. The RTT of path B will be evaluated precisely, with the help of the ACK delay. The RTT of path A will be evaluated less precisely. When the ACK of 2 and 1 is received, the sender cannot use the ACK delay to narrow the estimate of the RTT of path A. Same for packet 4.
Looks good.

uses a series of consecutive sequence numbers without creating holes. <ins> 7.1.2. The ACK Delay field of the ACK frame is relative to the largest acknowledged packet number (see Section 13.2.5 of QUIC-TRANSPORT). When using paths with different transmission delays, the reported host delay will most of the time relate to the path with the shortest latency.  To collect ACK delays on all the paths, hosts should rely on time stamps as described in QUIC-Timestamp. </ins> 7.2. If the multipath option is enabled with a value of 2, each path has


Multipath Extension for QUIC <del> draft-lmbdhk-quic-multipath-latest </del>  Abstract

Renames the markdown document and updates the README. The new links in the README that point to github-hosted pages might be dead for a while during the process.
Looks good.

Multipath Extension for QUIC  <ins> draft-ietf-quic-multipath-latest </ins> Abstract


risk of packets being dropped by middleboxes (which may only support QUIC version 1) <del> Congestion Control, RTT measurements and PMTU discovery should be per-path (following QUIC-TRANSPORT) </del>  A path is determined by the 4-tuple of source and destination IP address as well as source and destination port.  Therefore there

The sentence above is part of the introduction, but the should is misleading for congestion control at least (we have a MUST for per-path congestion control later). I would suggest to split the above sentence in two parts: Congestion Control must be performed per-path Path MTU should be performed per path We should probably list somewhere in the document the state that must/should be maintained on a per-path basis and the state that is related to the connection. This could clarify some of the mechanisms. Concerning PathMTU, I would suggest to stay with a should and not a must since an implementation could simply track the minimum MTU among the different paths without maintaining per-path MTU. Such an implementation would work and if the MTUs do not differ significantly, it would probably be simpler than an implementation than maintains per-path MTU and needs to adjust the packets based on the path where each packet is sent. Concerning RTT, we discuss the problem of measuring the RTT later with the packet number spaces discussion. Furthermore, some congestion control schemes need an accurate per-path RTT while others do not strictly require rtt estimations.
NAME this "should" in the intro was not meant to be normative. However, I anyway created PR to clarify. However, I didn't address your point about having a list of all state. Do we really need that? If so, where does that below? A new section somewhere? Can you provide such a list or create a PR? Or should we open a new separate issue for that?
I agree that specifying PMTUD per path is a good idea. We do have a slight ambiguity, because the draft uses the overlapping notions of "path" and "number space", with the number space tied to the DCID of packets. The DCID may change over time for privacy reasons, or because the server wants to retire old CIDs tied to a previous configuration of the load balancer. In these cases, there is no need to restart PMTUD or reset congestion control.

risk of packets being dropped by middleboxes (which may only support QUIC version 1)  <ins> Congestion Control must be per-path (following QUIC-TRANSPORT) which usually also requires per-path RTT measurements PMTU discovery should be performed per-path </ins> A path is determined by the 4-tuple of source and destination IP address as well as source and destination port.  Therefore there


This is a request for the entire body. This is used to confirm that the current set of MAX_PAYLOADS <del> payloads have been successfully received and that, upon receipt, the peer can continue to sent the next set of payloads.  This is the 'Continue' Q-Block-2. </del>  <del> This is a request for the remaining payloads in the current MAX_PAYLOADS set. </del>  If the request includes multiple Q-Block2 Options and these options overlap (e.g., combination of M being set (this and later blocks) and these missing blocks is as follows: The token to use for the response SHOULD be the token that was used <del> in the highest block number received payload.  Note that the use of any received token would work, but providing the one used in the highest received block number will aid any troubleshooting.  The client will use the token to match the request to find what Request- Tag value is currently being used. </del>  If the size of the 4.08 (Request Entity Incomplete) response packet is larger than that defined by Section 4.6 RFC7252, then the number

The draft describes this all as "just working through a proxy" -- original block-wise does not, and I'd like to hear how this is hoping to get away without the options being proxy-unsafe. Is there any point at all in using block-by-block proxying for these use cases? Unless a proxy is sitting right inside the link under attack, and unless the bodies get huge, it may be way easier for this to be purely hop-by-hop. Proxies would reassemble the full bodies and, for the next hop, just decide what works best there (be it regular block-wise, Block34 or BERT). (If there's a point in allowing block-by-block forwarding, we can talk it through and work through the initial ideas that will come up like "We use large enough Request-Tag values so they are unique", "how large does that need to be", "why do we end up having a Request-Tag in every request now" and "why do we not want that", but my gut feeling is we won't need to go there.)

This is a request for the entire body. This is used to confirm that the current set of MAX_PAYLOADS  <ins> payloads (the latest one having block number NUM-1) has been successfully received and that, upon receipt of this request, the server can continue to send the next set of payloads (the first one having block number NUM).  This is the 'Continue' Q-Block-2. </ins>  <ins> This is a request for that block and for all of the remaining blocks in the current MAX_PAYLOADS set. </ins> If the request includes multiple Q-Block2 Options and these options overlap (e.g., combination of M being set (this and later blocks) and these missing blocks is as follows: The token to use for the response SHOULD be the token that was used  <ins> in the highest block number received so far with the same Request-Tag value.  Note that the use of any received token with the same Request-Tag would work, but providing the one used in the highest received block number will aid any troubleshooting.  The client will use the token to determine what is the previously sent request to obtain the Request-Tag value to be used. </ins> If the size of the 4.08 (Request Entity Incomplete) response packet is larger than that defined by Section 4.6 RFC7252, then the number