to determine a suitable Local Endpoint), as well as all properties necessary for candidate selection. <del> The Initiate() Action consumes the Preconnection.  Once Initiate() has been called, no further properties may be added to the Preconnection, and no subsequent establishment call may be made on the Preconnection. </del>  Once Initiate is called, the candidate Protocol Stack(s) may cause one or more candidate transport-layer connections to be created to Preconnection during the pre-establishment phase with a Local Endpoint specifier, as well as all properties necessary for Protocol Stack selection.  A Remote Endpoint may optionally be specified, to <del> constrain what Connections are accepted.  The Listen() Action returns a Listener object.  Once Listen() has been called, properties added to the Preconnection have no effect on the Listener and the Preconnection can be disposed of or reused. </del>  Listening continues until the global context shuts down, or until the Stop action is performed on the Listener object: Endpoint to the Remote Endpoint.  This corresponds to a TCP simultaneous open, for example. <del> The Rendezvous() Action consumes the Preconnection.  Once Rendezvous() has been called, no further properties may be added to the Preconnection, and no subsequent establishment call may be made on the Preconnection. </del>  The RendezvousDone<> Event occurs when a Connection is established with the Remote Endpoint.  For Connection-oriented transports, this

Fixed text in the API draft to allow reusing Preconnections as discussed in . However, changes to the Preconnection MUST NOT have any effect on the Connections or Listeners created beforehand - I think normative language is useful here to really highlight this possible pitfall. I omitted any note on deep copy as that's just a way to implement this.
Great catch, it makes me wonder why we had closed issue ? Thanks for doing this!Thank you - fix achieved with minimal changes ;-)

to determine a suitable Local Endpoint), as well as all properties necessary for candidate selection.  <ins> The Initiate() Action returns a Connection object.  Once Initiate() has been called, any changes to the Preconnection MUST NOT have any effect on the Connection.  However, the Preconnection can be reused, e.g., to Initiate another Connection. </ins> Once Initiate is called, the candidate Protocol Stack(s) may cause one or more candidate transport-layer connections to be created to Preconnection during the pre-establishment phase with a Local Endpoint specifier, as well as all properties necessary for Protocol Stack selection.  A Remote Endpoint may optionally be specified, to  <ins> constrain what Connections are accepted. The Listen() Action returns a Listener object.  Once Listen() has been called, any changes to the Preconnection MUST NOT have any effect on the Listener.  The Preconnection can be disposed of or reused, e.g., to create another Listener. </ins> Listening continues until the global context shuts down, or until the Stop action is performed on the Listener object: Endpoint to the Remote Endpoint.  This corresponds to a TCP simultaneous open, for example.  <ins> The Rendezvous() Action returns a Connection object.  Once Rendezvous() has been called, any changes to the Preconnection MUST NOT have any effect on the Connection.  However, the Preconnection can be reused, e.g., for Rendezvous of another Connection. </ins> The RendezvousDone<> Event occurs when a Connection is established with the Remote Endpoint.  For Connection-oriented transports, this


Connections adds a third Connection to the Connection Group, and so on.  Connections in a Connection Group generally share Connection Properties.  However, there may be exceptions, such as "Priority <del> (Connection)", see conn-priority. </del>  In addition, incoming entangled Connections can be received by creating a Listener on an existing connection: not only inside endpoints but also across the end-to-end Internet path.  If the underlying Protocol Stack does not support cloning, or cannot create a new stream on the given Connection, then attempts to clone a Connection will result in a CloneError: inverse priority of this Connection relative to other Connections in the same Connection Group.  It has no effect on Connections not part of a Connection Group.  As noted in groups, this property is not <del> entangled when Connections are cloned. </del>  11.1.4.

Make clearer what "Priority is not entangled" means, following . Also added a note that Clone() may result in on-the-wire signaling.
Section 11.1.3 of the API says about Priority: "As noted in Section 6.4, this property is not entangled when Connections are cloned." Not sure what this means. I though when you clone a connection you would actually take the priority of the cloned connection but then you can change it independently for each connection in a group. Or what's the priority after cloning? Back to the default of 100? We should say this explicitly. Or maybe you even want to provide the priority while cloning? Actually looking at this now again, I think we would need to specify cloning a bit better anyway. I assume that calling clone() on a multiplexing capable connection would create a new stream; in case of quic is would only create local state but no on-the-wire singling. However if you call clone on a TCP connection, is that basically an initial() call that triggers the TCP handshake? We should say this explicitly.
I'd say it's exactly as you thought: When you clone a Connection, all Properties are copied (including the Priority), but then they are not entangled: If you change the Priority on the cloned Connection, the Priority on the original Connection stays as it was. If Priority was entangled, then it would change on both Connections if you change it on one of them. But yes, perhaps this is confusing and the text should be more explicit. The part on specifying cloning better: There's text on this in the Implementation draft on what happens with Clone() for each protocol. But maybe we can say in the API document that Clone() might trigger on-the-wire signaling or it might not, depending on the protocol.
Yes, both additions/clarification would be good.
LGTM - and thanks for adding this clarification!Thank you for the clarifications!

Connections adds a third Connection to the Connection Group, and so on.  Connections in a Connection Group generally share Connection Properties.  However, there may be exceptions, such as "Priority  <ins> (Connection)", see conn-priority.  Like all other Properties, Priority is copied to the new Connection when calling Clone(), but it is not entangled: Changing Priority on one Connection does not change it on the other Connections in the same Connection Group. </ins> In addition, incoming entangled Connections can be received by creating a Listener on an existing connection: not only inside endpoints but also across the end-to-end Internet path. <ins> Note that calling Clone() may result in on-the-wire signaling, e.g., to open a new connection, depending on the underlying Protocol Stack. </ins> If the underlying Protocol Stack does not support cloning, or cannot create a new stream on the given Connection, then attempts to clone a Connection will result in a CloneError: inverse priority of this Connection relative to other Connections in the same Connection Group.  It has no effect on Connections not part of a Connection Group.  As noted in groups, this property is not  <ins> entangled when Connections are cloned, i.e., changing the Priority on one Connection in a Connection Group does not change it on the other Connections in the same Connection Group. </ins> 11.1.4.


4.1.1. <del> Connection A Connection is the fundamental object used by an application for all interaction with a peer and data transfer.  It is generally capable of bi-directional communication.  It has state that represents the capability of being able to send or receive content between its local and remote endpoints.  This capability may be based merely on the existence of a path between the two endpoints and an application's permission to send and receive on its local endpoint; or may represent successful protocol handshakes at one or more layers that are pre-requisites to receiving content. Listener A Listener is any object that can be used to prepare a Connection with a remote endpoint without initiation by the local application.  That is, it can passively accept initiations for Connections made by another endpoint. </del>  4.1.2. <del> Endpoint An Endpoint represents one side of a transport connection.  It is a concept that represents how an application views itself or a peer that may vary in levels of specificity.  Examples include "IP address + port", "hostname + port", and "DNS service name". Remote Endpoint The Remote Endpoint in a properties [MICHAEL: I can't parse "in a properties" here.  Should this be removed?] represents the application's view of a peer that can participate in a transport connection. Local Endpoint The Local Endpoint in a properties [MICHAEL: I can't parse "in a properties" here.  Should this be removed?] represents the application's view of itself that it wants to use for transport connections.  This may be optional, or may be generic (a wild card address, for example). Path Selection Properties Path Selection Properties consist of the options that an application may set to influence the selection of path between itself and the Remote Endpoint.  These options can come in the form of requirements, prohibitions, or preferences.  Examples of options which may influence path selection include the interface type (such as a Wi-Fi Ethernet connection, or a Cellular LTE connection), characteristics of the path that are locally known like Maximum Transmission Unit (MTU), or expected throughput or latency. Protocol Selection Properties Protocol Selection Properties consist of the options that an application may set to influence the selection of transport protocol, configure the behavior of generic transport protocol features.  These options come in the form of requirements, prohibitions, and preferences.  Examples include reliability, service class, multipath support, and fast open support. Specific Protocol Properties Specific Protocol Properties refers to the subset of Protocol Properties options that apply to a single protocol (transport protocol, IP, or security protocol).  The presence of such Properties does not necessarily require that a specific protocol must be used, but that if this protocol is employed, a particular set of options should be used.  This is critical to allow compatibility with Protocol Properties on peers. </del>  4.1.3. <del> Initiate Initiate is the primary action that an application can take to create a Connection to a remote endpoint, and prepare any required local or remote state to be able to send and/or receive content.  For some protocols, this may initiate a server-to-client style handshake; for other protocols, this may just establish local state; and for peer-to-peer protocols, this may begin the process of a simultaneous open. Listen Listen is the action of marking a Listener as willing to accept incoming Connections. </del>  4.1.4. <del> Content Content is a unit of data that can be represented as bytes that can be transferred between two endpoints over a transport connection.  The bytes within a unit of Content are assumed to be ordered within the unit itself.  That is, if an application does not care about the order in which a peer receives two distinct spans of bytes, those spans of bytes are considered independent units of Content.  Content may or may not be usable if incomplete or corrupted.  Boundaries of Content may or may not be understood or transmitted by transport protocols.  That is, what one application considers to be two units of Content sent on a stream-based transport may be treated as a single unit of Content by the application on the other side. Send Send is the action to transmit Content over a Connection to a remote endpoint.  The interface to Send may include options specific to how this Content is to be sent.  Status of the Send operation may be delivered back to the application in an event events. Receive Receive is the indication that the application is ready to asynchronously accept Content over a Connection from a Remote Endpoint, while the Content itself will be delivered in an event events.  The interface to Receive may include options specific to the Content that is to be delivered to the application. </del>  4.1.5. exhaustive, but gives the top-level categories of events.  The API may expand this list. <del> Connection Ready A Connection Ready event signals to an application that a given Connection is ready to send and/or receive Content.  If the Connection relies on handshakes to establish state between peers, then it is assumed that these steps have been taken. Connection Finished A Connection Finished event signals to an application that a given Connection is no longer usable for sending or receiving Content.  This should deliver a useful error to the application. Connection Received A Connection Received event signals to an application that a given Listener has passively received a Connection. Content Received A Content Received event delivers received content to the application, based on a Receive action.  This may include an error if the action failed. Content Sent A Content Sent event notifies the application of the status of its Send action.  This may be an error, an indication that Content has been processed by the protocol stack, or potentially [MICHAEL: Why "potentially" - because you're not sure about it?  If it may be such an indication, "potentially" should be removed (this seems like the right thing to do for me).] an indication that Content has been acknowledged by a peer. Path Properties Changed A Path Properties Changed event notifies the application that some property of the Connection has changed that may influence how and where data is sent and/or received. </del>  4.1.6. <del> Close </del>  <del> Close is the action an application may take on a Connection to indicate that it no longer intends to send data, is no longer willing to receive data, and that the protocol should signal this state to the remote endpoint if applicable. Abort Abort is an action the application may take on a Connection to indicate a Close, but with the additional indication that the transport system should not attempt to deliver any outstanding data. </del>  4.2. functionality of transport networking, as required by the abstract interface. <del> Connection Group For multiplexing transport protocols, a Connection Group is a set of Connections that can be multiplexed together. Path A Path represents an available set of properties of a network route on which packets may be sent or received. Protocol Instance A Protocol Instance is a single instance of one protocol, including any state it has necessary to establish connectivity or send and receive content. Protocol Stack A Protocol Stack is a set of Protocol Instances (including relevant application, security, transport, or Internet protocols) that are used together to establish connectivity or send and receive content.  A single stack may be simple (a single transport protocol instance over IP), or complex (multiple application protocol streams going through a single security and transport protocol, over IP; or, a multi-path transport protocol over multiple transport sub-flows). System Policy A Transport Service System Policy defines the algorithm it uses to take connection properties from the application, and determine how it will gather candidate paths and protocols gathering and race the candidates during establishment racing. Association Cache The Association Cache holds the state that the implementation keeps for each set of associated endpoints that have been used previously.  This can include DNS results, TLS session state, previous success and quality of transport protocols over certain paths. </del>  4.2.1. <del> Path Candidate Identification/Gathering </del>  <del> Path Selection represents the act of choosing one or more paths that are available to use based on the Path Selection Properties provided by the application, and a Transport Services system's policies and heuristics.  [MICHAEL: I find it awkward that the terms and headline here talk about "Gathering" but then this description doesn't.  Maybe because "Path Selection" and "Protocol Selection" are indeed clearer, easier terms than this Identification/Gathering stuff?] Protocol Candidate Identification/Gathering Protocol Selection represents the act of choosing one or more sets of protocol options that are available to use based on the Protocol Properties provided by the application, and a Transport Services system's policies and heuristics.  [MICHAEL: Same as above, here...] </del>  4.2.2. <del> Protocol Option Racing [MICHAEL: About all the definitions below: I believe you can't "attempt a stack", you can only attempt to _use_ a stack.  Not easy to fix in a nice way because of the "scheduling attempts for", but I think as it stands, this is grammatically broken.  Then again, I'm no native speaker!] Protocol Racing is the act of attempting, or scheduling attempts for, multiple Protocol Stacks that differ based on the composition of protocols or the options used for protocols. Path Racing Path Racing is the act of attempting, or scheduling attempts for, multiple Protocol Stacks that differ based on a selection from the available Paths. Endpoint Racing </del>  <del> Endpoint Racing is the act of attempting, or scheduling attempts for, multiple Protocol Stacks that differ based on the specific representation of the remote and local endpoints, such as IP addresses resolved from a DNS hostname. </del>  5.

Incorporate Preconnection into the architecture and API draft.
Looks good. I pushed a commit to complete the change of the list style to include the implementation concepts section.

4.1.1.  <ins> Preconnection: A Preconnection object is a representation of a potential connection.  It has state that describes parameters of a Connection that might exist in the future: the Local Endpoint from which that Connection will be established, the Remote Endpoint to which it will connect, and Path Selection Properties, Protocol Selection Properties, and Specific Protocol Properties that influence the choice of transport that Connection will use.  A Preconnection can be fully specified and represent a single possible Connection, or it can be partially specified such that it represents a family of possible Connections.  The Local Endpoint MUST be specified if the Preconnection is used to Listen() for incoming connections, but is OPTIONAL if it is used to Initiate() connections.  The Remote Endpoint MUST be specified in the Preconnection is used to Initiate() connections, but is OPTIONAL if it is used to Listen() for incoming connections. Connection: A Connection object represents an active transport protocol instance that can send and/or receive Content between a Local Endpoint and a Remote Endpoint.  It holds state pertaining to the underlying transport protocol instance and any ongoing data transfer.  This represents, for example, an active connection in a connection-oriented protocol such as TCP, or a fully-specified 5-tuple for a connectionless protocol such as UDP. Listener: A Listener object accepts incoming transport protocol connections from Remote Endpoints and generates corresponding Connection objects.  It is created from a Preconnection object that specifies the type of incoming connections it will accept. </ins> 4.1.2.  <ins> Endpoint: An Endpoint represents one side of a transport connection.  It is a concept that represents how an application views itself or a peer that may vary in levels of specificity. Examples include "IP address + port", "hostname + port", and "DNS service name". Remote Endpoint: The Remote Endpoint represents the application's name for a peer that can participate in a transport connection. For example, a DNS name for the peer. Local Endpoint: The Local Endpoint represents the application's name for itself that it wants to use for transport connections. For example, a local interface name or address. Path Selection Properties: The Path Selection Properties consist of the options that an application may set on a Preconnection to influence the selection of path between itself and the Remote Endpoint.  These options can come in the form of requirements, prohibitions, or preferences.  Examples of options which may influence path selection include the interface type (such as a Wi- Fi Ethernet connection, or a Cellular LTE connection), characteristics of the path that are locally known like Maximum Transmission Unit (MTU), or expected throughput or latency. Protocol Selection Properties: The Protocol Selection Properties consist of the options that an application may set on a Preconnection to influence the selection of transport protocol, configure the behavior of generic transport protocol features. These options come in the form of requirements, prohibitions, and preferences.  Examples include reliability, service class, multipath support, and fast open support. Specific Protocol Properties: The Specific Protocol Properties refer to the subset of Protocol Properties options that apply to a single protocol (transport protocol, IP, or security protocol). The presence of such Properties on a Preconnection does not necessarily require that a specific protocol must be used when a Connection is established from that Preconnection, but that if this protocol is employed, a particular set of options should be used.  This is critical to allow compatibility with Protocol Properties on peers. </ins> 4.1.3.  <ins> Initiate() is the primary action that an application can take to create a Connection to a remote endpoint, and prepare any required local or remote state to be able to send and/or receive content. For some protocols, this may initiate a server-to-client style handshake; for other protocols, this may just establish local state; and for peer-to-peer protocols, this may begin the process of a simultaneous open.  The Initiate() action consumes a Preconnection object and creates a Connection object.  Resolution of the remote endpoint identifier occurs during the Initiate() call. Listen() is the action of marking a Listener as willing to accept incoming Connections.  It consumes a Preconnection object.  The Listener will then create Connection objects as incoming connections are accepted. </ins> 4.1.4.  <ins> Content: A Content object is a unit of data that can be represented as bytes that can be transferred between two endpoints over a transport connection.  The bytes within a unit of Content are assumed to be ordered within the unit itself.  That is, if an application does not care about the order in which a peer receives two distinct spans of bytes, those spans of bytes are considered independent units of Content.  Content may or may not be usable if incomplete or corrupted.  Boundaries of Content may or may not be understood or transmitted by transport protocols.  That is, what one application considers to be two units of Content sent on a stream-based transport may be treated as a single unit of Content by the application on the other side. Send() is the action to transmit Content over a Connection to a remote endpoint.  The interface to Send may include options specific to how this Content is to be sent.  Status of the Send operation may be delivered back to the application in an event events. Receive() is an action that indicates that the application is ready to asynchronously accept Content over a Connection from a Remote Endpoint, while the Content itself will be delivered in an event events.  The interface to Receive may include options specific to the Content that is to be delivered to the application. </ins> 4.1.5. exhaustive, but gives the top-level categories of events.  The API may expand this list.  <ins> Connection Ready: A Connection Ready event signals to an application that a given Connection is ready to send and/or receive Content.  If the Connection relies on handshakes to establish state between peers, then it is assumed that these steps have been taken. Connection Finished: A Connection Finished event signals to an application that a given Connection is no longer usable for sending or receiving Content.  This should deliver a useful error to the application. Connection Received: A Connection Received event signals to an application that a given Listener has passively received a Connection. Content Received: A Content Received event delivers received content to the application, based on a Receive action.  This may include an error if the action failed. Content Sent: A Content Sent event notifies the application of the status of its Send action.  This may be an error, an indication that Content has been processed by the protocol stack, or potentially [MICHAEL: Why "potentially" - because you're not sure about it?  If it may be such an indication, "potentially" should be removed (this seems like the right thing to do for me).] an indication that Content has been acknowledged by a peer. Path Properties Changed: A Path Properties Changed event notifies the application that some property of the Connection has changed that may influence how and where data is sent and/or received. </ins> 4.1.6.  <ins> Close() is the action an application may take on a Connection to indicate that it no longer intends to send data, is no longer willing to receive data, and that the protocol should signal this state to the remote endpoint if applicable. </ins>  <ins> Abort() is an action the application may take on a Connection to indicate a Close, but with the additional indication that the transport system should not attempt to deliver any outstanding data. </ins> 4.2. functionality of transport networking, as required by the abstract interface.  <ins> Connection Group: For multiplexing transport protocols, a Connection Group is a set of Connections that can be multiplexed together. Path: A Path represents an available set of properties of a network route on which packets may be sent or received. Protocol Instance: A Protocol Instance is a single instance of one protocol, including any state it has necessary to establish connectivity or send and receive content. Protocol Stack: A Protocol Stack is a set of Protocol Instances (including relevant application, security, transport, or Internet protocols) that are used together to establish connectivity or send and receive content.  A single stack may be simple (a single transport protocol instance over IP), or complex (multiple application protocol streams going through a single security and transport protocol, over IP; or, a multi-path transport protocol over multiple transport sub-flows). System Policy: A Transport Service System Policy defines the algorithm it uses to take connection properties from the application, and determine how it will gather candidate paths and protocols gathering and race the candidates during establishment racing. Association Cache: The Association Cache holds the state that the implementation keeps for each set of associated endpoints that have been used previously.  This can include DNS results, TLS session state, previous success and quality of transport protocols over certain paths. </ins> 4.2.1.  <ins> Path Selection: Path Selection represents the act of choosing one or more paths that are available to use based on the Path Selection Properties provided by the application, and a Transport Services system's policies and heuristics. </ins>  <ins> Protocol Selection: Protocol Selection represents the act of choosing one or more sets of protocol options that are available to use based on the Protocol Properties provided by the application, and a Transport Services system's policies and heuristics. </ins> 4.2.2.  <ins> Protocol Option Racing: Protocol Racing is the act of attempting to establish, or scheduling attempts to establish, multiple Protocol Stacks that differ based on the composition of protocols or the options used for protocols. Path Racing: Path Racing is the act of attempting to establish, or scheduling attempts to establish, multiple Protocol Stacks that differ based on a selection from the available Paths. </ins>  <ins> Endpoint Racing: Endpoint Racing is the act of attempting to establish, or scheduling attempts to establish, multiple Protocol Stacks that differ based on the specific representation of the remote and local endpoints, such as IP addresses resolved from a DNS hostname. </ins> 5.


is intended for immediate termination of a connection, without cleaning up state.  4.2. This section defines the set of objects used internally to a system transport service across a network, as required by the abstract interface. <del> Connection Group: A set of Connections that share properties and caches.  For multiplexing transport protocols, only Connections within the same Connection Group are allowed to be multiplexed together.  An application can explicitly define Connection Groups to control caching boundaries, as discussed in groups. </del> Path: Represents an available set of properties that a local system can use to communicate with a remote system, such as routes, addresses, and physical and virtual network interfaces.

This is fully editorial: I'm not sure if the term "Connection Group" is introduced in the right section. There might be no perfect fit but maybe section 2.2 is a bit better? Also I note that the terms "Connection Group" and "Path" do not not show up in any of the figures...
Make this a new subsection, i.e. 4.1.7.
Works - Thank you!

is intended for immediate termination of a connection, without cleaning up state. <ins> 4.1.7. A Connection Group is a set of Connections that share properties and caches.  For multiplexing transport protocols, only Connections within the same Connection Group are allowed to be multiplexed together.  An application can explicitly define Connection Groups to control caching boundaries, as discussed in groups. </ins> 4.2. This section defines the set of objects used internally to a system transport service across a network, as required by the abstract interface.  Path: Represents an available set of properties that a local system can use to communicate with a remote system, such as routes, addresses, and physical and virtual network interfaces.


where messageData is the data object to send. <del> The optional messageContext parameter supports per-message properties and is described in message-props.  It can be used to identify send events (see send-events) related to a specific message or to inspect meta-data related to the message sent (see msg-ctx). </del>  The optional endOfMessage parameter supports partial sending and is described in send-partial. Message (i.e., whether it has actually been transmitted, moved into a buffer on the network interface, moved into a kernel buffer, and so on) when the Sent Event occurs is implementation-specific.  The Sent <del> Event contains an implementation-specific reference to the Message to which it applies. </del>  Sent Events allow an application to obtain an understanding of the amount of buffering it creates.  That is, if an application calls the completion; i.e. when the Message was not sent before its Lifetime (see msg-lifetime) expired.  This is separate from SendError, as it is an expected behavior for partially reliable transports.  The <del> Expired Event contains an implementation-specific reference to the Message to which it applies. </del>  7.3.3. condition: an attempt to send a Message which is too large for the system and Protocol Stack to handle, some failure of the underlying Protocol Stack, or a set of Message Properties not consistent with <del> the Connection's transport properties.  The SendError contains an implementation-specific reference to the Message to which it applies. </del>  7.4. Message Properties can be set and queried using the Message Context: To get or set Message Properties, the optional scope parameter is <del> left empty, for framing meta-data, the framer is passed. </del>  For MessageContexts returned by send events (see send-events) and receive events (see receive-events), the application can query

Tentatively answering my own questions in : Removed the messageContext here because the RendezvousError does not necessarily relate to a specific sent Message. If it does, the "reason" parameter can point to that message and/or a messageContext. The beginning of Section 7 actually already said that this IS the messageContext. I removed the word "implementation-specific" because to me this word made it sound like this reference can't be the messageContext. Split into two sentences and changed the second sentence to active voice. Also, I slightly reworded the beginning of Section 7, which is now a bit clearer in my opinion.
When reading the current API draft, I found three minor issues related to Message Contexts. The RendezvousError (Section 6.3) includes a messageContext (which is even required), but neither the InitiateError nor the ListenError include one. Why? Should this messageContext at least be optional?In Section 7.3, it says "The Sent Event contains an implementation-specific reference to the Message to which it applies." - Is this in addition to the messageContext? If yes, why do we need both?In Section 9, it says "To get or set Message Properties, the optional scope parameter is left empty, for framing meta-data, the framer is passed." - I think this sentence is a bit hard to parse and it wasn't immediately obvious to me what kind of framing meta-data this refers to. Perhaps this refers to Section 10.2, so I think it's worth adding a reference to that section here. Also, I suggest turning this into two sentences, with the sentence on framing meta-data specifying that this is per-Message metadata to be used by the framer.
Thanks a lot for doing this! It clears things up nicely, I think.

where messageData is the data object to send.  <ins> The optional messageContext parameter allows adding Message Properties as described in message-props.  Moreover, the messageContext can be used to identify Send Events related to a specific Message (see send-events) or to inspect meta-data related to the Message sent (see msg-ctx). </ins> The optional endOfMessage parameter supports partial sending and is described in send-partial. Message (i.e., whether it has actually been transmitted, moved into a buffer on the network interface, moved into a kernel buffer, and so on) when the Sent Event occurs is implementation-specific.  The Sent  <ins> Event contains a reference to the Message to which it applies. </ins> Sent Events allow an application to obtain an understanding of the amount of buffering it creates.  That is, if an application calls the completion; i.e. when the Message was not sent before its Lifetime (see msg-lifetime) expired.  This is separate from SendError, as it is an expected behavior for partially reliable transports.  The  <ins> Expired Event contains a reference to the Message to which it applies. </ins> 7.3.3. condition: an attempt to send a Message which is too large for the system and Protocol Stack to handle, some failure of the underlying Protocol Stack, or a set of Message Properties not consistent with  <ins> the Connection's transport properties.  The SendError contains a reference to the Message to which it applies. </ins> 7.4. Message Properties can be set and queried using the Message Context: To get or set Message Properties, the optional scope parameter is  <ins> left empty.  To get or set meta-data for a Framer, the application has to pass a reference to this Framer as the scope parameter. </ins> For MessageContexts returned by send events (see send-events) and receive events (see receive-events), the application can query


5.2.12. <del> local-address-preference </del>  <del> Enumeration Default: For Listeners and Rendezvous Connections: Stable.  For other Connections: Temporary This property specifies whether Listeners and Connections should prefer the use of temporary addresses when possible.  This is generally used to prevent linking connections over time when a stable address is not needed.  Possible values are: </del>  <del> Prefer the use of stable (sometimes called "permanent") local addresses </del>  <del> Prefer the use of temporary (sometimes called "privacy") addresses </del>  5.2.13.

Following the suggestion in , made Local Address Preference a Set(Preference, Enumeration), so an application can now also Require or Prohibit stable addresses. Added a sentence warning that, e.g., preferring stable AND temporary addresses does not make sense - but there are less obvious ways to produce a conflicting set of Selection Properties, too, and we know how to deal with them.
Interim discussion: let's use prohibit/avoid/prefer/require temporary addresses. Default connections to prefer temporary. Default listeners to avoid temporary.
Note that it fails without temporary addresses if you require temporary; which makes IPv4 fail.
Simplified this as discussed in the interim. Please take another look.
Looks good to me – But I am unsure whether we should replace the temporary address with a stable address with Preference. Rational: For IPv6 with outgoing connections, temporary is default anyway, so requiring the special case instead of avoid/prohibit the default looks much more straight forward.
I've read this a few times but I'm still not sure: What alternative solution are you proposing here?
I am proposing to Invert the whole name and meaning of the property – instead of requesting a temporary address, request a stable address.
Thanks for clarifying. I think this Property is straightforward enough as is, no need to invert.
For Local Address Preference we only have the option to prefer stable or temporary. However, I guess we would also need a way to e.g. specify that stable is required, no?
Is this addressed by PR ?
I don't think so.
lgtm

5.2.12.  <ins> use-temporary-local-address </ins>  <ins> Preference </ins>  <ins> Avoid for Listeners and Rendezvous Connections.  Prefer for other Connections. </ins>  <ins> This property allows the application to express a preference for the use of temporary local addresses, sometimes called "privacy" addresses RFC4941.  Temporary addresses are generally used to prevent linking connections over time when a stable address, sometimes called "permanent" address, is not needed.  Note that if an application Requires the use of temporary addresses, the resulting Connection cannot use IPv4, as temporary addresses do not exist in IPv4. </ins> 5.2.13.


with RFC2914.  Also note that reliability is usually combined with congestion control in protocol implementations, rendering "reliable but not congestion controlled" a request that is unlikely to succeed. <del> The recommended default is to Require that the Connection is congestion controlled. </del>  5.2.10. specified via Provisioning Domain attributes (see prop-pvd) or another specific property.  5.2.11. pvd specific PvD instances or interface instances, and should be preferred over these options.  5.2.12. local-address-preference

Added defaults for interface and PvD following . Also, I caught one leftover "recommended default".
We require implementations to have default but we don't provide any recommendation for interface and PvD.
For Interface, this would the default be to set "Ignore" on all available interfaces, right? This indicates that the application has no preferences of its own, so the implementation can apply some default system policy. Also, if the application specified some local endpoint, this might constrain the actual interfaces used (related to ). If we try to phrase this as (Preference, Enumeration), it reads a bit weird though: ("Ignore", "Cellular") and ("Ignore", "WiFi"), for example. So perhaps we should just write text saying that the default is to set no application preference.
I think text is fine as well but we need to say something.
Thanks!

with RFC2914.  Also note that reliability is usually combined with congestion control in protocol implementations, rendering "reliable but not congestion controlled" a request that is unlikely to succeed.  <ins> The default is to Require that the Connection is congestion controlled. </ins> 5.2.10. specified via Provisioning Domain attributes (see prop-pvd) or another specific property. <ins> The default is to not set a preference for any interface. </ins> 5.2.11. pvd specific PvD instances or interface instances, and should be preferred over these options. <ins> The default is to not set a preference for any PvD. </ins> 5.2.12. local-address-preference


4.2.3. The Transport Services architecture defines a mechanism that allows <del> applications to easily use different network paths and Protocol Stacks.  In some cases, changing which Protocol Stacks or network paths are used will require updating the preferences expressed by the application that uses the Transport Services system.  For example, an application can enable the use of a multipath or multistreaming transport protocol by modifying the properties in its Pre-Connection configuration.  In some cases, however, the Transport Services system will be able to automatically change Protocol Stacks without an update to the application, either by selecting a new stack entirely, or by racing multiple candidate Protocol Stacks during connection establishment.  This functionality in the API can be a powerful driver of new protocol adoption, but needs to be constrained carefully to avoid unexpected behavior that can lead to functional or security problems. </del>  If two different Protocol Stacks can be safely swapped, or raced in parallel (see racing), then they are considered to be "equivalent".

In the architecture document, I think Section 4.2.3 Protocol Stack Equivalence could be misinterpreted to mean connection migration, i.e., changing network paths or protocol stacks during the lifetime of a single Connection. "The Transport Services architecture defines a mechanism that allows applications to easily use different network paths and Protocol Stacks. In some cases, changing which Protocol Stacks or network paths are used will require updating the preferences expressed by the application that uses the Transport Services system." It says "use different network paths and Protocol Stacks" and talks about changing these, but it does not say in what context or over what timeframe. Maybe here it would help to say something like "The Transport Services architecture defines a mechanism that allows applications to easily use different network paths and Protocol Stacks for their Connections or start using new protocols as they get implemented, without requiring major changes to the applications".
Agreed, sounds good
I don't think there's any particular reason a future transport shouldn't renegotiate paths and stack components on the fly (but then again I think the future of the Internet will see endpoints using TLS/TCP/IPv4 only to establish cryptographic and routing state, then dynamically negotiating connectivity beyond that, i.e. "legacy Internet as a bootstrap protocol alone" so my opinion should probably be regarded as a minority one and best ignored. :) )
+1 to Brian but I guess I was sharing an office with him for too long ;-)
So... This is a case where I wouldn't wish to speculate. It was controversial in the lead-up to TAPS for various reasons, and I'd really encourage we do not start thinking in this RFC about how that may turn in future. I do like Theresa's new para above. I would argue not to add more.
Thanks for fixing!

4.2.3. The Transport Services architecture defines a mechanism that allows  <ins> applications to easily make use of various network paths and Protocol Stacks without requiring major changes in application logic.  In some cases, changing which Protocol Stacks or network paths are used will require updating the preferences expressed by the application that uses the Transport Services system.  For example, an application can enable the use of a multipath or multistreaming transport protocol by modifying the properties in its Pre-Connection configuration.  In some cases, however, the Transport Services system will be able to automatically change Protocol Stacks without an update to the application, either by selecting a new stack entirely, or by racing multiple candidate Protocol Stacks during connection establishment. This functionality in the API can be a powerful driver of new protocol adoption, but needs to be constrained carefully to avoid unexpected behavior that can lead to functional or security problems. </ins> If two different Protocol Stacks can be safely swapped, or raced in parallel (see racing), then they are considered to be "equivalent".


Endpoint.  When receiving Messages, Message Properties can contain information about the received Message, such as metadata generated at the receiver and information signalled by the remote endpoint.  Send: The action to transmit a Message over a Connection to the remote system.  The interface to Send MAY include Message

arch doc sec 4.1.4: "When receiving Messages, Message Properties can contain information about the received Message, such as metadata generated at the receiver and information signalled by the remote endpoint." Maybe provide an example here...?
Suggestion: ECN, which is information given from UDP in the form of such metadata.
QUIC stream identifier example works
Final message is in API, just use that
LGTM modulo one nit - but see if you agree with my suggestion. If you don't, I'm ok with this anyway.

Endpoint.  When receiving Messages, Message Properties can contain information about the received Message, such as metadata generated at the receiver and information signalled by the remote endpoint. <ins> For example, a Message can be marked with a Message Property indicating that it is the final message on a connection if the peer sent a TCP FIN. </ins> Send: The action to transmit a Message over a Connection to the remote system.  The interface to Send MAY include Message


Properties after Connection establishment take effect on a best- effort basis.  4.1.3. Initiate: The primary action that an application can take to

Addresses
I wondering if we should to talk a bit more about authentication in the the arch doc. The API doc has the concept of security parameters, however there is no discussion about this in the arch document. Also, just double-checking: private keys and certificates/identify can be set as a security parameter of a pre-connection. However, is that really a connection parameter or rather a local endpoint property?
To match the API, add Security Properties to Section 4.1.2.
Try to associate with the endpoint?
Add to figure 4 too
I think?

Properties after Connection establishment take effect on a best- effort basis. <ins> Security Parameters: Security Parameters define an application's requirements for authentication and encryption on a Connection. They are used by Transport Security protocols (such as those described in I-D.ietf-taps-transport-security) to establish secure Connections.  Examples of parameters that can be set include local identities, private keys, supported cryptographic algorithms, and requirements for validating trust of remote identities.  Security Parameters are primarily associated with a Preconnection object, but properties related to identities can be associated directly with Endpoints. </ins> 4.1.3. Initiate: The primary action that an application can take to


will never show up when queuing the value of a preference - the effective preference must be returned instead. <del> The implementation MUST ensure a consistent outcome given the same Selection Properties.  Internally, it MUST exclude all protocols and paths that match a Prohibit and exclude all protocols and paths that do not match a Require.  It MUST then sort candidates according to Preferred properties, and then use Avoided properties as a tiebreaker. Note that the protocols and paths which are available on a specific system may vary, such that application preferences may conflict with each other.  For example, if an application indicates a preference for a specific path by specifying an interface, but also a preference for a protocol, a situation might occur in which the preferred protocol is not available on the preferred path.  In such cases, implementations MUST ensure a deterministic outcome by prioritizing Selection Properties that select paths over those that select protocols.  Therefore, the transport system MUST try the path first, ignoring the protocol preference if the protocol does not work on the path. </del>  Selection and Connection Properties, as well as defaults for Message Properties, can be added to a Preconnection to configure the section is the algorithm defining which of these options to try, when, and in what order.  4.1. The step of gathering candidates involves identifying which paths, properties, it should only keep branches that satisfy these requirements.  Finally, it should order branches according to preferred properties, and finally use avoided properties as a <del> tiebreaker. </del>  4.4.

In the , people found the normative language in too restrictive and agreed with the original suggestion in . Therefore, this PR moves the selection algorithm text to the implementation draft and removes some normative language in the interface draft. We still want to provide consistency to applications when possible, so there's a sentence in the implementation draft on this now as well.
LGTM modulo the suggestion from NAME which I would accept as-is Great with one minor knit.

will never show up when queuing the value of a preference - the effective preference must be returned instead.  <ins> The implementation MUST ensure an outcome that is consistent with application requirements as expressed using Require and Prohibit. While preferences expressed using Prefer and Avoid influence protocol and path selection as well, outcomes may vary given the same Selection Properties, as the available protocols and paths may vary across systems and contexts.  However, implementations are RECOMMENDED to aim to provide a consistent outcome to an application, given the same Selection Properties. Note that application preferences may conflict with each other.  For example, if an application indicates a preference for a specific path by specifying an interface, but also a preference for a protocol, a situation might occur in which the preferred protocol is not available on the preferred path.  In such cases, implementations SHOULD prioritize Selection Properties that select paths over those that select protocols.  Therefore, the transport system SHOULD race the path first, ignoring the protocol preference if the protocol does not work on the path. </ins> Selection and Connection Properties, as well as defaults for Message Properties, can be added to a Preconnection to configure the section is the algorithm defining which of these options to try, when, and in what order. <ins> During Candidate Gathering, an implementation first excludes all protocols and paths that match a Prohibit or do not match all Require properties.  Then, the implementation will sort branches according to Preferred properties, Avoided properties, and possibly other criteria. </ins> 4.1. The step of gathering candidates involves identifying which paths, properties, it should only keep branches that satisfy these requirements.  Finally, it should order branches according to preferred properties, and finally use avoided properties as a  <ins> tiebreaker.  When ordering branches, an implementation may give more weight to properties that the application has explicitly set than to properties that are default. As the available protocols and paths on a specific system and in a specific context may vary, the result of sorting and the outcome of racing may vary even given the same Selection and Connection Properties.  However, an implementation ought to aim to provide a consistent outcome to applications, e.g., by preferring protocols and paths that existing Connections with similar Properties are already using. </ins> 4.4.


an asynchronous operation, in which each call to Receive enqueues a request to receive new data from the connection.  Once data has been received, or an error is encountered, an event will be delivered to <del> complete the Receive request (see receive-events). As with sending, the type of the Message to be passed is dependent on the implementation, and on the constraints on the Protocol Stacks implied by the Connection's Transport Properties. </del>  8.1. or to assign the traffic to a lower-effort service.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling SHOULD assign the DSCP Less <del> than Best Effort LE-PHB PHB. </del>  The application is interactive, and prefers loss to latency. Response time should be optimized at the expense of bandwidth supported by the underlying transport; and so on.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD <del> assign the DSCP Expedited Forwarding RFC3246 PHB. </del>  The application prefers loss to latency but is not interactive. Response time should be optimized at the expense of bandwidth

: added a statement about data arriving before Receive is issued; : removed the confusing sentence; : carried out replacements suggested by NAME
This issue comes from other discussion at the interim of what it means to map to DSCP classes, and proposes to fix two topics. Replace reference OLD: DSCP Less than Best Effort [LE-PHB] PHB NEW: DSCP Less than Best Effort [RFC8622] PHB — Low Latency/Interactive: The current advice is wrong. An endpoint never should be assigning EF without understanding the implication of CAC and policing. OLD: Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD assign the DSCP Expedited Forwarding [RFC3246] PHB; NEW: Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD assign a DSCP Assured Forwarding (AF41,AF42,AF43,AF44) [RFC2597] PHB. Inelastic traffic that is expected to conform to the configured network service rate could be mapped to the DSCP Expedited Forwarding [RFC3246] or [RFC5865] PHBs. —
The new text resolve this issue.
I'm not sure I understand this sentence (in sec 8. - Receiving Data): "As with sending, the type of the Message to be passed is dependent on the implementation, and on the constraints on the Protocol Stacks implied by the Connection's transport parameters." Especially, what's meant by "type of Message"?

an asynchronous operation, in which each call to Receive enqueues a request to receive new data from the connection.  Once data has been received, or an error is encountered, an event will be delivered to  <ins> complete any pending Receive requests (see receive-events).  If Messages arrive at the transport system before Receive requests are issued, ensuing Receive requests will first operate on these Messages before awaiting any further Messages. </ins> 8.1. or to assign the traffic to a lower-effort service.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling SHOULD assign the DSCP Less  <ins> than Best Effort RFC8622 PHB. </ins> The application is interactive, and prefers loss to latency. Response time should be optimized at the expense of bandwidth supported by the underlying transport; and so on.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD  <ins> assign a DSCP Assured Forwarding (AF41,AF42,AF43,AF44) RFC2597 PHB.  Inelastic traffic that is expected to conform to the configured network service rate could be mapped to the DSCP Expedited Forwarding RFC3246 or RFC5865 PHBs. </ins> The application prefers loss to latency but is not interactive. Response time should be optimized at the expense of bandwidth


Abstract <del> The Transport Services architecture I-D.ietf-taps-arch defines a system that allows applications to use transport networking protocols flexibly.  This document serves as a guide to implementation on how to build such a system. </del>  1.

You cannot have references in the abstract. Initially I only wanted to remove that reference but then I also took the opportunity to rewrite the abstract a bit to not only talk about the architecture but an TAPS system and the interface that this guide helps with its implementation.
LGTM (I fixed a tiny nit directly with a commit)Thanks looks good.

Abstract  <ins> The Transport Services (TAPS) system enables applications to use transport protocols flexibly for network communication and defines a protocol-independent TAPS Application Programming Interface (API) that is based on an asynchronous, event-driven interaction pattern. This document serves as a guide to implementation on how to build such a system. </ins> 1.


4.1.1. Both Local and Remote Endpoint Candidates must be discovered during <del> connection establishment.  To support ICE, or similar protocols, that involve out-of-band indirect signalling to exchange candidates with the Remote Endpoint, it's important to be able to query the set of candidate Local Endpoints, and give the protocol stack a set of candidate Remote Endpoints, before it attempts to establish connections. </del>  4.1.1.1. If NAT traversal is required, the process of gathering Local Endpoints becomes broadly equivalent to the ICE candidate gathering <del> phase RFC5245.  The endpoint determines its server reflexive Local Endpoints (i.e., the translated address of a local, on the other side of a NAT) and relayed locals (e.g., via a TURN server or other relay), for each interface and network protocol.  These are added to the set of candidate Local Endpoints for this connection. </del>  Gathering Local Endpoints is primarily a local operation, although it might involve exchanges with a STUN server to derive server reflexive locals, or with a TURN server or other relay to derive relayed <del> locals.  It does not involve communication with the Remote Endpoint. </del>  4.1.1.2.

and spell out ICE.
I also updated a later reference and added references for STUN and TURN in this same PR now.

4.1.1. Both Local and Remote Endpoint Candidates must be discovered during  <ins> connection establishment.  To support Interactive Connectivity Establishment (ICE) RFC8445, or similar protocols, that involve out- of-band indirect signalling to exchange candidates with the Remote Endpoint, it's important to be able to query the set of candidate Local Endpoints, and give the protocol stack a set of candidate Remote Endpoints, before it attempts to establish connections. </ins> 4.1.1.1. If NAT traversal is required, the process of gathering Local Endpoints becomes broadly equivalent to the ICE candidate gathering  <ins> phase (see Section 5.1.1. of RFC8445).  The endpoint determines its server reflexive Local Endpoints (i.e., the translated address of a local, on the other side of a NAT, e.g via a STUN sever RFC5389) and relayed locals (e.g., via a TURN server RFC5766 or other relay), for each interface and network protocol.  These are added to the set of candidate Local Endpoints for this connection. </ins> Gathering Local Endpoints is primarily a local operation, although it might involve exchanges with a STUN server to derive server reflexive locals, or with a TURN server or other relay to derive relayed  <ins> locals.  However, it does not involve communication with the Remote Endpoint. </ins> 4.1.1.2.


will necessarily be delivered, so applications cannot rely on receiving them.  5.3. Most security parameters, e.g., TLS ciphersuites, local identity and

.. that gets in the way of using streams when cloning.
How does this interact with ?
NAME - this now states that it is irrelevant for Rendezvous(), and explicitly mentions the Initiate() and Listen() calls when talking about actively and passively opening a Connection. Why is it irrelevant? Remember, the problem is that you actively open a Connection that could be mapped to a stream, in which case an active Open doesn't do anything on the wire - and then, right away, you want to read. Then the peer never heard from you, you haven't even tried sending anything to it. Enabling this property will allow the TAPS system to carry out such a Connection = stream mapping. When calling rendezvous(), however, just using a stream is out of the question - you need signaling, and chances are that you're going to actively send stuff in any case. In other words, I don't think there's a mapping that would work with rendezvous() that enabling this property would allow, and hence the property is irrelevant for rendezvous().
Merging this now: I think I have addressed the comment by NAME We can always fix things later again if we must - but for now, this and it also relates to , which gates .... so it's probably better to move forward at this point.
Should ListenClone() fail for non-multiplexed transports How to specify if a top-level listener can accept cloned (multiplexed) streams Should ListenClone() be called ListenForClone()...?
Just a note to say that issue waits for this issue to be addressed.
Rename function Trying to do this without a multiplexing transport should fail
ping: issue is waiting for this...
Currently the "Clone" part of, e.g., TCP (section 10.1) only says: "Calling Clone on a TCP Connection creates a new Connection with equivalent parameters. The two Connections are otherwise independent." This text should reflect that equivalent behavior must be maintained upon later Connection configuration calls. And, there should be more text in general on cloning (see issue ).
This text must refer to a new "do I want a Connection that supports initiate followed by listen?" property. This property will be added to address - so I'll address this issue when the PR addressing has landed.
A couple of questions we might want to address regarding Connection Groups. 1) Can the application query the interface to figure if two connection in a group share the same transport connection but on different streams or not. Is that useful? I thinks we should explicitly say that if a multiplex protocol is used, calling Clone() MUST create a new stream (and not a new connection). 2) If a new connection is created by a Listener it is automatically part of a group, right? Can I detach that connection from the group? Otherwise how can I change a property only for that connection? Isn't is actually confusing that these connection are grouped? 2) Should we add a pointer to coupled congestion control (rmcat) in that section?
"Can the application query the interface to figure if two connection in a group share the same transport connection but on different streams or not." - no. "Is that useful?" - Good question. The behavior attained with coupled cc. for ongoing connections can be VERY similar to the behavior of multi-streaming - exposing such a detail to an application can then lead to unnecessarily protocol-specific code. Then again, it may be useful for an application to know if issuing Clone() comes with almost zero overhead, or if it produces a connection setup on the wire... so I lean towards "yes, it's useful". "I thinks we should explicitly say that if a multiplex protocol is used, calling Clone() MUST create a new stream (and not a new connection)." Fine by me. I think the query that you mention above, then, should be: "is a Connection multiplexing-capable?" (which will, due to your MUST, translate into "Clone() just means using a new stream"). The difference between Listening with and without cloning is explicit - normally, Listen does not produce a Connection that's grouped with some ongoing Connections. Rather, if you WANT to do a "listening clone", you need to issue "Listen" on an ongoing Connection, as described in Section 6.4. So it's a different "Listen" call. You can't detach Connections from groups. Yes! We've also done work on doing this with TCP connections, which works like a charm - I'll insert the coupled cc reference and cite the TCP coupling work too.
I agree with 1): having a query that indicates if a protocol provides multiplexing is probably sufficient (+ the MUST requirement) On 2) from how I read the document I understood that Connections that are derived from the same Preconnection are clones. If that is not the case of Listeners we need to clarify that and double-check the existing text.
On 2): that's what I meant in my answer, sorry if it wasn't clear. So what's the problem?
So are you saying Connection from the same Listeners are clones or not? And in each case shouldn't there be a way to also get the other?
Ahh, sorry, I misread, you talked about "the same Preconnection". No, that's not it. If you call Listen on a Preconnection, you get a new Connection that is not cloned. That's in: https://ietf-URL If, however, you call Listen on an existing Connection, you get a clone. That's in: https://ietf-URL
Just to add, I think its inconsistent that you would call listen() on an connection object. In all other parts of the draft it is only ever used on preconnections.
No that's also not what I was talking about. You call listen on a preconnnection. At some point someone connects and the listener creates a connection (A). Then another (or the same actually) peer connect to your server (again) and the same listener creates another connection (B). Are those connections(A and B) grouped or not?
The same listener = a listener that was called on a preconnection. So, in your example, you never called Listen on a Connection, which is semantically different. So, no, A and B would not be grouped - what makes you think they could be? Noone ever made the Listen call that would do that. What am I missing?
NAME that's just a naming thing. Using it on a connection is consistent with Clone() - it's just a different way of Cloning, as a passive instead of active open. We could call it something else - ListeningClone or something. Me, I like it as it stands because, to me, the semantics of "this is spawned off an existing connection" are nicely conveyed by the call being applied to a Connection instead of a Preconnection. But I wouldn't fight about the name if you think "ListeningClone" or something like that is better.
I agree, its mostly a naming thing but I think there are a few things here that need to be specified which aren't obvious. For example, what happens if you call listen on two connections that belong to the same group? Personally, I would assume that you can't and would get an error if you tried. I also agree with NAME that it is a bit confusing that listener objects, depending on how they have been created, sometimes group connections and sometimes they don't.
Why? The two connections are already both part of the same group, so any of these Listen calls should give you yet another connection that is in the same group. I don't think there's anything indicating any other semantics - such as the error you're describing. About this being confusing: this is just a matter of having a passive clone. How else would you do it?
But the listen wouldn't give you a new connection, it would give you a Listener object which in turn would give you new connection(s). Having multiple Listener objects for the same group seems wrong to me.
I think this whole discussion suggests that we should indeed use a different name, ListenClone or CloneListen seems the most natural choices to me. Perhaps update the name (and the text accordingly) and see if there is still any points for confusion left. While calling listen on a Preconnection and a Connection are semantically different, I can see that it is easy to get lost as to what Listen we are talking about in different parts of the text.
Yes!
I'm currently reading the Implementation draft again, where Section 4.8.3 talks about listening for multiplexed protocols. Interestingly, the implementation draft gives different answers to the questions here: The Implementation draft says that "The implementation should allow the application to introspect the Connection Group marked on the Connections to determine the grouping of the multiplexing." So I guess this would change the answer to question 1 here to "Yes", but the interface draft doesn't say how. "Protocols that provide multiplexing of streams into a single five-tuple can listen both for entirely new connections (a new HTTP/2 stream on a new TCP connection, for example) and for new sub-connections (a new HTTP/2 stream on an existing connection). If the abstraction of Connection presented to the application is mapped to the multiplexed stream, then the Listener should deliver new Connection objects in the same way for either case." - It doesn't say what "in the same way" means, but I guess it means "deliver a ConnectionReceived event". The sentence after that talks about querying Connection groups, cited above. So this sounds to me like this text presumes that Connections from new streams on existing connections are grouped, while Connections on entirely new connections are not. But I might be misreading this. In any case, it's not clear. This thread sounds to me like incoming Connections from a Listener created by Preconnection.Listen() are never grouped. So, I think either we should say explicitly that incoming Connections from that Listener may or may not be grouped (depending on whether there's an "entirely new connection" underlying or whether they're just a new stream), or we say they're never grouped and change the implementation draft later. However, in that latter case, we'd have outgoing multiplexed connections which are always Connection Groups, and we have incoming multiplexed connections which are never Connection Groups. That doesn't sound right.
Folks, this discussion and NAME last comment just made me remember something: I'm afraid we have designed something that just cannot work! I don't know about QUIC, but I assume it's the same: in SCTP, passively creating a new stream just doesn't exist - a stream comes into being as soon as a new message is sent on it. So, technically, the only thing that really works would be something much more restrictive, as for example: having a form of InitiateWithSend as an active clone, then, on the passive side, upon InitiateWithSend-cloning, Listener -> ConnectionReceived might just fire out of the blue, and only when that happens, Connections are cloned.
If initiate() is supposed to "open" new stream, calling initiate() will not lead to any action on the wire but also create some internal state that the next send of that connection will need a new stream (id). That's fine.
But is that a clean model? You call initiate, and because it's just like any other Connection, maybe you expect to possible be able to read something afterwards... but that will never be the case.
I agree that things get somewhat inconsistent here. When would a Connection created like this fire its ready event? Just immediately?
Felix Weinrank and Michael Tüxen described in detail how this works in case of NEAT in this paper: F. Weinrank and M. Tüxen. “Transparent Flow Mapping for NEAT“. In Workshop on Future of Internet Transport (FIT 2017), Stockholm, June 2017. URL For their implementation, the answer to your question is "yes". But that's just because we wanted to map Connections at any cost - it comes with the limitation of only being to send (not receive) later, so I'd say this is not fantastic API design. Being able to receive immediately after clone() + ready would require using a signal in the underlying protocol itself - the paper discusses that this could be done with the otherwise unused "Payload Protocol Identifier (PPID)", but the meaning of this signal (as: "I created a stream but I'm not sending anything") would of course have to be known to both sides too, requiring a new standard.
Opening a connection that just allocates stream ID state should be fine, no? I don't think it's necessarily a problem. As far as being able to read without sending, that's up to the protocol. No reason some future version of QUIC or similar couldn't support that.
Well that's true - although, why would future-QUIC do that? Just to support TAPS's communication model? It seems efficient to just implicitly create a new stream by transmitting data on it. Anyway, at the very least, we should warn the programmer that it may not be possible to initially read on a newly cloned connection, and provide this information as a readable connection property. Also note that we're now only discussing one aspect of the problem: is it ok to just have an "Initiate". The other aspect is listening: sure, we could make a general Listen map to things and have warnings and such, but what both QUIC and SCTP would currently be doing is completely different from that. I think it would be better to specify that Listener -> ConnectionReceived might just fire out of the blue, which would also indicate that the first data has probably (with current protocols: surely) already arrived at the stack.
No, not just to support TAPS! If there was some value in having a stream that a client can open, but not write, and have the server write, which presumably is the case you're concerned about here, and someone has a use case, they can use that. Even today, you can have a server for TCP that won't do anything until the client writes. I don't see any reason to have the API be restrictive based on particular protocol behavior, when there are cases that do allow that behavior.
Yes, that's the communication pattern I'm concerned about. It is rare - e.g. pub-sub systems, which in principle follow that pattern, will usually implement an application-level signal from the client to the server that at least says "I'm here". Because this pattern is rare, I doubt that it will be supported in future-QUIC-or-whatever. And that's fine! The problem is: if I write an application over TAPS and I decide to implement this "I listen, and then I'm the first to send" pattern, then: if the underlying protocol is TCP, this will work fine if the underlying protocol is a stream of QUIC or SCTP, my application simply gets stuck. So we should at least put a warning about this case in there, and let an application query if, after Listen, send-before-receive is possible.
Discussed in interim: this sounds like a property that does protocol selection that doesn't choose QUIC if you require being able to open a stream and then read without sending (TCP allows this, QUIC doesn't).
The Listen + ListenClone() discussion is moved to issue . I'll make a PR for the outcome above (protocol selection property).
NAME ping, will the protocol selection property land before the next interim?
ACK and YES. I had planned to do this today (not happening) or tomorrow (most likely) or the day after tomorrow (sigh, I hope tomorrow will work).
The "client opens the connection but server sends first" model is used in SMTP, right? It's not such an uncommon pattern.
lgtmWorks for me, but changing the name might help to get the pattern clearer

will necessarily be delivered, so applications cannot rely on receiving them. <ins> 5.2.17. activeReadBeforeSend Preference Ignore The most common client-server communication pattern involves the client actively opening a connection, then sending data to the server.  The server listens (passive open), reads, and then answers. This property specifies whether an application wants to diverge from this pattern - either by actively opening with Initiate(), immediately followed by reading, or passively opening with Listen(), immediately followed by writing.  This property is irrelevant for connection establishment using Rendezvous().  Requiring this property limits the choice of mappings to underlying protocols, which can reduce efficiency.  For example, it prevents the transport system from mapping Connections to SCTP streams, where the first transmitted data takes the role of an active open signal I-D.ietf-taps-impl. </ins> 5.3. Most security parameters, e.g., TLS ciphersuites, local identity and


by setting the "Safely Replayable" send parameter to true when sending the data.  In general, 0-RTT data may be replayed (for example, if a TCP SYN contains data, and the SYN is retransmitted, <del> the data will be retransmitted as well), but racing means that different leaf nodes have the opportunity to send the same data independently.  If data is truly safely replayable, this should be permissible. </del>  Once the application has provided its 0-RTT data, an implementation should keep a copy of this data and provide it to each new leaf node distributed pre-shared keys (PSKs).  Implementations should be aware of security concerns around using these tokens across multiple addresses or paths when racing.  In the case of TLS, any given ticket <del> or PSK should only be used on one leaf node.  If implementations have multiple tickets available from a previous connection, each leaf node attempt must use a different ticket.  In effect, each leaf node will send the same early application data, yet encoded (encrypted) differently on the wire. </del>  6.

I would further recommend to move this section into 4 somewhere as this section is more about connection establishment than sending messages
LGTM

by setting the "Safely Replayable" send parameter to true when sending the data.  In general, 0-RTT data may be replayed (for example, if a TCP SYN contains data, and the SYN is retransmitted,  <ins> the data will be retransmitted as well but may be considered as a new connection instead of a retransmission).  Also, when racing connections, different leaf nodes have the opportunity to send the same data independently.  If data is truly safely replayable, this should be permissible. </ins> Once the application has provided its 0-RTT data, an implementation should keep a copy of this data and provide it to each new leaf node distributed pre-shared keys (PSKs).  Implementations should be aware of security concerns around using these tokens across multiple addresses or paths when racing.  In the case of TLS, any given ticket  <ins> or PSK should only be used on one leaf node, since servers will likely reject duplicate tickets in order to prevent replays (see section-8.1 RFC8446).  If implementations have multiple tickets available from a previous connection, each leaf node attempt can use a different ticket.  In effect, each leaf node will send the same early application data, yet encoded (encrypted) differently on the wire. </ins> 6.


Message lengths can be defined, and Messages can be partially reliable. <del> Below, primitives in the style of "CATEGORY.[SUBCATEGORY].PRIMITIVENAME.PROTOCOL" (e.g., "CONNECT.SCTP") refer to the primitives with the same name in section 4 of RFC8303.  For further implementation details, the description of these primitives in RFC8303 points to section 3, which refers back to the specifications for each protocol.  This back-tracking method applies to all elements of I-D.ietf-taps-minset (see appendix D of I- D.ietf-taps-interface): they are listed in appendix A of I-D.ietf- taps-minset with an implementation hint in the same style, pointing back to section 4 of RFC8303. </del>  10.1.

Section 10 of impl draft: "Below, primitives in the style of "CATEGORY.[SUBCATEGORY].PRIMITIVENAME.PROTOCOL" (e.g., "CONNECT.SCTP") refer to the primitives with the same name in section 4 of [RFC8303]." I know it's specified like this in RFC8303 but PRIMITIVENAME does not seem to be explained or used. Maybe I didn't look careful enough but to be honest I also didn't find the use of this nomenclature from RFC8303 very helpful...
Just to answer the comment of usefulness of this nomenclature: this allows back-tracking to the relevant spec., which (I believe) is valuable for people implementing a TAPS system. In the example of CONNECT.SCTP, RFC 8303 describes this as consisting of: Interleaving' (optional), followed by 'Associate' Initialize and Associate are obvious; naturally, they come from RFC 4960. Considering to use 'Enable/Disable Interleaving' (which is very important for a TAPS system - to implement priorities between streams) is perhaps less obvious, and the text points back at RFC 8260.

Message lengths can be defined, and Messages can be partially reliable.  <ins> Below, terms in capitals with a dot (e.g., "CONNECT.SCTP") refer to the primitives with the same name in section 4 of RFC8303.  For further implementation details, the description of these primitives in RFC8303 points to section 3 of RFC8303 and section 3 of RFC8304, which refers back to the relevant specifications for each protocol. This back-tracking method applies to all elements of I-D.ietf-taps- minset (see appendix D of I-D.ietf-taps-interface): they are listed in appendix A of I-D.ietf-taps-minset with an implementation hint in the same style, pointing back to section 4 of RFC8303. </ins> 10.1.


The Local Endpoint MUST be specified if the Preconnection is used to Listen() for incoming Connections, but is OPTIONAL if it is used to <del> Initiate() connections.  The Remote Endpoint MUST be specified if the </del>  Preconnection is used to Initiate() Connections, but is OPTIONAL if it is used to Listen() for incoming Connections.  The Local Endpoint and the Remote Endpoint MUST both be specified if a peer-to-peer When an implementation is asked to Listen, it registers with the system to wait for incoming traffic to the Local Endpoint.  If no <del> Local Endpoint is specified, the implementation should either use an ephemeral port or generate an error. </del>  If the Selection Properties do not require a single network interface or path, but allow the use of multiple paths, the Listener object

Done.
Impl draft section 4.6 (Implementing listeners): "If no Local Endpoint is specified, the implementation should either use an ephemeral port or generate an error." Do we need an interface for the application to indicate that an ephemeral port should be used?
It's implicit in the old socket API - accept just gives me a new socket, which I can use to continue communication, but I don't see the port numbers associated with this socket and never have to say that they are ephermal. I believe it's similarly implicit in ours. So ... what's missing?
The text above says "use an ephemeral port or generate an error". So when would you throw an error? If the assumption is that always an ephemeral port should be used (no matter actually how the underlying interfaces works) that should be stated (in the API doc).
If you do not use an "ephemeral port", and choose to specify the port - then you may get the error if it can't be completed.
... or if the system is out of resources for some reasons. Just like "accept" can fail in the normal socket API.
That too.
NAME requested that I adapt the "error" text in the implementation draft too. In doing so, I noticed: I don't understand NAME comment above, about choosing the specify the port. I think this kind of error was possible with sockets somehow, but this text talks about not specifying a Local Endpoint. Technically, I think not specifying a Local Endpoint yet specifying a port is not possible.Regarding my own comment, any call may fail for reasons such as the system being out of resources; I think mentioning this in the text here doesn't add any value though. As a result, I'll remove the part about the error from the text in my update to PR .
This seems like what was agreed.This change is fine. You also need to adapt the implementation doc to remove the "or" part: "either use an ephemeral port or generate an error."

The Local Endpoint MUST be specified if the Preconnection is used to Listen() for incoming Connections, but is OPTIONAL if it is used to  <ins> Initiate() connections.  If no Local Endpoint is specified, the Transport System will assign an ephemeral local port to the Connection.  The Remote Endpoint MUST be specified if the </ins> Preconnection is used to Initiate() Connections, but is OPTIONAL if it is used to Listen() for incoming Connections.  The Local Endpoint and the Remote Endpoint MUST both be specified if a peer-to-peer When an implementation is asked to Listen, it registers with the system to wait for incoming traffic to the Local Endpoint.  If no  <ins> Local Endpoint is specified, the implementation should use an ephemeral port. </ins> If the Selection Properties do not require a single network interface or path, but allow the use of multiple paths, the Listener object


Preference: can take one of five values (Prohibit, Avoid, Ignore, Prefer, Require) for the level of preference of a given property <del> during protocol selection; see selection-props. </del>  For types Integer and Numeric, special values can be defined per property; it is up to implementations how these special values are Selection Properties are listed in the subsections below.  Note that many properties are only considered during establishment, and can not be changed after a Connection is established; however, they can be <del> queried.  Querying a Selection Property after establishment yields the value "Require" for properties of the selected protocol and path, Avoid for properties avoided during selection, and Ignore for all other properties. </del>  An implementation of this interface must provide sensible defaults for Selection Properties.  The defaults given for each property below

I chose boolean, because, when querying, it only matters whether a Selection Property was chosen or not (I believe).
From Kyle Rose: NAME says: I think this isn't very controversial, I believe we should just do this as he suggested. This issue is just a reminder for myself to do this.
Thanks

Preference: can take one of five values (Prohibit, Avoid, Ignore, Prefer, Require) for the level of preference of a given property  <ins> during protocol selection; see selection-props.  When querying, a Preference is of type Boolean, with "true" indicating that the Selection Property has been applied. </ins> For types Integer and Numeric, special values can be defined per property; it is up to implementations how these special values are Selection Properties are listed in the subsections below.  Note that many properties are only considered during establishment, and can not be changed after a Connection is established; however, they can be  <ins> queried.  The return type of a queried Selection Property is Boolean, where "true" means that the Selection Property has been applied and "false" means that the Selection Property has not been applied.  Note that "true" does not mean that a request has been honored.  For example, if "Congestion control" was requested with preference level "Prefer", but congestion control could not be supported, querying the "congestionControl" property yields the value "false".  If preference level "Avoid" was used for "Congestion control", and, as requested, the Connection is not congestion controlled, querying the "congestionControl" property also yields the value "false". </ins> An implementation of this interface must provide sensible defaults for Selection Properties.  The defaults given for each property below


Sending a Message with Message Properties inconsistent with the Selection Properties of the Connection yields an error.  The following Message Properties are supported: 7.5.1.

I inserted the text that I quoted in this issue, with a minor tweak.
List discussion: Kyle: NAME
I agree with the conclusion of NAME above - in which case we should leave things as they are. Other views?
I agree that if the application does nothing then communication should be reliable. But I understand Kyle's comment as asking about the relationship to the selection property Reliable Data Transfer (Connection). If you for example set that to Avoid and the TAPS system selects SCTP as the protocol, then if you do not set Reliable Data Transfer (Message) on a message I assume it will be sent as an unreliable message. So in that sense the reliability of a message defaults to whatever is set for the connection, unless it is overridden by setting Reliable Data Transfer (Message). And the default for Reliable Data Transfer (Connection) is Require so what NAME writes still holds but it is a different thing.
So if we agree upon Reliable Data Transfer (Connection), then it would seem less confusing for Reliable Data Transfer (Message) - to use the same logic. Does an apps writer need to know the data will be sent as an unreliable message? (I'm not sure I like "avoid reliable data transfer" as a meaningful concatenation of keywords BTW:-).
Let's see what needs to be done. Suggestion: the only textual change would be to add the following, at the beginning of Section 7.5, just before "The following Message Properties are supported": "Connection Properties describe the default behavior for all Messages on a Connection. If a Message Property contradicts a Connection Property, it overrides the Connection Property for the specific Message. For example, if "Reliable Data Transfer (Connection)" is set to "Avoid" and a protocol with configurable per-Message reliability is used, setting "Reliable Data Transfer (Message)" to "Prefer" for a particular Message will ensure that this Message is reliably delivered. If the underlying protocol can not support this per-Message behavior, this Message Property choice will yield an error."
Thanks! No more comments now :)

Sending a Message with Message Properties inconsistent with the Selection Properties of the Connection yields an error. <ins> Connection Properties describe the default behavior for all Messages on a Connection.  If a Message Property contradicts a Connection Property, and if this per-Message behavior can be supported, it overrides the Connection Property for the specific Message.  For example, if "Reliable Data Transfer (Connection)" is set to "Require" and a protocol with configurable per-Message reliability is used, setting "Reliable Data Transfer (Message)" to "false" for a particular Message will allow this Message to be unreliably delivered.  Note that changing the Reliable Data Transfer property on Messages is only possible for Connections that were established with the Selection Property "Configure Per-Message Reliability" enabled. </ins> The following Message Properties are supported: 7.5.1.


Transport Services architecture.  The Transport Services Application Programming Interface (API) is based on an asynchronous, event-driven interaction pattern.  It uses messages for representing data transfer <del> to applications, and it assumes an implementation that can use </del>  multiple IP addresses, multiple protocols, and multiple paths, and provide multiple application streams.  This document further defines common terminology and concepts to be used in definitions of There are key differences between the architecture of the Transport Services system and the architecture of the Socket API: the Transport Services API is asynchronous and event-driven; it uses messages for <del> representing data transfer to applications; and it assumes an implementation that can use multiple IP addresses, multiple protocols, multiple paths, and provide multiple application streams. </del>  2.1.

Address
I think the word /assumes/ is not a good choice of word in the arch draft. OLD: and it assumes an implementation that can use multiple IP addresses, multiple protocols, multiple paths, and provide multiple application streams NEW: and it can provide benefit when an implementation can use multiple IP addresses, multiple protocols, multiple paths, and provide multiple application streams.
Done
Exactly what I expected to see!

Transport Services architecture.  The Transport Services Application Programming Interface (API) is based on an asynchronous, event-driven interaction pattern.  It uses messages for representing data transfer  <ins> to applications, and it describes how implementations can use </ins> multiple IP addresses, multiple protocols, and multiple paths, and provide multiple application streams.  This document further defines common terminology and concepts to be used in definitions of There are key differences between the architecture of the Transport Services system and the architecture of the Socket API: the Transport Services API is asynchronous and event-driven; it uses messages for  <ins> representing data transfer to applications; and it describes how implementations can use multiple IP addresses, multiple protocols, multiple paths, and provide multiple application streams. </ins> 2.1.


application.  This requires the implementation to understand where messages end, either via a supplied deframer or because the top-level protocol in the established Protocol Stack preserves message <del> boundaries; if, on the other hand, the top-level protocol only supports a byte-stream and no deframers were supported, the application must specify the minimum number of bytes of Message content it wants to receive (which may be just a single byte) to control the flow of received data. </del>  If a Connection becomes finished before a requested Receive action can be satisfied, the implementation should deliver any partial

As described in , I removed the condition that requires the application to specify the length upon receiving from this example.
Impl draft section 5.2 Receiving Messages: "if, on the other hand, the top-level protocol only supports a byte-stream and no deframers were supported, the application must specify the minimum number of bytes of Message content it wants to receive (which may be just a single byte) to control the flow of received data." This text says the application MUST specify minIncompleteLength, however, the API draft says that this values is optional. This should be further clarified and more guidance given in case the value is not set.
Having looked at this and the API carefully, my suggestion would be to turn the semicolon preceding this statement into a full stop and remove all of this "if, on the other hand, ...." text in the implementation draft.
WFM.
This was a long text block - I had a few editorial NiTs you may like to address.

application.  This requires the implementation to understand where messages end, either via a supplied deframer or because the top-level protocol in the established Protocol Stack preserves message  <ins> boundaries.  If the top-level protocol only supports a byte-stream and no framers were supported, the application can control the flow of received data by specifying the minimum number of bytes of Message content it wants to receive at one time. </ins> If a Connection becomes finished before a requested Receive action can be satisfied, the implementation should deliver any partial