want to always "Prohibit Cellular". The set of interface types is expected to change over time as new <del> access technologies become available. </del>  Interface types should not be treated as a proxy for properties of interfaces such as metered or unmetered network access.  If an

from Kyle: Philipp Tiesel:
I'm not sure what Philipp's response means. Where is this registry, and why was it not useful?
Kyle Rose's review: URL
This will be system/implementation specific in general, I'd say.
Add a sentence to 5.2.10 to indicate that this is implementation specific
Mention that these should be documented? I've completed my review of draft-05, inline below. I will also submit a PR with minor changes that I recommend be addressed via cherry-picking agreed-upon hunks from the diff. Terminology and Notation "The method for dispatching and handling Events is an implementation detail, with the caveat that the interface for receiving Messages must require the application to invoke the Connection.Receive() Action once per Message to be received (see Section 8)." This isn't true if ReceivedPartial events are triggered, right? Also, it's not really clear that any of this discussion belongs in a section titled "Terminology and Notation". This issue is alluded to later in the doc, and probably mostly belongs in the implementation doc. The notation "Sender -> Event" says nothing about who receives the event. Is it just assumed to be the application in the context of the particular event (e.g., a thread processing a particular connection)? Interface Design Principles What does "long-term caching of cryptographic identities" mean? The implication seems to be that credentials and transport security parameters can be configured once and used repeatedly. I'm not sure time interval ("long-term") is the right modifier here, and the word "caching" might be misunderstood as it means something subtly different in the realm of computer hardware and software than the dictionary definition implies. Maybe something like: "URL for configuration of cryptographic identities and transport security parameters persistent across multiple Connections" "An application primarily interacts with this interface..." This seems awkward. An interface is the means by which an application interacts with the underlying transports. Transport Property Names The use of dashes in property names means they will necessarily look different in almost every widely-used language, in which dash is an operator. If we're not currently asking IANA to create a registry of property names or namespaces, should we provide a recommendation that such symbols not listed explcitly in this document be prefixed with some experimental identifier? Scope of the Interface Definition "Implementations of this interface SHOULD implement each Selection Property, Connection Property, and Message Context Property specified in this document, exclusive of appendices. Each interface SHOULD be implemented even when this will always result in no operation, e.g. there is no action when the API specifies a Property that is not available in a transport protocol implemented on a specific platform." This seems like it implies that a property that is unavailable should silently fail to be activated, which conflicts with the "Require" preference referenced later. There may be some way to reword this to be clear that it's referring to implementing all of the properties in the API so that compile-time or runtime errors don't spuriously occur when some symbol hasn't been defined. Also: why "exclusive of appendices"? Specifying Transport Properties There are a lot of choices being made about how users will want to prioritize transport protocol selection. How confident are we that, for instance, path selection should take priority over protocol selection? I think that's right, but I wonder if it might not make more sense to have two interfaces: one that provides a purely numeric priority ordering of preferences, and one (based on the existing 5-level preferences) that maps into it. "As preference typed selection properties..." I can't parse this. Using the same enum (Require(d[sic])/Avoid/Ignore) for the queried output of selected properties seems like a shortcut that will lead to some confusion. Use 0-RTT Session Establishment with an Idempotent Message I'll repeat the same statement I made at the mic a few years ago: idempotency != replay-safe. DELETE is idempotent, but not safe for replay because someone might have done a PUT or POST in the meantime. Interface Instance or Type These type symbols really deserve an actual registry, or at least the start of one. Otherwise, we are likely to end up with a mess. Provisioning Domain Instance or Type What about ordering of similar interfaces? I have a 2-SIM cellphone with wifi. Connection Establishment Callbacks What constitutes trust verification prior to the handshake? Connection Groups "An ideal transport system" for whom? The capacity partitioning recommended here seems purely subjective. I'd just remove it. Sent It seems like an abstract API would be helpful for the reference to the Message to which a Sent event applies: this seems like something many, many applications would need to do. With callbacks, the application can always curry in a reference to the original message (that's what my event system from ~2001 did), so maybe that should be the recommendation and the message reference removed...? I don't have a strong feeling here other than that if something is included then its use should be more well-defined. SendError Is there a reason why a single messageContext object is used for both sends and receives? I probably missed the original discussion about this, but I'd like to understand the reasoning. Reliable Data Transfer (Message) The default isn't "true", it's whatever the underlying connection had, right? Singular Transmission Might be worth pointing out there's no guarantee here, either, considering resegmenting TCP middleboxes. Receiving Data A connection can be used to receive data if it is not configured as unidirectional transmit. Receive Events "A call to Receive will be paired with a single Receive Event" or possibly multiple ReceivedPartial Events? Also, can the draft be consistent about Receive vs. Received, et al.? Message Contexts "To get or set Message Properties, the optional scope parameter is left empty, for framing meta-data, the framer is passed." I can't parse this. General comments There's a lot of UNIXisms here that should probably be excised in favor of a more abstract presentation. The most obvious is the use of -1 to mean something other than the integer -1. In Python, for instance, you might want to use None for this purpose. I would specify non-numeric values symbolically, and maybe indicate in the appendix that a UNIX C implementation might want to use -1 to represent such values. Priority (Connection) I'm not sure what "relative inverse priority" means. Is P1 higher priority than P5, and therefore lower inverse priority than P5? Or the other way around? Capacity Profile Very little of this section is about capacity. Traffic Profile? "High Throughput Data" might be better phrased as "Capacity-Seeking". Soft Errors Should the SoftError Event carry some information about e.g. the ICMP message received?

want to always "Prohibit Cellular". The set of interface types is expected to change over time as new  <ins> access technologies become available.  The taxonomy of interface types on a given Transport Services system is implementation- specific. </ins> Interface types should not be treated as a proxy for properties of interfaces such as metered or unmetered network access.  If an


1. <del> This document specifies a modern abstract programming interface (API) atop the high-level architecture for transport services defined in I- D.ietf-taps-arch.  It supports the asynchronous, atomic transmission of messages over transport protocols and network paths dynamically selected at runtime.  It is intended to replace the traditional BSD sockets API as the common interface to the transport layer, in environments where endpoints select from multiple interfaces and potential transport protocols. </del>  As applications adopt this interface, they will benefit from a wide set of transport features that can evolve over time, and ensure that

Resolving Editorial NiTs in section 1
Section 1: This section contains a RFC2119 keyword that precedes the definition! We use the words /abstract programming interface/ and the acronym /API/ but never tie these together. NiTs: /provide applications a way to/provide a way for applications to/ /as it comes at the cost of reduced portability/because it can reduce the portability./ /How these abstract concepts/The way these abstract concepts/ /is largely dependent on/mostly depends on/
A PR for this issue has already been merged.
Looks good to me, thanks for doing this!

1.  <ins> This document specifies a modern abstract application programming interface (API) atop the high-level architecture for transport services defined in I-D.ietf-taps-arch.  It supports the asynchronous, atomic transmission of messages over transport protocols and network paths dynamically selected at runtime.  It is intended to replace the traditional BSD sockets API as the common interface to the transport layer, in environments where endpoints select from multiple interfaces and potential transport protocols. </ins> As applications adopt this interface, they will benefit from a wide set of transport features that can evolve over time, and ensure that


The most basic form of sending on a connection involves enqueuing a single Data block as a complete Message, with default Message <del> Properties.  Message data is transferred as an array of bytes, and the resulting object contains both the byte array and the length of the array. </del>  The interpretation of a Message to be sent is dependent on the implementation, and on the constraints on the Protocol Stacks implied

Looks fine to me. I wonder if we want to add some text and given some implementation hints. Not sure if that is needed but it could be useful...?
Looks good.
In section 7.1 of API it says: messageData := "hello".bytes() Connection.Send(messageData) For some languages that are dynamic typed (i.e. python) this limits the usability of the API as it adds an unnecessary extra step to each send. To me this seems like it should be an implementation detail and not specified in the API.
What are the options other than raw bytes?
In python you could just pass any type that gets automatically converted inside the API. For example, in our implementation you could just pass a string or integer that gets converted to bytes inside the implementation before sending. Also, if there is a TLV framer, you could just pass a [type,value] tuple without having to go through the message context.
We generally consider this a case for framers. However, I think it's reasonable to assume that any of the objects used in a TAPS implementation should provide some way for the app to get at the underlying bytes (and therefore, at least implicitly, the length). I don't read this text as prohibiting the use of some object that is able.
I agree that this is an implementation detail that's irrelevant to the abstract API. Can we close this issue?
Requiring the data be accessible as a byte array is overly restrictive. Can we just remove "Message data is transferred as an array of bytes, and the resulting object contains both the byte array and the length of the array." and the call?

The most basic form of sending on a connection involves enqueuing a single Data block as a complete Message, with default Message  <ins> Properties. </ins> The interpretation of a Message to be sent is dependent on the implementation, and on the constraints on the Protocol Stacks implied


"Interface Instance or Type": If the application specifies an interface type to be preferred or avoided, implementations should <del> rank paths accordingly.  If the application specifies an interface type to be required or prohibited, we expect an implementation to not include the non-conforming paths into the three. </del>  <del> "Capacity Profile": An implementation may use the Capacity Profile to prefer paths optimized for the application's expected traffic pattern according to cached performance estimates, see </del>  performance-caches: Scavenger: Prefer paths with the highest expected available <del> bandwidth, based on observed maximum throughput </del>  Low Latency/Interactive: Prefer paths with the lowest expected <del> Round Trip Time Constant-Rate Streaming: Prefer paths that can satisfy the requested Stream Send or Stream Receive Bitrate, based on observed maximum throughput Implementations process properties in the following order: Prohibit, Require, Prefer, Avoid.  If Selection Properties contain any prohibited properties, the implementation should first purge branches containing nodes with these properties.  For required properties, it should only keep branches that satisfy these requirements.  Finally, it should order branches according to preferred properties, and finally use avoided properties as a tiebreaker.  When ordering branches, an implementation may give more weight to properties that the application has explicitly set than to properties that are default. As the available protocols and paths on a specific system and in a specific context may vary, the result of sorting and the outcome of racing may vary even given the same Selection and Connection Properties.  However, an implementation ought to aim to provide a </del>  consistent outcome to applications, e.g., by preferring protocols and <del> paths that existing Connections with similar Properties are already using. </del>  4.2.

Not intended to change meaning, but intended to improve reading, see issue .
Scavenger: This reads wrong to me, this is unutilised capacity /Highest available bandwidth/available capacity/ Why is /satisfy./ used for CR streaming? To me this implies admission-based QoS. Do we really mean this? Or should we say /expected/ here also? /Implementations process properties/ It would really have helped me to read this if we insert /the/ before properties. /As the available protocols and paths on a specific system and in a specific context may vary,/ would seem better as: /The available protocols and paths on a specific system and in a specific context can vary, therefore/ /, by preferring protocols and paths that existing Connections with similar Properties are already using./ should be something like: /, by preferring protocols and paths that are already used by existing Connections that specified similar Properties./
See PR (which includes text polishing).

"Interface Instance or Type": If the application specifies an interface type to be preferred or avoided, implementations should  <ins> accordingly rank the paths.  If the application specifies an interface type to be required or prohibited, an implementation is expeceted to not include the non-conforming paths. </ins>  <ins> "Capacity Profile": An implementation can use the Capacity Profile to prefer paths that match an application's expected traffic pattern.  This match will use cached performance estimates, see </ins> performance-caches: Scavenger: Prefer paths with the highest expected available  <ins> capacity, based on the observed maximum throughput; </ins> Low Latency/Interactive: Prefer paths with the lowest expected  <ins> Round Trip Time, based on observed round trip time estimates; Constant-Rate Streaming: Prefer paths that can are expected to satisy the requested Stream Send or Stream Receive Bitrate, based on the observed maximum throughput. Implementations process the Properties in the following order: Prohibit, Require, Prefer, Avoid.  If Selection Properties contain any prohibited properties, the implementation should first purge branches containing nodes with these properties.  For required properties, it should only keep branches that satisfy these requirements.  Finally, it should order the branches according to the preferred properties, and finally use any avoided properties as a tiebreaker.  When ordering branches, an implementation can give more weight to properties that the application has explicitly set, than to the properties that are default. The available protocols and paths on a specific system and in a specific context can change; therefore, the result of sorting and the outcome of racing may vary, even when using the same Selection and Connection Properties.  However, an implementation ought to provide a </ins> consistent outcome to applications, e.g., by preferring protocols and  <ins> paths that are already used by existing Connections that specified similar Properties. </ins> 4.2.


that Message are outstanding, there will be two Expired events delivered.  7.3.1. The Sent Event occurs when a previous Send Action has completed, provide backpressure to the transport stack when it is temporarily not ready to receive messages.  8.2.1. A Received event indicates the delivery of a complete Message.  It

Apple does this: URL URL it may be more convenient than what we currently have (or do we have it unspecified? My interpretation of our text would have been to use only one receive handler associated to the Connection). E.g., if an application expects 3 different types of messages to be received in sequence, it can hand over three different receive handlers in Apple's case, and that seems useful. Apple also has a "completion" handler per sent event, handed over as a parameter to "send". Is this necessary? Who needs 3 different "Sent" handlers for 3 different messages - shouldn't the "sent" handler rather be a singular per-Connection thing?
I think this gets into an implementation-specific area. If the language has an easy way to provide a unique completion/closure for a given invocation of Receive(), then I think it is preferable to do this to having a single function. If anything, it may be nice to make a not for Received() and Sent() that they should, if supported by the language, provide this kind of linking between calls and events.
I agree, and I assign this to Tommy since he offered (in an email) to write text. Thanks a lot!

that Message are outstanding, there will be two Expired events delivered. <ins> The interface should allow the application to correlate which Send Action resulted in a particular Send Event.  The manner in which this correlation is indicated is implementation-specific. </ins> 7.3.1. The Sent Event occurs when a previous Send Action has completed, provide backpressure to the transport stack when it is temporarily not ready to receive messages. <ins> The interface should allow the application to correlate which call to Receive resulted in a particular Receive Event.  The manner in which this correlation is indicated is implementation-specific. </ins> 8.2.1. A Received event indicates the delivery of a complete Message.  It


7.7. To reduce the overhead of sending multiple small Messages on a <del> Connection, the application may want to batch several Send actions </del>  together.  This provides a hint to the system that the sending of these Messages should be coalesced when possible, and that sending any of the batched Messages may be delayed until the last Message in the batch is enqueued.  7.8. For application-layer protocols where the Connection initiator also

batch sending, as currently specified, means that you already have to have all messages ready in the application because you can declare a patch. That means you have to potentially "buffer" some message in the application. Wouldn't it may be better/easier to have a message property that indicates that a certain message should wait (probably up until a maximum time) for the next message and then buffer it in the transport instead?
We can just clarify the text that the batch may be an arbitrarily long-running block of code, and not require knowing up front all of the data.
Could also allow begin batch/end batch. Explain use case around optimizing send performance.

7.7. To reduce the overhead of sending multiple small Messages on a  <ins> Connection, the application may want to batch several Send Actions </ins> together.  This provides a hint to the system that the sending of these Messages should be coalesced when possible, and that sending any of the batched Messages may be delayed until the last Message in the batch is enqueued. <ins> The semantics for starting and ending a batch can be implementation- specific, but need to allow multiple Send Actions to be enqueued. </ins> 7.8. For application-layer protocols where the Connection initiator also


4.2.1. Transport Properties are referred to by property names.  These names <del> are lower-case strings whereby words are separated by hyphens.  These names serve two purposes: </del>  <del> Allow different components of a TAPS implementation to pass </del>  Transport Properties, e.g., between a language frontend and a policy manager, or as a representation of properties retrieved from a file or other storage. <del> Make code of different TAPS implementations look similar. </del>  Transport Property Names are hierarchically organized in the form [<Namespace>.]<PropertyName>. <del> The Namespace part MUST be empty for well-known, generic </del>  properties, i.e., for properties that are not specific to a protocol and are defined in an RFC. Vendor or implementation specific properties MUST use a string identifying the vendor or implementation as Namespace. <del> Namespaces for the keywords provided in the IANA protocol numbers registry (see https://www.iana.org/assignments/protocol-numbers/ protocol-numbers.xhtml) are reserved for Protocol Specific Properties and MUST not be used for vendor or implementation specific properties. </del>  4.2.2.

Clarify how the IANA protocol numbers registry keywords (largely upper-case) textually relate to Transport Property Names. Refine some of the surrounding text.

4.2.1. Transport Properties are referred to by property names.  These names  <ins> are lower-case alphanumeric strings in which words may be separated by hyphens.  These names serve two purposes: </ins>  <ins> Allowing different components of a TAPS implementation to pass </ins> Transport Properties, e.g., between a language frontend and a policy manager, or as a representation of properties retrieved from a file or other storage.  <ins> Making code of different TAPS implementations look similar. </ins> Transport Property Names are hierarchically organized in the form [<Namespace>.]<PropertyName>.  <ins> The Namespace component MUST be empty for well-known, generic </ins> properties, i.e., for properties that are not specific to a protocol and are defined in an RFC. Vendor or implementation specific properties MUST use a string identifying the vendor or implementation as Namespace.  <ins> Namespaces for each of the keywords provided in the IANA protocol numbers registry (see https://www.iana.org/assignments/protocol- numbers/protocol-numbers.xhtml), translated to lower-case and with characters other than letters, numbers, and hyphens deleted, are reserved for Protocol Specific Properties and MUST not be used for vendor or implementation-specific properties. </ins> 4.2.2.


if the chosen transport supports using multiple paths. The connection will negotiate the use of multiple paths if the <del> chosen transport supports this.  It will actively bring up paths based on the multipath policy (multipath-policy) selected by the application, as limited by the Interface Instance and Provisioning Domain Instance properties (see prop-interface and prop-pvd). </del>  The connection will support the use of multiple paths if the <del> remote endpoint requests it.  The connection will not actively initiate the use of any new paths, but will accept paths initiated by the remote endpoint. </del>  The policy for using multiple paths is specified using the separate "multipath-policy" property, see multipath-policy below.  To enable

Sorry we just merged the multipath part into the API doc but I think there is still an issue about this part: 'Active: : The connection will negotiate the use of multiple paths if the chosen transport supports this. It will actively bring up paths based on the multipath policy ({{multipath-policy}}) selected by the application, as limited by the Interface Instance and Provisioning Domain Instance properties (see {{prop-interface}} and {{prop-pvd}}). Passive: : The connection will support the use of multiple paths if the remote endpoint requests it. The connection will not actively initiate the use of any new paths, but will accept paths initiated by the remote endpoint.' This mixes two things which should be separated: requesting MP support actively (or just do it if requested by the other end) and opening of new subflows when MP support was negotiated successfully. I thought we wanted to take the later out (opening sub flows) and discuss that separately....?
Just created PR to remove the part about opening new subflows. Do we need to further discuss when/how to open subflows or do we leave this as open for now?
Agree that interim (June 19) to not address opening of subflows in the API

if the chosen transport supports using multiple paths. The connection will negotiate the use of multiple paths if the  <ins> chosen transport supports this. </ins> The connection will support the use of multiple paths if the  <ins> remote endpoint requests it. </ins> The policy for using multiple paths is specified using the separate "multipath-policy" property, see multipath-policy below.  To enable


7.5.9. <del> singularTransmission </del>  Boolean false This property specifies that a message should be sent and received as <del> a single packet without transport-layer segmentation or network-layer fragmentation, if possible.  Attempts to send a message with this property set with a size greater to the transport's current estimate of its maximum transmission segment size will result in a "SendError".  When used with transports supporting this functionality and running over IP version 4, the Don't Fragment bit will be set. </del>  7.6.

When I initial read the heading I thought this is again about reliability. May we can call this single-segment or something...?
Well it's a term - I don't like it much. Single segment doesn't mean it's all in one IP packet (it means it's all in one transport segment). We could just say sending as a single IP packet.
Interim: proposing to rename to something more simple like Don't Fragment? We don't have a good name.
Don't Fragment seems good too me - and tidy this sentence: OLD: /Attempts to send a message with this property set with a size greater to the transport's current estimate of its maximum transmission segment size will result in a "SendError"./ NEW: /Setting this property causes a "SendError" when an attempt is made to send a message with a size greater than the transport's current estimate of the maximum packet size./

7.5.9.  <ins> noFragmentation </ins> Boolean false This property specifies that a message should be sent and received as  <ins> a single packet network-layer fragmentation, if possible.  Attempts to send a message with this property set with a size greater to the transport's current estimate of its maximum transmission segment size will result in a "SendError".  When used with transports supporting this functionality and running over IP version 4, the Don't Fragment bit will be set. </ins> 7.6.


transport-security, many transport security protocols require specific security parameters and constraints from the client at the time of configuration and actively during a handshake.  These <del> configuration parameters are created as follows: </del>  Security configuration parameters and sample usage follow: system.  Parameters include: ciphersuites, supported groups, and signature algorithms. <del> Session cache management: Used to tune cache capacity, lifetime, re-use, and eviction policies, e.g., LRU or FIFO.  Constants and policies for these interfaces are implementation-specific. </del> Pre-Shared Key import: Used to install pre-shared keying material established out-of-band.  Each pre-shared keying material is associated with some identity that typically identifies its use or has some protocol-specific meaning to the Remote Endpoint.  5.3.2. Security decisions, especially pertaining to trust, are not static.

... while we use (property, values) pairs to set other properties? Is there a reason for this? Otherwise I suggest we agree to one interaction model!
Related I also think this text isn't really that correct: "Security decisions, especially pertaining to trust, are not static. Once configured, parameters may also be supplied during connection establishment. These are best handled as client-provided callbacks." You can update some security parameter anytime during the connection using the same interface as before. However the two callbacks we have defined require actually actions that follow an event, so they have to be callbacks.
"However the two callbacks we have defined require actually actions that follow an event, so they have to be callbacks." - that's it, these need to be callbacks. NAME made them like that because using them requires a tighter interaction model. See issue .
As discussed in interim, Security Parameters are a bit different from selection properties. We should review the security parameter interface, but it will likely be independent.
I did a closer review on these and there are two main differences between Security Parameters and the other Preferences / Events: Use of individual calls on security parameters instead of add with keywor. These could easily be integrated, e.g., by turning into Use of Callbacks instead of Events – closed as Issue . Despite there is no technical reason to not turn the Security Callbacks into events, events that "need to do something", i.e., accept, reject, or return an identity, really look wired. Let's discuss which way to take this one forward: Leave as is Align the interface of SecurityParameters to the Transport Parameters, but keep them as a separate object and leave SecurityCallbacks as they are Integrate the interface of SecurityParameters into the Transport Parameters, but keep SecurityCallbacks roughly the same Integrate the interface of SecurityParameters into the Transport Parameters, make SecurityCallbacks more event-like. I am happy to make a PR for whatever we decide, but not for all of them.
We can do the minor option to change AddCiphersuite to Add(ciphersuite...)
Maybe add clarification text to say that these are pre-connection only parameters
Consider cutting cache lifetime, etc, from this list
We really should explicitly add a sentence to clearly say that the security functions are configured using add functions and not via Properties. This is so useful to say, so that someone does not come along and do this because they think we just didn't see this.
closed with
lgtm, will let NAME merge it though.

transport-security, many transport security protocols require specific security parameters and constraints from the client at the time of configuration and actively during a handshake.  These  <ins> configuration parameters need to be specified in the pre-connection phase and are created as follows: </ins> Security configuration parameters and sample usage follow: system.  Parameters include: ciphersuites, supported groups, and signature algorithms.  Pre-Shared Key import: Used to install pre-shared keying material established out-of-band.  Each pre-shared keying material is associated with some identity that typically identifies its use or has some protocol-specific meaning to the Remote Endpoint. <ins> Session cache management: Used to tune cache capacity, lifetime, re-use, and eviction policies, e.g., LRU or FIFO.may also me changed, but are implementation-specific. </ins> 5.3.2. Security decisions, especially pertaining to trust, are not static.


4.2.1. Transport Properties are referred to by property names.  These names <del> are lower-case alphanumeric strings in which words may be separated by hyphens.  These names serve two purposes: </del>  Allowing different components of a TAPS implementation to pass Transport Properties, e.g., between a language frontend and a

This doesn't change the idea of translating keywords from the IANA protocol numbers registry to lower-case; it just removes the requirement for all Transport Property Names to be lower-case. Why? Right now, they're camlCase, and I guess that reads better than changing them all into hardtoreadstrings or using-hyphens-everywhere ...

4.2.1. Transport Properties are referred to by property names.  These names  <ins> are alphanumeric strings in which words may be separated by hyphens. These names serve two purposes: </ins> Allowing different components of a TAPS implementation to pass Transport Properties, e.g., between a language frontend and a


occur before the Ready Event for Connections established using Initiate. <del> An InitiateError occurs either when the set of transport properties and security parameters cannot be fulfilled on a Connection for initiation (e.g. the set of available Paths and/or Protocol Stacks meeting the constraints is empty) or reconciled with the local and/or remote Endpoints; when the remote specifier cannot be resolved; or when no transport-layer connection can be established to the remote Endpoint (e.g. because the remote Endpoint is not accepting connections, the application is prohibited from opening a Connection by the operating system, or the establishment attempt has timed out for any other reason). </del>  See also initiate-and-send to combine Connection establishment and transmission of the first message in a single action. higher value.  By default, this value is Infinite.  The caller is also able to reset the value to Infinite at any point. <del> A ListenError occurs either when the Properties and Security </del>  Parameters of the Preconnection cannot be fulfilled for listening or cannot be reconciled with the Local Endpoint (and/or Remote Endpoint, if specified), when the Local Endpoint (or Remote Endpoint, if contained within the RendezvousDone<> Event, and is ready to use as soon as it is passed to the application via the Event. <del> An RendezvousError occurs either when the Properties and Security </del>  Parameters of the Preconnection cannot be fulfilled for rendezvous or cannot be reconciled with the Local and/or Remote Endpoints, when the Local Endpoint or Remote Endpoint cannot be resolved, when no

No need to have a different type of Event for initiate, listen and rendezvous.
On a PreConnection, an application uses EITHER Initiate OR Listen OR Rendezvous. Therefore, it seems unnecessary to distinguish between three types of errors. Shouldn't we have a general per-PreConnection "UnfulfilledError" instead of the InitiateError + ListenError + RendezvousError, just to shorten things a bit? An application needs more details (reasons), but it gets them as a parameter anyway when the new "UnfulfilledError" would occur.

occur before the Ready Event for Connections established using Initiate.  <ins> An EstablishmentError occurs either when the set of transport properties and security parameters cannot be fulfilled on a Connection for initiation (e.g. the set of available Paths and/or Protocol Stacks meeting the constraints is empty) or reconciled with the local and/or remote Endpoints; when the remote specifier cannot be resolved; or when no transport-layer connection can be established to the remote Endpoint (e.g. because the remote Endpoint is not accepting connections, the application is prohibited from opening a Connection by the operating system, or the establishment attempt has timed out for any other reason). </ins> See also initiate-and-send to combine Connection establishment and transmission of the first message in a single action. higher value.  By default, this value is Infinite.  The caller is also able to reset the value to Infinite at any point.  <ins> A EstablishmentError occurs either when the Properties and Security </ins> Parameters of the Preconnection cannot be fulfilled for listening or cannot be reconciled with the Local Endpoint (and/or Remote Endpoint, if specified), when the Local Endpoint (or Remote Endpoint, if contained within the RendezvousDone<> Event, and is ready to use as soon as it is passed to the application via the Event.  <ins> An EstablishmentError occurs either when the Properties and Security </ins> Parameters of the Preconnection cannot be fulfilled for rendezvous or cannot be reconciled with the Local and/or Remote Endpoints, when the Local Endpoint or Remote Endpoint cannot be resolved, when no


false This property specifies that a message should be sent and received as <del> a single packet network-layer fragmentation, if possible.  Attempts to send a message with this property set with a size greater to the transport's current estimate of its maximum transmission segment size will result in a "SendError".  When used with transports supporting this functionality and running over IP version 4, the Don't Fragment bit will be set. </del>  7.6. can always support this property, but once a Message has been sent into the send buffer of a protocol, only certain protocols may support removing a message.  For example, an implementation cannot <del> cannot remove bytes from a TCP send buffer, while it can remove data from a SCTP send buffer using the partial reliability extension RFC8303.  When there is no standing queue of Messages within the system, and the Protocol Stack does not support the removal of a Message from the stack's send buffer, this property may be ignored. </del>  Priority: this represents the ability to prioritize a Message over other Messages.  This can be implemented by the system re-ordering Safely Replayable: when this is true, this means that the Message can be used by mechanisms that might transfer it multiple times - e.g., as a result of racing multiple transports or as part of TCP <del> Fast Open. </del>  Final: when this is true, this means that a transport connection can be closed immediately after transmission of the message. Corruption Protection Length: when this is set to any value other <del> than -1, it sets the minimum protection in protocols that allow limiting the checksum length (e.g.  UDP-Lite). Singular Transmission: When set, this property limits the message size to the Maximum Message Size Before Fragmentation or Segmentation (see Section 10.1.7 of I-D.ietf-taps-interface). Messages larger than this size generate an error.  Setting this avoids transport-layer segmentation or network-layer fragmentation.  When used with transports running over IP version 4 the Don't Fragment bit will be set to avoid on-path IP fragmentation (RFC8304). </del>  5.1.2.

I also added a link to NEATPy in the overview of implementations, and inserted a missing word in interface (a missing "without", must have been a nit)
It might be worth to better align the sections on Message Properties in the API and Implementation draft. Note that e.g. "Final" has more implementation details in the API draft than in the implementation draft.

false This property specifies that a message should be sent and received as  <ins> a single packet without network-layer fragmentation, if possible. Attempts to send a message with this property set with a size greater to the transport's current estimate of its maximum transmission segment size will result in a "SendError".  When used with transports supporting this functionality and running over IP version 4, the Don't Fragment bit will be set. </ins> 7.6. can always support this property, but once a Message has been sent into the send buffer of a protocol, only certain protocols may support removing a message.  For example, an implementation cannot  <ins> remove bytes from a TCP send buffer, while it can remove data from a SCTP send buffer using the partial reliability extension RFC8303.  When there is no standing queue of Messages within the system, and the Protocol Stack does not support the removal of a Message from the stack's send buffer, this property may be ignored. </ins> Priority: this represents the ability to prioritize a Message over other Messages.  This can be implemented by the system re-ordering Safely Replayable: when this is true, this means that the Message can be used by mechanisms that might transfer it multiple times - e.g., as a result of racing multiple transports or as part of TCP  <ins> Fast Open.  Also, protocols that do not protect against duplicated messages, such as UDP, can only be used with Messages that are Safely Replayable. </ins> Final: when this is true, this means that a transport connection can be closed immediately after transmission of the message. Corruption Protection Length: when this is set to any value other  <ins> than "Full Coverage", it sets the minimum protection in protocols that allow limiting the checksum length (e.g.  UDP-Lite). Reliable Data Transfer (Message): When true, the property specifies that the Message must be reliably transmitted.  When false, and if unreliable transmission is supported by the underlying protocol, then the Message should be unreliably transmitted.  If the underlying protocol does not support unreliable transmission, the Message should be reliably transmitted. Message Capacity Profile Override: When true, this expresses a wish to override the Generic Connection Property "Capacity Profile" for this Message.  Depending on the value, this can, for example, be implemented by changing the DSCP value of the associated packet (note that the he guidelines in Section 6 of RFC7657 apply; e.g., the DSCP value should not be changed for different packets within a reliable transport protocol session or DCCP connection). No Fragmentation: When set, this property limits the message size to the Maximum Message Size Before Fragmentation or Segmentation (see Section 10.1.7 of I-D.ietf-taps-interface).  Messages larger than this size generate an error.  Setting this avoids transport- layer segmentation or network-layer fragmentation.  When used with transports running over IP version 4 the Don't Fragment bit will be set to avoid on-path IP fragmentation (RFC8304). </ins> 5.1.2.


Since sending a Message may involve a context switch between the application and the transport system, sending patterns that involve multiple small Messages can incur high overhead if each needs to be <del> enqueued separately.  To avoid this, the application should have a way to indicate a batch of Send actions, during which time the implementation will hold off on processing Messages until the batch is complete.  This can also help context switches when enqueuing data in the interface driver if the operation can be batched. </del>  5.2.

In the implementation draft section 5.1.3 says that there should be an interface for the application to indicate batch sending. Section 7.7 specifies such an interface. Should the section in the implementation draft be updated or removed?
I think it should be removed. Mainly, it points out that batching reduces overhead, which is also stated in the API draft in section 7.7.
Perhaps it is still important for the implementation to know that it needs to support batching sends though? I would suggest adjusting the text so it reads from the implementations perspective. Something like
I agree; this is a nice text proposal. PR replaces the current text with your proposed one.

Since sending a Message may involve a context switch between the application and the transport system, sending patterns that involve multiple small Messages can incur high overhead if each needs to be  <ins> enqueued separately.  To avoid this, the application can indicate a batch of Send actions through the API.  When this is used, the implementation should hold off on processing Messages until the batch is complete. </ins> 5.2.


During pre-establishment the application specifies the Endpoints to be used for communication as well as its preferences via Selection Properties and, if desired, also Connection Properties.  Generally, <del> Connection Properties should be configured as early as possible, as they may serve as input to decisions that are made by the implementation (the Capacity Profile may guide usage of a protocol offering scavenger-type congestion control, for example).  In the remainder of this document, we only refer to Selection Properties because they are the more typical case and have to be handled by all implementations. The implementation stores these objects and properties as part of the </del>  Preconnection object for use during connection establishment.  For Selection Properties that are not provided by the application, the implementation must use the default values specified in the Transport The transport system should have a list of supported protocols available, which each have transport features reflecting the capabilities of the protocol.  Once an application specifies its <del> Transport Parameters, the transport system should match the required and prohibited properties against the transport features of the available protocols. </del>  In the following cases, failure should be detected during pre- establishment: <del> The application requested Protocol Properties that include </del>  requirements or prohibitions that cannot be satisfied by any of the available protocols.  For example, if an application requires "Configure Reliability per Message", but no such protocol is <del> available on the host running the transport system, e.g., because SCTP is not supported by the operating system, this should result in an error. The application requested Protocol Properties that are in conflict with each other, i.e., the required and prohibited properties cannot be satisfied by the same protocol.  For example, if an application prohibits "Reliable Data Transfer" but then requires "Configure Reliability per Message", this mismatch should result in an error. It is important to fail as early as possible in such cases in order to avoid allocating resources, e.g., to endpoint resolution, only to find out later that there is no protocol that satisfies the requirements. </del>  3.2. The properties specified during pre-establishment have a close <del> connection to system policy.  The implementation is responsible for </del>  combining and reconciling several different sources of preferences when establishing Connections.  These include, but are not limited to: If all of the leaf nodes fail to connect during racing, i.e. none of the configurations that satisfy all requirements given in the <del> Transport Parameters actually work over the available paths, then the </del>  transport system should notify the application with an InitiateError event.  An InitiateError event should also be generated in case the transport system finds no usable candidates to race.

Implementation draft sec 3: "In the remainder of this document, we only refer to Selection Properties because they are the more typical case and have to be handled by all implementations. The implementation stores these objects and properties as part of the Preconnection object for use during connection establishment." I don't understand these two sentences. In the API doc it seems we use the term Transport Properties for both selection and configuration properties. Why do we only talk about selection properties in this doc? Also in the next sentence what's meant by "objects and properties"? I guess the object(s) that hold the properties? But then this is saying twice the same and we should only use one term to avoid confusion.
Also in the next section 3.1: "Once an application specifies its Transport Parameters" I guess that should be Transport Properties instead? Maybe it would also be good to say here "Once an application specifies its Transport Properties and Preference (required, prefer, ignore, avoid, prohibit - see section 5.2. of [draft-taps-arch])".
I agree; I suspect that this is ancient leftover text that should be cleaned up.
Some small changes suggested

During pre-establishment the application specifies the Endpoints to be used for communication as well as its preferences via Selection Properties and, if desired, also Connection Properties.  Generally,  <ins> Connection Properties should be configured as early as possible, because they can serve as input to decisions that are made by the implementation (e.g., the Capacity Profile can guide usage of a protocol offering scavenger-type congestion control). The implementation stores these properties as a part of the </ins> Preconnection object for use during connection establishment.  For Selection Properties that are not provided by the application, the implementation must use the default values specified in the Transport The transport system should have a list of supported protocols available, which each have transport features reflecting the capabilities of the protocol.  Once an application specifies its  <ins> Transport Properties, the transport system matches the required and prohibited properties against the transport features of the available protocols. </ins> In the following cases, failure should be detected during pre- establishment:  <ins> A request by an application for Protocol Properties that include </ins> requirements or prohibitions that cannot be satisfied by any of the available protocols.  For example, if an application requires "Configure Reliability per Message", but no such protocol is  <ins> available on the host running the transport system this should result in an error, e.g., when SCTP is not supported by the operating system. A request by an application for Protocol Properties that are in conflict with each other, i.e., the required and prohibited properties cannot be satisfied by the same protocol.  For example, if an application prohibits "Reliable Data Transfer" but then requires "Configure Reliability per Message", this mismatch should result in an error. To avoid allocating resources, it is important that such cases fail as early as possible, e.g., to endpoint resolution, only to find out later that there is no protocol that satisfies the requirements. </ins> 3.2. The properties specified during pre-establishment have a close  <ins> relationship to system policy.  The implementation is responsible for </ins> combining and reconciling several different sources of preferences when establishing Connections.  These include, but are not limited to: If all of the leaf nodes fail to connect during racing, i.e. none of the configurations that satisfy all requirements given in the  <ins> Transport Properties actually work over the available paths, then the </ins> transport system should notify the application with an InitiateError event.  An InitiateError event should also be generated in case the transport system finds no usable candidates to race.


4.2.1. <del> Transport Properties are referred to by property names.  These names are alphanumeric strings in which words may be separated by hyphens. These names serve two purposes: </del>  Allowing different components of a TAPS implementation to pass Transport Properties, e.g., between a language frontend and a policy manager, or as a representation of properties retrieved from a file or other storage. <del> Making code of different TAPS implementations look similar. </del>  Transport Property Names are hierarchically organized in the form [<Namespace>.]<PropertyName>. Namespaces for each of the keywords provided in the IANA protocol numbers registry (see https://www.iana.org/assignments/protocol- <del> numbers/protocol-numbers.xhtml), translated to lower-case and with characters other than letters, numbers, and hyphens deleted, are reserved for Protocol Specific Properties and MUST not be used for vendor or implementation-specific properties. </del>  4.2.2.

Updates PR to remove reference to lower-case in namespaces corresponding to IANA protocol names. Express that strict adherence to the doc's naming convention is not required, but that symbols in different languages should be visually similar.
NAME NAME Last minute PR

4.2.1.  <ins> Transport Properties are referred to by property names.  For the purposes of this document, these names are alphanumeric strings in which words may be separated by hyphens.  These names serve two purposes: </ins> Allowing different components of a TAPS implementation to pass Transport Properties, e.g., between a language frontend and a policy manager, or as a representation of properties retrieved from a file or other storage.  <ins> Making code of different TAPS implementations look similar.  While individual programming languages may preclude strict adherence to the aforementioned naming convention (for instance, by prohibiting the use of hyphens in symbols), users interacting with multiple implementations will still benefit from the consistency resulting from the use of visually similar symbols. </ins> Transport Property Names are hierarchically organized in the form [<Namespace>.]<PropertyName>. Namespaces for each of the keywords provided in the IANA protocol numbers registry (see https://www.iana.org/assignments/protocol-  <ins> numbers/protocol-numbers.xhtml), reformatted where necessary to conform to an implementation's naming conventions, are reserved for Protocol Specific Properties and MUST not be used for vendor or implementation-specific properties. </ins> 4.2.2.


There are three different approaches to racing the attempts for different nodes of the connection establishment tree: <del> Immediate </del>  <del> Delayed </del>  Failover Each approach is appropriate in different use-cases and branch types. However, to avoid consuming unnecessary network resources, <del> implementations should not use immediate racing as a default </del>  approach. The timing algorithms for racing should remain independent across 4.2.1. <del> Immediate racing is when multiple alternate branches are started </del>  without waiting for any one branch to make progress before starting the next alternative.  This means the attempts are effectively <del> simultaneous.  Immediate racing should be avoided by implementations, since it consumes extra network resources and establishes state that might not be used. </del>  4.2.2. <del> Delayed racing can be used whenever a single node of the tree has </del>  multiple child nodes.  Based on the order determined when building the tree, the first child node will be initiated immediately, followed by the next child node after some delay.  Once that second initiated, or one of the child nodes successfully completes its negotiation. <del> Delayed racing attempts occur in parallel.  Implementations should </del>  not terminate an earlier child connection attempt upon starting a secondary child.

Opening this to follow up on a comment by NAME on that is now closed. "Simultaneous" and "Staggered" would be better names than "Immediate" and "Delayed". Just from these terms, I would normally think that "Immediate" means to start racing right away, whereas "Delayed" means that nothing would happen for some time. So these two terms are slightly misleading in my opinion.)
I agree this would be an improvement.
LGTM (of course :-) )Thanks, looks good!

There are three different approaches to racing the attempts for different nodes of the connection establishment tree:  <ins> Simultaneous </ins>  <ins> Staggered </ins> Failover Each approach is appropriate in different use-cases and branch types. However, to avoid consuming unnecessary network resources,  <ins> implementations should not use simultaneous racing as a default </ins> approach. The timing algorithms for racing should remain independent across 4.2.1.  <ins> Simultaneous racing is when multiple alternate branches are started </ins> without waiting for any one branch to make progress before starting the next alternative.  This means the attempts are effectively  <ins> simultaneous.  Simultaneous racing should be avoided by implementations, since it consumes extra network resources and establishes state that might not be used. </ins> 4.2.2.  <ins> Staggered racing can be used whenever a single node of the tree has </ins> multiple child nodes.  Based on the order determined when building the tree, the first child node will be initiated immediately, followed by the next child node after some delay.  Once that second initiated, or one of the child nodes successfully completes its negotiation.  <ins> Staggered racing attempts occur in parallel.  Implementations should </ins> not terminate an earlier child connection attempt upon starting a secondary child.


actual thresholds to declare a path unusable are implementation specific. <del> The connection should attempt to use multiple paths in parallel in order to minimize loss and delay.  The actual strategy is implementation specific. </del>  The connection should attempt to use multiple paths in parallel in order to maximize bandwidth and possibly overcome bandwidth

Get a better/more formal explanation of Interactive mode for multipath based on our experience with Siri, etc.
Proposal from Christoph Paasch: The connection should attempt to minimize the latency for interactive traffic patterns by transmitting data across multiple paths. The goal of minimizing the latency will be balanced against the cost fo each of these paths, meaning that depending on the cost of the lower latency path the scheduling might choose to use a higher latency path. Finally, the scheduling of the traffic can be done such that data may be transmitted on multiple paths in parallel to achieve the lowest latency possible.
Thanks for following up on this one.LGTM

actual thresholds to declare a path unusable are implementation specific.  <ins> The connection should attempt to minimize the latency for interactive traffic patterns by transmitting data across multiple paths when it is beneficial to do so.  The goal of minimizing the latency will be balanced against the cost of each of these paths, meaning that depending on the cost of the lower-latency path, the scheduling might choose to use a higher-latency path.  Traffic can be scheduled such that data may be transmitted on multiple paths in parallel to achieve the lowest latency possible.  The specific scheduling algorithm is implementation-specific. </ins> The connection should attempt to use multiple paths in parallel in order to maximize bandwidth and possibly overcome bandwidth


10. Each protocol that can run as part of a Transport Services <del> implementation defines both its API mapping as well as implementation details.  API mappings for a protocol apply most to Connections in which the given protocol is the "top" of the Protocol Stack.  For example, the mapping of the "Send" function for TCP applies to Connections in which the application directly sends over TCP.  If HTTP/2 is used on top of TCP, the HTTP/2 mappings take precendence. </del>  Each protocol has a notion of Connectedness.  Possible values for Connectedness are: in appendix A of I-D.ietf-taps-minset with an implementation hint in the same style, pointing back to section 4 of RFC8303.  10.1. Connectedness: Connected 10.2.  Connectedness: Unconnected Data Unit: Datagram Calling "Abort" on a UDP Connection (ABORT.UDP(-Lite)) is identical to calling "Close". <del> 10.3. </del>  Connectedness: Unconnected Calling "Abort" on a UDP Multicast Receive Connection (ABORT.UDP(- Lite)) is identical to calling "Close". <del> 10.4. The mapping of a TLS stream abstraction into the application is equivalent to the contract provided by TCP (see tcp), and builds upon many of the actions of TCP connections. Connectedness: Connected Data Unit: Byte-stream Connection objects represent a single TLS connection running over a TCP connection between two hosts. Calling "Initiate" on a TLS Connection causes it to first initiate a TCP connection.  Once the TCP protocol is Ready, the TLS handshake will be performed as a client (starting by sending a "client_hello", and so on). Early safely replayable data is supported by TLS 1.3, and sends encrypted application data in the first TLS message when performing session resumption.  For older versions of TLS, or if a session is not being resumed, the initial data will be delayed until the TLS handshake is complete.  TCP Fast Open can also be enabled automatically. A TLS Connection is ready once the underlying TCP connection is Ready, and TLS handshake is also complete and keys have been established to encrypt application data. In addition to TCP initiation errors, TLS can generate errors during its handshake.  Examples of error include a failure of the peer to successfully authenticate, the peer rejecting the local authentication, or a failure to match versions or algorithms. TLS connections will generate TCP errors, or errors due to failures to rekey or decrypt received messages. Calling "Listen" for TLS listens on TCP, and sets up received connections to perform server-side TLS handshakes. TLS Listeners will deliver new connections once they have successfully completed both TCP and TLS handshakes. As with TCP, calling "Clone" on a TLS Connection creates a new Connection with equivalent parameters.  The two Connections are otherwise independent. Like TCP, TLS does not preserve message boundaries.  Although application data is framed natively in TLS, there is not a general guarantee that these TLS messages represent semantically meaningful application stream boundaries.  Rather, sending data on a TLS Connection only guarantees that the application data will be transmitted in an encrypted form.  Marking Messages as Final causes a "close_notify" to be generated once the data has been written. Like TCP, TLS delivers a stream of bytes without any Message delineation.  The data is decrypted prior to being delivered to the application.  If a "close_notify" is received, the stream-wide Message will be delivered with EndOfMessage set. Calling "Close" on a TLS Connection indicates that the Connection should be gracefully closed by sending a "close_notify" to the peer and waiting for a corresponding "close_notify" before delivering the "Closed" event. Calling "Abort" on a TCP Connection indicates that the Connection should be immediately closed by sending a "close_notify", optionally preceded by "user_canceled", to the peer. Implementations do not need to wait to receive "close_notify" before delivering the "Closed" event. 10.5. DTLS follows the same behavior as TLS (tls), with the notable exception of not inheriting behavior directly from TCP.  Differences from TLS are detailed below, and all cases not explicitly mentioned should be considered the same as TLS. Connectedness: Connected Data Unit: Datagram Connection objects represent a single DTLS connection running over a set of UDP ports between two hosts. Calling "Initiate" on a DTLS Connection causes it reserve a UDP local port, and begin sending handshake messages to the peer over UDP.  These messages are reliable, and will be automatically retransmitted. A DTLS Connection is ready once the TLS handshake is complete and keys have been established to encrypt application data. Sending over DTLS does preserve message boundaries in the same way that UDP datagrams do.  Marking a Message as Final does send a "close_notify" like TLS. Receiving over DTLS delivers one decrypted Message for each received DTLS datagram.  If a "close_notify" is received, a Message will be delivered that is marked as Final. </del> 10.6. <del> HTTP requests and responses map naturally into Messages, since they are delineated chunks of data with metadata that can be sent over a transport.  To that end, HTTP can be seen as the most prevalent framing protocol that runs on top of streams like TCP, TLS, etc. In order to use a transport Connection that provides HTTP Message support, the establishment and closing of the connection can be treated as it would without the framing protocol.  Sending and receiving of Messages, however, changes to treat each Message as a well-delineated HTTP request or response, with the content of the Message representing the body, and the Headers being provided in Message metadata. Connectedness: Multiplexing Connected Data Unit: Message Connection objects represent a flow of HTTP messages between a client and a server, which may be an HTTP/1.1 connection over TCP, or a single stream in an HTTP/2 connection. Calling "Initiate" on an HTTP connection intiates a TCP or TLS connection as a client. Calling "Clone" on an HTTP Connection opens a new stream on an existing HTTP/2 connection when possible.  If the underlying version does not support multiplexed streams, calling "Clone" simply creates a new parallel connection. When an application sends an HTTP Message, it is expected to provide HTTP header values as a MessageContext in a canonical form, along with any associated HTTP message body as the Message data.  The HTTP header values are encoded in the specific version format upon sending. HTTP Connections deliver Messages in which HTTP header values attached to MessageContexts, and HTTP bodies in Message data. Calling "Close" on an HTTP Connection will only close the underlying TLS or TCP connection if the HTTP version does not support multiplexing.  For HTTP/2, for example, closing the connection only closes a specific stream. 10.7. QUIC provides a multi-streaming interface to an encrypted transport. Each stream can be viewed as equivalent to a TLS stream over TCP, so a natural mapping is to present each QUIC stream as an individual Connection.  The protocol for the stream will be considered Ready whenever the underlying QUIC connection is established to the point that this stream's data can be sent.  For streams after the first stream, this will likely be an immediate operation. Closing a single QUIC stream, presented to the application as a Connection, does not imply closing the underlying QUIC connection itself.  Rather, the implementation may choose to close the QUIC connection once all streams have been closed (often after some timeout), or after an individual stream Connection sends an Abort. Connectedness: Multiplexing Connected Data Unit: Stream Connection objects represent a single QUIC stream on a QUIC connection. 10.8. Similar to QUIC (quic), HTTP/2 provides a multi-streaming interface. This will generally use HTTP as the unit of Messages over the streams, in which each stream can be represented as a transport Connection.  The lifetime of streams and the HTTP/2 connection should be managed as described for QUIC. It is possible to treat each HTTP/2 stream as a raw byte-stream instead of a carrier for HTTP messages, in which case the Messages over the streams can be represented similarly to the TCP stream (one Message per direction, see tcp). Connectedness: Multiplexing Connected Data Unit: Stream Connection objects represent a single HTTP/2 stream on a HTTP/2 connection. 10.9. </del> Connectedness: Connected Data Unit: Message

Please merge. Completeness really isn't necessary. If this spec is used, the published RFC CAN be updated or added to, at the moment, I'd prefer to ship a spec sooner, rather than add too much, so let's converge:-).
NAME Brought back multicast for now. Good to merge?
I think we can merge this and break out the remaining parts as separate issues as suggested by NAME .
See discussion on
Provide a complete template that covers all functions/properties from the API. Indicate how the template should be used (e.g. leave out the parts that are not relevant for a particular protocol or include all fields and mark fields that are not relevant as such.
I'm wondering if this is really something that's in scope that we should cover. We do have the message properties section that explains how to implement various properties, but I think adding each property to every protocol mapping will be a huge amount of text that will be difficult to get exactly right.
I agree, we have added additional properties since we opened this issue to check the mapping and make sure we have a complete coverage. We should still be consistent in how we treat things though, and perhaps add a sentence that we do not describe all the properties for instance. I have made a pass through to check the mapping from the API to impl. draft for all the protocol mappings as a basis for making things consistent. Here is the summary. We cover the same actions and events for all the protocols with the exceptions that UDP Multicast does not cover the ConnectionError event and SCTP covers the Priority connection property. And MPTCP and UDP-lite are special as they refer back to the base protocols. There is one terminology inconsistency where the impl. draft refers to InitiateError and this has been generalized to EstablishmentError in the API Unless I missed something, there are only two actions that can be made on a connection that are not covered, addRemote and removeRemote. This is probably fine as they do not mean much for most of the protocols (or we could add them to be complete). We should perhaps mention how they relate to MPTCP though if we leave them out. We also do not cover the Preconnection.Rendezvous action. Again this may be fine as it is a bit special (or we can mention it for consistency.) For the events, we cover a subset with main focus on the connection related events. The Closed Event and some events related to connection setup are missing. As we cover ConnectionError we should perhaps cover the Closed event as well? We do not cover any of the Send or Receive events which seems fine. In relation to Connection and Message properties, we do not cover any of them except for the Connection priority covered for SCTP and we talk about some properties in connection to UDP-lite and MPTCP. SCTP also has support for partial reliability and unordered messages so not sure why the priority part was singled out and added. NAME any particular reasoning behind this part? Possibly we should remove the priority part for SCTP or discuss it in running text in a similar fashion as for UDP-lite and MPTCP We may want to update the UDP-lite description to also say something about what properties it supports over UDP. The current description sounds like it is a subset. Once we agree on how to deal with the bullets above, only smaller modifications should be needed to the text.
ConnectionError should be added; SCTP priority should be removed Fix the inconsistency Add addRemote and removeRemote text related to MPTCP No action Mention Closed Event See item 1, no further action Update UDP-Lite description
Currently, the implementation document does not have mappings for MPTCP, even though MPTCP is part of minset. Should be boring, right? :)
Should be taken care of by , yes
closed byPR
Right now, the per-protocol behavior is specified in Implementation, which is an Informational document; however, this stuff really should be specified normatively. So either they should be split out into (one or more?) separate documents, or Implementation should be made Normative just for the sake of one of its sections (which imo would be weird).
I guess having a separate document for the per-protocol behaviour makes a lot of sense. This document than can also define names for protocol specific properties.
Interim discussion: considering splitting off a non-informational document for mapping definitions
Last discussion ended in agreement on splitting out protocol mappings. Two options were discussed 1) taking out all protocol mappings, 2) leaving TCP and UDP as the two basic protocols and moving the other mappings. Good if we can conclude on which path to take.
I suggest that keeping UDP and TCP in here could be a simple expedient, especially if we make it clear that other RFCs can specify additional mappings. These two are very long-established transport protocols, and I think we can rapidly conclude this spec using them. Whilst I would be very happy to also see bindings for other important transports, that seems like something that could delay the group.
Leave minset protocols (UDP, TCP, SCTP, MPTCP), and define the template for new mappings. Split out drafts: (TLS + DTLS), QUIC (and fallback to TLS, maybe join with TLS), HTTP
Just a small correction: minset also includes UDP-Lite, and the LEDBAT congestion control (the latter should probably be skipped since its encapsulation still isn't spec'd).
We should apply this as soon as possible as it makes the implementation consistent with minset, but deleting UDP multicast really hurts .Can we re-add the UDP multicast part, especially as this does not seem to be enough for an individual mapping document. Let's merge and fix the rest separately.

10. Each protocol that can run as part of a Transport Services  <ins> implementation should have a well-defined API mapping.  API mappings for a protocol apply most to Connections in which the given protocol is the "top" of the Protocol Stack.  For example, the mapping of the "Send" function for TCP applies to Connections in which the application directly sends over TCP. </ins> Each protocol has a notion of Connectedness.  Possible values for Connectedness are: in appendix A of I-D.ietf-taps-minset with an implementation hint in the same style, pointing back to section 4 of RFC8303. <ins> This document defines the API mappings for protocols defined in I- D.ietf-taps-minset.  Other protocol mappings can be provided as separate documents, following the mapping template appendix-mapping- template. </ins> 10.1. Connectedness: Connected 10.2. <ins> Connectedness: Connected Data Unit: Byte-stream API mappings for MPTCP are identical to TCP.  MPTCP adds support for multipath properties, such as "Multi-Paths Transport" and "Policy for using Multi-Path Transports". 10.3. </ins> Connectedness: Unconnected Data Unit: Datagram Calling "Abort" on a UDP Connection (ABORT.UDP(-Lite)) is identical to calling "Close".  <ins> 10.4. Connectedness: Unconnected Data Unit: Datagram API mappings for UDP-Lite are identical to UDP.  Properties that require checksum coverage are not supported by UDP-Lite, such as "Corruption Protection Length", "Full Checksum Coverage on Sending", "Required Minimum Corruption Protection Coverage for Receiving", and "Full Checksum Coverage on Receiving". 10.5. </ins> Connectedness: Unconnected Calling "Abort" on a UDP Multicast Receive Connection (ABORT.UDP(- Lite)) is identical to calling "Close".  10.6.  Connectedness: Connected Data Unit: Message


initiated, or one of the child nodes successfully completes its negotiation. <del> Staggered racing attempts occur in parallel.  Implementations should not terminate an earlier child connection attempt upon starting a secondary child. The delay between starting child nodes should be based on the properties of the previously started child node.  For example, if the first child represents an IP address with a known route, and the second child represents another IP address, the delay between starting the first and second IP addresses can be based on the expected retransmission cadence for the first child's connection (derived from historical round-trip-time).  Alternatively, if the first child represents a branch on a Wi-Fi interface, and the second child represents a branch on an LTE interface, the delay should be based on the expected time in which the branch for the first interface would be able to establish a connection, based on link quality and historical round-trip-time. Any delay should have a defined minimum and maximum value based on the branch type.  Generally, branches between paths and protocols should have longer delays than branches between derived endpoints. The maximum delay should be considered with regards to how long a user is expected to wait for the connection to complete. If a child node fails to connect before the delay timer has fired for the next child, the next child should be started immediately. </del>  4.2.3.

NAME NAME please take a look at the updated text!
Section 4.2.1 on delayed racing: "For example, if the first child represents an IP address with a known route, and the second child represents another IP address, the delay between starting the first and second IP addresses can be based on the expected retransmission cadence for the first child's connection (derived from historical round-trip-time). Alternatively, if the first child represents a branch on a Wi-Fi interface, and the second child represents a branch on an LTE interface, the delay should be based on the expected time in which the branch for the first interface would be able to establish a connection, based on link quality and historical round-trip-time." What the reasoning for these two recommendations? I'm actually not sure what exactly is meant. Is the first case 3xRTT (-> initial RTO) and the second 1xRTT? Why? Also in the next paragraph: "Generally, branches between paths and protocols should have longer delays than branches between derived endpoints." What? Why? And then "The maximum delay should be considered with regards to how long a user is expected to wait for the connection to complete." How do I know this?
Also, maybe it's just me, but I can't parse "retransmission cadence", and the phrase with "historical round-trip time" is 1) confusing (I don't get what it means, but maybe that's due to me misunderstanding "cadence"?), and 2) seems to be missing an article.
I read this as examples of things to consider when implementing a happy-eyeballs style algorithm, rather than explicit recommendations. Maybe this fix is to make this more vague, rather than more explicit?
I think NAME is right and the intention here is that these are just examples of things to consider. We should conclude if anything more is needed beyond making it more clear in the text that these are examples and fixing the other language issues.
The issue I agreed with was a desire to find simpler terms to describe these examples.
Refer to Happy Eyeballs for exact timing algorithm as the concrete reference (or ICE, etc).
It would be helpful to add some of the reasoning behind, e.g., "Generally, branches between paths and protocols should have longer delays than branches between derived endpoints."
Thanks, looks good. (Just marked a nit.)

initiated, or one of the child nodes successfully completes its negotiation.  <ins> Staggered racing attempts can proceed in parallel.  Implementations should not terminate an earlier child connection attempt upon starting a secondary child. If a child node fails to connect before the delay time has expired for the next child, the next child should be started immediately. Staggered racing between IP addresses for a generic Connection should follow the Happy Eyeballs algorithm described in RFC8305.  RFC8421 provides guidance for racing when performing Interactive Connectivity Establishment (ICE). Generally, the delay before starting a given child node ought to be based on the length of time the previously started child node is expected to take before it succeeds or makes progress in connection establishment.  Algorithms like Happy Eyeballs choose a delay based on how long the transport connection handshake is expected to take. When performing staggered races in multiple branch types (such as racing between network interfaces, and then racing between IP addresses), a longer delay may be chosen for some branch types.  For example, when racing between network interfaces, the delay should also take into account the amount of time it takes to prepare the network interface (such as radio association) and name resolution over that interface, in addition to the delay that would be added for a single transport connection handshake. Since the staggered delay can be chosen based on dynamic information, such as predicted round-trip time, implementations should define upper and lower bounds for delay times.  These bounds are implementation-specific, and may differ based on which branch type is being used. </ins> 4.2.3.


is not entangled: Changing Priority on one Connection does not change it on the other Connections in the same Connection Group. <del> In addition, incoming entangled Connections can be received by creating a Listener on an existing connection: ListenClone() creates a Listener that listens on the same LocalEndpoint as the one the cloned Connection is using.  Any new Connection received by this Listener will be entangled with the cloned Connection.  Changing one of the Connection Properties on one Connection in the group changes it for all others.  Message Properties, however, are not entangled.  For example, changing "Timeout for aborting Connection" (see conn-timeout) on one Connection in a group will automatically change this Connection Property for all Connections in the group in the same way.  However, changing "Lifetime" (see msg-lifetime) of a Message will only affect a single Message on a single Connection, entangled or not. </del>  If the underlying protocol supports multi-streaming, it is natural to use this functionality to implement Clone.  In that case, entangled

Proposal: replace this with a way to query for a connection what the other connections in a group are (which will only happen passively for multi streamed protocols today).
Implemented ; NAME does this work?
I'm merging this now so I can proceed with re-structuring. It reflects what we discussed at the interim, so I don't think there should be more concerns.
See the discussion in PR . I think this is best dealt with by merging PR after a small fix, then doing a separate PR to address this. IMO the easiest way to do this is to allow to query a group number (a simple integer). Without grouping, every new Connection simply gets a new number. Having the same group number as any previous Connection(s) indicates group membership. This is how we did it in NEAT too. If people agree with this, I can take care of this.
I remember the discussion in NEAT, to me it seems a good way to resolve the issue.
PR now resolves this differently. It doesn't matter much, IMO the method in this PR also works fine.
I merged PR , which closes this issue.
Should ListenClone() fail for non-multiplexed transports How to specify if a top-level listener can accept cloned (multiplexed) streams Should ListenClone() be called ListenForClone()...?
Just a note to say that issue waits for this issue to be addressed.
Rename function Trying to do this without a multiplexing transport should fail
ping: issue is waiting for this...
Currently the "Clone" part of, e.g., TCP (section 10.1) only says: "Calling Clone on a TCP Connection creates a new Connection with equivalent parameters. The two Connections are otherwise independent." This text should reflect that equivalent behavior must be maintained upon later Connection configuration calls. And, there should be more text in general on cloning (see issue ).
This text must refer to a new "do I want a Connection that supports initiate followed by listen?" property. This property will be added to address - so I'll address this issue when the PR addressing has landed.
I'm ok with this modulo one suggested change

is not entangled: Changing Priority on one Connection does not change it on the other Connections in the same Connection Group.  <ins> It is also possible to check which Connections belong to the same Connection Group.  Calling GroupedConnections() on a specific Connection returns a set of all Connections in the same group. Connections will be in the same group if the application previously called Clone.  Passive Connections can also be added to the same group - e.g., when a Listener receives a new Connection that is just a new stream of an already active multi-streaming protocol instance. Changing one of the Connection Properties on one Connection in the group changes it for all others.  Message Properties, however, are not entangled.  For example, changing "Timeout for aborting Connection" (see conn-timeout) on one Connection in a group will automatically change this Connection Property for all Connections in the group in the same way.  However, changing "Lifetime" (see msg- lifetime) of a Message will only affect a single Message on a single Connection, entangled or not. </ins> If the underlying protocol supports multi-streaming, it is natural to use this functionality to implement Clone.  In that case, entangled


protocol stack(s).  They are collected into a TransportProperties object to be passed into a Preconnection object: <del> Individual properties are then added to the TransportProperties Object: </del>  Selection Properties of type "Preference" can be frequently used. Implementations MAY therefore provide additional convenience

If I understand correctly, we're staying with the current master. Therefore, here's my PR to - I think we always want to override, and if we override, calling it Set() makes this clearer than Add(). For Security Parameters, perhaps it should be possible to add multiple identities, ciphersuites, etc. But that's out of scope for this PR.
NAME - I agree with the response by NAME in , which concludes: "If I misunderstood, I think the best approach is to indeed allow specifying multiple (Preference, Enumeration) in a single Set() call." KISS, I say :)
If calling Set() for Security Parameters overrides (), then the API should allow setting a Security Parameter to multiple values, e.g., using a list. This would be needed, e.g., to set multiple cipher suites. I don't think we need to make this overly formal. Adding one sentence to the Security Parameters section should be enough. Right?
We should also consider whether setting multiple values is not only needed for Security Properties, but also for Transport Properties like Interface and PvD. This one is more complex, as it comes with a tuple. Not sure whether just adding a sentence and making accept a list auf values is the right approach or if we should have an call for that – The former is nicer in the API document, the latter seems more appropriate when thinking about how one would use the API
Agree that we want to enable setting multiple values for Interface and PvD, but I thought that's why their Type is "Set (Preference, Enumeration)" - I took this to mean that it's multiple Interfaces, each with a Preference. But reading the text again, maybe I misunderstood. If I misunderstood, I think the best approach is to indeed allow specifying multiple (Preference, Enumeration) in a single Set() call.
Reopening this because I still think we should add a sentence to Section 5.3.1 of the API draft, perhaps simply saying that the implementation should support setting multiple values for a single Security Parameter. Still not sure about the Interface and PvD properties - I think that making set accept a list auf values, as suggested by NAME is the right approach, but perhaps it's worth adding a sentence to explain this to their sections as well.
Interface and PvD are already "sets of tuples". We could have a sentence to clarify that this is a "collection", or maybe just call it "collection". Maybe also add a sentence to the data types section to indicate that they can have collections. Mention in text that security parameters can take collections.
We should rename these if they are really setting and not adding.
The original idea to call it was to emphasise that we add these properties to the set the system needs to consider. As we removed all Intents and decided to be much more conservative here, I agree is the better choice for the current API
Regarding continuing with the current master: yes, correct. About this PR: looks fine to me! Thanks a lot for doing thisThank you for changing this – makes sense given that we removed the Intents. After applying this PR, we should consider whether setting multiple values is not only needed for Security Properties (as stated in ), but also for Transport Properties like Interface and PvD.

protocol stack(s).  They are collected into a TransportProperties object to be passed into a Preconnection object:  <ins> Individual properties are then set on the TransportProperties Object. Setting a Transport Property to a value overrides the previous value of this Transport Property. </ins> Selection Properties of type "Preference" can be frequently used. Implementations MAY therefore provide additional convenience


7.2. <del> When a path change occurs, the Transport Services implementation is responsible for notifying Protocol Instances in the Protocol Stack. If the Protocol Stack includes a transport protocol that supports multipath connectivity, an update to the available paths should inform the Protocol Instance of the new set of paths that are permissible based on the Selection Properties passed by the application.  A multipath protocol can establish new subflows over new paths, and should tear down subflows over paths that are no longer available.  Pooled Connections pooled-connections may add or remove underlying transport connections in a similar manner.  If the Protocol Stack includes a transport protocol that does not support multipath, but support migrating between paths, the update to available paths can be used as the trigger to migrating the connection.  For protocols that do not support multipath or migration, the Protocol Instances may be informed of the path change, but should not be forcibly disconnected if the previously used path becomes unavailable.  An exception to this case is if the System Policy changes to prohibit traffic from the Connection based on its properties, in which case the Protocol Stack should be disconnected. </del>  8.

This is a first attempt to work on the section "Handling Path Changes" (see issue ), however, I think more work is needed. To be honest I don't fully understand this part and it would be could if someone else could provide more input about the System Policy part: "For protocols that do not support multipath or migration, the Protocol Instances should be informed of the path change, but should not be forcibly disconnected if the previously used path becomes unavailable. An exception to this case is if the System Policy changes to prohibit traffic from the Connection based on its properties, in which case the Protocol Stack should be disconnected." Also the last part about pooled connection should say more but I'm not sure what. We currently have now only this sentence left: "Pooled Connections {{pooled-connections}} may add or remove underlying transport connections in a similar manner." And finally the originally issue was also mentioning fallback which I didn't address yet but I have this feeling that that might need a separate section. Should I add a separate issue for that or is that already addressed somewhere and I missed it?
II like your change - makes the text much clearer.
NAME : Is the path change problem resolved here - there's been stuff added on paths and policy since this.
Section 7.2 on Handling Path Changes in the implementation draft needs to be better aligned to the interface we are currently still discussing for multipath protocols. Currently it reads like an taps implementation can freely decide which paths to use when. That's probable not the case but in any case needs also more specific guidance. However, we probably need to leave this issue as a reminder for later when we have decided on the final interface for MP protocols. I would eventually like to separate sections for handover (two parallel paths), migration (move from one to the other but keep same protocol instance) and fallback (move form one protocol instance to another).

7.2.  <ins> When a path change occurs, e.g., when the IP address of an interface changes or a new interface becomes available, the Transport Services implementation is responsible for notifying the application of the change.  The path change may interrupt connectivity on a path for an active connection or provide an opportunity for a transport that supports multipath or migration to adapt to the new paths. For protocols that do not support multipath or migration, the Protocol Instances should be informed of the path change, but should not be forcibly disconnected if the previously used path becomes unavailable. If the Protocol Stack includes a transport protocol that also supports multipath connectivity with migration support, the Transport Services implementation should also inform the Protocol Instance of potentially new paths that become permissible based on the Selection Properties passed by the application.  A protocol can then establish new subflows over new paths while an active path is still available or, if migration is supported, also after a break has been detected, and should attempt to tear down subflows over paths that are no longer used.  The Transport Services API provides an interface to set a multipath policy that indicates when and how different paths should be used.  However, detailed handling of these policies is still implementation-specific.  The decision about when to create a new path or to announce a new path or set of paths to the remote endpoint, e.g., in the form of additional IP addresses, is implementation-specific or could be be supported by future API extensions.  If the Protocol Stack includes a transport protocol that does not support multipath, but does support migrating between paths, the update to the set of available paths can trigger the connection to be migrated. In case of Pooled Connections pooled-connections, the transport system may add connections over new paths or different protocols to the pool if permissible based on the multipath policy and Selection Properties.  In case a previously used path becomes unavailable, the transport system may disconnect all connections that require this path, but should not disconnect the pooled connection object exposed to the application.  The strategy how is implementation-specific, but should be consistent with the behavior of multipath transports. </ins> 8.


5.3. Several protocols allow sending higher-level protocol or application <del> data within the first packet of their protocol establishment, such as TCP Fast Open RFC7413 and TLS 1.3 RFC8446.  This approach is referred to as sending Zero-RTT (0-RTT) data.  This is a desirable property, but poses challenges to an implementation that uses racing during connection establishment. </del>  If the application has 0-RTT data to send in any protocol handshakes, it needs to provide this data before the handshakes have begun.  When

Removed phrasing that implies 0-RTT data is just one packet and added a reference to the Maximum Message Size with Connection Establishment property. Added a sentence on how/when an implementation can set this property.
The opening para reads to me like there is just one first packet. ... or is this actually more than one packets before the handshake complete?
It is one packet for TCP, but it can be more for QUIC, TLS 1.3 (etc)
Reference Connection Property "Maximum Message Size Concurrent with Connection Establishment"
Seems like a useful addition that resolves my issue.

5.3. Several protocols allow sending higher-level protocol or application  <ins> data during their protocol establishment, such as TCP Fast Open RFC7413 and TLS 1.3 RFC8446.  This approach is referred to as sending Zero-RTT (0-RTT) data.  This is a desirable property, but poses challenges to an implementation that uses racing during connection establishment. The amount of data that can be sent as 0-RTT data varies by protocol and can be queried by the application using the "Maximum Message Size Concurrent with Connection Establishment" Connection Property.  An implementation can set this property according to the protocols that it will race based on the given Selection Properties when the application requests to establish a connection. </ins> If the application has 0-RTT data to send in any protocol handshakes, it needs to provide this data before the handshakes have begun.  When


use of temporary local addresses, sometimes called "privacy" addresses RFC4941.  Temporary addresses are generally used to prevent linking connections over time when a stable address, sometimes called <del> "permanent" address, is not needed.  Note that if an application Requires the use of temporary addresses, the resulting Connection cannot use IPv4, because temporary addresses do not exist in IPv4. </del>  5.2.13.

NAME PTAL
sec 5.2.12 Using temporary local addresses may break various mechanisms that prove address ownership (e.g QUIC resumption tokens) and therefore impair performance as the client has to re-verify its address.
The default is to use a temporary address for client local address; and to not use it for a server local address. This is the same as most socket APIs. What are you suggesting in the text, NAME
right, a QUIC resumption token will generally contain the client's IP address so that future connections no longer have to go through Retry and anti-amplification mechanisms, which reduce throughput. so IIUC 5.2.12 could say something like "Clients that use temporary local addresses might invalidate server-provided tokens that validate the client's IP addresses (e.g. Sec 8.1.3 of quic-transport), increasing latency."
Add a one-sentence warning that using a temporary address has a potential performance impact on resumption.
Do we wish to refer to RFC 7721?
This looks pretty close ... check these words and submit.

use of temporary local addresses, sometimes called "privacy" addresses RFC4941.  Temporary addresses are generally used to prevent linking connections over time when a stable address, sometimes called  <ins> "permanent" address, is not needed.  There are some caveats to note when specifying this property.  First, if an application Requires the use of temporary addresses, the resulting Connection cannot use IPv4, because temporary addresses do not exist in IPv4.  Second, temporary local addresses might involve trading off privacy for performance. For instance, temporary addresses can interfere with resumption mechanisms that some protocols rely on to reduce initial latency. </ins> 5.2.13.