As with sending, the type of the Message to be passed is dependent on the implementation, and on the constraints on the Protocol Stacks <del> implied by the Connection's transport parameters.  The Message may also contain metadata from protocols in the Protocol Stack; which metadata is available is Protocol Stack dependent.  In particular, when this information is available, the value of the Explicit Congestion Notification (ECN) field is contained in such metadata. This information can be used for logging and debugging purposes, and for building applications which need access to information about the transport internals for their own operation. </del>  The Message Object must provide some method to retrieve an octet array containing application data, corresponding to a single message 8.1.  The application may check a property to determine if a received Message is the Final Message on a Connection.  For any Message that is marked as Final, the application can assume that there will be no Any calls to "Receive" once the Final Message has been delivered will result in errors. <del> 8.2. </del>  The Receive Event is intended to be fired once per application-layer Message sent by the remote endpoint; i.e., it is a desired property

And add an early data metadata value.
I'll dig up relevant discussions from the TLS WG.
See, e.g.: URL
Looks like a good candidate for a read-only recv property…
NAME Aren't all receives read-only?
Looks good - minor nits noted in URL lgtmMuch better I think!

As with sending, the type of the Message to be passed is dependent on the implementation, and on the constraints on the Protocol Stacks  <ins> implied by the Connection's transport parameters. </ins> The Message Object must provide some method to retrieve an octet array containing application data, corresponding to a single message 8.1. <ins> Each Message may also contain metadata from protocols in the Protocol Stack; which metadata is available is Protocol Stack dependent.  The following metadata values are supported: 8.1.1. When available, Message metadata carries the value of the Explicit Congestion Notification (ECN) field.  This information can be used for logging and debugging purposes, and for building applications which need access to information about the transport internals for their own operation. 8.1.2. In some cases it may be valuable to know whether data was read as part of early data streams.  This is useful if applications need to treat early data separately, e.g., if early data has different security properties than data sent after connection establishment. In the case of TLS 1.3, client early data can be replayed maliciously (see I-D.ietf-tls-tls13).  Thus, receivers may wish to perform additional checks for early data to ensure it is idempotent or not replayed.  If TLS 1.3 is available and the recipient Message was sent as part of early data, the corresponding metadata carries a flag indicating as such.  If early data is enabled, applications should check this metadata field for Messages received during connection establishment and respond accordingly. 8.2. </ins> The application may check a property to determine if a received Message is the Final Message on a Connection.  For any Message that is marked as Final, the application can assume that there will be no Any calls to "Receive" once the Final Message has been delivered will result in errors.  <ins> 8.3. </ins> The Receive Event is intended to be fired once per application-layer Message sent by the remote endpoint; i.e., it is a desired property


Remote Endpoint and the Local Endpoint, such as IP addresses resolved from a DNS hostname. <del> 4.2.3. </del>  While some transports expose a byte stream abstraction, most higher level protocols impose some structure onto that byte stream.  That

Satisfies , including text for both generic protocol equivalence and security protocols. We may want to move the section around organizationally later, as per some of Gorry's comments on structure. However, I reference terms from our concept definitions, so it is easier to write in the latter portion of the document.
And add a subsection specifically for security protocols. Determining what security protocols (and configurations) are equivalent is nontrivial and incredibly important if we are to race, e.g., TLS and gQUIC.
I think the racing/gathering section needs a tiny little explanation text and this seems reasonable to add - if you add to this section, can you also make sure it mentions something of the terms "policy" and "cached state" as a part of this update?
NAME yes, will do!
Addressed by .
tiny editorial nits on section refs otherwise this is good to goLGTM!

Remote Endpoint and the Local Endpoint, such as IP addresses resolved from a DNS hostname.  <ins> 4.3. The Transport Services architecture defines a mechanism that allows applications to easily use different network paths and Protocol Stacks.  Transitioning between different Protocol Stacks may in some cases be controlled by properties that only change when application code is updated.  For example, an application may enable the use of a multipath or multistreaming transport protocol by modifying the properties in its Pre-Connection configuration.  In some cases, however, the Transport Services system will be able to automatically change Protocol Stacks without an update to the application, either by selecting a new stack entirely, or racing multiple candidate Protocol Stacks during connection establishment.  This functionality can be a powerful driver of new protocol adoption, but must be constrained carefully to avoid unexpected behavior that can lead to functional or security problems. If two different Protocol Stacks can be safely swapped, or raced in parallel (see racing), then they are considered to be "equivalent". Equivalent Protocol Stacks must meet the following criteria: Both stacks must offer the same interface to the application for connection establishment and data transmission.  For example, if one Protocol Stack has UDP as the top-level interface to the application, then it is not equivalent to a Protocol Stack that runs TCP as the top-level interface.  Among other differences, the UDP stack would allow an application to read out message boundaries based on datagrams sent from the Remote Endpoint, whereas TCP does not preserve message boundaries on its own. Both stacks must offer the same transport services, as required by the application.  For example, if an application specifies that it requires reliable transmission of data, then a Protocol Stack using UDP without any reliability layer on top would not be allowed to replace a Protocol Stack using TCP.  However, if the application does not require reliability, then a Protocol Stack that adds unnecessary reliability might be allowed as an equivalent Protocol Stack as long as it does not conflict with any other application-requested properties. Both stacks must offer the same security properties.  See the security protocol equivalence section below for futher discussion (security-equivalence). 4.3.1. The inclusion of transport security protocols I-D.ietf-taps- transport-security in a Protocol Stack adds extra restrictions to Protocol Stack equivalence.  Security features and properties, such as cryptographic algorithms, peer authentication, and identity privacy vary across security protocols, and across versions of security protocols.  Protocol equivalence should not be assumed for different protocols or protocol versions, even if they offer similar application configuration options. To ensure that security protocols are not incorrectly swapped, Transport Services systems should only automatically generate equivalent Protocol Stacks when the transport security protocols within the stacks are identical.  Specifically, a system should consider protocols identical only if they are of the same type and version.  For example, the same version of TLS running over two different transport protocol stacks may be considered equivalent, whereas TLS 1.2 and TLS 1.3 I-D.ietf-tls-tls13 should not be considered equivalent. 4.4. </ins> While some transports expose a byte stream abstraction, most higher level protocols impose some structure onto that byte stream.  That


local interfaces that are connected to different access networks. Internally, the transport system will first exclude all protocols and <del> paths that match a Prohibit, then only keep candidates that match a Require, then sort candidates according to Preferred properties, and then use Avoided properties as a tiebreaker.  In case of conflicts between Protocol and Path Selection Properties, Path Selection Properties take precedence.  For example, if an application indicates a preference for a specific path, but also a preference for a protocol not available on this path, the transport system will try the path first, so the Protocol Selection Property might not have an effect. </del>  All Transport Properties used in the pre-establishment phase are collected in a TransportProperties Object that is passed to the The Preconnection Object must be specified with both a Local Endpoint and a Remote Endpoint, and also the transport properties and security <del> parameters needed for Protocol Stack selection.  The Rendezvous() Action causes the Preconnection to listen on the Local Endpoint for an incoming Connection from the Remote Endpoint, while simultaneously trying to establish a Connection from the Local Endpoint to the Remote Endpoint.  This corresponds to a TCP simultaneous open, for example. </del>  The Rendezvous() Action consumes the Preconnection.  Once Rendezvous() has been called, no further properties may be added to represent the concrete addresses, local and server reflexive, on which a Rendezvous() for the Preconnection will listen for incoming Connections.  This list can be passed to a peer via a signalling <del> protocol, such as SIP or WebRTC, to configure the remote. </del>  6.4. i.e., when the data derived from the Message has been passed down or through the underlying Protocol Stack and is no longer the responsibility of the implementation of this interface.  The exact <del> disposition of the Message when the Sent Event occurs is specific to the implementation and the constraints on the Protocol Stacks implied by the Connection's Transport Properties.  The Sent Event contains an implementation-specific reference to the Message to which it applies. </del>  Sent Events allow an application to obtain an understanding of the amount of buffering it creates.  That is, if an application calls the

quickly adding this to fix NAME nits before -01 submission
Text: Internally, the transport system will first exclude all protocols and paths that match a Prohibit, then only keep candidates that match a Require, then sort candidates according to Preferred properties, and then use Avoided properties as a tiebreaker. does not read correct for me... I understand excluding prohibit, but the action on require seems wrong. I expected to remove candidates with prohibit and remove candidates that do NOT have require. What is written is different. —- I would prefer a para break before: The Rendezvous() Action causes the to show the change of topic. — Refs needed: SIP or WebRTC — maximum Message size should this be capitalised? What is maxlen? The same value? Of different? —- I could not finally understand what is intended by this text; The exact disposition of the Message when the Sent Event occurs is specific to the implementation and the constraints on the Protocol Stacks implied by the Connection's Transport Properties.

local interfaces that are connected to different access networks. Internally, the transport system will first exclude all protocols and  <ins> paths that match a Prohibit, then exclude all protocols and paths that do not match a Require, then sort candidates according to Preferred properties, and then use Avoided properties as a tiebreaker.  In case of conflicts between Protocol and Path Selection Properties, Path Selection Properties take precedence.  For example, if an application indicates a preference for a specific path, but also a preference for a protocol not available on this path, the transport system will try the path first, so the Protocol Selection Property might not have an effect. </ins> All Transport Properties used in the pre-establishment phase are collected in a TransportProperties Object that is passed to the The Preconnection Object must be specified with both a Local Endpoint and a Remote Endpoint, and also the transport properties and security  <ins> parameters needed for Protocol Stack selection. The Rendezvous() Action causes the Preconnection to listen on the Local Endpoint for an incoming Connection from the Remote Endpoint, while simultaneously trying to establish a Connection from the Local Endpoint to the Remote Endpoint.  This corresponds to a TCP simultaneous open, for example. </ins> The Rendezvous() Action consumes the Preconnection.  Once Rendezvous() has been called, no further properties may be added to represent the concrete addresses, local and server reflexive, on which a Rendezvous() for the Preconnection will listen for incoming Connections.  This list can be passed to a peer via a signalling  <ins> protocol, such as SIP RFC3261 or WebRTC RFC7478, to configure the remote. </ins> 6.4. i.e., when the data derived from the Message has been passed down or through the underlying Protocol Stack and is no longer the responsibility of the implementation of this interface.  The exact  <ins> disposition of the Message (i.e., whether it has actually been transmitted, moved into a buffer on the network interface, moved into a kernel buffer, and so on) when the Sent Event occurs is implementation-specific.  The Sent Event contains an implementation- specific reference to the Message to which it applies. </ins> Sent Events allow an application to obtain an understanding of the amount of buffering it creates.  That is, if an application calls the


representations. Multiple endpoint identifiers can be specified for each Local <del> Endpoint and RemoteEndoint.  For example, a Local Endpoint could be </del>  configured with two interface names, or a Remote Endpoint could be specified via both IPv4 and IPv6 addresses.  These multiple identifiers refer to the same transport endpoint. <del> The transport services API will resolve names internally, when the </del>  Initiate(), Listen(), or Rendezvous() method is called establish a <del> Connection.  The API does not need the application to resolve names, and premature name resolution can damage performance by limiting the scope for alternate path discovery during Connection establishment. The Resolve() method is, however, provided to resolve a Local Endpoint or a Remote Endpoint in cases where this is required, for example with some Network Address Translator (NAT) traversal protocols (see rendezvous). </del>  5.2. Local Endpoint will be configured with some method of discovering NAT bindings, e.g., a Session Traversal Utilities for NAT (STUN) server. In this case, the Local Endpoint may resolve to a mixture of local <del> and server reflexive addresses.  The Resolve() method on the </del>  Preconnection can be used to discover these bindings: The Resolve() call returns a list of Preconnection Objects, that

as I started reviewing issues in advance of the interim, I saw that was already mostly fixed, but I made a few editorial changes to the language there.

representations. Multiple endpoint identifiers can be specified for each Local  <ins> Endpoint and Remote Endpoint.  For example, a Local Endpoint could be </ins> configured with two interface names, or a Remote Endpoint could be specified via both IPv4 and IPv6 addresses.  These multiple identifiers refer to the same transport endpoint.  <ins> The transport services API resolves names internally, when the </ins> Initiate(), Listen(), or Rendezvous() method is called establish a  <ins> Connection.  The API explicitly does not require the application to resolve names, though there is a tradeoff between early and late binding of addresses to names.  Early binding allows the API implementation to reduce connection setup latency, at the cost of potentially limited scope for alternate path discovery during Connection establishment, as well as potential additional information leakage about application interest when used with a resolution method (such as DNS without TLS) which does not protect query confidentiality. The Resolve() action on Preconnection can be used by the application to force early binding when required, for example with some Network Address Translator (NAT) traversal protocols (see rendezvous). </ins> 5.2. Local Endpoint will be configured with some method of discovering NAT bindings, e.g., a Session Traversal Utilities for NAT (STUN) server. In this case, the Local Endpoint may resolve to a mixture of local  <ins> and server reflexive addresses.  The Resolve() action on the </ins> Preconnection can be used to discover these bindings: The Resolve() call returns a list of Preconnection Objects, that


Preconnection Object with a Remote Endpoint specifier, optionally a Local Endpoint specifier (if not specified, the system will attempt to determine a suitable Local Endpoint), as well as all properties <del> necessary for candidate selection.  After calling Initiate, no further properties may be added to the Preconnection.  The Initiate() call consumes the Preconnection and creates a Connection Object.  A Preconnection can only be initiated once. </del>  Once Initiate is called, the candidate Protocol Stack(s) may cause one or more candidate transport-layer connections to be created to connections, or the application is prohibited from opening a Connection by the operating system).  6.2. Passive open is the Action of waiting for Connections from remote invalid to send a Message over a UDP connection that is larger than the available datagram sending size. <del> If Send is called on a Connection which has not yet been established, an Initiate Action will be implicitly performed simultaneously with the Send.  Together with the Idempotent property (see send- idempotent), this can be used to send data during establishment for 0-RTT session resumption on Protocol Stacks that support it. </del> 7.2. Like all Actions in this interface, the Send Action is asynchronous. 7.6.  Sender-side framing allows a caller to provide the interface with a function that takes a Message of an appropriate application-layer type and returns an array of octets, the on-the-wire representation

This is my attempt to address with something that's (IMO) less clunky in the common case than . I'm not a big fan of this way of doing things either -- specifically, I think the semantics of RendezvousAndSend(), while consistent, are probably wrong.
After some discussion with Brian I agree that the only solution is to provide some initial date to the preconnection when calling initiate. However, why do we need to have two different function. If we only have initiate and provide idempotent data as an optional input parameter that should be fine, right?
NAME your question is now
In commentary on , NAME asks: However, why do we need to have two different function. If we only have initiate and provide idempotent data as an optional input parameter that should be fine, right? The argument here is that you really want to be explicit about the fact that the data you pass to is idempotent, so you have to put it in the name. Another way to do this, though, would be to require a with the property to be passed when message data is passed to on pain of an error.
No, you can also put it in the name of the input parameter of the initiate action: 'Initiate(idempotentData)' Do we even need a messageContext here?
I think both are OK. We've visited this in the past I recall. I favour: InitateWithIdempotentData For exactly the reason you say, because it makes it very explicit. If Apple's API does it different, I would listen to why.... Gorry
now covered by InitiateWithSend(), closing.
How is idempotent data, to be send in the zero-RTT connection establishment, specified? The Use 0-RTT session establishment with an idempotent Message property "specifies whether an application would like to supply a Message to the transport protocol before Connection establishment" which suggests the idempotent message is set on the and used later when the is established, but it might be more natural to specify the idempotent data as a parameter to /?
This also relates to Section 7, which says: This seems incompatible with the / split, since the only way to get a on which to call is by calling , , or , which starts the process of establishing the connection. (Edit: and section 7.1.1.4)
paging NAME to discuss the pros and cons of the Apple and Linux approaches to TFO...
There are three top-level approaches that we’ve used/tried: Mark fast open in parameters (Preconnection) such that connect() (Initiate) becomes a fake action, and you then send after that, and the first send kicks off the real Initiate. Add your initial fast-open data to the parameters (Preconnection), such that it’s already known by the time Initiate is called. Allow marking sends as idempotent, and allow enqueuing sends after create and before start. (1) gets tricky if you want to monitor when connections actually complete, because they’re marked as Connected/Ready too early. This is a bad application contract, especially when there’s racing involved. (2) is not great since it requires an entirely new way of specifying data that isn’t Send(). Neither (1) nor (2) clearly support sending multiple chunks of idempotent data into the stack as fast open data. Thus, we chose (3). However, Preconnection has certainly messed this up a bit. My suggestion is to no longer take Preconnection as an argument to Initiate(), but instead use a NewConnection(Preconnection) -> Initiate() pattern, which you can insert Sends in between. There are many other reasons to do this as well—if I want to set my callbacks for events for this connection, etc, in languages like C that’s a lot easier if I can call create() and then set my event handlers, and then start once those are set. Note: I already removed the text in the architecture that referred to Preconnection being passed to calls like Initiate and Listen, since that was API, not architecture, and since I didn’t quite agree :)
So you create a from a , then call on the ? That would work, provided we're clear when the parameters of that connection lock in (when it's created or when it's initiated)
Yes, the parameters from the Preconnection are immutable once the Connection is created.
Okay, makes sense. I'll put a pull request together to address this (although it'll most likely be tomorrow now)
I created a branch to implement the change to create a from a , then call on the , rather than . However, I'm increasingly unconvinced this is the right approach. The approach: Gives a much clearer split between the phases Makes it clearer what parameters affect connection establishment and which affect ongoing connections (i.e., we don't have to specify the effect of setting connection properties on a connection that has been created but not established) Makes it clearer how works, in particular around resolving candidates Prevents the , , error case, since you can't call on a And, in strongly typed implementations, can prevent errors due to trying to call , , or more than once on a , forgetting to call them, or calling / on an unestablished connection – having taught introductory networking recently, these are all common bugs, and can be made compile time errors if we get the / split right. So, please review the branch, but I don't think we should merge it. Rather, I think the right way to handle idempotent data is to add an optional idempotent to and . Event handlers seem like something that should be specified on initiate, etc., too.
NAME That's not exactly how I imagined the flow being allowed—I don't think we should allow Connection to turn into a Listen or Rendezvous, etc. A listener vends Connections, for example, and isn't a Listen itself. I was thinking that with the current terminology, Preconnection could Create an outbound connection, which could be initiated later. The interaction with listen taking a preconnection would be the same as it is today. Perhaps instead we can just change the terms here, and add an equivalent of a "start"/"resume" for the objects. Thus, we would have: Preconnection.Initiate() -> Connection (not started) Connection.SetEventHandler(...) Connection.Send(Message) Connection.Start() And for listening, Preconnection.Listen() -> Listener (not started) Listener.SetNewConnectionHandler(...) Listener.Start() This is pretty much how our implementation works today.
NAME I really don't like the model of adding many optional parameters to Initiate to handle the various callbacks or extra data that might need to be sent—this list will continually grow and shrink, and most languages don't cleanly support having a single symbol/function/method support an extensible list of parameters. These items also don't belong on the preconnection, since I should be able to have the same Preconnection across many connections to make them look similar, without requiring the same part of the app to handle their events, and to have the same initial data.
NAME hm. from an API expressiveness standpoint, I'm not wild about this. Metacomment: I think we need to meet in person to converge on this. Can we stay with what we have (which gets a little twitchy with more than one Message on 0RTT), keep NAME branch, and defer to post-London?
NAME Sure, I'm fine to defer to converge more. Specifically, what's the issue from the API expressiveness issue? Is it that you'd prefer to have a single call that takes all possible parameters and initial work to schedule? We actually did start with that for our APIs in Objective-C, but moved to the delayed start model to better support languages like C, and to make it cleaner to be clear about which events you want to handle, and who handles them. It allows someone to create a Connection from a Preconnection with certain setup, and then hand it to another part of the app/framework to in turn use it and schedule the event handlers on it. If you require all of that to be fore-known (both the properties of the connection AND the runtime "who gets this callback" state), it's more limiting, since where the events get delivered is not really a fundamental property. This isn't an issue in languages that support an arbitrary number of event watchers without registering, etc, but that's not true across all languages.
I updated the branch, to sketch out how this might look. I don't think it's too bad now.
i should have said elegance instead of expressiveness; it's late. I really don't like the idea, aesthetically, that every connection has a two-phase startup , more or less just to support the (uncommon) case where you want to send more than one idempotent message per RTT. It seems to me we have two overarching design goals (at least, I do): The common case ("I don't care, just send my damn packets") should be as easy or easier to deal with in the transport services API as it is in the sockets API. Code written to the common case should benefit from advanced features in shiny new protocols (e.g. 0RTT) by default if possible. 0RTT is a special snowflake here, because it actually changes the order of operations at startup, and maybe we can't get away from leaking that complexity into the don't care case. But it'd be neat if we could find a way to make send-on-initiate work via a protocol (selection) property anyway.
NAME I opened this as so we can comment on the details of the wording over there.
discussion continues in , propose to defer after -00 and tag this post-London if no objections...
Expanding from my last thought on , here's what I think my current proposal for this is, designed to let application developers not really care all that much about 0RTT but still get benefits from it when available: Setting a protocol selection property that prefers 0RTT-capable transports changes the semantics of and : an will be delayed until either: an initial call, in order to allow idempotent send on startup; or a (fairly aggressively short) timeout, selected such that if the doesn't happen until after the timeout, then the sender is so overloaded or otherwise slow that it wouldn't realize any material speedup from cutting an RTT out of establishment. This allows 0RTT for one message. An application that wants to bundle multiple Messages into a single 0RTT flight can set a new third state on the send parameter (or a new send parameter that is mutually exclusive with , doesn't matter). When is set, a Message will be held until the following Message on send; during 0RTT this will mean (explicitly) that the Initiate should also be delayed until the end of the run. All this behavior can be overridden with an call, or with an parameter on (which would default to false). I can write this up in more detail in an alternate PR (after submitting -00) if y'all don't think it's insane.
That could work for , although I'm not sure I like the invisible change in semantics, but I don't think it works for . The approach in has the advantage of having very similar APIs for zero-RTT data for both and .
Can you walk me through why it doesn't work for ? (Are you somehow callable at the moment?)
I don't think it works for because there's no to call on until the rendezvous has completed, and by then it's too late for zero-RTT. (At home due to a snow day, but Skype me if a call is useful)
(for posterity, I'm going to go off and think about how my suggestion above might work in a non ugly way with Rendezvous, and if I get time before London, write that up in another PR targeted to land in this branch)
For in-person discussion in Montreal; we should turn the suggestions in this issue and in into a message to the list.
NAME and NAME go into the thunderdome first week of June.
this was bikeshedding is now in

Preconnection Object with a Remote Endpoint specifier, optionally a Local Endpoint specifier (if not specified, the system will attempt to determine a suitable Local Endpoint), as well as all properties  <ins> necessary for candidate selection. The Initiate() Action consumes the Preconnection.  Once Initiate() has been called, no further properties may be added to the Preconnection, and no subsequent establishment call may be made on the Preconnection. </ins> Once Initiate is called, the candidate Protocol Stack(s) may cause one or more candidate transport-layer connections to be created to connections, or the application is prohibited from opening a Connection by the operating system). <ins> See also initiate-and-send to combine Connection establishment and transmission of the first message in a single action. </ins> 6.2. Passive open is the Action of waiting for Connections from remote invalid to send a Message over a UDP connection that is larger than the available datagram sending size.  7.2. Like all Actions in this interface, the Send Action is asynchronous. 7.6. <ins> For application-layer protocols where the Connection initiator also sends the first message, the InitiateWithIdempotentSend() action combines Connection initiation with a first Message sent, provided that message is idempotent. Without a message context (as in send-basic): With a message context (as in message-props): The message passed to InitiateWithIdempotentSend() is, as suggested by the name, considered to be idempotent (see send-idempotent) regardless of declared message properties or defaults.  If protocol stacks supporting 0-RTT establishment with idempotent data are available on the Preconnection, then 0-RTT establishment may be used with the given message when establishing candidate connections.  For a non-idemponent initial message, or when the selected stack(s) do not support 0-RTT establishment, InitiateWithIdempotentSend is identical to Initiate() followed by Send(). Neither partial sends nor send batching are supported by InitiateWithIdempotentSend(). The Events that may be sent after InitiateWithIdempotentSend() are equivalent to those that would be sent by an invocation of Initate() followed immediately by an invocation of Send(), with the caveat that a send failure that occurs because the Connection could not be established will not result in a SendError separate from the InitiateError signaling the failure of Connection establishment. 7.7. </ins> Sender-side framing allows a caller to provide the interface with a function that takes a Message of an appropriate application-layer type and returns an array of octets, the on-the-wire representation


The application can set a minIncompleteLength value to indicates the smallest partial Message data size in bytes that should be delivered in response to this Receive.  By default, this value is infinite, <del> which means that only complete Messages should be delivered.  If this value is set to some smaller value, the associated receive event will be triggered only when at least that many bytes are available, or the Message is complete with fewer bytes, or the system needs to free up memory.  Applications should always check the length of the data delivered to the receive event and not assume it will be as long as minIncompleteLength in the case of shorter complete Messages or memory issues. </del>  The maxLength argument indicates the maximum size of a Message in bytes the application is currently prepared to receive.  The default Multiple invocations of ReceivedPartial deliver data for the same Message by passing the same messageContext, until the endOfMessage <del> flag is delivered.  All partial blocks of a single Message are delivered in order without gaps.  This event does not support delivering discontiguous partial Messages. </del>  If the minIncompleteLength in the Receive request was set to be infinite (indicating a request to receive only complete Messages),

1 tiny fix suggested to make the text a little clearer (because the default "complete message" delivery may need a de-framer, or the whole Connection may function as a message), and pointed at a possible error when delivering partial data (it's already written elsewhere, but the suggestion makes a sentence more correct, I believe).
Looks good, thanks!
lgtm

The application can set a minIncompleteLength value to indicates the smallest partial Message data size in bytes that should be delivered in response to this Receive.  By default, this value is infinite,  <ins> which means that only complete Messages should be delivered (see receive-partial and receive-framing for more information on how this is accomplished).  If this value is set to some smaller value, the associated receive event will be triggered only when at least that many bytes are available, or the Message is complete with fewer bytes, or the system needs to free up memory.  Applications should always check the length of the data delivered to the receive event and not assume it will be as long as minIncompleteLength in the case of shorter complete Messages or memory issues. </ins> The maxLength argument indicates the maximum size of a Message in bytes the application is currently prepared to receive.  The default Multiple invocations of ReceivedPartial deliver data for the same Message by passing the same messageContext, until the endOfMessage  <ins> flag is delivered or a ReceiveError occurs.  All partial blocks of a single Message are delivered in order without gaps.  This event does not support delivering discontiguous partial Messages. </ins> If the minIncompleteLength in the Receive request was set to be infinite (indicating a request to receive only complete Messages),


9. <del> At any point, the application can query Connection Properties.  It can also set per-connection Protocol Properties. </del>  Depending on the status of the connection, the queried Connection Properties will include different information: For Connections that are Establishing: Transport Properties that the application specified on the Preconnection, see connection- <del> props.  Selection Properties of a Connection can only be queried, not set. </del>  For Connections that are Established, Closing, or Closed (TODO: double-check if closed belongs here): Transport Properties of the Property that an application specified as Preferred may not actually be present in the chosen protocol stack because none of the currently available transport protocols had this <del> feature.  Selection Properties of a Connection can only be queried. </del>  Protocol Properties of the protocol stack in use (see protocol- <del> props below).  These can be queried and set.  Certain specific Procotol Properties may be read-only, on a protocol- and property-specific basis. </del>  For Connections that are Established, properties of the path(s) in use.  These properties can be derived from the local provisioning domain RFC7556, measurements by the Protocol Stack, or other <del> sources.  They can only be queried. </del>  10. aborted the Connection; however, there is no guarantee that an Abort will indeed be signaled. <del> A SoftError can inform the application about the receipt of an ICMP error message that does not force termination of the connection, if the underlying protocol stack supports access to soft errors; however, even if the underlying stack supports it, there is no guarantee that a soft error will be signaled. </del> 11. As this interface is designed to be independent of an

Rather than saying "this can only be queried" everywhere, I split this into setting and querying now, and I also moved SoftErrors in there. It's not precisely "querying" for a Property, but it's the closest match - better than "Connection Termination", I think (this is where SoftErrors were before).
NAME what about "Connection Maintenance: Setting Properties and Obtaining Information" ?
Maintenance sounds a bit strange to me, "Managing Connections: Setting Properties and Obtaining Information" ?
I like it (and I agree about "maintenance" being slightly strange)
Managing sounds good to me too.
Ack, done
lgtm, thanks!

9.  <ins> The application can set per-connection Protocol Properties (see protocol-props).  Certain Procotol Properties may be read-only, on a protocol- and property-specific basis.  ~~~ Connection.SetProperty(property, value) ~~~ At any point, the application can query Connection Properties.  ~~~ ConnectionProperties := Connection.GetProperties() ~~~ </ins> Depending on the status of the connection, the queried Connection Properties will include different information: For Connections that are Establishing: Transport Properties that the application specified on the Preconnection, see connection-  <ins> props. </ins> For Connections that are Established, Closing, or Closed (TODO: double-check if closed belongs here): Transport Properties of the Property that an application specified as Preferred may not actually be present in the chosen protocol stack because none of the currently available transport protocols had this  <ins> feature. </ins> Protocol Properties of the protocol stack in use (see protocol-  <ins> props). </ins> For Connections that are Established, properties of the path(s) in use.  These properties can be derived from the local provisioning domain RFC7556, measurements by the Protocol Stack, or other  <ins> sources. Slightly different from "querying", a SoftError Event can also occur, informing the application about the receipt of an ICMP error message related to the Connection.  This will only happen if the underlying protocol stack supports access to soft errors; however, even if the underlying stack supports it, there is no guarantee that a soft error will be signaled. </ins> 10. aborted the Connection; however, there is no guarantee that an Abort will indeed be signaled.  11. As this interface is designed to be independent of an


An Action creates an Object:  An Action is performed on an Object: An Object sends an Event: The Resolve() call returns a list of Preconnection Objects, that represent the concrete addresses, local and server reflexive, on which a Rendezvous() for the Preconnection will listen for incoming <del> Connections.  This list can be passed to a peer via a signalling protocol, such as SIP RFC3261 or WebRTC RFC7478, to configure the remote. </del>  6.4.

section 6.3 (Rendezvous) has the following line: PreconnectionBinding := Preconnection.Resolve() If i read the text correctly, I'd propose to change this to Preconnection<> := Preconnection.Resolve()
Agreed, not sure what a is...
Agreed; the text would need to change to compensate as well. NAME is the on an array of preconnections? I don't think we have a notation for this yet ( are event params), but we probably should.
Agree with the change. are new objects derived from the initial by a process like ICE. E.g., once the resolve call has done a STUN exchange to find the external address of a NAT you're behind, you'll get back an object that represents that external address.
lgtm

An Action creates an Object: <ins> An Action creates an array of Objects: </ins> An Action is performed on an Object: An Object sends an Event: The Resolve() call returns a list of Preconnection Objects, that represent the concrete addresses, local and server reflexive, on which a Rendezvous() for the Preconnection will listen for incoming  <ins> Connections.  These resolved Preconnections will share all other Properties with the Preconnection from which they are derived, though some Properties may be made more-specific by the resolution process. This list can be passed to a peer via a signalling protocol, such as SIP RFC3261 or WebRTC RFC7478, to configure the remote. </ins> 6.4.


12.3.19. <del> Intent [TODO: Discuss] </del>  Enumeration Preconnection, Connection, Message <del> This property specifies the application's expectation of the dominating traffic pattern for this Connection.  This implies that the transport system should optimize for the capacity profile specified.  This can influence path and protocol selection.  The following values are valid for the Capacity Profile: The application makes no request about its expected capacity profile.  No special optimizations of the tradeoff between delay, delay variation, and bandwidth efficiency should be made when selecting and configuring stacks. Response time (latency) should be optimized at the expense of bandwidth efficiency and delay variation when sending this message.  This can be used by the system to disable the coalescing of multiple small Messages into larger packets (Nagle's algorithm); to prefer immediate acknowledgment from the peer endpoint when supported by the underlying transport; to signal a preference for lower-latency, higher-loss treatment; and so on. The application expects to send/receive data at a constant rate after Connection establishment.  Delay and delay variation should be minimized at the expense of bandwidth efficiency.  This implies that the Connection may fail if the desired rate cannot be maintained across the Path.  A transport may interpret this capacity profile as preferring a circuit breaker RFC8084 to a rate-adaptive congestion controller. The application is not interactive.  It expects to send/receive a large amount of data, without any urgency.  This can, for example, be used to select protocol stacks with scavenger transmission control, to signal a preference for less-than-best-effort treatment, or to assign the traffic to a lower-effort service. </del>  12.3.20.

This is a (work in progress) attempt to clean up Capacity Profile as a selection property along the lines of the state of the discussion in . NAME you had a suggestion to add "High Throughput Data" as a separate profile mapping to AF2n, but I don't know anything else about what this would actually imply for transport selection; can you help me out here?
+1 to this
NAME - "Please give me the maximum available bandwidth, possibly at the cost of latency" is not how this is defined. See RFC 4594, section 4.8: "The High-Throughput Data service class is RECOMMENDED for elastic applications that require timely packet forwarding of variable rate traffic sources and, more specifically, is configured to provide good throughput for TCP longer-lived flows." I don't interpret "require timely packet forwarding" as "possibly at the cost of latency". I'm also in favor of keeping it - because it has a defined meaning and implementation method as described in at least one RFC, and is hence perhaps even already used by some ISPs to treat traffic in a certain way.
NAME Yes, but for a high-throughput flow, "timely packet forwarding" will more likely be achieved on a high-bandwidth path with higher latency than on a low-latency path with lower bandwidth, that's what I mean. In RFC 4594, Figure 2, it says that High-Throughput data has a "Medium-High tolerance to Delay", I think that means the same - for this traffic, you don't optimize for low latency on the path, but for high bandwidth.
just committed text for High Throughput, making it clear that the only thing the transport system can do with this is stick a DSCP mark on it. Is this ready to go?
NAME is this good to merge?
Seemed you reached the correct conclusion scavenger =/= bulk.
I like the way this is phrased now. I'm in favor of keeping High Throughput: The Transport System could use this for Path Selection imo, if it has an idea of the available bandwidth over the paths. I think it makes sense that the application is able to say "Please give me the maximum available bandwith, possibly at the cost of latency" here.Please also change Scavenger/Bulk to Scavenger as per Anna's request. I'd do it myself but I don't want to create a conflict

12.3.19.  <ins> Selection Property </ins> Enumeration Preconnection, Connection, Message  <ins> This property specifies the desired network treatment for traffic sent by the application and the tradeoffs the application is prepared to make in path and protocol selection to receive that desired treatment.  When the capacity profile is set to a value other than Default, the transport system should select paths and profiles to optimize for the capacity profile specified.  The following values are valid for the Capacity Profile: Default: : The application makes no representation about its expected capacity profile.  No special optimizations of the tradeoff between delay, delay variation, and bandwidth efficiency should be made when selecting and configuring transport protocol stacks.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD assign the DSCP Default Forwarding RFC2474 PHB; when the Connection is multiplexed, the guidelines in section 6 of RFC7657 apply. Scavenger: : The application is not interactive.  It expects to send and/or receive data without any urgency.  This can, for example, be used to select protocol stacks with scavenger transmission control and/or to assign the traffic to a lower-effort service.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD assign the DSCP Less than Best Effort LE-PHB PHB; when the Connection is multiplexed, the guidelines in section 6 of RFC7657 apply. Low Latency/Interactive: : The application is interactive, and prefers loss to latency.  Response time should be optimized at the expense of bandwidth efficiency and delay variation when sending on this connection.  This can be used by the system to disable the coalescing of multiple small Messages into larger packets (Nagle's algorithm); to prefer immediate acknowledgment from the peer endpoint when supported by the underlying transport; and so on.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD assign the DSCP Expedited Forwarding RFC3246 PHB; when the Connection is multiplexed, the guidelines in section 6 of RFC7657 apply. Low Latency/Non-Interactive: : The application prefers loss to latency but is not interactive.  Response time should be optimized at the expense of bandwidth efficiency and delay variation when sending on this connection.Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD assign a DSCP Assured Forwarding (AF21,AF22,AF23,AF24) RFC2597 PHB; when the Connection is multiplexed, the guidelines in section 6 of RFC7657 apply. Constant-Rate Streaming: : The application expects to send/receive data at a constant rate after Connection establishment.  Delay and delay variation should be minimized at the expense of bandwidth efficiency.  This implies that the Connection may fail if the desired rate cannot be maintained across the Path.  A transport may interpret this capacity profile as preferring a circuit breaker RFC8084 to a rate-adaptive congestion controller.  Transport system implementations that map the requested capacity profile onto per- connection DSCP signaling without multiplexing SHOULD assign a DSCP Assured Forwarding (AF31,AF32,AF33,AF34) RFC2597 PHB; when the Connection is multiplexed, the guidelines in section 6 of RFC7657 apply. High Throughput Data: : The application expects to send/receive data at the maximum rate allowed by its congestion controller over a relatively long period of time.  Transport system implementations that map the requested capacity profile onto per-connection DSCP signaling without multiplexing SHOULD assign a DSCP Assured Forwarding (AF11,AF12,AF13,AF14) RFC2597 PHB per section 4.8 of RFC4594.  When the Connection is multiplexed, the guidelines in section 6 of RFC7657 apply. </ins> 12.3.20.


[TODO: turn this list into actual properties or move up into the explaining text] <del> The status of the connection, which can be one of the following: </del>  Establishing, Established, Closing, or Closed. Whether the connection can be used to send data.  A connection can the application specified on the Preconnection, see connection- props. <del> For Connections that are Established, Closing, or Closed (TODO: double-check if closed belongs here): Transport Properties of the actual protocols that were selected and instantiated.  These features correspond to the properties given in transport-props and include Selection Properties and Protocol Properties. </del>  Selection Properties indicate whether or not the Connection has or offers a certain Selection Property.  Note that the actually As this interface is designed to be independent of an implementation's concurrency model, the details of how exactly actions are handled, and on which threads/callbacks events are <del> dispatched, are implementation dependent.  However, the interface does provide the following guarantees about the ordering of operations: </del>  <del> Received<> will never occur on a Connection before a Ready<> event on that Connection, or a ConnectionReceived<> or RendezvousDone<> containing that Connection. </del>  <del> No events will occur on a Connection after a Closed<> event, an InitiateError<> or ConnectionError<> on that connection.  To ensure this ordering, Closed<> will not occur on a Connection while other events on the Connection are still locally outstanding (i.e., known to the interface and waiting to be dealt with by the application).  ConnectionError<> may occur after Closed<>, but the interface must gracefully handle the application ignoring these errors. </del>  Sent<> events will occur on a Connection in the order in which the Messages were sent (i.e., delivered to the kernel or to the network interface, depending on implementation).  12. Having discussed Selection Properties in selection-props, Connection

Connections have states, and the document specifies a contract for ordering operations. It seems like the latter of these should probably refer to the former.
Reading through the API, I noted that there is, amongst the various connection states, a state that is defined but never mentioned again in any of the drafts: "Closing". I am unclear how this state is useful to the application, and how an implementation would determine it. If the connection is closed from the remote side, like a RST, then we'll move right to Closed. If the connection is closed from the local side, I guess we can move into the Closing state before getting an ack that the remote side knows that we've closed, but at that point, I'm not going to be able send or receive any more on the connection anyhow. Do we have a good use case for Closing? If so, can we get more text? If not, can we remove it? For reference, we don't have any corresponding state in our implementation, and it's never come up as a need.
I also don't see a good use case and would rather see it removed. I guess the only case worth even thinking about is the local close one. Here, our "close" semantics don't promise any use for any Closing state anyway, and since (as you say) you can't send or receive anymore on a closing connection, the only point is resource optimization - to avoid opening a new connection when we can wait until the previous one is closed, stuff like that. Even that idea creates a protocol dependency in the application (because some protocols will have delay on closing, but some will just terminate), which IMO makes the state a bad idea.
Right, if you wanted to know you were locally 'closing', you could just know that you called and haven't gotten the state yet.
So for me, the state is between someone called and the sending of the event – might be interesting for concurrent threads to not fall into race issues, but besides this not very useful. Side question: can I clean up all my connection related state in the event handler and is the event guaranteed to fire eventually?
This seems implementation-internal, which indicates to me we can probably get rid of the state. I think that's a question for / which I haven't got around to thinking about yet.
NAME yes, that seems to be the only meaning of the "closing" state, but it doesn't really even provide any threading benefit. If you want to make sure that the connection has closed, just wait for the state—anything after you invoke Close() before the transition to is effectively . With regards to the question about the final event, I think there MUST be some final event that means that everything is cleaned up from the application's perspective so the memory can be released, etc. In the current model, seems to be that correct event. In our implementation, we call this , which only happens after the client of the API calls the / function, to distinguish from a remote-initiated close. So, to be most safe, you may want a state for invalidated/cancelled/never-delivering-any-events-again.
(catching-up on old discussions) I'm confused: we surely need a name for the time between calling and getting the event. What would this be, other than the state? I don't understand how we can remove this.
I agree with NAME Depending on the case, it may not map to much going on in code, you may just get an immediate "Closed" instead after calling Close() - but not always. So then, what's the reason to not give this time period a name, and what else is it but a state?
Maybe we should add a hint to this state, that it is not particularly useful from a programmers perspective, but necessary in the presence of concurrency.
In , I've added a section on ordering of API events (please have a look at it). It talks about guarantees the API makes about which events must occur in which order, but it doesn't talk about guarantees the application must make in its ordering of operations. If we added this, we could talk about the state machine there. It does seem to me that if we add such a discussion, that after is not allowed. This suggests that is a state.
I don't see a need to have a state be set to enforce that must not be called after . If everything is an asynchronous operation, and states are signaled asynchronously, etc, then there still may be a race if you call then immediately; you most likely will not have received the state transition to yet, but it's still an invalid sequence of calls, and the send should fail. NAME while I agree that closing is a state that may exist internally for some protocols, I don't see any value to delivering a signal to an application that they are in the closing state. That's an extra event delivery that necessitates to action and provides no new information. It also is a source of confusion if some protocols use a closing state, but others don't (because UDP should just go to immediately).
I agree with NAME that conceptually you always pass through the Closing state after calling Close() and before receiving the Closed event, even if this transition may sometimes be practically immediate as when you use UDP. Even if you use UDP the TAPS system may need to perform some internal cleanup before triggering the Closed event though. I agree with NAME that you do not need to signal that you are in the Closing state to the application, but I also do not see that this is suggested anywhere. If the application asks what state it is in, then Closing is a possible return value, but that does not create an extra event delivery? So I do not see that the Closing state causes any problem or extra overhead.
I think that state changes do need to be events signaled to the application. Knowing that a connection is ready should not be polled—it is an event. Similar for closed, etc. there should not be a special case for closing, but closing should not be a fired event at all!
Why not have a special case? True, I struggle to imagine other state changes that an application doesn't care about, but so be it - "closing" is no less of a state just because it's not important enough to apps to be signalled to them. (side note: I tag this message as "I offer my opinion but I also don't care much about this")
Why would we not signal to the application? We need some signal that the stack is acting on the request, since this implies that events pertaining to an active connection will no longer be delivered and the application can free up those resources. Especially since, for some protocols, the event could come some significant time after .
Splitting this into and could simplify tge However, I'm a little confused about this: happens after is called, but before happens. ISTM you can still receive things after (i.e., there can still be events fired from pending receive calls), but is it legal to call and ?
NAME If I some data then call , when do I enter the closing state? The event could mean "I got your close request, and will send any outstanding data then close the connection" or it could mean "I've send all outstanding data and am in the process of closing the connection". If the former, I'd expect to get Message Sent events after , if the latter I'd not expect such events.
I thought you enter the Closing state immediately after calling Close() and you can no longer call Send() or Receive() after this point and a Closing<> event would have the former meaning ("I got your close request, and will send any outstanding data then close the connection"). That's why I did not see the need to signal this state transition to the application. I agree we need to signal the application when all outstanding data has been sent and resources can be freed. I thought this would be captured by the Closed<> event. But if the Closed<> event relies on the other end it may be good to separate them. In relation to the need for signaling all state transitions, do we not have a similar situation for Establishing. The Establishing state is I assume entered when an application calls Initiate(), Listen() or Rendezvous(). We do not signal this state transition to the application as far as I understand. We signal when the connection is ready and we move to the Established state, correct?
Linking this to operation ordering as in might make the whole concept of application-exposed connection state make more sense.
having submitted , it's pretty clear that the Established and Closing states are identical from the application's standpoint with respect to which events are received. It may be that Closing is simply equivalent to Established s.t. both "can send" and "can receive" are False. That raises the quesiton of whether or not we should split Established into four states: Bidirectional Send-Only Receive-Only Closing
I don't see the point of this state split proposal - but if you like this better I don't mind it either.
NAME I agree with the overall assessment that Closing is the same as Established, with the distinction that I/O doesn't work. There are other cases in which I/O may not work temporarily in the middle of a connection too, and I wonder if that should be related. For example, if you have a migratable protocol like MPTCP or QUIC, and you lose all interfaces, you can't actually send or receive for a time (although sends and receive calls may be buffered), and you may end up timing out and closing, or you may end up re-establishing over another link.
ahhhh yep. "no I/O available" can actually happen for a range of stack/transport system implementation specific reasons, not just because it's closing, so I do think we need a state but it needs a different name. It also makes sense to keep "can send" and "can receive" separate read-only booleans, because they happen for different reasons which are not necessarily state-related. We have four states then: Establishing: connection is not yet up and running; local allocation is complete and resolution/racing is in progress, or a listener is up and waiting for connections. Established: connection is up and running and I/O is possible in at least one direction; there exists at least one transport stack attached to the connection Dormant: the transport system is maintains state for a connection, but there exist no active transport stacks attached to the connection, and the transport system is not currently attempting to re-establish the connection. Modeling this probably goes hand in hand with the text for . Closed: the transport system has shut down the connection; it is dead, and eligible for GC etc.
and are different though: the connection doesn't come back from , but it does from . Also: is it or do we just transition back to in this case? Is it just for migration?
Dormant might also happen in an environment where the transport system knows it can resume. Migration, yes, but maybe migration during sleep (i.e., you closed your laptop on a download then landed in another country and opened it again -- if the endpoints know about each other they can still resume). As to whether Dormant isn't just Establishing, it depends on whether we need the application to distinguish between "the transport system has at least one candidate it is racing, sends will be buffered and receives will queue" and "the transport system has no candidates, but attempts to send or receive may result in a transition to Establishing". Rereading what I wrote above, I think I got the equivalence between Dormant and Closing wrong -- they're actually totally separate. But I still don't think I see the utility of Closing on its own from a properties management standpoint-- you're Established, you Close (or the remote peer does), then you're Closed (and subject to GC if you're that sort of language :) ).
Whether or not has utility, it exists as a state: for many transports there is a time period between starting to tear down the connection, and having completed tearing down the connection. That's the state, whether or not its useful. I don't see that pretending it doesn't exist is a good idea.
ok i buy that. So we have five: establishing, established, dormant (which might just be establishing), closing and closed (which act the same from the interface's standpoint, but are semantically different...)
It seems this discussion has concluded, without a need to add new text (because we have establishing, established, closing and closed already covered).
This LGTM

[TODO: turn this list into actual properties or move up into the explaining text]  <ins> The connection state, which can be one of the following: </ins> Establishing, Established, Closing, or Closed. Whether the connection can be used to send data.  A connection can the application specified on the Preconnection, see connection- props.  <ins> For Connections that are Established, Closing, or Closed: Transport Properties of the actual protocols that were selected and instantiated.  These features correspond to the properties given in transport-props and include Selection Properties and Protocol Properties. </ins> Selection Properties indicate whether or not the Connection has or offers a certain Selection Property.  Note that the actually As this interface is designed to be independent of an implementation's concurrency model, the details of how exactly actions are handled, and on which threads/callbacks events are  <ins> dispatched, are implementation dependent. Each transition of connection state is associated with one of more events: Ready<> occurs when a Connection created with Initiate() or InitiateWithIdempotentData() transitions to Established state. ConnectionReceived<> occurs when a Connection created with Listen() transitions to Established state. RendezvousDone<> occurs when a Connection created with Rendezvous() transitions to Established state. </ins>  <ins> Closed<> occurs when a Connection transitions to Closed state without error. </ins>  <ins> InitiateError<> occurs when a Connection created with Initiate() transitions from Establishing state to Closed state due to an error. ConnectionError<> occurs when a Connection transitions to Closed state due to an error in all other circumstances. The interface provides the following guarantees about the ordering of operations: </ins> Sent<> events will occur on a Connection in the order in which the Messages were sent (i.e., delivered to the kernel or to the network interface, depending on implementation). <ins> Received<> will never occur on a Connection before it is Established; i.e.  before a Ready<> event on that Connection, or a ConnectionReceived<> or RendezvousDone<> containing that Connection. No events will occur on a Connection after it is Closed; i.e., after a Closed<> event, an InitiateError<> or ConnectionError<> on that connection.  To ensure this ordering, Closed<> will not occur on a Connection while other events on the Connection are still locally outstanding (i.e., known to the interface and waiting to be dealt with by the application).  ConnectionError<> may occur after Closed<>, but the interface must gracefully handle all cases where application ignores these errors. </ins> 12. Having discussed Selection Properties in selection-props, Connection


[TODO: This is inconsistent with {prop-cap-profile}} - needs to be fixed]  7.4. It is not always possible for an application to send all data

Alternate fix for fragmentation/segmentation prohibited in a single message property, per discussion. Merge only one of these. :)
Discussion on identified that we probably want a Message (and/or Connection) property that forces DF on for IPv4 (and, I guess, ensures that datagram sends on IPv6 below MTU will fail, by disabling sender fragmentation, as well).
This should also prevent IPv4 sender fragmentation. It's possible to generate IPv4 fragments from the stack with the DF set. I think the property should default to do this, and I'd suggest "ALLOW Network Fragementation" as a suggested name.
This has to be a Message Property, however, we expect programmers to typically set this property on a Connection or Preconnection (as default for all messages).
NAME so you say "network fragmentation". This excludes fragmentation in the host, which we may also want to control if we want to send unfragmented PMTU-sized packets. On the other hand, if this includes fragmentation in the host, your suggested default will always fail for TCP when I send messages > PMTU, yet in case of a TCP-like service I really don't want to care about this. => My suggestion would be to just to drop the word "network" and (alas) NOT make forbid-fragmentation the default. NAME it has to be a Message Property, yes. But what makes you say that you'd expect programmers to typically set this on a Connection or a Preconnection?
NAME the only use case to set this on a per-message basis I can image is PMTU discovery and I hope this is usually done by the transport system implementaion
NAME I agree with you but don't see the relationship with setting that on a Connection or Preconnection.
Agreed to the general point that this is a useful property! To the discussion of where it goes, I agree with NAME We had discussed letting defaults for Messages be set connection-wide for convenience, so I think the point is that this would commonly be set as a convenient default, rather than having the application set it over and over.
After reading again, I get the feeling we actually need two properties: Allow Fragmentation for the Messages passed to the transport service. Allow IP level Fragmentation for the Packets generated by the transport protocol. Now it becomes ugly: For UDP, both are equivalent. For TCP/QUIC, the former set to false means that sending a message should fail if the exceeds the PMTU after passing the framer. This also implies that a message should be sent within a single packet. For TCP/QUIC, the latter set to false means that the transport system should set the DF flag for v4 and disable local fragmentation for v6, but the transport protocol can still split a message into multiple packets. While the former definitely works as discussed so far, I have no idea what should happen if the latter is set to false on a message but the former is set to true…
Not sure about QUIC, but what you say here about TCP isn't right: none of this should matter to TCP, you can't control DF with it and the PMTU shouldn't be visible to the application.
More on philsbln's note above: I'm putting TCP aside, and assuming he's right about QUIC: 1) The former set to false, but the latter to true: "a message should be sent within a single packet" - but then we want to be able to allow network-layer fragmentation? That seems strange to me. 2) The latter set to false, but the former to true: so here we want the DF flag, but it's still ok to split messages across multiple packets. Again, that seems strange to me. => I struggle to see a use case for any of these combinations. My proposal would be to have only one boolean property that controls both IP- and transport-level fragmentation (allowing or forbidding both).
while I was writing , I also came to think that this should be a single message context property whose implementation was context-dependent made more sense. I'm happy to rework along these lines.
I may have confused people - but I too think it is OK to express this as one property. I think there are subtle differences, but such details are not important to TAPS. (Any app needing finer control will likely need to control more than this).
this alternative is in
Very nice! Thanks a lot for doing this!

[TODO: This is inconsistent with {prop-cap-profile}} - needs to be fixed] <ins> 7.3.9. Boolean This property specifies that a message should be sent and received as a single packet without transport-layer segmentation or network-layer fragmentation.  Attempts to send a message with this property set with a size greater to the transport's current estimate of its maximum transmission segment size will result in a "SendError".  When used with transports supporting this functionality and running over IP version 4, the Don't Fragment bit will be set. </ins> 7.4. It is not always possible for an application to send all data


Transport Services API I-D.ietf-taps-interface and Implementation I- D.ietf-taps-impl documents.  2. The Transport Services architecture is based on the survey of

Address issue Add more intro section, with friendly diagrams Highlight why we use events, messages, and racing
To me the document is in very good shape, but there are several things that just appear, and a few short paras and maybe a simple picture could help avoid people giving-up before they know the document is useful to them. We seem to not introduce it nicely, I'd suggest: This isn't scarey (draw a simple familiar figure to start our story, that the reader immediately recognises) Sockets v Streams ... we speak of messages, let's explain early that they ae all you need for any app. Events ... we know these are "nromal" - but let's say we provide an eevent-driven interface. it would even be worth saying why thsi is much better (Tommy - Apple have been doing this from the start, can you perhaps say why it is beautiful?) Comments follow: Section: Background. The last part of this section suddenly introduces messages. I know it is just an example, but readers may be already looking here at whether this is an interface to TCP or UDP or something else ... and I think we do not really tackle that earlier enough. One way to help that would be to draw a diagram that looks something like this, in this section: +-----------------------------------------------------+ +------------------------+----------------------------+ | +------------------------|----------------------------+ +------------------------|----------------------------+ | +------------------------|----------------------------+ +------------------------|----------------------------+ +------------------------|----------------------------+ +-----------------------------------------------------+ And the say that the Transport Services API provides message-oriented communication using transport protocols (e.g. UDP, TCP, etc). I'd also mention the words /data transfer/ somewhere. Providing a simpler earlier diagram may also be slightly easier on the new reader. In the same way, the intro should explain a little of the idea of /events/ - common enough to some, but may be unexpected for a new reader.
I suspect you're already writing this sort of approach: I think it would be super nice if the background/intro text clearly had a para expalining that this is a call-back based API. I do suspect that many people coming to this will be atrracted because the new API is simple and provides a more modern interface. The rationale for why we design it to enable multiple transports, etc, is the "detail inside the engine" the fact it delivers what they want is the first point to make:-).
NAME does close this issue?
NAME Yes, I think so.
lgtm modulo NAME commentsLGTM!

Transport Services API I-D.ietf-taps-interface and Implementation I- D.ietf-taps-impl documents. <ins> 1.1. The model for using sockets for networking can be represented as follows: applications create connections and transfer data using the socket API, which provides the interface to the implementations of UDP and TCP (typically implemented in the system's kernel), which in turn send data over the available network layer interfaces. The Transport Services architecture maintains this general model of interaction, but aims to both modernize the API surface exposed for transport protocols and enrich the capabilities of the transport system implementation. The Transport Services API I-D.ietf-taps-interface defines the mechanism for an application to create and monitor network connections, and transfer data.  The Implementation I-D.ietf-taps- impl is responsible for mapping the API into the various available transport protocols and managing the available network interfaces and paths. There are a few key departures that Transport Services makes from the sockets API: it presents an asynchronous, event-driven API; it uses messages for respresenting data transfer to applications; and it assumes an implementation that can use multiple IP addresses, multiple protocols, multiple paths, and provide multiple application streams. 1.2. Originally, sockets presented a blocking interface for establishing connections and transferring data.  However, most modern applications interact with the network asynchronously.  When sockets are presented as an asynchronous interface, they generally use a try-and-fail model.  If the application wants to read, but data has not yet been received from the peer, the call to read will fail.  The application then waits for a notification that it should try again. All interaction with a Transport Services system is expected to be asynchronous, and use an event-driven model unlike sockets events. For example, if the application wants to read, its call to read will not fail, but will deliver an event containing the received data once it is available. The Transport Services API also delivers events regarding the lifetime of a connection and changes to available network links, which were not previously made explicit in sockets. Using asynchronous events allows for a much simpler interaction model when establishing connections and transferring data.  Events in time more closely reflect the nature of interactions over networks, as opposed to how sockets represent network resources as file system objects that may be temporarily unavailable. 1.3. Sockets provide a message interface for datagram protocols like UDP, but provide an unstructured stream abstraction for TCP.  While TCP does indeed provide the ability to send and receive data as streams, most applications need to interpret structure within these streams. HTTP/1.1 uses character delimiters to segment messages over a stream; TLS record headers carry a version, content type, and length; and HTTP/2 uses frames to segment its headers and bodies. In order to more closely match the way applications use the network, the Transport Services API respresents data as messages.  Messages seamlessly work with transport protocols that support datagrams or records, but can also be used over a stream by defining the application-layer framing being used framing. 1.4. Sockets, for protocols like TCP, are generally limited to connecting to a single address over a single interface.  They also present a single stream to the application.  Software layers built upon sockets often propagate this limitation of a single-address single-stream model.  The Transport Services architecture is designed to handle multiple candidate endpoints, protocols, and paths; and support multipath and multistreaming protocols. Transport Services implementations are meant to be flexible at connection establishment time, considering many different options and trying to select the most optimal combinations (gathering and racing).  This requires applications to provide higher-level endpoints than IP addresses, such as hostnames and URLs, which are used by a Transport Services implementation for resolution, path selection, and racing. Flexibility after connection establishment is also important. Transport protocols that can migrate between multiple network layer interfaces need to be able to process and react to interface changes. Protocols that support multiple application-layer streams need to support initiating and receiving new streams using existing connections. </ins> 2. The Transport Services architecture is based on the survey of


1. <del> Many APIs to perform transport networking have been deployed, perhaps the most widely known and imitated being the BSD socket() POSIX interface.  The names and functions between these APIs are not consistent, and vary depending on the protocol being used.  For example, sending and receiving on a stream of data is conceptually the same between operating on an unencrypted Transmission Control Protocol (TCP) stream and operating on an encrypted Transport Layer Security (TLS) I-D.ietf-tls-tls13 stream over TCP, but applications cannot use the same socket send() and recv() calls on top of both kinds of connections.  Similarly, terminology for the implementation of protocols offering transport services vary based on the context of the protocols themselves.  This variety can lead to confusion when trying to understand the similarities and differences between protocols, and how applications can use them effectively. </del>  The goal of the Transport Services architecture is to provide a common, flexible, and reusable interface for transport protocols.  As multiple sets of protocols and protocol features, it can use these protocols across multiple paths that may have different performance and functional characteristics, and it can communicate with different <del> Remote Endpoints to optimize performance.  Beyond these, if the API for the system remains the same over time, new protocols and features may be added to the system's implementation without requiring changes in applications for adoption. </del>  The following considerations were used in the design of this architecture. 3.2. <del> Since applications will often need to control fine-grained details of transport protocols to optimize their behavior and ensure compatibility with remote peers, a Transport Services system also needs to allow more specialized protocol features to be used.  The interface for these specialized options should be exposed differently from the common options to ensure flexibility. </del>  <del> A specialized feature may be required by an application only when </del>  using a specific protocol, and not when using others.  For example, <del> if an application is using UDP, it may require control over the </del>  checksum or fragmentation behavior for UDP; if it used a protocol to frame its data over a byte stream like TCP, it would not need these options.  In such cases, the API should expose the features in such a way that they take effect when a particular protocol is selected, but <del> do not imply that only that protocol may be used if there are </del>  equivalent options. Other specialized features, however, may be strictly required by an possible that some very constrained devices might not have, for example, a full TCP implementation. <del> In order to preserve flexibility and compatibility with future protocols, top-level features in the Transport Services API should avoid referencing particular transport protocols.  Mappings of these API features in the Implementation document, on the other hand, must explain the ramifications of each feature on existing protocols.  It is expected that the Implementation document will be updated and supplemented as new protocols and protocol features are developed. </del>  It is important to note that neither the Transport Services API nor the Implementation document defines new protocols that require any <del> changes on remote hosts.  The Transport Services system must be </del>  deployable on one side only, as a way to allow an application to make better use of available capabilities on a system and protocol features that may be supported by peers across the network.

From /Introudction/ Please define/API/ in /Many APIs to perform/. First Para mentions TCP, but not UDP - some people could draw conclusions (very silly perhaps) that this is about TCP. It would be super nice to include an e.g. with Datagrams mentioning QUIC and UDP. ... From /Design principles/ Curent text: /and it can communicate with different Remote Endpoints to optimize performance./ My new proposed text: /and it can communicate with different Remote Endpoints to optimize performance, robustness to failure, or some other metric/. From /Access to Specialized Features/ This says: /Since applications will often need to control fine-grained details of transport protocols to optimize their behavior and ensure compatibility with remote peers, a Transport Services system also needs to allow more specialized protocol features to be used./ I quibble. I don’t believe this is /often/, and anyway I do not think this is the most prevalent case, just and important usage, so I would suggest we write something: /There are applications that will need to control fine-grained details of transport protocols to optimize their behavior and ensure compatibility with remote peers,. A Transport Services system will therefore also needsto allow more specialized protocol features to be used./ From /Access to Specialized Features/ This says: /A specialized feature may be required by an application only when using a specific protocol, / I suggest: /A specialized feature could be required by an application only when it uses a specific transport protocol, / and change: /For example, if an application is using UDP, it may require/ to: /For example, if an application is using UDP, it could require/ Please remove /In order to preserve flexibility and compatibility .../ Suggest: /To preserve flexibility and compatibility .../ Please change: /Mappings of these API features in the Implementation document, on the other hand, must explain the ramifications of each feature on existing protocols. It is expected that the Implementation document will be updated and supplemented as new protocols and protocol features are developed./ to this: /The mappings of these API features is explained in the TAPS Implementation [REF], which also explain the implications of the feature provided by existing protocols. It is expected that this documentation will be updated and supplemented as new protocols and protocol features are developed./ Please change: /require any changes on remote hosts./ suggest: /require any changes to a remote host./ Please change: /Transport protocols such as SCTP provide/ add commas: /Transport protocols, such as SCTP, provide/
lgtm with one nit

1.  <ins> Many application programming interfaces (APIs) to perform transport networking have been deployed, perhaps the most widely known and imitated being the BSD socket() POSIX interface.  The names and functions between these APIs are not consistent, and vary depending on the protocol being used.  For example, sending and receiving on a stream of data is conceptually the same between operating on an unencrypted Transmission Control Protocol (TCP) stream and operating on an encrypted Transport Layer Security (TLS) I-D.ietf-tls-tls13 stream over TCP, but applications cannot use the same socket send() and recv() calls on top of both kinds of connections.  Similarly, terminology for the implementation of protocols offering transport services vary based on the context of the protocols themselves.  This variety can lead to confusion when trying to understand the similarities and differences between protocols, and how applications can use them effectively. </ins> The goal of the Transport Services architecture is to provide a common, flexible, and reusable interface for transport protocols.  As multiple sets of protocols and protocol features, it can use these protocols across multiple paths that may have different performance and functional characteristics, and it can communicate with different  <ins> Remote Endpoints to optimize performance, robustness to failure, or some other metric.  Beyond these, if the API for the system remains the same over time, new protocols and features may be added to the system's implementation without requiring changes in applications for adoption. </ins> The following considerations were used in the design of this architecture. 3.2.  <ins> There are applications that will need to control fine-grained details of transport protocols to optimize their behavior and ensure compatibility with remote peers,. A Transport Services system will therefore also needs to allow more specialized protocol features to be used.  The interface for these specialized options should be exposed differently from the common options to ensure flexibility. </ins>  <ins> A specialized feature could be required by an application only when </ins> using a specific protocol, and not when using others.  For example,  <ins> if an application is using UDP, it could require control over the </ins> checksum or fragmentation behavior for UDP; if it used a protocol to frame its data over a byte stream like TCP, it would not need these options.  In such cases, the API should expose the features in such a way that they take effect when a particular protocol is selected, but  <ins> do not imply that only that protocol could be used if there are </ins> equivalent options. Other specialized features, however, may be strictly required by an possible that some very constrained devices might not have, for example, a full TCP implementation.  <ins> To preserve flexibility and compatibility with future protocols, top- level features in the Transport Services API should avoid referencing particular transport protocols.  The mappings of these API features to specific implementations of each feature is explained in the TAPS- IMPL, which also explain the implications of the feature provided by existing protocols.  It is expected that this document will be updated and supplemented as new protocols and protocol features are developed. </ins> It is important to note that neither the Transport Services API nor the Implementation document defines new protocols that require any  <ins> changes to a remote host.  The Transport Services system must be </ins> deployable on one side only, as a way to allow an application to make better use of available capabilities on a system and protocol features that may be supported by peers across the network.


API in a given language on a given platform is largely dependent on the features of the language and the platform.  Actions could be implemented as functions or method calls, for instance, and Events <del> could be implemented via callback passing or other asynchronous calling conventions.  The method for registering callbacks and handlers is left as an implementation detail, with the caveat that the interface for receiving Messages must require the application to invoke the Connection.Receive() Action once per Message to be received (see receiving). </del>  This specification treats Events and errors similarly.  Errors, just as any other Events, may occur asynchronously in network written to a single API to make use of transport protocols in terms of the features they provide; <del> Explicit support for security properties as first-order transport features, and for long-term caching of cryptographic identities and parameters for associations among endpoints; </del>  Asynchronous Connection establishment, transmission, and <del> reception, allowing most application interactions with the transport layer to be Event-driven, in line with developments in modern platforms and programming languages; </del>  Explicit support for multistreaming and multipath transport protocols, and the grouping of related Connections into Connection Groups through cloning of Connections, to allow applications to take full advantage of new transport protocols supporting these <del> features; and Atomic transmission of data, using application-assisted framing and deframing where the underlying transport does not provide these. </del>  4. The Transport Services Interface is the basic common abstract application programming interface to the Transport Services <del> Architecture defined in I-D.ietf-taps-arch.  An application primarily interacts with this interface through two Objects, Preconnections and Connections.  A Preconnection represents a set of properties and constraints on the selection and configuration of paths and protocols to establish a Connection with a remote endpoint.  A Connection represents a transport Protocol Stack on which data can be sent to and/or received from a remote endpoint (i.e., depending on the kind of transport, connections can be bi-directional or unidirectional). Connections can be created from Preconnections in three ways: by initiating the Preconnection (i.e., actively opening, as in a client), through listening on the Preconnection (i.e., passively opening, as in a server), or rendezvousing on the Preconnection (i.e. peer to peer establishment). </del>  Once a Connection is established, data can be sent on it in the form of Messages.  The interface supports the preservation of message Prefer, Require) for the level of preference of a given property during protocol selection; see selection-props.  5. The pre-establishment phase allows applications to specify properties

Now that the document is targeted for the standards track, its purpose (i.e. what exactly it standardizes, and what it does not) should be made clear. This is made a little more difficult because, as an abstract API we intend to be implementable in any reasonable language (i.e. no brainfuck kthxbye), Here's what I think the scope is: message-orientation asynchronicity of event and reception asynchronicity of resolution and initiation the "shape" of the API: objects and events, and interactions between them, with a RECOMMENDATION that you use the same names ("unless this facility is implemented in terms of a networking interface that already has different names or naming conventions for equivalent objects" or something like that). an MTI set of selection, connection, and message properties, and definitions (though not names) of these properties. a larger fully defined set of selection, connection, and message properties, such that if a system implements a given property it must implement it as described.
Why would we not want to make a recommendation on the names of properties? We'll need to name them in the drafts anyway, if we're to cross-reference them from elsewhere, and at that point there's no benefit in not recommending re-use of those names in concrete API implementations.
I think providing consistent naming across different implementations will be a key argument for the adoption of the abstract interface. The strongest argument for sticking to BSD Sockets is that they somehow look the same on all platforms. We should not miss the change to at least give recommendations about how to name the individual objects and properties. Implementations can and should extend and divert from it where it makes sense, but recommending names should help the individual APIs look at least similar.
There are different kinds of things we can recommend names for. objects actions events properties We have pretty good recommendations for the first three (though different languages have e.g. different capitalization conventions). Names of properties are descriptive, though, and would make bad code names, so if we want to do the fourth we have some work to do. What I would propose to do for Bangkok: assume we recommend names for the first three, open an issue for recommended property names (which is necessary for NAME registry project anyway).
Looks good! Just one comment in re-reading a section more carefully, not blocking.I especially like the "Scope of the Interface Definition" text! Really nice and useful to have IMO

API in a given language on a given platform is largely dependent on the features of the language and the platform.  Actions could be implemented as functions or method calls, for instance, and Events  <ins> could be implemented via callbacks, communicating sequential processes, or other asynchronous calling conventions.  The method for dispatching and handling Events is left as an implementation detail, with the caveat that the interface for receiving Messages must require the application to invoke the Connection.Receive() Action once per Message to be received (see receiving). </ins> This specification treats Events and errors similarly.  Errors, just as any other Events, may occur asynchronously in network written to a single API to make use of transport protocols in terms of the features they provide;  <ins> Message- as opposed to stream-orientation, using application- assisted framing and deframing where the underlying transport does not provide these; </ins> Asynchronous Connection establishment, transmission, and  <ins> reception, allowing concurrent operations during establishment and supporting event-driven application interactions with the transport layer, in line with developments in modern platforms and programming languages; Explicit support for security properties as first-order transport features, and for long-term caching of cryptographic identities and parameters for associations among endpoints; and </ins> Explicit support for multistreaming and multipath transport protocols, and the grouping of related Connections into Connection Groups through cloning of Connections, to allow applications to take full advantage of new transport protocols supporting these  <ins> features. </ins> 4. The Transport Services Interface is the basic common abstract application programming interface to the Transport Services  <ins> Architecture defined in I-D.ietf-taps-arch. An application primarily interacts with this interface through two Objects, Preconnections and Connections.  A Preconnection represents a set of properties and constraints on the selection and configuration of paths and protocols to establish a Connection with a remote endpoint.  A Connection represents a transport Protocol Stack on which data can be sent to and/or received from a remote endpoint (i.e., depending on the kind of transport, connections can be bi- directional or unidirectional).  Connections can be created from Preconnections in three ways: by initiating the Preconnection (i.e., actively opening, as in a client), through listening on the Preconnection (i.e., passively opening, as in a server), or rendezvousing on the Preconnection (i.e.  peer to peer establishment). </ins> Once a Connection is established, data can be sent on it in the form of Messages.  The interface supports the preservation of message Prefer, Require) for the level of preference of a given property during protocol selection; see selection-props. <ins> 4.2. This document defines a language- and platform-independent interface to a Transport Services system.  Given the wide variety of languages and language conventions used to write applications that use the transport layer to connect to other applications over the Internet, this independence makes this interface necessarily abstract.  While there is no interoperability benefit to tightly defining how the interface be presented to application programmers in diverse platforms, maintaining the "shape" of the abstract interface across these platforms reduces the effort for programmers who learn the transport services interface to apply their knowledge in multiple platforms.  We therefore make the following recommendations: Actions, Events, and Errors in implementations of this interface SHOULD carry the names given for them in the document, subject to capitalization and punctuation conventions in the language of the implementation, unless the implementation itself uses different names for substantially equivalent objects for networking by convention. Implementations of this interface SHOULD implement each Selection Property, Connection Property, and Message Context Property specified in this document, exclusive of appendices, even if said implementation is a non-operation, e.g.  because transport protocols implementing a given Property are not available on the platform. </ins> 5. The pre-establishment phase allows applications to specify properties