operation of the selected Protocol Stacks. All Transport Parameters are organized within a single namespace <del> shared with Send Parameters (see send-params).  All transport parameters take paremeter-specific values.  Protocol and Path Selection properties additionally take one of five preference levels, though not all preference levels make sense with all such properties. </del>  Note that it is possible for a set of specified transport parameters <del> to be internally inconsistent, or for preferences to be inconsistent with the later use of the API by the application.  Application developers can reduce inconsistency by only using the most stringent preference levels when failure to meet a preference would break the application's functionality (e.g. the Reliable Data Transfer preference, which is a core assumption of many application protocols).  Implementations of this interface should also raise any detected errors in configuration as early as possible, to help ensure these inconsistencies are caught early in the development process. </del>  The protocol(s) and path(s) selected as candidates during Connection establishment are determined by a set of properties.  Since there tied to path selection.  This may involve choosing between multiple local interfaces that are connected to different access networks. <del> To reflect the needs of an individual Connection, they can be specified with five different preference levels: </del>  An implementation of this interface must provide sensible defaults for protocol and path selection properties.  The defaults given for 5.2.1. <del> This boolean property specifies whether the application wishes to use a transport protocol that that provides mechanisms to help ensure that all data is received and without corruption on the other side. This also entails being notified when a Connection is closed or aborted.  This property applies to Connections and Connection Groups. This is a strict requirement.  The default is to enable Reliable Data </del>  Transfer. 5.2.2. <del> This boolean property specifies whether the application wishes to use a transport protocol that provides mechanisms to ensure that data is </del>  received by the application on the other end in the same order as it was sent.  This property applies to Connections and Connection Groups.  This is a strict requirement.  The default is to preserve 5.2.3. <del> This boolean property specifies whether an application considers it useful to indicate its reliability requirements on a per-Message basis.  This property applies to Connections and Connection Groups. This is not a strict requirement.  The default is to not have this option. </del>  5.2.4. <del> This boolean property specifies whether an application would like to supply a Message to the transport protocol before Connection establishment, which will then be reliably transferred to the other side before or during Connection establishment, potentially multiple times.  See also send-idempotent.  This is a strict requirement.  The default is to not have this option. </del>  5.2.5. <del> This boolean property specifies that the application would prefer multiple Connections within a Connection Group to be provided by streams of a single underlying transport connection where possible. This is not a strict requirement.  The default is to not have this option. </del>  5.2.6. <del> This boolean property specifies whether an application considers it useful to be informed in case sent data was retransmitted more often than a certain threshold.  This property applies to Connections and Connection Groups.  This is not a strict requirement.  The default is to have this option. </del>  5.2.7. <del> This boolean property specifies whether an application considers it useful to be informed when an ICMP error message arrives that does not force termination of a connection.  This property applies to Connections and Connection Groups.  Received ICMP errors will be available as SoftErrors.  Note that even if a protocol supporting this property is selected, not all ICMP errors will necessarily be delivered, so applications cannot rely on receiving them.  This is not a strict requirement.  The default is not to have this option. </del>  5.2.8. <del> This boolean property specifies whether the application considers it useful to enable / disable / configure a checksum when sending data, or decide whether to require a checksum or not when receiving data. </del>  This property applies to Connections and Connection Groups.  This is not a strict requirement, as it signifies a reduction in reliability. The default is full checksum coverage without being able to change 5.2.9. <del> This enumerated property specifies which kind of access network interface, e.g., WiFi, Ethernet, or LTE, to prefer over others for this Connection, in case they are available.  In general, Interface Types should be used only with the "Prefer" and "Prohibit" preference level.  Specifically, using the "Require" preference level for Interface Type may limit path selection in a way that is detrimental to connectivity.  The default is to use the default interface configured in the system policy. </del>  5.2.10. <del> This enumerated property specifies the application's expectation of the dominating traffic pattern for this Connection.  The Capacity Profile should only be used with the "Prefer" preference level; other preference levels make no sense for profiles.  The following values are valid for Capacity Profile: </del>  The application makes no representation about its expected capacity profile.  No special optimizations of the tradeoff

This commit fixes bug by making transport parameter types explicit. It also fixes Path- and Protocol Selection Properties from "binary + preference" to preferences type. With the old boolean properties, one could do things like "prohibit Reliable Transport false" - is that the same as "prefer Reliable Transport true"?
To the questions in NAME review summary: For the general case: I believe there will be boolean properties (that are not preferences). The way I understand "Excessive Retransmit Notifications" and "ICMP soft error message", I think booleans are the more appropriate type. Not for -00, but for -01: I want to define all types more rigorously in -01 --- we have Boolean, Enumeration, Boolean in the Transport Parameters right now, we also have some Integers in the Send Parameters. I am not sure that the frontmatter of this section is the right place to do so.
General clarification: This pull request changed "preference" to a data type a value can have rather than being something weakly defined that can be specified along with every Transport Parameter. So we go from a Key / Preference / Value tripel where Preference or Value is pointless for (n-1) cases to a Key / Value pair with typed values. The only Transport Parameter (including the Intents in the Appendix) that uses a parameter and a preference is the "Interface Type". I would like to land this for -00 as this is good enough for discussion while the old state is confusing. ToDos for -01: Explain types clearly make "Interface Type" easier in -01 discuss extendability
If we're doing booleans, the document needs to define a concise and generally applicable rule for how booleans interact with preferences. ISTM that boolean True == preference Prefer and boolean False == preference Avoid (in which case, why not just say Prefer and Avoid instead of True and False?) I might be wrong here though. Will defer to NAME
also NAME Please file everything in the comments on this PR that should be addressed in -01 as an issue (if you haven't already)... thanks!
Ok - now I got your point. These properties (Notification of excessive retransmissions and Notification of ICMP soft error message arrival) are actually doing two things: Enable sending these events Indicate that we prefer protocols stacks that can send these events The question is which one is more important. As I think the former is more important, I leave it boolean an add remarks to the individual properties that this implies preferring protocols that can provide these events for now. As this does not hide things, this should provide the better feedback for -00
This should be ready for landing. – NAME please merge if you agree Open Issues for Milestone itef-01: Restructure Transport Parameters Section Explain data types explicitly Rewrite Interface Types to be useful and implementable
Transport and Send Parameter have different data types. These types are used in the document in a more or less intuitive way but are never defined explicitly. May need a new section next to the pseudocode definition.
This has been addressed in PR
This is a fork of Issue – The part of it that does not need discussion. Annotate "data type" for each Transport Parameter Change "boolean" Protocol and Path Selection Parameters to type "Preference" Remove editing artifacts
still not convinced by the boolean argument, but it's time to ship it.

operation of the selected Protocol Stacks. All Transport Parameters are organized within a single namespace  <ins> shared with Send Parameters (see send-params).  These transport parameters take values of parameter-specific types. </ins> Note that it is possible for a set of specified transport parameters  <ins> to be internally inconsistent, or to be inconsistent with the later use of the API by the application.  Application developers can reduce inconsistency by only using the most stringent preference levels when failure to meet a preference would break the application's functionality (e.g. the Reliable Data Transfer preference, which is a core assumption of many application protocols).  Implementations of this interface should also raise any detected errors in configuration as early as possible, to help ensure these inconsistencies are caught early in the development process. </ins> The protocol(s) and path(s) selected as candidates during Connection establishment are determined by a set of properties.  Since there tied to path selection.  This may involve choosing between multiple local interfaces that are connected to different access networks.  <ins> The type of most Protocol and Path Selection properties is "preference" and has with five different preference levels: Internally, the transport system will first exclude all protocols and paths that match a Prohibit, then only keep candidates that match a Require, then sort candidates according to Preferred properties, and then use Avoided properties as a tiebreaker.  In case of conflicts between protocol and path selection properties, path selection properties take precedence.  For example, if an application indicates a preference for a specific path, but also a preference for a protocol not available on this path, the transport system will try the path first, so the protocol selection property might not have an effect. </ins> An implementation of this interface must provide sensible defaults for protocol and path selection properties.  The defaults given for 5.2.1.  <ins> Type: Preference This property specifies whether the application wishes to use a transport protocol that that provides mechanisms to help ensure that all data is received and without corruption on the other side.  This also entails being notified when a Connection is closed or aborted. This property applies to Connections and Connection Groups.  This is a strict requirement.  The default is to enable Reliable Data </ins> Transfer. 5.2.2.  <ins> Type: Preference This property specifies whether the application wishes to use a transport protocol that provides mechanisms to ensure that data is </ins> received by the application on the other end in the same order as it was sent.  This property applies to Connections and Connection Groups.  This is a strict requirement.  The default is to preserve 5.2.3.  <ins> Type: Preference This property specifies whether an application considers it useful to indicate its reliability requirements on a per-Message basis.  This property applies to Connections and Connection Groups.  This is not a strict requirement.  The default is to not have this option. </ins> 5.2.4.  <ins> Type: Preference This property specifies whether an application would like to supply a Message to the transport protocol before Connection establishment, which will then be reliably transferred to the other side before or during Connection establishment, potentially multiple times.  See also send-idempotent.  This is a strict requirement.  The default is to not have this option. </ins> 5.2.5.  <ins> Type: Preference This property specifies that the application would prefer multiple Connections within a Connection Group to be provided by streams of a single underlying transport connection where possible.  This is not a strict requirement.  The default is to not have this option. </ins> 5.2.6.  <ins> Type: Boolean This property specifies whether an application considers it useful to be informed in case sent data was retransmitted more often than a certain threshold.  When set to true, the effect is twofold: The application may receive events in case excessive retransmissions.  In addition, the transport system considers this as a preference to use transports stacks that can provide this notification.  This is not a strict requirement.  If set to false, no notification of excessive retransmissions will be sent and this transport feature is ignored for protocol selection. This property applies to Connections and Connection Groups.  The default is to have this option. </ins> 5.2.7.  <ins> Type: Boolean This property specifies whether an application considers it useful to be informed when an ICMP error message arrives that does not force termination of a connection.  When set to true, received ICMP errors will be available as SoftErrors.  Note that even if a protocol supporting this property is selected, not all ICMP errors will necessarily be delivered, so applications cannot rely on receiving them.  Setting this option also implies a preference to prefer transports stacks that can provide this notification.  If not set, no events will be sent for ICMP soft error message and this transport feature is ignored for protocol selection. This property applies to Connections and Connection Groups.  The default is not to have this option. </ins> 5.2.8.  <ins> Type: Preference This property specifies whether the application considers it useful to enable / disable / configure a checksum when sending data, or decide whether to require a checksum or not when receiving data. </ins> This property applies to Connections and Connection Groups.  This is not a strict requirement, as it signifies a reduction in reliability. The default is full checksum coverage without being able to change 5.2.9.  <ins> Type: Tuple (Enumeration, Preference) This property specifies which kind of access network interface, e.g., WiFi, Ethernet, or LTE, to prefer over others for this Connection, in case they are available.  In general, Interface Types should be used only with the "Prefer" and "Prohibit" preference level. Specifically, using the "Require" preference level for Interface Type may limit path selection in a way that is detrimental to connectivity.  The default is to use the default interface configured in the system policy.  The valid values for the access network interface kinds are implementation specific. </ins> 5.2.10.  <ins> Type: Enumeration This property specifies the application's expectation of the dominating traffic pattern for this Connection.  This implies that the transport system should optimize for the capacity profile specified.  This can influence path and protocol selection.  The following values are valid for Capacity Profile: </ins> The application makes no representation about its expected capacity profile.  No special optimizations of the tradeoff


first.  Implementations should order the branches to reflect the preferences expressed by the application for its new connection, including Protocol and Path Selection Properties, which are specified <del> in I-D.trammell-taps-interface.  In addition to the properties provided by the application, an implementation may include additional criteria such as cached performance estimates, see performance- caches, or system policy, see role-of-system-policy, in the ranking. Two examples of how the Protocol and Path Selection Properties may be used to sort branches are provided below: </del>  Interface Type: If the application specifies an interface type to be preferred or avoided, implementations should rank paths [Note: See branch-sorting-non-consensus for additional examples related to Properties under discussion.]  4.4. The primary goal of the Candidate Racing process is to successfully To reflect the needs of an individual Connection, they can be specified with five different preference levels:  An implementation of this interface must provide sensible defaults for protocol and path selection properties.  The defaults given for each property below represent a configuration that can be implemented

Added text about what happens when an application specifies both protocol and path selection properties. Path selection properties have precedence. I added examples to both API and implementation draft to illustrate this. Relates to
In section 5.2. in the API doc: "Internally, the transport system will first exclude all protocols and paths that match a Prohibit, then exclude all protocols and paths that do not match a Require, then sort candidates according to Preferred properties, and then use Avoided properties as a tiebreaker. Selection Properties that select paths take preference over those that select protocols. For example, if an application indicates a preference for a specific path by specifying an interface, but also a preference for a protocol not available on this path, the transport system will try the path first, ignoring the preference." This sounds rather like an example algorithm that should maybe go into the implementation doc instead? Or is there any part of this that we need to specify normatively in order to ensure that all systems implement the same?
This goes back to issue on "conflicting policies". At the time, we decided that there must be a deterministic outcome, therefore, text was added to both the API and the Implementation document, see . If we keep it that way, perhaps the text here should point out that the goal is to resolve conflicting policies, so we achieve a deterministic outcome.
Ah okay, thanks for the background info; missed that. I think we need to use normative language here then. Maybe also a separate subsection makes this more prominent.
This reads to me like the rules for processing polices rather than an example algorithm. I think it is part of the API... Normative language would be good and a stronger opening that explains why the rules exist.
I don't think a separate subsection would make this a better read - this fits fine as it stands, as an intro to the transport property specification IMO. Regarding normative language, that's a separate discussion. So, can we close this issue?
Agree that we don't need a separate subsection. I can try to tweak the beginning of this part a bit to explain why the rules exist as NAME suggested. Regarding normative language - we definitely have it in the API draft, right? So why not have the discussion on it here as well? if we're serious about deterministic outcomes, I guess the whole thing needs to be a MUST, right?
Sorry, true, that normative language debate was about the arch draft, and we definitely have it in the API draft. So yes, let's discuss it here. I also think a MUST would make sense.
So the order of filtering out require/prohibit is not important (outcome is the same) - making the order a MUST does not make sense. Also, first sorting by preference and then filtering has the same outcome as doing it the other way around. the only thing we need to say is that filtering MUST happen and sorting SHOULD happen.
ok - I have to admit I didn't think this through, I just agree that we want a deterministic outcome from this.
Why did we require a deterministic behavior in first place? Does this prevent TAPS implementations from running experiments like Apple did with TCP SACK and ECN? Does this prevent probabilistic path probing?
This was discussed in the TAPS BoF? ... I thought it was about developers being assured that X will happen if they choose Y and not finding the ground shifting as they use the interface.
Interim: Prefers and avoids allow the implementation to decide the ordering and weighting; require and prohibit are the only hard requirements. Applications should be able to cope with any possibility that isn't prohibited. Likely should add implementation text to say how preferences weight racing.
During the interim discussion, we found that was too strict and limiting, so we opted for another approach that moves the selection algorithm to the Implementation draft. I'm preparing another PR implementing these changes.
The text that ended up in the implementation draft is not fully consistent with the racing text, opening a new issue for this and closing this issue.
See thread at URL Suggested resolution: implementation must have a deterministic tiebreaker, current recommendation is that path properties take precedence. API should note expected behavior. There should be additional text on candidate selection and ordering.
Give example for path candidate gathering and sorting (get selection properties, get available paths in system preferred order, walk through them cancelling out the prohibited ones, and ranking preferences; get ultimate list). Do the same for protocols likely.
Added text to API and Implementation drafts on branch "conflicting-policies". API has an example, implementation still needs one. Feel free to add, otherwise I'll come back to it later today or tomorrow.
I guess that works for -00 For -01, we might want to integrate the resolution a little bit more into the decision process.
Resolved by PR
this is good, thanks! Hi, I made this comment in the past in QUIC when discussing migration. One policy may be prefer wifi over cellular network, Second policy prefers quic cove TCP Now I walk into my house and my cellphone has HTTP over QUIC. At home will switch to wifi but quic is not supported on the wifi so will fall back to TCP. This contradicts the second policy. So two options wifi TCP or cellular QUIC , how to decide? Roni Even

first.  Implementations should order the branches to reflect the preferences expressed by the application for its new connection, including Protocol and Path Selection Properties, which are specified  <ins> in I-D.trammell-taps-interface. In addition to the properties provided by the application, an implementation may include additional criteria such as cached performance estimates, see performance-caches, or system policy, see role-of-system-policy, in the ranking.  Two examples of how the Protocol and Path Selection Properties may be used to sort branches are provided below: </ins> Interface Type: If the application specifies an interface type to be preferred or avoided, implementations should rank paths [Note: See branch-sorting-non-consensus for additional examples related to Properties under discussion.] <ins> Implementations should process properties in the following order: Prohibit, Require, Prefer, Avoid.  If Protocol or Path Selection Properties contain any prohibited properties, the implementation should first purge branches containing nodes with these properties. For required properties, it should only keep branches that satisfy these requirements.  Finally, it should order branches according to preferred properties, and finally use avoided properties as a tiebreaker. For Require and Avoid, Path Selection Properties take precedence over Protocol Selection Properties.  For example, if the application has indicated both a preference for WiFi over LTE and for a feature only available in SCTP, branches will be first sorted accord to the Path Selection Property, with WiFi at the top.  Then, branches with SCTP will be sorted to the top within their subtree according to the Protocol Selection Property.  However, if the implementation has cached the information that SCTP is not available on the path over WiFi, there is no SCTP node in the WiFi subtree.  Here, the path over WiFi will be tried first, and, if connection establishment succeeds, TCP will be used.  So the Path Selection Property of preferring WiFi takes precedence over the Protocol Selection Property of preferring SCTP. </ins> 4.4. The primary goal of the Candidate Racing process is to successfully To reflect the needs of an individual Connection, they can be specified with five different preference levels: <ins> Internally, the transport system will first exclude all protocols and paths that match a Prohibit, then only keep candidates that match a Require, then sort candidates according to Preferred properties, and then use Avoided properties as a tiebreaker.  In case of conflicts between protocol and path selection properties, path selection properties take precedence.  For example, if an application indicates a preference for a specific path, but also a preference for a protocol not available on this path, the transport system will try the path first, so the protocol selection property might not have an effect. </ins> An implementation of this interface must provide sensible defaults for protocol and path selection properties.  The defaults given for each property below represent a configuration that can be implemented


TCP, this may correspond to when the data has entered the send buffer.  5.1.2. Similar to sending, Receiving a Message is determined by the top- 7.2.  Sender-side framing allows a caller to provide the interface with a function that takes a Message of an appropriate application-layer type and returns an array of octets, the on-the-wire representation

See issue The one question I have here is the semantics around batching. In our implementation, we use block semantics (from clang), to batch looks like: connectionbatch(^{ connectionsend(...); connection_send(...); }); where the block is called inline, and the batch is over when the block returns. I'm not sure what the right style for the pseudo-code is for the draft.
In aligning the implementation with the API for the initial draft, we removed this text since we still need text in the API: Since sending a Message may involve a context switch between the application and the transport system, sending patterns that involve multiple small Messages can incur high overhead if each needs to be enqueued separately. To avoid this, the application should have a way to indicate a batch of Send actions, during which time the implementation will hold off on processing Messages until the batch is complete. This can also help context switches when enqueuing data in the interface driver if the operation can be batched.
This seems equivalent to the idea of PUSH in existing systems and would make sense for lots of short write calls.
+1. The higher-level question here is how to support high-performance applications. Send batching is one thing you can do here. Receive batching would be another obvious one. Zero-copy is another, though that's probably so tied to implementation and platform that it's not useful to standardize. But there should be some way for this interface to coexist with it...
+1 from me too
+1 In my opinion, support for high-performance applications is nothing we can add to the API, but rather something we must sure to not prevent or make hard to implement. So we should ask ourselves for every point in the API "can this be implemented zero-copy" or "can an implementation batch this" – if the answer is no, we have to fix that.
this one seems to have consensus and does not seem to be too much work. NAME if you can get a PR in in the next week or so let's ship it with ietf-00...
I'll try to get this in soon!
fixed with merge of
LGTM, we can worry about the right way to pseudocode this later.

TCP, this may correspond to when the data has entered the send buffer. <ins> 5.1.1.3. Since sending a Message may involve a context switch between the application and the transport system, sending patterns that involve multiple small Messages can incur high overhead if each needs to be enqueued separately.  To avoid this, the application should have a way to indicate a batch of Send actions, during which time the implementation will hold off on processing Messages until the batch is complete.  This can also help context switches when enqueuing data in the interface driver if the operation can be batched. </ins> 5.1.2. Similar to sending, Receiving a Message is determined by the top- 7.2. <ins> In order to reduce the overhead of sending multiple small Messages on a Connection, the application may want to batch several Send actions together.  This provides a hint to the system that the sending of these Messages should be coalesced when possible, and that sending any of the batched Messages may be delayed until the last Message in the batch is enqueued. 7.3. </ins> Sender-side framing allows a caller to provide the interface with a function that takes a Message of an appropriate application-layer type and returns an array of octets, the on-the-wire representation


Type: Tuple (Enumeration, Preference) <del> This property specifies which kind of access network interface, e.g., WiFi, Ethernet, or LTE, to prefer over others for this Connection, in case they are available.  In general, Interface Types should be used only with the "Prefer" and "Prohibit" preference level. Specifically, using the "Require" preference level for Interface Type may limit path selection in a way that is detrimental to connectivity.  The default is to use the default interface configured in the system policy.  The valid values for the access network interface kinds are implementation specific. </del>  5.2.10.  Type: Enumeration This property specifies the application's expectation of the Path Properties of the path(s) in use, once the Connection has been established.  These properties can be derived from the local <del> provisioning domain, measurements by the Protocol Stack, or other sources.  They can only be queried. </del>  9.1.

Implements
tagging discuss since NAME and I, at least, apparently have a fairly fundamental disagreement on flexibility versus classification of layers 2.
NAME - I don't think I'll fight for this one, but to begin with, tell me: what do you think is wrong about my view?
Thanks for the hearty discussion! On this point, I do agree with NAME that we're not going to achieve full portability across systems for interfaces and PvDs, and it likely doesn't need to be a goal. If I'm writing an implementation in ten years for some new system that only uses Wi-Fi and [INSERT FANCY NEW RADIO TYPE HERE], why would I need to add support for this wired ethernet thing that I don't have any devices for? Since moving between different implementations isn't likely going to be 100% portable anyhow due to language differences and symbol differences, applications will need to change. However, a specific implementation that is portable (like NEAT, or like Apple's if we make it cross-platform) should endeavor to offer the same types across systems.
Fine by me, go ahead
NAME NAME please take a look at the update! NAME I assume based on your last comment that you're okay with the change going in now?
Yes, I agree with it!
let's land this one then...
Works for me now. Sorry for the delay!in my opinion, almost there… only minor comments to address left

Type: Tuple (Enumeration, Preference)  <ins> This property allows the application to select which specific network interfaces or categories of interfaces it wants to "Require", "Prohibit", "Prefer", or "Avoid". If a system supports discovery of specific interface identifiers, such as "en0" or "eth0" on Unix-style systems, an implemention should allow using these identifiers to define path preferences.  Note that marking a specific interface as "Required" strictly limits path selection to a single interface, and leads to less flexible and resilient connection establishment. The set of valid interface types is implementation- and system- specific.  For example, on a mobile device, there may be "Wi-Fi" and "Cellular" interface types available; whereas on a desktop computer, there may be "Wi-Fi" and "Wired Ethernet" interface types available. Implementations should provide all types that are supported on some system to all systems, in order to allow applications to write generic code.  For example, if a single implementation is used on both mobile devices and desktop devices, it should define the "Cellular" interface type for both systems, since an application may want to always "Prohibit Cellular".  Note that marking a specific interface type as "Required" limits path selection to a small set of interfaces, and leads to less flexible and resilient connection establishment. The set of interface types is expected to change over time as new access technologies become available. Interface types should not be treated as a proxy for properties of interfaces such as metered or unmetered network access.  If an application needs to prohibit metered interfaces, this should be specified via Provisioning Domain attributes prop-pvd or another specific property. </ins> 5.2.10. <ins> Type: Tuple (Enumeration, Preference) Similar to interface instances and types prop-interface, this property allows the application to control path selection by selecting which specific Provisioning Domains or categories of Provisioning Domains it wants to "Require", "Prohibit", "Prefer", or "Avoid".  Provisioning Domains define consistent sets of network properties that may be more specific than network interfaces RFC7556. The indentification of a specific Provisioning Domain (PvD) is defined to be implementation- and system-specific, since there is not a portable standard format for a PvD identitfier.  For example, this identifier may be a string name or an integer.  As with requiring specific interfaces, requiring a specific PvD strictly limits path selection. Categories or types of PvDs are also defined to be implementation- and system-specific.  These may be useful to identify a service that is provided by a PvD.  For example, if an application wants to use a PvD that provides a Voice-Over-IP service on a Cellular network, it can use the relevant PvD type to require some PvD that provides this service, without needing to look up a particular instance.  While this does restrict path selection, it is more broad than requiring specific PvD instances or interface instances, and should be preferred over those options. 5.2.11. </ins> Type: Enumeration This property specifies the application's expectation of the Path Properties of the path(s) in use, once the Connection has been established.  These properties can be derived from the local  <ins> provisioning domain RFC7556, measurements by the Protocol Stack, or other sources.  They can only be queried. </ins> 9.1.


7.1.5.  This numeric property specifies the length of the section of the Message, starting from byte 0, that the application assumes will be received without corruption due to lower layer errors.  It is used to be used to indicate the default.  Only full coverage is guaranteed, any other requests are advisory. <del> 7.1.6. </del>  This enumerated property specifies the application's preferred tradeoffs for sending this Message; it is a per-Message override of 8.1.  The Receive Event is intended to be fired once per application-layer Message sent by the remote endpoint; i.e., it is a desired property of this interface that a Send at one end of a Connection maps to

Satisfies
Okay, addressed capitalization and took Brian's suggested paragraph (with one tiny change).
Fine with me - go ahead
NAME NAME any further comments on this one?
ship it
Sorry, seems I should have clicked "approve" earlier. Still being clumsy in github :(

7.1.5. <ins> Final is a boolean property.  If true, this Message is the last one that the application will send on a Connection.  This allows underlying protocols to indicate to the Remote Endpoint that the Connection has been effectively closed in the sending direction.  For example, TCP-based Connections can send a FIN once a Message marked as Final has been completely sent.  Protocols that do not support signalling the end of a Connection in a given direction will ignore this property. Note that a Final Message must always be sorted to the end of a list of Messages.  The Final property overrides Niceness and any other property that would re-order Messages.  If another Message is sent after a Message marked as Final has already been sent on a Connection, the new Message will report an error. 7.1.6. </ins> This numeric property specifies the length of the section of the Message, starting from byte 0, that the application assumes will be received without corruption due to lower layer errors.  It is used to be used to indicate the default.  Only full coverage is guaranteed, any other requests are advisory.  <ins> 7.1.7. </ins> This enumerated property specifies the application's preferred tradeoffs for sending this Message; it is a per-Message override of 8.1. <ins> The application may check a property to determine if a received Message is the Final Message on a Connection.  For any Message that is marked as Final, the application can assume that there will be no more Messages received on the Connection once the Message has been completely delivered.  This corresponds to the Final property that may be marked on a sent Message send-final. Some transport protocols and peers may not support signaling of the Final property.  Applications therefore should not rely on receiving a Message marked Final to know that the other endpoint is done sending on a connection. Any calls to "Receive" once the Final Message has been delivered will result in errors. 8.2. </ins> The Receive Event is intended to be fired once per application-layer Message sent by the remote endpoint; i.e., it is a desired property of this interface that a Send at one end of a Connection maps to


scope for alternate path discovery during Connection establishment. The Resolve() method is, however, provided to resolve a Local Endpoint or a Remote Endpoint in cases where this is required, for <del> example with some NAT traversal protocols (see rendezvous). </del>  5.2. be minimized at the expense of bandwidth efficiency.  This implies that the Connection may fail if the desired rate cannot be maintained across the Path.  A transport may interpret this <del> capacity profile as preferring a circuit breaker RFC8084 to a rate adaptive congestion controller. </del>  The application is not interactive.  It expects to send/receive a <del> large amount of data, without any urgency.  This can be used to select protocol stacks with scavenger transmission control, to signal a preference for less-than-best-effort treatment, and so on. </del>  5.3. established to the Remote Endpoint, or when the application is prohibited from rendezvous by policy. <del> When using some NAT traversal protocols, e.g., ICE RFC5245, it is expected that the Local Endpoint will be configured with some method of discovering NAT bindings, e.g., a STUN server.  In this case, the Local Endpoint may resolve to a mixture of local and server reflexive addresses.  The Resolve() method on the Preconnection can be used to discover these bindings: </del>  The Resolve() call returns a list of Preconnection Objects, that represent the concrete addresses, local and server reflexive, on Connection's Niceness value, M is the maximum Niceness value used by all Connections in the group and C is the total available capacity. However, the niceness setting is purely advisory, and no guarantees <del> are given about capacity allocation and each implementation is free to implement exact capacity allocation as it sees fit. </del>  7. Send Parameters may be inconsistent with the properties of the Protocol Stacks underlying the Connection on which a given Message is <del> sent.  For example, infinite Lifetime is not possible on a Message over a Connection not providing reliability.  Sending a Message with Send Properties inconsistent with the Transport Preferences on the Connection yields an error. </del>  The following send parameters are supported: 7.1.2. <del> Niceness represents an unbounded hierarchy of priorities of Messages, relative to other Messages sent over the same Connection and/or Connection Group (see groups).  It is most naturally represented as a non-negative integer.  A Message with Niceness 0 will yield to a Message with Niceness 1, which will yield to a Message with Niceness 2, and so on.  Niceness may be used as a sender-side scheduling construct only, or be used to specify priorities on the wire for Protocol Stacks supporting prioritization. </del>  <del> Note that this inversion of normal schemes for expressing priority has a convenient property: priority increases as both Niceness and Lifetime decrease. </del>  7.1.3. this message. Response time (latency) should be optimized at the expense of <del> bandwidth efficiency and delay variation when sending this </del>  message.  This can be used by the system to disable the coalescing of multiple small Messages into larger packets (Nagle's algorithm); to prefer immediate acknowledgment from the peer endpoint when supported by the underlying transport; to signal a preference for lower-latency, higher-loss treatment; and so on. <del> Delay and delay variation should be minimized at the expense of bandwidth efficiency. </del>  This Message may be sent at the system's leisure.  This can be used to signal a preference for less-than-best-effort treatment, <del> to delay sending until lower-cost paths are available, and so on. </del>  7.2. the maximum Message size that can be received.  This property is read-only.  In order to specify Specific Protocol Properties, Transport System implementations may offer applications to attach a set of options to the Preconnection Object, associated with a specific protocol.  For

and addresses
Section: Low Latency: : I really dislike the term /bandwidth efficiency/ - because we are not talking here of bandwidth in MHz, and certainly of not how efficiently the bandwidth is used from a radio perspective. Could we say /efficiently of using the available capacity/? I also have problems undrestaing what was intended by /and delay variation/ - because it doesn’t read correctly to me. What is intended? Section: Constant Rate: : still dislike /bandwidth efficiency/ - see above. /rate adaptive/ should this be /rate-adaptive/ ? Section: Scavenger/Bulk: : /for less-than-best-effort treatment, and so on./ can we explicitly point to the scavenger traffic class, rather than just the transport behaviour? e.g. add /or to assign the traffic to a lower effort service/ ?? Should we expand these abbreviations on first use?: NAT, ICE, STUN, SIP Section: Connection Groups {#groups} Could we consider removing the following clause ... I am not sure that I buy the specification text that reads: /An ideal transport system implementation would assign the Connection the capacity share (M-N) x C / M, where N is the Connection's Niceness value, M is the maximum Niceness value used by all Connections in the group and C is the total available capacity. / I’d prefer to feel comfortable that we have running code that does this - NEAT does not implement this approach, it could do something I am sure - but do we have to write this as if we specify this in the API? I do not like the words /about capacity allocation/ --- my rationale is that this doesn’t imply using an allocation method - I’d much prefer /about the way capacity is shared/ Similarly: /exact capacity allocation/ -- I’d prefer: /the way an implementation shares capacity/ /For example, infinite Lifetime is not possible on a Message over a Connection not providing reliability. / This didn’t read quite correct to me, is this better: /For example, it is not possible to set an infinite Lifetime for a Message over a Connection that does not provide reliability. / or /For example, a Connection must provide reliability to allow a Message to set an infinite Lifetime. / The concept of /an infinite Lifetime/ is just a tiny little odd, does this mean the value is set to infinite - or do we really expect persistent non-volatile storage of the message until it is sent?, I'd perhaps geel happier if we explained this as /an infinitie value for the lifetime property means that ..../ Section: Niceness {#send-niceness} The term /normal/ /Note that this inversion of normal schemes for expressing priority has a convenient property: priority increases as both Niceness and Lifetime decrease./ I wonder if this actually is clearer just to say: /This encoding of the priority has a convenient property that the priority increases as both Niceness and Lifetime decrease./ /It is most naturally represented as a non-negative integer./ the other terms are defined more concretely. Is there any reason we do not start the niceness section by saying Niceness is a numeric (non-negative integer) quantity. I didn't read the appendexes because I thought they were still being edited - happy to read when asked.
I'll try to take care of these early next week, together with a few other updates I planned for that period.
About this one: I'd rather keep this: otherwise it's not clarified anywhere what the real meaning of a niceness value is - is it a linear priority, is it like doubling priorities per value (as Harald was forced to invent here, for lack of any other available definition: URL ), or what? And BTW, indeed we do implement it like this in NEAT - AFAIK both the SCTP scheduler and Safiqul's coupled cc. kernel patch do this.
We need to distinguish between sending unreliably, but congestion controlled, and sending unreliably without congestion control (the application must know if it has to perform congestion control or not). This should influence at least the protocol and path selection properties.
lgtmLooks good

scope for alternate path discovery during Connection establishment. The Resolve() method is, however, provided to resolve a Local Endpoint or a Remote Endpoint in cases where this is required, for  <ins> example with some Network Address Translator (NAT) traversal protocols (see rendezvous). </ins> 5.2. be minimized at the expense of bandwidth efficiency.  This implies that the Connection may fail if the desired rate cannot be maintained across the Path.  A transport may interpret this  <ins> capacity profile as preferring a circuit breaker RFC8084 to a rate-adaptive congestion controller. </ins> The application is not interactive.  It expects to send/receive a  <ins> large amount of data, without any urgency.  This can, for example, be used to select protocol stacks with scavenger transmission control, to signal a preference for less-than-best-effort treatment, or to assign the traffic to a lower-effort service. 5.2.13. Type: Preference This property specifies whether the application would like the Connection to be congestion controlled or not.  Note that if a Connection is not congestion controlled, an application using such a Connection should itself perform congestion control in accordance with RFC2914.  Also note that reliability is usually combined with congestion control in protocol implementations, rendering "reliable but not congestion controlled" a request that is unlikely to succeed. </ins> 5.3. established to the Remote Endpoint, or when the application is prohibited from rendezvous by policy.  <ins> When using some NAT traversal protocols, e.g., Interactive Connectivity Establishment (ICE) RFC5245, it is expected that the Local Endpoint will be configured with some method of discovering NAT bindings, e.g., a Session Traversal Utilities for NAT (STUN) server. In this case, the Local Endpoint may resolve to a mixture of local and server reflexive addresses.  The Resolve() method on the Preconnection can be used to discover these bindings: </ins> The Resolve() call returns a list of Preconnection Objects, that represent the concrete addresses, local and server reflexive, on Connection's Niceness value, M is the maximum Niceness value used by all Connections in the group and C is the total available capacity. However, the niceness setting is purely advisory, and no guarantees  <ins> are given about the way capacity is shared.  Each implementation is free to implement a way it shares capacity that it sees fit. </ins> 7. Send Parameters may be inconsistent with the properties of the Protocol Stacks underlying the Connection on which a given Message is  <ins> sent.  For example, a Connection must provide reliability to allow setting an infinitie value for the lifetime property of a Message. Sending a Message with Send Properties inconsistent with the Transport Preferences on the Connection yields an error. </ins> The following send parameters are supported: 7.1.2.  <ins> Niceness is a numeric (non-negative) value that represents an unbounded hierarchy of priorities of Messages, relative to other Messages sent over the same Connection and/or Connection Group (see groups).  A Message with Niceness 0 will yield to a Message with Niceness 1, which will yield to a Message with Niceness 2, and so on. Niceness may be used as a sender-side scheduling construct only, or be used to specify priorities on the wire for Protocol Stacks supporting prioritization. </ins>  <ins> This encoding of the priority has a convenient property that the priority increases as both Niceness and Lifetime decrease. </ins> 7.1.3. this message. Response time (latency) should be optimized at the expense of  <ins> efficiently using the available capacity when sending this </ins> message.  This can be used by the system to disable the coalescing of multiple small Messages into larger packets (Nagle's algorithm); to prefer immediate acknowledgment from the peer endpoint when supported by the underlying transport; to signal a preference for lower-latency, higher-loss treatment; and so on.  <ins> Delay should be minimized at the expense of efficiently using the available capacity. </ins> This Message may be sent at the system's leisure.  This can be used to signal a preference for less-than-best-effort treatment,  <ins> assign the traffic to a lower effort service, delay sending until lower-cost paths are available, and so on. </ins> 7.2. the maximum Message size that can be received.  This property is read-only. <ins> Congestion control: This boolean property informs about the protocol carrying out congestion control or not.  This property is read-only. </ins> In order to specify Specific Protocol Properties, Transport System implementations may offer applications to attach a set of options to the Preconnection Object, associated with a specific protocol.  For


further parameters may be bound to the Preconnection, and no subsequent establishment call may be made on the Preconnection.  The ConnectionReceived Event occurs when a Remote Endpoint has established a transport-layer connection to this Preconnection (for Connection-oriented transport protocols), or when the first Message if specified) cannot be resolved, or when the application is prohibited from listening by policy.  6.3. Simultaneous peer-to-peer Connection establishment is supported by 11.  RFC-EDITOR: Please remove this section before publication. This document has no Actions for IANA. <del> 12. </del>  This document describes a generic API for interacting with a transport services (TAPS) system.  Part of this API includes

Suggestion to address and in the API doc.
Reading through the API, I noted that there is, amongst the various connection states, a state that is defined but never mentioned again in any of the drafts: "Closing". I am unclear how this state is useful to the application, and how an implementation would determine it. If the connection is closed from the remote side, like a RST, then we'll move right to Closed. If the connection is closed from the local side, I guess we can move into the Closing state before getting an ack that the remote side knows that we've closed, but at that point, I'm not going to be able send or receive any more on the connection anyhow. Do we have a good use case for Closing? If so, can we get more text? If not, can we remove it? For reference, we don't have any corresponding state in our implementation, and it's never come up as a need.
I also don't see a good use case and would rather see it removed. I guess the only case worth even thinking about is the local close one. Here, our "close" semantics don't promise any use for any Closing state anyway, and since (as you say) you can't send or receive anymore on a closing connection, the only point is resource optimization - to avoid opening a new connection when we can wait until the previous one is closed, stuff like that. Even that idea creates a protocol dependency in the application (because some protocols will have delay on closing, but some will just terminate), which IMO makes the state a bad idea.
Right, if you wanted to know you were locally 'closing', you could just know that you called and haven't gotten the state yet.
So for me, the state is between someone called and the sending of the event – might be interesting for concurrent threads to not fall into race issues, but besides this not very useful. Side question: can I clean up all my connection related state in the event handler and is the event guaranteed to fire eventually?
This seems implementation-internal, which indicates to me we can probably get rid of the state. I think that's a question for / which I haven't got around to thinking about yet.
NAME yes, that seems to be the only meaning of the "closing" state, but it doesn't really even provide any threading benefit. If you want to make sure that the connection has closed, just wait for the state—anything after you invoke Close() before the transition to is effectively . With regards to the question about the final event, I think there MUST be some final event that means that everything is cleaned up from the application's perspective so the memory can be released, etc. In the current model, seems to be that correct event. In our implementation, we call this , which only happens after the client of the API calls the / function, to distinguish from a remote-initiated close. So, to be most safe, you may want a state for invalidated/cancelled/never-delivering-any-events-again.
(catching-up on old discussions) I'm confused: we surely need a name for the time between calling and getting the event. What would this be, other than the state? I don't understand how we can remove this.
I agree with NAME Depending on the case, it may not map to much going on in code, you may just get an immediate "Closed" instead after calling Close() - but not always. So then, what's the reason to not give this time period a name, and what else is it but a state?
Maybe we should add a hint to this state, that it is not particularly useful from a programmers perspective, but necessary in the presence of concurrency.
In , I've added a section on ordering of API events (please have a look at it). It talks about guarantees the API makes about which events must occur in which order, but it doesn't talk about guarantees the application must make in its ordering of operations. If we added this, we could talk about the state machine there. It does seem to me that if we add such a discussion, that after is not allowed. This suggests that is a state.
I don't see a need to have a state be set to enforce that must not be called after . If everything is an asynchronous operation, and states are signaled asynchronously, etc, then there still may be a race if you call then immediately; you most likely will not have received the state transition to yet, but it's still an invalid sequence of calls, and the send should fail. NAME while I agree that closing is a state that may exist internally for some protocols, I don't see any value to delivering a signal to an application that they are in the closing state. That's an extra event delivery that necessitates to action and provides no new information. It also is a source of confusion if some protocols use a closing state, but others don't (because UDP should just go to immediately).
I agree with NAME that conceptually you always pass through the Closing state after calling Close() and before receiving the Closed event, even if this transition may sometimes be practically immediate as when you use UDP. Even if you use UDP the TAPS system may need to perform some internal cleanup before triggering the Closed event though. I agree with NAME that you do not need to signal that you are in the Closing state to the application, but I also do not see that this is suggested anywhere. If the application asks what state it is in, then Closing is a possible return value, but that does not create an extra event delivery? So I do not see that the Closing state causes any problem or extra overhead.
I think that state changes do need to be events signaled to the application. Knowing that a connection is ready should not be polled—it is an event. Similar for closed, etc. there should not be a special case for closing, but closing should not be a fired event at all!
Why not have a special case? True, I struggle to imagine other state changes that an application doesn't care about, but so be it - "closing" is no less of a state just because it's not important enough to apps to be signalled to them. (side note: I tag this message as "I offer my opinion but I also don't care much about this")
Why would we not signal to the application? We need some signal that the stack is acting on the request, since this implies that events pertaining to an active connection will no longer be delivered and the application can free up those resources. Especially since, for some protocols, the event could come some significant time after .
Splitting this into and could simplify tge However, I'm a little confused about this: happens after is called, but before happens. ISTM you can still receive things after (i.e., there can still be events fired from pending receive calls), but is it legal to call and ?
NAME If I some data then call , when do I enter the closing state? The event could mean "I got your close request, and will send any outstanding data then close the connection" or it could mean "I've send all outstanding data and am in the process of closing the connection". If the former, I'd expect to get Message Sent events after , if the latter I'd not expect such events.
I thought you enter the Closing state immediately after calling Close() and you can no longer call Send() or Receive() after this point and a Closing<> event would have the former meaning ("I got your close request, and will send any outstanding data then close the connection"). That's why I did not see the need to signal this state transition to the application. I agree we need to signal the application when all outstanding data has been sent and resources can be freed. I thought this would be captured by the Closed<> event. But if the Closed<> event relies on the other end it may be good to separate them. In relation to the need for signaling all state transitions, do we not have a similar situation for Establishing. The Establishing state is I assume entered when an application calls Initiate(), Listen() or Rendezvous(). We do not signal this state transition to the application as far as I understand. We signal when the connection is ready and we move to the Established state, correct?
Linking this to operation ordering as in might make the whole concept of application-exposed connection state make more sense.
having submitted , it's pretty clear that the Established and Closing states are identical from the application's standpoint with respect to which events are received. It may be that Closing is simply equivalent to Established s.t. both "can send" and "can receive" are False. That raises the quesiton of whether or not we should split Established into four states: Bidirectional Send-Only Receive-Only Closing
I don't see the point of this state split proposal - but if you like this better I don't mind it either.
NAME I agree with the overall assessment that Closing is the same as Established, with the distinction that I/O doesn't work. There are other cases in which I/O may not work temporarily in the middle of a connection too, and I wonder if that should be related. For example, if you have a migratable protocol like MPTCP or QUIC, and you lose all interfaces, you can't actually send or receive for a time (although sends and receive calls may be buffered), and you may end up timing out and closing, or you may end up re-establishing over another link.
ahhhh yep. "no I/O available" can actually happen for a range of stack/transport system implementation specific reasons, not just because it's closing, so I do think we need a state but it needs a different name. It also makes sense to keep "can send" and "can receive" separate read-only booleans, because they happen for different reasons which are not necessarily state-related. We have four states then: Establishing: connection is not yet up and running; local allocation is complete and resolution/racing is in progress, or a listener is up and waiting for connections. Established: connection is up and running and I/O is possible in at least one direction; there exists at least one transport stack attached to the connection Dormant: the transport system is maintains state for a connection, but there exist no active transport stacks attached to the connection, and the transport system is not currently attempting to re-establish the connection. Modeling this probably goes hand in hand with the text for . Closed: the transport system has shut down the connection; it is dead, and eligible for GC etc.
and are different though: the connection doesn't come back from , but it does from . Also: is it or do we just transition back to in this case? Is it just for migration?
Dormant might also happen in an environment where the transport system knows it can resume. Migration, yes, but maybe migration during sleep (i.e., you closed your laptop on a download then landed in another country and opened it again -- if the endpoints know about each other they can still resume). As to whether Dormant isn't just Establishing, it depends on whether we need the application to distinguish between "the transport system has at least one candidate it is racing, sends will be buffered and receives will queue" and "the transport system has no candidates, but attempts to send or receive may result in a transition to Establishing". Rereading what I wrote above, I think I got the equivalence between Dormant and Closing wrong -- they're actually totally separate. But I still don't think I see the utility of Closing on its own from a properties management standpoint-- you're Established, you Close (or the remote peer does), then you're Closed (and subject to GC if you're that sort of language :) ).
Whether or not has utility, it exists as a state: for many transports there is a time period between starting to tear down the connection, and having completed tearing down the connection. That's the state, whether or not its useful. I don't see that pretending it doesn't exist is a good idea.
ok i buy that. So we have five: establishing, established, dormant (which might just be establishing), closing and closed (which act the same from the interface's standpoint, but are semantically different...)
It seems this discussion has concluded, without a need to add new text (because we have establishing, established, closing and closed already covered).
As far as I can tell there's no way for an application to indicate that it wants to stop listening for an incoming connection. As with my , this will need a way to know that it's complete, in case there's a race with Preconnection -> ConnectionReceived<> on another thread.
fixed by
An application needs a way to know when its Close() or Abort() call is complete, guaranteeing it won't receive any further events on the connection and all current event processing is done. This is necessary so the application can clean up any application state related to the connection. Note that in a multi-threaded environment, events can be executing on other threads at the same time that the Close() or Abort() call is called; in addition, an application can potentially call Close() or Abort() from within an event processing function itself.
The intention (in the API) is that the Closed<> and Aborted<> events fire after these are complete, whether the close or abort was locally or remotely initiated.
fixed by
The API document needs to specify the relative timing and ordering (i.e., for appropriate thread models, the threading properties) of events. E.g., I think it should be guaranteed that a connection's Received<> event isn't called until its initiating event (Connected<>, ConnectionReceived<>, or RendezvousDone<>) has finished processing; and only one Received<> event is invoked at a time for a given connection, with the events being invoked with the appropriate logical order is required by the connection semantics. Similarly, only one of a connection's Sent<> events should be invoked at a time. However, I don't think there needs to be any restriction against Sent<> and Received<> being invoked simultaneously on the same connection. No other events should be received for a connection after Closed<> or ConnectionError<>. For different connections, It should be allowed for their events to be invoked simultaneously, even if the connections are entangled with each other.
One particular consideration that came up in the May 16th call is that this last event (presumably Closed<> or ConnectionError<> or Aborted<> or whatever) needs to be enqueued to be executed after previous callbacks complete, rather than being executed synchronously on the stack of another callback.
fixed by

further parameters may be bound to the Preconnection, and no subsequent establishment call may be made on the Preconnection. <ins> Listening continues until the global context shuts down, or until the Stop action is performed on the same Preconnection: After Stop() is called, the preconnection can be disposed of. </ins> The ConnectionReceived Event occurs when a Remote Endpoint has established a transport-layer connection to this Preconnection (for Connection-oriented transport protocols), or when the first Message if specified) cannot be resolved, or when the application is prohibited from listening by policy. <ins> A Stopped event occurs after the Preconnection has stopped listening. </ins> 6.3. Simultaneous peer-to-peer Connection establishment is supported by 11. <ins> As this interface is designed to be independent of concurrency model, the details of how exactly actions are handled, and on which threads/ callbacks events are dispatched, are implementation dependent. However, the interface does provide the following guarantees about the ordering of operations: Received<> will never occur on a Connection before a Ready<> event on that Connection, or a ConnectionReceived<> or RendezvousDone<> containing that Connection. No events will occur on a Connection after a Closed<> event, an InitiateError<> or ConnectionError<> on that connection.  To ensure this ordering, Closed<> will not occur on a Connection while other events on the Connection are still locally outstanding (i.e., known to the interface and waiting to be dealt with by the application).  ConnectionError<> may occur after Closed<>, but the interface must gracefully handle the application ignoring these errors. Sent<> events will occur on a Connection in the order in which the Messages were sent (i.e., delivered to the kernel or to the network interface, depending on implementation). 12. </ins> RFC-EDITOR: Please remove this section before publication. This document has no Actions for IANA.  <ins> 13. </ins> This document describes a generic API for interacting with a transport services (TAPS) system.  Part of this API includes


1. Many APIs to perform transport networking have been deployed, perhaps <del> the most widely known and imitated being the BSD socket() interface. The names and functions between these APIs are not consistent, and vary depending on the protocol being used.  For example, sending and receiving on a stream of data is conceptually the same between operating on an unencrypted Transmission Control Protocol (TCP) stream and operating on an encrypted Transport Layer Security (TLS) I-D.ietf-tls-tls13 stream over TCP, but applications cannot use the same socket send() and recv() calls on top of both kinds of connections.  Similarly, terminology for the implementation of protocols offering transport services vary based on the context of </del>  the protocols themselves.  This variety can lead to confusion when trying to understand the similarities and differences between protocols, and how applications can use them effectively. that an application can pass to describe its intent, requirements, prohibitions, and preferences for its networking operations.  For any system that provides generic Transport Services, these <del> properties should primarily offer knobs that are applicable to multiple transports.  Properties may have a large impact on the rest of the aspects of the interface: they can modify how establishment occurs, they can influence the expectations around data transfer, and they determine the set of events that will be supported. </del>  Establishment (establishment) focuses on the actions that an application takes on the basic objects to prepare for data state or updating maintenance options. Termination (termination) focuses on the methods by which data <del> transmission is ceased, and state is torn down in the transport. </del>  The diagram below provides a high-level view of the actions taken during the lifetime of a connection. Endpoint: An Endpoint represents one side of a transport connection.  Endpoints can be Local Endpoints or Remote Endpoints, and respectively represent an identity that the application uses <del> for the source or destination of a connection.  Endpoint can vary in levels of specificity, and can be resolved to more concrete identities. </del>  Remote Endpoint: The Remote Endpoint represents the application's name for a peer that can participate in a transport connection. service name/port. Local Endpoint: The Local Endpoint represents the application's <del> name for itself that it wants to use for transport connections. For example, a local IP address and port. </del>  Path Selection Properties: The Path Selection Properties consist of the options that an application may set to influence the <del> selection of paths between itself and the Remote Endpoint.  These options can come in the form of requirements, prohibitions, or preferences.  Examples of options which may influence path selection include the interface type (such as a Wi-Fi Ethernet connection, or a Cellular LTE connection), characteristics of the path that are locally known like Maximum Transmission Unit (MTU) or discovered like Path MTU (PMTU), or predicted based on cached information like expected throughput or latency. </del>  Protocol Selection Properties: The Protocol Selection Properties consist of the options that an application may set to influence the selection of transport protocol, or to configure the behavior <del> of generic transport protocol features.  These options come in the form of requirements, prohibitions, and preferences.  Examples </del>  include reliability, service class, multipath support, and fast open support. The presence of such Properties does not necessarily require that a specific protocol must be used when a Connection is established, but that if this protocol is employed, a particular set of options <del> should be used. </del>  4.1.3. <del> Initiate is the primary action that an application can take to create a Connection to a remote endpoint, and prepare any required </del>  local or remote state to be able to send and/or receive Messages. For some protocols, this may initiate a client-to-server style handshake; for other protocols, this may just establish local resolution of the Remote Endpoint, occurs in response the Initiate call. <del> Listen is the action of marking a Listener as willing to accept </del>  incoming Connections.  The Listener will then create Connection objects as incoming connections are accepted (events). <del> Rendezvous is the action of establishing a peer-to-peer connection with a remote endpoint.  It simultaneously attempts to initiate a connection to a remote endpoint whilst listening for an incoming </del>  connection from that endpoint.  This corresponds, for example, to <del> a TCP simultaneous open.  The process of identifying options for the connection, such as resolution of the Remote Endpoint, occurs during the Rendezvous call.  If successful, the rendezvous call returns a Connection object to represent the established peer-to- peer connection. </del>  4.1.4. assumed to be ordered within the Message.  If an application does not care about the order in which a peer receives two distinct spans of bytes, those spans of bytes are considered independent <del> Messages.  Messages may or may not be usable if incomplete or corrupted.  Boundaries of a Message may or may not be understood or transmitted by transport protocols.  Specifically, what one application considers to be two Messages sent on a stream-based transport may be treated as a single Message by the application on the other side. Send is the action to transmit a Message or partial Message over a </del>  Connection to a Remote Endpoint.  The interface to Send may include options specific to how the Message's content is to be sent.  Status of the Send operation may be delivered back to the application in an event (events). <del> Receive is an action that indicates that the application is ready to asynchronously accept a Message over a Connection from a Remote </del>  Endpoint, while the Message content itself will be delivered in an event (events).  The interface to Receive may include options specific to the Message that is to be delivered to the 4.1.6. <del> Close is the action an application may take on a Connection to </del>  indicate that it no longer intends to send data, is no longer willing to receive data, and that the protocol should signal this state to the remote endpoint if applicable. <del> Abort is an action the application may take on a Connection to </del>  indicate a Close, but with the additional indication that the transport system should not attempt to deliver any outstanding data. The Transport System Implementation Concepts define the set of objects used internally to a system or library to provide the <del> functionality of transport networking, as required by the abstract interface. Connection Group: A Connections Group is a set of Connections that share properties.  For multiplexing transport protocols, the Connection Group defines the set of Connections that can be multiplexed together. Path: A Path represents an available set of properties of a network route on which packets may be sent or received. Protocol Instance: A Protocol Instance is a single instance of one protocol, including any state it has necessary to establish connectivity or send and receive Messages. Protocol Stack: A Protocol Stack is a set of Protocol Instances (including relevant application, security, transport, or Internet protocols) that are used together to establish connectivity or send and receive Messages.  A single stack may be simple (a single transport protocol instance over IP), or complex (multiple application protocol streams going through a single security and transport protocol, over IP; or, a multi-path transport protocol over multiple transport sub-flows). System Policy: System Policy represents the input from an operating system or other global preferences that can constrain or influence how an implementation will gather candidate paths and protocols (gathering) and race the candidates during establishment (racing).  Specific aspects of the System Policy may apply to all </del>  Connections, or only certain ones depending on the runtime context and properties of the Connection. <del> Cached State: Cached State is the state and history that the implementation keeps for each set of associated endpoints that have been used previously.  This can include DNS results, TLS session state, previous success and quality of transport protocols over certain paths. </del>  4.2.1.

Integrate Gorry's comments from This does not yet address Gorry's two other issues, which cover the background section, or potentially moving the transport implementation earlier in the document.
These issues mostly relate to the ARCH document wording: Section: Introduction It looks to me like we should have a reference after:/the BSD socket() interface./ Section: Background. > Section: Transport Services Architecture and Concepts The Application box seems one character wider than the TAPS boxes. i know apps are important to APIs, but suspect this is just a typo with extra '-' characters! /primarily offer knobs that are applicable to multiple transports/ Understand /knobs/, but suspect someone who does not speak English as a first language could be most puzzled, can we use a different word please? This may be something: /by which data transmission is ceased/ To me this doesn't read as common English and would be: /by which data transmission is stopped/ /by which data transmission ceases/ /Endpoint can vary in levels of specificity/ /Endpoint/ and /vary..specificity/ ... does it really vary, this seems difficult to pin down. Is this better as: /An Endpoint may be specified at various levels/? /that it wants to use for transport connections/ ... surely it does use it, or is more intended (if so, what)? Is it simpler as: /that it uses for transport connections/ /paths between itself and the Remote Endpoint/ I think strictly speaking, it is not between the App and the remote endpoint, but the local and remote endpoints, so could I suggest: /paths between the Local Endpoint and the Remote Endpoint/ /These options can come in the form of / could this be written: /These options can take the form of / -- appears multiple times in the ID. /Examples of options which may influence/ should be: /Examples of options that may influence/ /but that if this protocol is employed, a particular set of options should be used./ understood, is it even clearer (or not) as: /but that if this protocol is employed, a particular set of options should then be used./ In Section: Establishment Actions and Data Transfer Objects and Actions and Termination Actions:: I read this as definitions of terms, but there is no syntax that helps someone understand this. Would it look foolish to include a colon or something after each term (as for the case currently defining message), so it is super clear that this is the defining para? In Section: Establishment Actions /Connection to a remote endpoint/ Could remote endpoint be capitalised here to be consistent with earlier? /for example, to a TCP simultaneous open./ do we need an info reference here. (Sadly, many people will not know what this is). In Section: Data Transfer Objects and Actions / Messages may or may not be usable if incomplete or corrupted. / Agree, but not entirely clear, so I think we should explain to the reader. Could we say: /If a received Message is incomplete or corrupted, it may or may not be usable by a specific application./ In Section: Transport System Implementation Concepts /the functionality of transport networking/ what is transport networking? - either define or rephrase, e.g. as: /the functionality required to provide a transport service across a network/ In Section: Connection Group: /Connection Group: A Connections Group/ mixed single and plural, suggest: /Connection Group: A Connection Group/ /Path: A Path represents an available set of properties of a network route on which packets may be sent or received./ I'm not so happy with this. I know people use terms in various ways but to me this is not a route. I'd prefer something like: /Path: A Path represents an available set of properties that a Local Endpoint may use to send or receive packets with a Remote Endpoint./ I think we need to define /candidate/ in some way, e.g. something like: a path or protocol identified in the gathering stage over which a connection may be attempted in the racing stage/. Also I'd much prefer to actually the two terms (as we currently do in the earlier figure): /Candidate Gathering/ and /Candidate Racing/ The present contents of, provides a set of definitions which are fine... /Transport System Implementation Concepts/ but I really wonder if I could suggest something slightly different. I wonder if we could call this /Transport System Concepts/ and move this way forward in the document to provide a nice early introduction of these terms (with forward refs to sections where these are discussed - if that appeals. Things I'd like to see in this section: /Transport System Implementation Concepts/ Can we add a couple of sentences on gathering and racing, just to explain this using the terms used in the /lifetime of a connection/ figure. Can we create small subsections on /Cached State/ and /System Policy/ I think the subsection section titles below should be changed: /Gathering//Candidate Gathering/ and /Candidate Racing/Candidate Racing/ /### Message Framing, Parsing, and Serialization {#framing}/ This subsection appears out-of sequence. (Actually this could really help the reader if made a little easier to read and placed ahead of discussing the details). >
Closing this, we can discuss details in PR
lgtm

1. Many APIs to perform transport networking have been deployed, perhaps  <ins> the most widely known and imitated being the BSD socket() POSIX interface.  The names and functions between these APIs are not consistent, and vary depending on the protocol being used.  For example, sending and receiving on a stream of data is conceptually the same between operating on an unencrypted Transmission Control Protocol (TCP) stream and operating on an encrypted Transport Layer Security (TLS) I-D.ietf-tls-tls13 stream over TCP, but applications cannot use the same socket send() and recv() calls on top of both kinds of connections.  Similarly, terminology for the implementation of protocols offering transport services vary based on the context of </ins> the protocols themselves.  This variety can lead to confusion when trying to understand the similarities and differences between protocols, and how applications can use them effectively. that an application can pass to describe its intent, requirements, prohibitions, and preferences for its networking operations.  For any system that provides generic Transport Services, these  <ins> properties should primarily be defined to apply to multiple transports.  Properties may have a large impact on the rest of the aspects of the interface: they can modify how establishment occurs, they can influence the expectations around data transfer, and they determine the set of events that will be supported. </ins> Establishment (establishment) focuses on the actions that an application takes on the basic objects to prepare for data state or updating maintenance options. Termination (termination) focuses on the methods by which data  <ins> transmission is stopped, and state is torn down in the transport. </ins> The diagram below provides a high-level view of the actions taken during the lifetime of a connection. Endpoint: An Endpoint represents one side of a transport connection.  Endpoints can be Local Endpoints or Remote Endpoints, and respectively represent an identity that the application uses  <ins> for the source or destination of a connection.  An Endpoint may be specified at various levels, and an Endpoint with wider scope (such as a hostname) can be resolved to more concrete identities (such as IP addresses). </ins> Remote Endpoint: The Remote Endpoint represents the application's name for a peer that can participate in a transport connection. service name/port. Local Endpoint: The Local Endpoint represents the application's  <ins> name for itself that it uses for transport connections.  For example, a local IP address and port. </ins> Path Selection Properties: The Path Selection Properties consist of the options that an application may set to influence the  <ins> selection of paths between the Local Endpoint and the Remote Endpoint.  These options can take the form of requirements, prohibitions, or preferences.  Examples of options that may influence path selection include the interface type (such as a Wi- Fi Ethernet connection, or a Cellular LTE connection), characteristics of the path that are locally known like Maximum Transmission Unit (MTU) or discovered like Path MTU (PMTU), or predicted based on cached information like expected throughput or latency. </ins> Protocol Selection Properties: The Protocol Selection Properties consist of the options that an application may set to influence the selection of transport protocol, or to configure the behavior  <ins> of generic transport protocol features.  These options can take the form of requirements, prohibitions, and preferences.  Examples </ins> include reliability, service class, multipath support, and fast open support. The presence of such Properties does not necessarily require that a specific protocol must be used when a Connection is established, but that if this protocol is employed, a particular set of options  <ins> should then be used.. </ins> 4.1.3.  <ins> Initiate: The primary action that an application can take to create a Connection to a Remote Endpoint, and prepare any required </ins> local or remote state to be able to send and/or receive Messages. For some protocols, this may initiate a client-to-server style handshake; for other protocols, this may just establish local resolution of the Remote Endpoint, occurs in response the Initiate call.  <ins> Listen: The action of marking a Listener as willing to accept </ins> incoming Connections.  The Listener will then create Connection objects as incoming connections are accepted (events).  <ins> Rendezvous: The action of establishing a peer-to-peer connection with a Remote Endpoint.  It simultaneously attempts to initiate a connection to a Remote Endpoint whilst listening for an incoming </ins> connection from that endpoint.  This corresponds, for example, to  <ins> a TCP simultaneous open RFC0793.  The process of identifying options for the connection, such as resolution of the Remote Endpoint, occurs during the Rendezvous call.  If successful, the rendezvous call returns a Connection object to represent the established peer-to-peer connection. </ins> 4.1.4. assumed to be ordered within the Message.  If an application does not care about the order in which a peer receives two distinct spans of bytes, those spans of bytes are considered independent  <ins> Messages.  If a received Message is incomplete or corrupted, it may or may not be usable by certain applications.  Boundaries of a Message may or may not be understood or transmitted by transport protocols.  Specifically, what one application considers to be two Messages sent on a stream-based transport may be treated as a single Message by the application on the other side. Send: The action to transmit a Message or partial Message over a </ins> Connection to a Remote Endpoint.  The interface to Send may include options specific to how the Message's content is to be sent.  Status of the Send operation may be delivered back to the application in an event (events).  <ins> Receive: An action that indicates that the application is ready to asynchronously accept a Message over a Connection from a Remote </ins> Endpoint, while the Message content itself will be delivered in an event (events).  The interface to Receive may include options specific to the Message that is to be delivered to the 4.1.6.  <ins> Close: The action an application may take on a Connection to </ins> indicate that it no longer intends to send data, is no longer willing to receive data, and that the protocol should signal this state to the remote endpoint if applicable.  <ins> Abort: The action the application may take on a Connection to </ins> indicate a Close, but with the additional indication that the transport system should not attempt to deliver any outstanding data. The Transport System Implementation Concepts define the set of objects used internally to a system or library to provide the  <ins> functionality required to provide a transport service across a network, as required by the abstract interface. Connection Group: A set of Connections that share properties.  For multiplexing transport protocols, the Connection Group defines the set of Connections that can be multiplexed together. Path: Represents an available set of properties that a Local Endpoint may use to send or receive packets with a Remote Endpoint. Protocol Instance: A single instance of one protocol, including any state it has necessary to establish connectivity or send and receive Messages. Protocol Stack: A set of Protocol Instances (including relevant application, security, transport, or Internet protocols) that are used together to establish connectivity or send and receive Messages.  A single stack may be simple (a single transport protocol instance over IP), or complex (multiple application protocol streams going through a single security and transport protocol, over IP; or, a multi-path transport protocol over multiple transport sub-flows). Candidate Path: One path that is available to an application and conforms to the Path Selection Properties and System Policy. Candidate Paths are identified during the gathering phase (gathering) and may be used during the racing phase (racing). Candidate Protocol Stack: One protocol stack that may be used by an application for a connection, of which there may be several. Candidate Protocol Stacks are identified during the gathering phase (gathering) and may be started during the racing phase (racing). System Policy: Represents the input from an operating system or other global preferences that can constrain or influence how an implementation will gather candidate paths and protocol stacks (gathering) and race the candidates during establishment (racing). Specific aspects of the System Policy may apply to all </ins> Connections, or only certain ones depending on the runtime context and properties of the Connection.  <ins> Cached State: The state and history that the implementation keeps for each set of associated endpoints that have been used previously.  This can include DNS results, TLS session state, previous success and quality of transport protocols over certain paths. </ins> 4.2.1.


5.3.  Common parameters such as TLS ciphersuites are known to <del> implementations.  Clients SHOULD use common safe defaults for these </del>  values whenever possible.  However, as discussed in I-D.ietf-taps- transport-security, many transport security protocols require specific security parameters and constraints from the client at the time of configuration and actively during a handshake.  These <del> configuration parameters are created as follows </del>  Security configuration parameters and sample usage follow: Local identity and private keys: Used to perform private key operations and prove one's identity to the Remote Endpoint. (Note, if private keys are not available, e.g., since they are <del> stored in HSMs, handshake callbacks MUST be used.  See below for </del>  details.) Supported algorithms: Used to restrict what parameters are used by underlying transport security protocols.  When not specified, <del> these algorithms SHOULD default to known and safe defaults for the </del>  system.  Parameters include: ciphersuites, supported groups, and signature algorithms. <del> Session cache: Used to tune cache capacity, lifetime, re-use, and eviction policies, e.g., LRU or FIFO. </del>  <del> Pre-shared keying material: Used to install pre-shared keying material established out-of-band.  Each pre-shared keying material is associated with some identity that typically identifies its use or has some protocol-specific meaning to the Remote Endpoint. </del>  Security decisions, especially pertaining to trust, are not static. <del> Thus, once configured, parameters must also be supplied during live handshakes.  These are best handled as client-provided callbacks. Security handshake callbacks include: </del>  Trust verification callback: Invoked when a Remote Endpoint's trust must be validated before the handshake protocol can proceed. is required, e.g., when local authentication is requested by a remote. <del> Like transport parameters, security parameters are inherited during cloning (see groups). </del> 6. Before a Connection can be used for data transfer, it must be

Make sure all the examples in the API and interface drafts use constants agnostic to argument type and syntax. From Dave Thaler: On 2 May 2018, at 03:07, Dave Thaler wrote:
will fix this when it lands
Addressed by .
Security properties should be synchronized with (the next rev of) URL
Addressed by .
lgtm

5.3. <ins> Most security parameters, e.g., TLS ciphersuites, local identity and private key, etc., may be configured statically.  Others are dynamically configured during connection establishment.  Thus, we partition security parameters and callbacks based on their place in the lifetime of connection establishment.  Similar to transport parameters, both parameters and callbacks are inherited during cloning (see groups). 5.3.1. </ins> Common parameters such as TLS ciphersuites are known to  <ins> implementations.  Clients should use common safe defaults for these </ins> values whenever possible.  However, as discussed in I-D.ietf-taps- transport-security, many transport security protocols require specific security parameters and constraints from the client at the time of configuration and actively during a handshake.  These  <ins> configuration parameters are created as follows: </ins> Security configuration parameters and sample usage follow: Local identity and private keys: Used to perform private key operations and prove one's identity to the Remote Endpoint. (Note, if private keys are not available, e.g., since they are  <ins> stored in HSMs, handshake callbacks must be used.  See below for </ins> details.) Supported algorithms: Used to restrict what parameters are used by underlying transport security protocols.  When not specified,  <ins> these algorithms should default to known and safe defaults for the </ins> system.  Parameters include: ciphersuites, supported groups, and signature algorithms.  <ins> Session cache management: Used to tune cache capacity, lifetime, re-use, and eviction policies, e.g., LRU or FIFO.  Constants and policies for these interfaces are implementation-specific. </ins>  <ins> Pre-Shared Key import: Used to install pre-shared keying material established out-of-band.  Each pre-shared keying material is associated with some identity that typically identifies its use or has some protocol-specific meaning to the Remote Endpoint. 5.3.2. </ins> Security decisions, especially pertaining to trust, are not static.  <ins> Once configured, parameters may also be supplied during connection establishment.  These are best handled as client-provided callbacks. Security handshake callbacks that may be invoked during connection establishment include: </ins> Trust verification callback: Invoked when a Remote Endpoint's trust must be validated before the handshake protocol can proceed. is required, e.g., when local authentication is requested by a remote.  6. Before a Connection can be used for data transfer, it must be