To generate the report, the client begins by initializing the report nonce.  Specifically, the client first sets "Nonce.rand" to 16 random bytes output from a cryptographically secure random number generator. <del> It then sets "Nonce.time" to the current UNIX time rounded down to the nearest multiple of "min_batch_duration".  This truncation is done to ensure that the report's timestamp cannot be used to link the report back to the originating client. </del>  The client then finishes the report generation by sharding its measurement into a sequence of input shares as specified by the VDAF

We might want to consider renaming to something based on what it's actually doing, which is helping with replay protection. Maybe ?
~Wait, what? Replay is enforced on the basis of Nonce in its entirety, which currently consists of a (1) timestamp and (2) a random value.~ Chatted to clear this up in Slack
Not sure if DAP enforces precision of : But with fine-grained timestamp, it is easier for attacker to associate a report with the user sent that report. For example, if timestamp is accurate to second, and leader is compromised, attacker can compare users that uploaded at a given second (perhaps from a routing server) and reports' timestamp, then figure out if a particular user has contributed to a task. With a coarse-grained timestamp, however, we can minimise this threat with a secure routing server. Anti-replay can still be satisfied with the UUID part of Nonce.
This is a valid concern. One reasonable change would be to modify the report timestamp to align with the batch window start (minbatchduration), and then bump the size of the nonce to offset the increase change of collisions. Specifically, this would mean we update the Nonce like so: NAME would this work? NAME NAME NAME thoughts?
NAME I think this should work.
Great! I'll prep a PR.
I would add the can also be of any length, so aligning Nonce with it will make it coarse-grained for use cases where is large, say in hours, but won't be as effective for very small , but I think that should probably be addressed separately.
I'm happy with this change, though 16 bytes will be sufficient since clients are supposed to choose this at random. (Note that we chose 32 bytes for the various "IDs" (i.e., task ID, aggregation-job ID) because we anticipated some deployments using a hash of something as the ID.
Yep
We currently have an open issue for tracking the consensus protocol needed for this system to work. In working through this text, we should make note of what sort of synchronization we assume between different participants of the system.
In the 3/17 design call we agreed that for purposes of the design doc and probably the initial deployments of Prio, we would assume that shared parameters (e.g., arithmetic circuits, primes, aggregation identifiers) would be exchanged between all parties (i.e., clients, aggregators, collector, leader) before the start of the protocol over some unspecified secure channel. However, eventually, it should be possible for a client that wishes to use Prio to programmatically: discover participating servers (i.e., a directory of aggregators) discover the capabilities of participating servers (i.e., what algorithms do they support for secret sharing or validity proofs or aggregations? What optional protocol features do they support?) configure a Prio aggregation (i.e., select some number of aggregation servers, getting a unique handle, configuring field size, etc.)
One thing that's still TBD is how to choose the joint randomness. It could be as simple as having the leader choose a seed for a PRG and distributing it to the aggregators over secure channels. It could be as complicated as querying at some point after the shares are received by the aggregators.
To close the loop on this issue: the PDA parameters are distributed to all entities out-of-band.
Chris W. and I hashed this out . Chris convinced me that as written, the rounding of is optional for the client, and aggregators don't need to do anything special to handle unrounded nonces: if that happens aggregators can still use the unrounded nonce to correctly identify a bucket to put the report in, so this is something that a client should do in order to improve privacy, but nothing breaks if they fail to. With that in mind, I don't think we need new text to explain what aggregators should do with nonces that aren't rounded correctly, nor does the new text for the client need a MUST.

To generate the report, the client begins by initializing the report nonce.  Specifically, the client first sets "Nonce.rand" to 16 random bytes output from a cryptographically secure random number generator.  <ins> It then sets "Nonce.time" to the number of seconds elapsed since the start of the UNIX epoch, rounded down to the nearest multiple of "min_batch_duration".  This truncation is done to ensure that the report's timestamp cannot be used to link the report back to the originating client. </ins> The client then finishes the report generation by sharding its measurement into a sequence of input shares as specified by the VDAF


The type of each measurement. <del> The aggregation function to compute (e.g., sum, mean, etc.) and an optional aggregation parameter. </del>  The set of aggregators and necessary cryptographic keying material to use. "Report.encrypted_input_shares" is equal to the index of the aggregator in the task's "aggregator_endpoints" to which the AggregateInitializeReq is being sent.  The "agg_param" field is an <del> opaque, VDAF-specific aggregation parameter.  The "job_id" parameter contains the leader's chosen AggregationJobID. </del>  Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its "batch_interval", the request's batch interval. "agg_param", an aggregation parameter for the VDAF being executed.  Depending on the VDAF scheme and how the leader is configured, the leader and helper may already have prepared all the reports falling "batch_interval" is the batch interval of the request. "agg_param", an aggregation parameter for the VDAF being executed.  "report_count" is the number of reports included in the aggregation.

Similar to , it should not be possible for a share to be included in more than one batch. The timestamp currently gives us some way of ensuring that reports map to at most batch, but not a way of ensuring that reports aren't included two overlapping batch windows. It's not immediately clear if include a single report in two overlapping batch windows is a problem, or if a problem only arises when these windows are disjoint. We should probably clarify what are the inter-batch requirements here and what sort of replay prevention is required.
The aggregators (either the leader or helper) can prevent the collector from getting outputs from overlapping batches by remembering batch intervals. For example, suppose the leader responded successfully to a collect request for time interval . Then suppose that, some time later, the collector requests output shares for , where . The potential privacy violation occurs because the sets of reports in each batch are non-disjoint. To prevent this, the leader might respond to the second request with an error, or it might just include the reports in .
If aggregators are meant to use the report timestamp to protect against replay, then I think it needs to appear in the AEAD-ed message constructed by the client (i.e. ). Otherwise a leader could insert arbitrary timestamps into and trick helpers into exceeding privacy budgets or double counting inputs. Additionally, a batching client could insert arbitrary values into .
I created a PR that addresses this issue partially. What's left to do is allow multiple collect requests over the same batch, as required by the PDA protocol (e.g., Hits). I have an idea for how to do this, I'll follow up with a PR.
Closed by URL
LGTM -- thanks!

The type of each measurement.  <ins> The aggregation function to compute (e.g., sum, mean, etc.). </ins> The set of aggregators and necessary cryptographic keying material to use. "Report.encrypted_input_shares" is equal to the index of the aggregator in the task's "aggregator_endpoints" to which the AggregateInitializeReq is being sent.  The "agg_param" field is an  <ins> opaque, VDAF-specific aggregation parameter provided during a collection flow.  The "job_id" parameter contains the leader's chosen AggregationJobID. [[OPEN ISSUE: Check that this handling of "agg_param" is appropriate when the definition of Poplar is done.]] </ins> Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its "batch_interval", the request's batch interval. "agg_param", an aggregation parameter for the VDAF being executed. <ins> This is the same value as in "AggregateInitializeReq" (see leader- init). </ins> Depending on the VDAF scheme and how the leader is configured, the leader and helper may already have prepared all the reports falling "batch_interval" is the batch interval of the request. "agg_param", an aggregation parameter for the VDAF being executed. <ins> This is the same value as in "AggregateInitializeReq" (see leader- init) and in "CollectReq" (see collect-init). </ins> "report_count" is the number of reports included in the aggregation.


prevents additional reports from being aggregated after its batch has already been collected. <del> Check if the report has already been aggregated.  If this check fails, the input share is marked as invalid with the error "report-replayed".  This is the case if the report was used in a previous aggregate request and is therefore a replay.  An aggregator may also choose to mark an input share as invalid with the error "report-dropped" under the conditions prescribed in anti-replay. </del>  If both checks succeed, the input share is not marked as invalid.

Previously, report-replayed would be returned if the same client report was aggregated more than once. However, some VDAFs (e.g. poplar1) will require repeatedly aggregating a given client report with varying aggregation parameters. This commit relaxes the condition on which report-replayed will be returned to allow aggregating the same client report multiple times, as long as each aggregation has a distinct aggregation parameter.
This is odd because the aggregation parameter is opaque and it's not clear if any change to it should allow aggregating a report again. For example, if aggregation is allowed only once for privacy reasons but some kind of shared random number is needed as agg_param this will be an issue. I think checking if another aggregation is allowed has to be done depending on the VDAF and the task.
There is a separate check, under the section "Validating Batch Parameters", that each report is used in at most aggregate shares. This should be set to 1 for Prio3 and VDAFs like you describe. If for a task is 1, then the present change will make no difference, but if for a task is greater than one, we need this change in order to allow that second batch.
Currently, the report share error is returned "if the report has already been aggregated" (). Some VDAFs, like , will require repeatedly aggregating the same client reports with a varying aggregation parameter. I think the specification language would effectively block this use-case currently. I think the fix is to return a error if the report has already been aggregated with the same aggregation parameter. Note that this change would not allow a malicious leader to potentially aggregate a different set of reports with a different aggregation parameter: the existing error would be returned if a client attempted to upload another report after the first collection attempt. (Given the small size of the proposed fix, I'll send a PR shortly, but discussion on this issue/the WG mailing list takes precedence -- I'll update the PR to follow any changes from discussion.)
Thanks NAME -- this change LGTM.

prevents additional reports from being aggregated after its batch has already been collected.  <ins> Check if the report has already been aggregated with this aggregation parameter.  If this check fails, the input share is marked as invalid with the error "report-replayed".  This is the case if the report was used in a previous aggregate request and is therefore a replay.  An aggregator may also choose to mark an input share as invalid with the error "report-dropped" under the conditions prescribed in anti-replay. </ins> If both checks succeed, the input share is not marked as invalid.


(described in hpke-config); see compliance for information about the HPKE configuration algorithms. <del> "vdaf_verify_param": The aggregator's VDAF verification parameter output by the setup algorithm computed jointly by the aggregators before the start of the DAP protocol VDAF).  [OPEN ISSUE: This is yet to be specified.  See issue#161.] </del>  The helper stores a bearer token used to authenticate HTTP requests from the leader.  Likewise, the leader stores a bearer token to Each aggregator runs this procedure for a given input share with corresponding nonce as follows: <del> "vdaf_verify_param" is the public VDAF parameter, and "agg_param" is the opaque aggregation parameter.  If either step fails, the aggregator marks the report as invalid with error "vdaf-prep-error". </del>  <del> Otherwise, the value "out" is interpreted as follows.  If this is the last round of the VDAF, then "out" is the aggregator's output share. Otherwise, "out" is the pair "(prep_state, prep_msg)". </del>  4.3.2. each share in the candidate report set until the underlying VDAF moves into a terminal state, yielding an output share for all aggregators or an error.  This phase may involve multiple rounds of <del> interaction depending on the underlying VDAF parameters.  Each round trip is initiated by the leader. </del>  4.3.2.1.

The next VDAF draft is out. It includes some minor changes that we'll need to adopt here. URL

(described in hpke-config); see compliance for information about the HPKE configuration algorithms.  <ins> "vdaf_verify_key": The VDAF verification key shared by the aggregators.  This key is used in the aggregation sub-protocol (aggregate-flow).  [OPEN ISSUE: The manner in which this key is distributed may be relevant to the VDAF's security.  See issue#161.] </ins> The helper stores a bearer token used to authenticate HTTP requests from the leader.  Likewise, the leader stores a bearer token to Each aggregator runs this procedure for a given input share with corresponding nonce as follows:  <ins> "vdaf_verify_key" is the VDAF verification key shared by the aggregators; "agg_id" is the aggregator ID ("0x00" for the Leader and "0x01" for the helper); and "agg_param" is the opaque aggregation parameter distributed to the aggregtors by the collector. </ins>  <ins> If either step fails, the aggregator marks the report as invalid with error "vdaf-prep-error".  Otherwise, the value "out" is interpreted as follows.  If this is the last round of the VDAF, then "out" is the aggregator's output share.  Otherwise, "out" is the pair "(prep_state, prep_msg)". </ins> 4.3.2. each share in the candidate report set until the underlying VDAF moves into a terminal state, yielding an output share for all aggregators or an error.  This phase may involve multiple rounds of  <ins> interaction depending on the underlying VDAF.  Each round trip is initiated by the leader. </ins> 4.3.2.1.


HTTP GET requests to the collect job's URI with HTTP status code 200 OK and a body consisting of a "CollectResp": <del> The "encrypted_agg_shares" field is the vector of encrypted aggregate shares.  They MUST appear in the same order as the aggregator endpoints list of the task parameters. </del>  If obtaining aggregate shares fails, then the leader responds to subsequent HTTP GET requests to the collect job URI with an HTTP

This adds a field to collect responses with the number of reports that contributed to the aggregation. Collectors will likely want this count to aid in the interpretation of aggregate results. (see ) There is also a change in draft-irtf-cfrg-vdaf-02 that opens the door for VDAFs that would require the report count during unsharding. (see cfrg/draft-irtf-cfrg-vdaf) Note that while this field is not part of the adjacent s, it will be protected by to the confidentiality of HTTPS and the leader-collector bearer token authentication.
I'm open to pushing it into the ciphertext, but my thinking was that a tampered reportcount would only affect robustness, not privacy, and thus we only need to arrive at the correct reportcount when all aggregators follow the protocol.
NAME can you please rebase? Sorry!
Collectors will want to know how many inputs are included in the aggregations they receive, for a number of reasons: they may wish to compute a mean over a privately aggregated sum, or track metrics on how many clients are providing inputs. struct should include a count of how many contributions went into an aggregation.
The linked PR only added a to the structure, which communicates it to the helper. I'm reopening this issue because the collector does not yet receive this information. In addition to the statistics-related motivation above, a recent PR to the VDAF spec, cfrg/draft-irtf-cfrg-vdaf, expects the number of reports to be available when unsharding.
This was completed in .
OK, seems reasonable to me.

HTTP GET requests to the collect job's URI with HTTP status code 200 OK and a body consisting of a "CollectResp":  <ins> "report_count" is the number of reports included in the aggregation. "encrypted_agg_shares" is the vector of encrypted aggregate shares.  They MUST appear in the same order as the aggregator endpoints list of the task parameters. </ins> If obtaining aggregate shares fails, then the leader responds to subsequent HTTP GET requests to the collect job URI with an HTTP


subsequent HTTP GET requests to the collect job URI with an HTTP error status and a problem document as described in errors. <del> The collector may send an HTTP DELETE request to the collect job URI, </del>  to which the leader MUST respond with HTTP status 204 No Content. The leader MAY respond with HTTP status 204 No Content for requests to a collect job URI which has not received a DELETE request, for

The previous wording required the leader to store collect results until the collector sent an explicit DELETE request to the collect job's URI. Now, the relevant text is reworded to allow the leader to remove collect results for other reasons (for example, a leader can now garbage-collect old results after a suitable period of time). This fell out of discussion on .
OK -- I left the motivating example out as I didn't want to be too prescriptive to implementations. I would also be interested in comments from folks more experienced with RFC writing; I've added the motivating example for now.
When the Collector issues a collect request to the Leader, the Leader responds with a URI the Collector can poll later on to get the (encrypted) aggregate result. This is done via an HTTP GET to the collect URI returned by the Leader. My understanding is that the Leader is free to remove the aggregate results from storage thereafter. (See URL for discussion.) The spec doesn't require this request to be authenticated, so it's possible for someone to cause the Leader to delete this state before the Collector has a chance to fetch it. This issue can be solved by: requiring the Leader to store results for some period of time; making the collect URI hard to predict (e.g., by sticking a random number in the URI string); or requiring authentication for this endpoint. (2.) seems easiest to implement, but may be tricky to spell. Any thoughts on this?
That's not what says: That text is consistent with . So I argue the current situation is (1). We do have to solve the problem of how long a leader can wait before tossing collect results -- or maybe we don't, since ACME doesn't bother to. But we could use this issue to discuss that question. re: (2): I think we should avoid as much as possible imposing requirements on the construction of URIs and relative paths[^1]. An implementation should feel free to generate unpredictable collect job URIs (Janus randomly generates a v4 UUID) but I don't think DAP should mandate it, just so we don't have to write the text to spell it out. re: (3): I have a hard time imagining a deployment where the endpoint and collect job URIs wouldn't be authenticated[^2], but my position on authn in DAP is that we shouldn't specify anything, allowing implementations to use any authn scheme they want that is composable with HTTP. At most we ought to state requirements (e.g., "the channel between aggregators must be mutually authenticated and confidential") but avoid mandating how those requirements are met (e.g., "the aggregators must use mTLS"). which in turn guides clients to the URIs to hit to create orders, or accounts, or whatever. That said, we shouldn't do exactly what ACME does because (1) AFAIK the BCP that mandated this part of ACME's design has been revised and (2) forcing a directory lookup (that is, an extra HTTP round trip) every time an ACME subscriber wants to issue a new cert (roughly once per 90 days) is not so bad, but forcing a directory lookup every time a DAP client wants to upload a measurement seems wasteful. [^2]: For argument's sake: maybe a deployment wants to make the results of some aggregation public, but in such settings, the collector could combine the aggregate shares into results and then stick them into a CDN instead of making it the leader's problem. Not to mention that the collector is probably most capable of transforming aggregate results into some other format that's better suited for public consumption.
I agree with this. I don't think DAP should specify or even mandate authentication, for the reasons Tim mentioned, but also because the aggregate outputs are encrypted only to the intended collector. Giving those encrypted aggregate shares to an unauthorized party seems fine based on the properties of HPKE.
FWIW, the current DAP spec text: implies (at least to my reading) that the leader must indeed retain the results until an explicit is sent from the collector but inconsistent (IMO) with the current specification text. If I'm not missing anything, I can prep a small PR fixing this up. , the leader could choose to send a representation indicating that the results are gone; but strictly following the text of DAP, the leader would still be required to retain the results. Given that these results would be inaccessible from an outside observer, this is a somewhat academic distinction -- IMO another good reason to make a change here.
NAME we took a change here to improve the text discussing how long leaders should hold on to collect job results, and I think questions about authentication of the collect job URIs is captured by . Are there any outstanding concerns about whether polling collect job URIs causes state changes?
Nope :)
Approved as this change achieves the intended mechanical purpose, though I agree with Simon's point about clarity, and others more experienced in the RFC writing style might comment further.

subsequent HTTP GET requests to the collect job URI with an HTTP error status and a problem document as described in errors.  <ins> The collector can send an HTTP DELETE request to the collect job URI, </ins> to which the leader MUST respond with HTTP status 204 No Content. The leader MAY respond with HTTP status 204 No Content for requests to a collect job URI which has not received a DELETE request, for


coerce an aggregator into diverting from the protocol specified (e.g., by divulging its input shares to the attacker). <del> In the presence of this adversary, DAP aims to achieve the following high-level secure aggregation goals: Privacy.  Clients trust that some aggregator is honest.  That is, as long as at least one aggregator executes the protocol faithfully, the parties learn nothing beyond the aggregate result (i.e., the output of the aggregation function computed over the honest measurements). Correctness.  The collector trusts that the aggregators execute the protocol correctly.  That is, as long as the aggregators execute the protocol faithfully, a malicious client can skew the aggregate result only by reporting a false (untruthful) measurement.  The result cannot be influenced in any other way. </del>  Currently, the specification does not achieve these goals.  In particular, there are several open issues that need to be addressed [[OPEN ISSUE: The threat model for Prio -- as it's described in the original paper and BBCGGI19 -- considers <del> a malicious client (attacking soundness) </del>  a malicious subset of aggregators (attacking privacy).  In <del> particular, soundness isn't guaranteed if any one of the aggregators </del>  is malicious; in theory it may be possible for a malicious client and <del> aggregator to collude and break soundness.  Is this a contingency we </del>  need to address?  There are techniques in BBCGGI19 that account for this; we need to figure out if they're practical.]]

NAME let's squash-then-merge this change.
contains a nice definition and discussion of the notions of privacy (meaning protecting any individual client's measurement from being seen) and robustness (meaning that the aggregates delivered to a collector are accurate aggregations over the inputs). DAP's security considerations has definitions of privacy and correctness, and we refer to robustness and soundness in a few places. We should align our definitions with VDAF (or just refer to VDAF's definitions) and consistently use the term "robustness" throughout DAP.
PR, please!
I'm going to get this done for draft-02, hopefully early next week.

coerce an aggregator into diverting from the protocol specified (e.g., by divulging its input shares to the attacker).  <ins> In the presence of this adversary, DAP aims to achieve the privacy and robustness security goals described in VDAF's Security Considerations section. </ins> Currently, the specification does not achieve these goals.  In particular, there are several open issues that need to be addressed [[OPEN ISSUE: The threat model for Prio -- as it's described in the original paper and BBCGGI19 -- considers  <ins> a malicious client (attacking robustness) </ins> a malicious subset of aggregators (attacking privacy).  In  <ins> particular, robustness isn't guaranteed if any one of the aggregators </ins> is malicious; in theory it may be possible for a malicious client and  <ins> aggregator to collude and break robustness.  Is this a contingency we </ins> need to address?  There are techniques in BBCGGI19 that account for this; we need to figure out if they're practical.]]


Aggregators can control this cached lifetime with the Cache-Control header, as follows: <del> Clients SHOULD follow the usual HTTP caching RFC7234 semantics for </del>  key configurations. Note: Long cache lifetimes may result in clients using stale HPKE

I-D Nits complains of outdated reference for: ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) We should update tit.
Thanks!

Aggregators can control this cached lifetime with the Cache-Control header, as follows:  <ins> Clients SHOULD follow the usual HTTP caching RFC9111 semantics for </ins> key configurations. Note: Long cache lifetimes may result in clients using stale HPKE


The time-interval query type is described in time-interval-query; the fixed-size query type is described in fixed-size-query.  Future specifications can introduce new query types as needed (see query- <del> type-reg.  A query includes parameters used by the Aggregators to </del>  select a batch of reports specific to the given query type.  A query is defined as follows:

NAME I think we should wait for one of NAME or NAME before merging.
s4.1 includes the following: Future specifications can introduce new query types as needed. That means this I-D needs an IANA consideration to address registering a query type.

The time-interval query type is described in time-interval-query; the fixed-size query type is described in fixed-size-query.  Future specifications can introduce new query types as needed (see query-  <ins> type-reg).  A query includes parameters used by the Aggregators to </ins> select a batch of reports specific to the given query type.  A query is defined as follows:


known HPKE configuration.  When clients receive an 'outdatedConfig' error, they SHOULD invalidate any cached aggregator "HpkeConfigList" and retry with a freshly generated Report.  If this retried report <del> does not succeed, clients MUST abort and discontinue retrying. </del>  The Leader MUST ignore any report pertaining to a batch that has <del> already been collected.  (See input-share-validation for details.) </del>  Otherwise, comparing the aggregate result to the previous aggregate <del> result may result in a privacy violation.  (Note that the Helpers enforce this as well.)  The Leader MAY ignore any reports whose timestamp is past the task's "task_expiration".  When it does so, the leader SHOULD abort the upload protocol and alert the client with error "reportTooLate".  Client MAY choose to opt out of the task if its own clock has passed "task_expiration". </del>  Leaders can buffer reports while waiting to aggregate them.  The leader SHOULD NOT accept reports whose timestamps are too far in the future.  Implementors MAY provide for some small leeway, usually no more than a few minutes, to account for clock skew.  If the leader rejects a report for this reason, it SHOULD abort the upload protocol <del> and alert the client with error "reportTooEarly". </del>  If the Report contains an unrecognized extension, or of two extensions have the same ExtensionType, then the Leader MAY abort the

We didn't have any text that actually instructed leaders to reject uploads deemed to be replays based on their nonce. The text in {{anti-replay}} says leaders should do this, but then refers to the uploads section, which doesn't contain any text about checking nonces or what the leader should do. We now define a new problem document type "reportRejected", which replaces the previous "reportTooLate", previously used in the case where the leader detects a report pertaining to an already-collected batch. The rationale for using a less specific error type for these two cases is twofold: In these cases, we don't expect the client to do anything but give up on the report, so nothing more specific than a generic rejection is needed; If we did provide more specific error codes, we'd be giving attackers an oracle for what nonces have already been uploaded or what reports have already been collected. While I was in this text, I also: resolved . clarified that the client should do if it gets "reportTooEarly". If the client shouldn't do anything special in that situation, then I argue the error should be the generic "reportRejected". downgraded a MUST to a SHOULD in the case where the client gets "outdatedConfig" twice. There's no way the protocol can meaningfully enforce that MUST, and if the client continues to upload reports referencing an outdated or non-existent HPKE config, the leader can just continue to ignore the reports.
I don't think we should hold up draft-02 for this, unless it takes long enough for the fixed-task stuff to land anyway.
Responding to David-- I see your point, and maybe we need to invent some new terms to clearly distinguish between anti-replay as in "you tried to use the same report ID with different timestamps or content" and anti-replay as in "you tried to aggregate a single report too many times". But I do think the helper needs to defend against the former, not just the latter, and so the {{anti-replay}} section does seem pertinent to it. In my latest commit I added text to this effect. I agree. The API changes proposed in would mean that multiple PUT requests for the same report will all yield 200 OK (or 201 Created, I guess). So if you send the same multiple times, that's OK. But I do think we want to enable (though not require, to Chris' point about allowing implementations to defer checking anti-replay to the aggregation sub-protocol) leaders alert clients if it's detected that they've tried to use an existing report ID with a new timestamp or input share.
NAME can you please rebase this?
I've updated this change. The section on handling uploads no longer references {{input-share-validation}}, per NAME argument that leader checks during upload and aggregator checks during aggregation are not the same thing. Instead we plainly state in the text of "Upload Requests" that leaders MUST ignore replays and MAY alert the client.
Section 4.2.2: "Clients SHOULD NOT upload the same measurement value in more than one report if the leader responds with status 200 and an empty body.”" Is a non-empty body understood to signal an error condition based on some other context? What if the client got status 200 from the leader but a non-empty body? Should the client retry as this text suggests? I don't know enough about HTTP semantics to know for sure, but it seems like specifying an empty body here is unnecessary. It seems fine to specify that the leader should send an empty body for 200, but maybe be less strict on the consumption side and drop the empty body requirement?

known HPKE configuration.  When clients receive an 'outdatedConfig' error, they SHOULD invalidate any cached aggregator "HpkeConfigList" and retry with a freshly generated Report.  If this retried report  <ins> does not succeed, clients SHOULD abort and discontinue retrying. If a report's ID matches that of a previously uploaded report, the leader MUST ignore it and the leader MAY alert the client with error "reportRejected".  See the implementation note in input-share- validation. </ins> The Leader MUST ignore any report pertaining to a batch that has  <ins> already been collected (see input-share-validation for details). </ins> Otherwise, comparing the aggregate result to the previous aggregate  <ins> result may result in a privacy violation.  Note that this is enforced by all Aggegators, not just the leader.  In addition, the leader MAY abort the upload protocol and alert the client with error "reportRejected". The Leader MAY ignore any reports whose timestamp is past the task's "task_expiration".  When it does so, the leader SHOULD abort the upload protocol and alert the client with error "reportRejected". Client MAY choose to opt out of the task if its own clock has passed "task_expiration". </ins> Leaders can buffer reports while waiting to aggregate them.  The leader SHOULD NOT accept reports whose timestamps are too far in the future.  Implementors MAY provide for some small leeway, usually no more than a few minutes, to account for clock skew.  If the leader rejects a report for this reason, it SHOULD abort the upload protocol  <ins> and alert the client with error "reportTooEarly".  In this situation, clients MAY re-upload the report later on. </ins> If the Report contains an unrecognized extension, or of two extensions have the same ExtensionType, then the Leader MAY abort the


This message is called the Client's report.  It consists of the task ID, report metadata, the "public share" output by the VDAF's input- distribution algorithm, and the encrypted input share of each of the <del> Aggregators.  The metadata consists of the following fields: </del>  A timestamp representing the time at which the report was generated.  Specifically, the "time" field is set to the number of

cc/ NAME
The presentation syntax is: opaque public_share; Are there ever going to be no public shares in a Report?
EDITED: Yes, Prio3 has an empty public share. This is mainly needed for Poplar1. From the :
NAME if you want, we can add text that makes sure your question is answered. Otherwise I don't think there's any action to take here.
I think the answer should be: "Yes, Prio3 doesn't need public shares."
Thanks for the update.

This message is called the Client's report.  It consists of the task ID, report metadata, the "public share" output by the VDAF's input- distribution algorithm, and the encrypted input share of each of the  <ins> Aggregators.  (Note that the public share might be empty, depending on the VDAF.  For example, Prio3 has an empty public share, but Poplar1 does not.  See VDAF.)  The header consists of the task ID and report "metadata".  The metadata consists of the following fields: </ins> A timestamp representing the time at which the report was generated.  Specifically, the "time" field is set to the number of


and discontinue retrying. The Leader MUST ignore any report pertaining to a batch that has <del> already been collected.  (See input-share-batch-validation for </del>  details.)  Otherwise, comparing the aggregate result to the previous aggregate result may result in a privacy violation.  (Note that the Helpers enforce this as well.)  The Leader MAY ignore any reports input-share-decryption. Check that the resulting input share is valid as described in <del> input-share-batch-validation. </del>  Initialize VDAF preparation as described in input-share-prep. input-share-decryption. Check that the resulting input share is valid as described in <del> input-share-batch-validation. </del>  Initialize VDAF preparation and initial outputs as described in input-share-prep. <del> [[OPEN ISSUE: consider moving the helper nonce check into #input- share-batch-validation]] </del>  Once the helper has processed each valid report share in "AggregateInitializeReq.report_shares", the helper then creates an of failure, the input share is marked as invalid with a corresponding ReportShareError error. <del> The validation checks are as follows. </del>  Check if the report has never been aggregated but is contained by a batch that has been collected.  If this check fails, the input aggregation parameter.  If this check fails, the input share MUST be marked as invalid with the error "report-replayed".  This is the case if the report was used in a previous aggregate request <del> and is therefore a replay.  An aggregator may also choose to mark an input share as invalid with the error "report-dropped" under the conditions prescribed in anti-replay. </del>  Depending on the query type for the task, additional checks may be applicable: "task_expiration" time, if so the Aggregator MAY mark the input share as invalid with the error "task-expired".  4.4.1.5. Input share preparation consists of running the preparation-state nonce is in this set, it either ignores it or aborts the upload sub- protocol as described in upload-flow.  A Helper who receives an encrypted input share whose nonce is in this set rejects the report <del> as described in input-share-batch-validation. </del>  [OPEN ISSUE: This has the potential to require aggreagtors to store nonce sests indefinitely.  See issue#180.] leaders must be at least as capable as helpers, where helpers are generally required to: <del> Support the collect protocol, which includes validating and </del>  aggregating reports; and Publish and manage an HPKE configuration that can be used for the them.  In particular, aggregators must store a batch as long as the batch has not been queried more than "max_batch_lifetime" times. However, it is not always necessary to store the reports themselves. <del> For schemes like Prio3 VDAF in which the input-validation protocol is only run once per report, each aggregator only needs to store its aggregate share for each possible batch interval, along with the number of times the aggregate share was used in a batch.  This is due to the requirement that the batch interval respect the boundaries defined by the DAP parameters.  (See batch-validation.) </del>  Furthermore, the aggregators must store data related to a task as long as the current time has not passed this task's

Clarify that is used when it's impossible to determine if an input share is valid. One situation we need this for is when an Aggregator drops some anti-replay state in order to reduce storage costs. This should result in data loss in the worst case, not a loss of privacy.
Since , aggregators are required to store report nonces for batch intervals that have not yet been collected. This is necessary in order to prevent replay attacks. The problem is that nothing guarantees that a batch interval will be collected in a timely manner: In theory, it's possible that aggregators will have to store nonce sets indefinitely. To prevent this, there needs to be a way for the aggregators to agree on when to stop accepting reports for a given batch interval. The strawman solution is to allow each aggregator to, at its own discretion, ignore a report if it can't confirm if the report was replayed or not. The problem is that this potentially leads to unnecessary data loss.
I propose resolving this with .

and discontinue retrying. The Leader MUST ignore any report pertaining to a batch that has  <ins> already been collected.  (See early-input-share-validation for </ins> details.)  Otherwise, comparing the aggregate result to the previous aggregate result may result in a privacy violation.  (Note that the Helpers enforce this as well.)  The Leader MAY ignore any reports input-share-decryption. Check that the resulting input share is valid as described in  <ins> early-input-share-validation. </ins> Initialize VDAF preparation as described in input-share-prep. input-share-decryption. Check that the resulting input share is valid as described in  <ins> early-input-share-validation. </ins> Initialize VDAF preparation and initial outputs as described in input-share-prep.  <ins> [[OPEN ISSUE: consider moving the helper nonce check into #early- input-share-validation]] </ins> Once the helper has processed each valid report share in "AggregateInitializeReq.report_shares", the helper then creates an of failure, the input share is marked as invalid with a corresponding ReportShareError error.  <ins> Before beginning the preparation step, Aggregators are required to perform the following generic checks. </ins> Check if the report has never been aggregated but is contained by a batch that has been collected.  If this check fails, the input aggregation parameter.  If this check fails, the input share MUST be marked as invalid with the error "report-replayed".  This is the case if the report was used in a previous aggregate request  <ins> and is therefore a replay. </ins> Depending on the query type for the task, additional checks may be applicable: "task_expiration" time, if so the Aggregator MAY mark the input share as invalid with the error "task-expired". <ins> Finally, if an Aggregator cannot determine if an input share is valid. it MUST mark the input share as invalid with error "report- dropped".  This situation arises if, for example, the Aggregator has evicted from long-term storage the state required to perform the check.  (See reducing-storage-requirements for details.) If all of the above checks succeed, the input share is not marked as invalid. </ins> 4.4.1.5. Input share preparation consists of running the preparation-state nonce is in this set, it either ignores it or aborts the upload sub- protocol as described in upload-flow.  A Helper who receives an encrypted input share whose nonce is in this set rejects the report  <ins> as described in early-input-share-validation. </ins> [OPEN ISSUE: This has the potential to require aggreagtors to store nonce sests indefinitely.  See issue#180.] leaders must be at least as capable as helpers, where helpers are generally required to:  <ins> Support the aggregate sub-protocol, which includes validating and </ins> aggregating reports; and Publish and manage an HPKE configuration that can be used for the them.  In particular, aggregators must store a batch as long as the batch has not been queried more than "max_batch_lifetime" times. However, it is not always necessary to store the reports themselves.  <ins> For schemes like Prio3 VDAF in which reports are verified only once, each aggregator only needs to store its aggregate share for each possible batch interval, along with the number of times the aggregate share was used in a batch.  This is due to the requirement that the batch interval respect the boundaries defined by the DAP parameters. (See batch-validation.) However, Aggregators are also required to implement several per- report checks that require retaining a number of data artificts.  For example, to detect replay attacks, it is necessary for each Aggregator to retain the set of nonces of reports that have been aggregated for the task so far.  Depending on the task lifetime and report upload rate, this can result in high storage costs.  To elliviate this burden, DAP allows Aggregators to drop this state as needed, so long as reports are droppped properly as described in early-input-share-validation.  Aggregators SHOULD take steps to mitigate the risk of dropping reports (e.g., by evicting the oldest data first). </ins> Furthermore, the aggregators must store data related to a task as long as the current time has not passed this task's


Information used to bind the aggregate result to the query.  For fixed-size tasks, this includes the batch ID assigned to the batch <del> by the Leader.  The indicatd query type MUST match the task's </del>  query type. [OPEN ISSUE: What should the Collector do if the query type

Currently parsing AggreateInitializeReq, AggregateshareReq, CollectReq, and CollectResp requires the query type associated with the task. The query type is determined by the task, but it is necessary to parse the task ID from the message before the remainder can be parsed. This change ensures that parsing of these messages does not require the query type.
The query type is determined by the task config, which in turn is determined by the task ID. The task ID is encoded by the message, so it is necessary to parse the task ID before the Aggregator knows how to parse it. PR URL introduced the following problem: If an Aggregator implements multiple query types ("time-interval" and "fixed-size", say), then a message's format depends on the query type for the task. This means it is necessary to parse the task ID before the rest of the message. This is perfectly implementable, but it has two downsides: This likely breaks abstraction boundaries in current implementations and will require refactoring. (For example, . Notice that the message is parsed in one step.) Going forward, we'll need to make sure that the task ID always appears first in any DAP message so that it can be easily parsed. This is already true today, but we might want to add some text to Operational Considerations that spells this requirement out so that we minimize the chance of regression. Here's my question for folks: Do we want to try and do something about this? I can think of a couple of solutions: Add the query type byte to messages whose format depends on this. One downside is that it creates a new error case for the Aggregator to check, namely, that the query type indicated in the message matches the query type of the task. (In fact, we tried to avoid this in .) Move the task ID out of the message payload and into the URL query string, or perhaps the HTTP header. The only downside to this is that it complicates request authentication for deployments that want to sign the payload and bind it to the task config. (This may be desirable from a security perspective.) One possible upshot is that it might put us in a better position to address . (NAME please weigh in here!)
I noticed this issue while working on the implementation of "fixed-size". I think my preference would be to do nothing here, i.e., shift the burden to implementations and maybe make a note in Operational Considerations.
I thought about this too, but I was waiting to finish Janus' DAP-02 implementation to see how impactful it was. (implementation is still ongoing, I'll update this bug w/ my thoughts once we're closer to done) If we do decide to do something, I think I'd prefer solution . We've had a few issues now of the form "correctly parsing this message requires knowing the task ID, which requires parsing part of the message." This is technically feasible (as you note), but is fairly awkward from an implementation standpoint (IMO). Moving the task ID out from the message sidesteps this problem, and IMO fits well with . I think solution would allow implementations to have simpler parsing logic, but I'm not sure this upside is worth the cost of an additional byte on the wire & an additional error case. Though I suppose this new error case exists either way -- but without a discriminator byte, the error case would most likely present itself as a message parsing failure. (e.g. "message has too many/too few bytes")
After conferring with NAME NAME et al offline, we discovered another consideration here which makes me lean towards making protocol messages self-describing (solution (1.)). We're working on an . At a high-level, an Aggregator that supports this extension uses information provided by the client (in a Report extension) to derive the task ID. This information includes, among other things, the query type to use for the task. The problem is that decoding of the depends on the query type, which, when this extension is in-use, is not known until parsing the report extensions: If we take solution (1.), then is encoded by , thus resolving this bug. There are other ways to resolve this. I can think of a few possibilities if there were an extension field in . However there are lots of things to consider for this sort of change, and I would prefer we wait until later. Does anyone object to taking solution (1.)? I will have a PR ready for this by the end of the week. I would like to land this in DAP-02 if at all possible.
PR is up, would appreciate review.
I think that the right solution in the long run is Chris' , namely putting the task ID, and anything else needed to parse the request body, into the HTTP request path. But I'm OK with taking for now in the interest of making implementable. Sure, we'll have to take the query type byte out of these messages later, but there are quite a few messages that currently have a or member that will get moved up into the request path in draft-03 () I think there's plenty of prior art on including things like HTTP headers, path or query params in request signatures. See for instance , which explains how to canonicalize an HTTP request into a string that can be "signed" (well, it's an HMAC, but AWS insists that it's a signature).
Option (2.) doesn't quite solve my extension problem (URL), but I have no objection to moving the task ID (and maybe agg job ID) out of the message in any case. Seems to me like a way cleaner design that would likely solve other problems.
LGTM, one typo needs fixingThis accomplishes what it sets out to, modulo some typos.

Information used to bind the aggregate result to the query.  For fixed-size tasks, this includes the batch ID assigned to the batch  <ins> by the Leader.  The indicated query type MUST match the task's </ins> query type. [OPEN ISSUE: What should the Collector do if the query type


"(prep_state, prep_msg)". To carry out this step, for each PrepareStep in <del> AggregateContinueReq.prepare_shares received from the leader, the </del>  helper performs the following check to determine if the report share should continue being prepared: "prep_msg" is its next VDAF message. This output message for each report in <del> AggregateContinueReq.prepare_shares is then sent to the leader in an </del>  AggregateContinueResp message, structured as follows: <del> The order of AggregateContinueResp.prepare_shares MUST match that of the PrepareStep values in "AggregateContinueReq.prepare_shares".  The </del>  helper's response to the leader is an HTTP status code 200 OK whose body is the AggregateContinueResp and media type is "message/dap- aggregate-continue-resp".  The helper then awaits the next message

The prepare_shares fields were always of type PrepareStep. To match the convention used for most other field names, I change the name of the field to match its type. (There is no "PrepareShare" type.)

"(prep_state, prep_msg)". To carry out this step, for each PrepareStep in  <ins> AggregateContinueReq.prepare_steps received from the leader, the </ins> helper performs the following check to determine if the report share should continue being prepared: "prep_msg" is its next VDAF message. This output message for each report in  <ins> AggregateContinueReq.prepare_steps is then sent to the leader in an </ins> AggregateContinueResp message, structured as follows:  <ins> The order of AggregateContinueResp.prepare_steps MUST match that of the PrepareStep values in "AggregateContinueReq.prepare_steps".  The </ins> helper's response to the leader is an HTTP status code 200 OK whose body is the AggregateContinueResp and media type is "message/dap- aggregate-continue-resp".  The helper then awaits the next message


The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted <del> "enc_share", for a given batch interval, denoted "batch_interval", </del>  decryption works as follows: where "sk" is the HPKE secret key, "task_id" is the task ID for the

The Aggregate Share Encryption section still had a few references to "batch interval", but the encryption process now uses the "batch selector".

The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted  <ins> "enc_share", for a given batch selector, denoted "batch_selector", </ins> decryption works as follows: where "sk" is the HPKE secret key, "task_id" is the task ID for the


encrypted input share whose report ID is in this set rejects the report as described in early-input-share-validation. <del> [OPEN ISSUE: This has the potential to require aggreagtors to store </del>  report ID sets indefinitely.  See issue#180.] A malicious aggregator may attempt to force a replay by replacing the (See batch-validation.) However, Aggregators are also required to implement several per- <del> report checks that require retaining a number of data artificts.  For </del>  example, to detect replay attacks, it is necessary for each Aggregator to retain the set of report IDs of reports that have been aggregated for the task so far.  Depending on the task lifetime and

Historically, the Nonce included a time component. Now that the time component is split out & Nonce is effectively an opaque identifier, ReportID is clearer & more closely matches similar identifiers (TaskID, AggregationJobID, BatchID).
Note this PR includes a single wire-format change: had its fields reordered so that the comes first, since the ID is the primary identifier. I am happy to revert this part of the PR if folks would prefer a purely-editorial PR.
LGTM. Any thoughts on replacing and with a Nonce so then it has both?
I'm pretty happy leaving & as separate fields -- for practical purposes, IMO binds them together closely enough (along with the report extensions). Conceptually, DAP is changing such that the primary identifier for a report is its opaque ID (with this PR, called ). Previously, the primary identifier for a report was . The technical aspects of this change were made by cjpatton in his changes to introduce tasks. If we do decide to introduce a struct that is effectively , I'd prefer if it was called something other than -- while we do expect that this value is a "value to be used only once", it's no longer used as a nonce in any cryptographic operations. And I'd argue there's no better justification to call the report ID (or ) a nonce than, say, the aggregation job ID/batch ID/task ID. That is, IMO "report ID" more accurately names what this type is than "nonce" -- "nonce" is not technically incorrect, just less clear.
Currently, the DAP specification has a number of opaque values used to identify some DAP entity: (identifying a client report) (identifying a task) (identifying an aggregation job) (identifying a batch) 's name differs from the rest. I think this is because, historically, the nonce also included a time component. Now that the time component has been split out, I think we could rename to for clarity of purpose. Thoughts? (and I probably don't need to say this, but there is no reason to push this into DAP-02 -- it could easily land in DAP-03 instead.)
Yup, I think this is a good idea! I'm tempted to say let's stick this in 02, since it's editorial, but it's actually somewhat invasive because we use the term "nonce set" in so many places. If you want to send a PR, I'd review it. But we should make sure there's broad consensus before merging.
Nice change Yeah I think this is a big win :) good idea NAME

encrypted input share whose report ID is in this set rejects the report as described in early-input-share-validation.  <ins> [OPEN ISSUE: This has the potential to require aggregators to store </ins> report ID sets indefinitely.  See issue#180.] A malicious aggregator may attempt to force a replay by replacing the (See batch-validation.) However, Aggregators are also required to implement several per-  <ins> report checks that require retaining a number of data artifacts.  For </ins> example, to detect replay attacks, it is necessary for each Aggregator to retain the set of report IDs of reports that have been aggregated for the task so far.  Depending on the task lifetime and


too many times.  This is determined by the maximum number of times a batch can be queried, "max_batch_query_count".  Unless the query has been issued less than "max_batch_query_count" times, the Aggregator <del> MUST abort with error of type "batchLifetimeExceeded". </del>  Finally, the Aggregator checks that the batch does not contain a report that was included in any previous batch.  If this batch

NAME please have another look
This seems useful, since it disambiguates "tasklifetime" and "batchlifetime". The former, as i understand it, is a timestamp; and the latter is a number of queries. cc/ NAME for thoughtful review here

too many times.  This is determined by the maximum number of times a batch can be queried, "max_batch_query_count".  Unless the query has been issued less than "max_batch_query_count" times, the Aggregator  <ins> MUST abort with error of type "batchQueriedTooManyTimes". </ins> Finally, the Aggregator checks that the batch does not contain a report that was included in any previous batch.  If this batch


This document uses the verbs "abort" and "alert with "[some error message]"" to describe how protocol participants react to various <del> error conditions. </del>  4. the task ID obtained from the task parameters, encoded in Base 64 with URL and filename safe alphabet with no padding, as specified in sections 5 and 3.2 of RFC4648.  If the aggregator does not recognize <del> the task ID, then it responds with HTTP status code 400 Bad Request and an error of type "unrecognizedTask". </del>  An aggregator is free to use different HPKE configurations for each task with which it is configured.  If the task ID is missing from a measurement value in more than one report if the leader responds with HTTP status code 200 OK and an empty body. <del> If the leader does not recognize the task ID, then it responds with HTTP status code 400 Bad Request and an error of type "unrecognizedTask". </del>  The leader responds to requests whose leader encrypted input share uses an out-of-date "HpkeConfig.id" value, indicated by <del> "HpkeCiphertext.config_id", with HTTP status code 400 Bad Request and an error of type 'outdatedConfig'.  Clients SHOULD invalidate any cached aggregator "HpkeConfig" and retry with a freshly generated Report.  If this retried report does not succeed, clients MUST abort and discontinue retrying. </del>  The Leader MUST ignore any report pertaining to a batch that has already been collected.  (See early-input-share-validation for VDAF-specific message for each report share. To begin this process, the helper first checks if it recognizes the <del> task ID, if not then it responds with HTTP status code 400 Bad Request and an error of type "unrecognizedTask".  Then the helper checks that the report IDs in "AggregateInitializeReq.report_shares" are all distinct.  If two ReportShare values have the same report ID, then the helper MUST abort with error "unrecognizedMessage".  If this check succeeds, the helper then attempts to recover each input share in "AggregateInitializeReq.report_shares" as follows: </del>  Decrypt the input share for each report share as described in input-share-decryption. 4.4.2.2. <del> If the helper does not recognize the task ID, then it aborts preparation with HTTP status code 400 Bad Request and an error of type "unrecognizedTask". </del>  Otherwise, the helper continues with preparation for a report share by combining the leader's input message in "AggregateContinueReq" and The named parameters are: "task_id", the DAP task ID.  If the Leader does not recognize the <del> task ID, it MUST abort with HTTP status code 400 Bad Request and error "unrecognizedTask". </del>  "query", the Collector's query.  The indicated query type MUST match the task's query type.  Otherwise, the Leader MUST abort The message contains the following parameters: The DAP task ID.  If the Helper does not recognize the task ID, it <del> MUST abort with HTTP status code 400 Bad Request and error "unrecognizedTask". </del>  The "batch selector", which encodes parameters used to determine the batch being aggregated.  The value depends on the query type

Addresses issue URL
I don't think that's true. : >DAP servers can return responses with an HTTP error response code (4XX or 5XX). For example, if the client submits a request using a method not allowed in this document, then the server MAY return HTTP status code 405 Method Not Allowed. So maybe we do need to specify that the status should be 400 (client error) and not 500 (server error). FWIW, has several places where it dictates HTTP 400 inline in the text, so I think it's OK for us to do that. I agree we should be consistent about using MUST throughout the document. Curiously, ACME has some examples where the 400 error response is a MUST, but the 200 success response has no all-caps applied to it. :
NAME NAME thanks for the comments, I've specified 400 in the errors section and shortened the words elsewhere.
Error types are either double quoted or code formatted, the text seems to encourage both: But these format are not consistent, some examples: It would be much clearer if we choose one style and make it consistent across the text.
The current protocol text doesn't specify what participants should do when a task ID is not recognized, there are two places unrecognized task are mentioned: In the table, it has: mentions "404 Not Found and an error of type unrecognizedTask", which seems to conflict with most implementation's choice of error code 400 (NAME ) The text should specify error 400 with type is used in every API that receives unrecognized task ID.
This would be useful I think. Would you mind sending a PR, NAME
NAME will do, create this issue to remind myself.
PR merged

This document uses the verbs "abort" and "alert with "[some error message]"" to describe how protocol participants react to various  <ins> error conditions.  This implies HTTP status code 400 Bad Request unless explicitly specified otherwise. </ins> 4. the task ID obtained from the task parameters, encoded in Base 64 with URL and filename safe alphabet with no padding, as specified in sections 5 and 3.2 of RFC4648.  If the aggregator does not recognize  <ins> the task ID, then it MUST abort with error "unrecognizedTask". </ins> An aggregator is free to use different HPKE configurations for each task with which it is configured.  If the task ID is missing from a measurement value in more than one report if the leader responds with HTTP status code 200 OK and an empty body.  <ins> If the leader does not recognize the task ID, then it MUST abort with error "unrecognizedTask". </ins> The leader responds to requests whose leader encrypted input share uses an out-of-date "HpkeConfig.id" value, indicated by  <ins> "HpkeCiphertext.config_id", with error of type 'outdatedConfig'. Clients SHOULD invalidate any cached aggregator "HpkeConfig" and retry with a freshly generated Report.  If this retried report does not succeed, clients MUST abort and discontinue retrying. </ins> The Leader MUST ignore any report pertaining to a batch that has already been collected.  (See early-input-share-validation for VDAF-specific message for each report share. To begin this process, the helper first checks if it recognizes the  <ins> task ID, if not then it MUST abort with error "unrecognizedTask". Then the helper checks that the report IDs in "AggregateInitializeReq.report_shares" are all distinct.  If two ReportShare values have the same report ID, then the helper MUST abort with error "unrecognizedMessage".  If this check succeeds, the helper then attempts to recover each input share in "AggregateInitializeReq.report_shares" as follows: </ins> Decrypt the input share for each report share as described in input-share-decryption. 4.4.2.2.  <ins> If the helper does not recognize the task ID, then it MUST abort with error "unrecognizedTask". </ins> Otherwise, the helper continues with preparation for a report share by combining the leader's input message in "AggregateContinueReq" and The named parameters are: "task_id", the DAP task ID.  If the Leader does not recognize the  <ins> task ID, it MUST abort with error "unrecognizedTask". </ins> "query", the Collector's query.  The indicated query type MUST match the task's query type.  Otherwise, the Leader MUST abort The message contains the following parameters: The DAP task ID.  If the Helper does not recognize the task ID, it  <ins> MUST abort with error "unrecognizedTask". </ins> The "batch selector", which encodes parameters used to determine the batch being aggregated.  The value depends on the query type


Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its AggregateInitializeReq message as the payload.  The media type is <del> "message/dap-aggregate-initialize-req". </del>  4.4.1.2. is finished for the report share. The helper's response to the leader is an HTTP status code 200 OK <del> whose body is the AggregateInitializeResp and media type is "message/ dap-aggregate-initialize-resp". </del>  Upon receipt of a helper's AggregateInitializeResp message, the leader checks that the sequence of PrepareStep messages corresponds "AggregateContinueReq.task_id"'s parameters except its own, the leader sends a POST request to "[aggregator]/aggregate" with AggregateContinueReq as the payload and the media type set to <del> "message/dap-aggregate-continue-req". </del>  4.4.2.2. The order of AggregateContinueResp.prepare_steps MUST match that of the PrepareStep values in "AggregateContinueReq.prepare_steps".  The helper's response to the leader is an HTTP status code 200 OK whose <del> body is the AggregateContinueResp and media type is "message/dap- </del>  aggregate-continue-resp".  The helper then awaits the next message from the leader.

In IANA all media types seem to be of "application" type instead of "message" type, this PR is to align the mismatch in the protocol description and adapt to the media types specified in IANA.
Nice catch!

Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its AggregateInitializeReq message as the payload.  The media type is  <ins> "application/dap-aggregate-initialize-req". </ins> 4.4.1.2. is finished for the report share. The helper's response to the leader is an HTTP status code 200 OK  <ins> whose body is the AggregateInitializeResp and media type is "application/dap-aggregate-initialize-resp". </ins> Upon receipt of a helper's AggregateInitializeResp message, the leader checks that the sequence of PrepareStep messages corresponds "AggregateContinueReq.task_id"'s parameters except its own, the leader sends a POST request to "[aggregator]/aggregate" with AggregateContinueReq as the payload and the media type set to  <ins> "application/dap-aggregate-continue-req". </ins> 4.4.2.2. The order of AggregateContinueResp.prepare_steps MUST match that of the PrepareStep values in "AggregateContinueReq.prepare_steps".  The helper's response to the leader is an HTTP status code 200 OK whose  <ins> body is the AggregateContinueResp and media type is "application/dap- </ins> aggregate-continue-resp".  The helper then awaits the next message from the leader.


"AggregateShareReq" is done as follows: where "pk" is the HPKE public key encoded by the collector's HPKE <del> key, "server_role" is "0x02" for the leader and "0x03" for a helper. </del>  The "SealBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted <del> "enc_share", for a given batch selector, denoted "batch_selector", decryption works as follows: where "sk" is the HPKE secret key, "task_id" is the task ID for the collect request, and "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper).  The value of "batch_selector" is computed by the Collector from its query and the response to its query: </del>  For time_interval tasks, the batch selector is the batch interval specified in the query.

This revises the sections dealing with encryption and decryption of input shares and aggregate shares, rewriting the construction of each AAD input to be in terms of RFC 8446-style structs instead of byte string concatenation. This
I squashed this back into two commits and then rebased to fix some conflicts.
NAME would you mind taking a quick look?
Currently, the AAD for input shares is defined as . The variable is narratively introduced in 4.4.1.3 as "the public share sent to each aggregator", referring to a part of the report share. Two implementations interpreted the construction of the AAD differently, as either or a 32-bit length suffix preceding the public share. While it would likely be okay to have a single variable-length byte array in the AAD without a length prefix, as the resulting AAD would still be unambiguous, I think it would be prudent to change this to explicitly include a length prefix, to simplify future changes. We should define new structs for both kinds of AAD strings, and specify that the AAD is the result of encoding the corresponding struct. For example:
Places in DAP-02 is used without a length prefix: Section 4.3.2, in the AAD to . Section 4.4.1.3, in the AAD to . I think all of the places where we might unexpectedly not have a length prefix will live in the info or AAD parameters to or , since I think all other encoding is specified via RFC8446 struct notation. I like the idea of specifying info/AAD values as encodings of an RFC8446 struct, as suggested in the OP: this would be unambiguous, and would lower the odds of regressions if we introduced additional variable-length fields.
I interpreted these sections as including the length prefix. The only section where we definitely can't pass the length prefix is URL, where the VDAF is invoked.
I also support this approach in OP, I think all AAD string should be defined in structs and make it explicit that OpenBase and SealBase takes the encoding of such struct for AAD, that means AAD used in Aggregate share encrypt/decrypt in URL should also have a struct definition.
Regarding the VDAF call, am I right TLS is not specified for the procedure calls with VDAF? so publicshare can be any type/encoding that VDAF implementation understands, therefore how length of publicshare is passed in the below call doesn't have to be constrained by how AAD is used.
As input to VDAF, is the opaque byte string carried by the , excluding the prefix. Just as , it is decoded as specified by the VDAF in use.
LGTMSolid implementation, and a clear win

"AggregateShareReq" is done as follows: where "pk" is the HPKE public key encoded by the collector's HPKE  <ins> key, "server_role" is "0x02" for the leader and "0x03" for a helper, and "agg_share_aad" is a value of type "AggregateShareAad" with its values set from the corresponding fields of the "AggregateShareReq". </ins> The "SealBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted  <ins> "enc_share", for a given batch selector, decryption works as follows: where "sk" is the HPKE secret key, "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper), and "agg_share_aad" is an "AggregateShareAad" message constructed from the task ID in the collect request and a batch selector.  The value of the batch selector used in "agg_share_aad" is computed by the Collector from its query and the response to its query as follows: </ins> For time_interval tasks, the batch selector is the batch interval specified in the query.


"time_precision" in order to ensure that that the timestamp cannot be used to link a report back to the Client that generated it. <del> A list of extensions to be included with the report.  (See upload- extensions.) </del>  <del> To generate a report, the Client first shards its measurement into input shares as specified by the VDAF.  It then encrypts each input share as follows: </del>  where "pk" is the aggregator's public key; "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the <del> helper), "input_share" is the Aggregator's input share, and "input_share_aad" is an encoded message of type "InputShareAad", constructed from the same values as the corresponding fields in the report.  The "SealBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. </del>  The order of the encrypted input shares appear MUST match the order of the task's "aggregator_endpoints".  That is, the first share rejects a report for this reason, it SHOULD abort the upload protocol and alert the client with error "reportTooEarly".  4.3.3. <del> Each Report carries a list of extensions that clients may use to convey additional, authenticated information in the report.  [OPEN ISSUE: The extensions aren't authenticated.  It's probably a good idea to be a bit more clear about how we envision extensions being used.  Right now this includes client attestation for defeating Sybil attacks.  See issue#89.]  Each extension is a tag-length encoded value of the following form: </del>  <del> "extension_type" indicates the type of extension, and </del>  "extension_data" contains information specific to the extension.  4.3.4. The contents of each input share must be kept confidential from 4.4.1.3. Each report share has a corresponding task ID, report metadata <del> (report ID, timestamp, and extensions), the public share sent to each Aggregator, and the recipient's encrypted input share.  Let "task_id", "metadata", "public_share", and "encrypted_input_share" denote these values, respectively.  Given these values, an aggregator decrypts the input share as follows.  First, the aggregator looks up the HPKE config and corresponding secret key indicated by </del>  "encrypted_input_share.config_id".  If not found, then it marks the report share as invalid with the error "hpke_unknown_config_id". Otherwise, it constructs an "InputShareAad" message from "task_id", the aggregator ("0x02" for the leader and "0x03" for the helper).  If decryption fails, the aggregator marks the report share as invalid with the error "hpke_decrypt_error".  Otherwise, it outputs the <del> resulting "input_share".  The "OpenBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. </del>  4.4.1.4. "task_expiration" time, if so the Aggregator MAY mark the input share as invalid with the error "task_expired".  Finally, if an Aggregator cannot determine if an input share is valid. it MUST mark the input share as invalid with error "report_dropped".  This situation arises if, for example, the "0x01" for the helper); "agg_param" is the opaque aggregation parameter distributed to the aggregators by the collector; "public_share" is the public share generated by the client and <del> distributed to each aggregator; and "input_share" is the aggregator's input share. </del>  If either step fails, the aggregator marks the report as invalid with error "vdaf_prep_error".  Otherwise, the value "out" is interpreted

Rebased and squashed.
Rebased and squashed.
NAME I've removed the shared extensions as discussed. Can you take another look and re-approve?
As noted by NAME in URL, we need to decide how to handle duplicate extensions. One simple idea is to treat duplicate extensions as decoding failures, i.e., abort with "unrecognizedMessage".
Are unknown/unsupported extensions ignored?
First off, just for clarity's sake, I'm assuming Martin is talking about . I think there are three ways to solve this problem. First, unknown extensions could be ignored by the aggregators. I believe this is the intent of the existing text (though I acknowledge that the protocol text would need to be updated to make that clear). I think this would work for the two report extension use cases I'm aware of at the moment, which are client authentication to the helper and . In either case, the aggregator can ignore the extension, proceed with the protocol and I don't think anything disastrous happens. If the aggregator doesn't support client auth extensions, it simply proceeds with processing the report upload after applying whatever other security policy it wants to the report. If the aggregator doesn't support the taskprov extension, then the upload will fail gracefully and reliably because the aggregator doesn't recognize the task ID referenced by the report. This is simple and flexible, but it does mean that all extensions ever deployed in DAP need to be designed such that they can be gracefully ignored by aggregators. The second way to handle this would be to use something like (ugh) X.509's extension criticality bit, which would appear alongside an extension and if set to true, would indicate to servers that they should abort processing of an upload with some error like , the idea being that further handling of the report without handling the extension is too dangerous. However, anytime I'm reaching out to (ugh) X.509 for ideas, I take that as a sign that maybe we're solving the wrong problem or doing something too complicated. But I think the reason X.509 ends up needing a criticality bit is because at the time of issuance of a certificate, it's not known (and not knowable) what is the set of implementations of relying parties, servers, CT logs, audit tools, etc. that might ever see the cert, and so special care must be taken to account for the diversity of implementations. But in DAP, when a client is putting extensions into its reports, it's doing so with knowledge of the specific pair of aggregators to which it is uploading the report, so I think it's more reasonable to expect the client to know precisely what extensions the aggregators support. We have discussed having servers advertise what HPKE algorithms they support in , so advertising supported extensions is a minor step beyond that. So the third option is to treat all extensions as critical, and require servers to abort if they don't recognize them. This is the least flexible, but I also think it's the easiest one to reason about, both for the DAP protocol and eventual extension authors. All that said, I think I'll tentatively vote for all extensions implicitly being critical, but I'm open to having my mind changed.
I would prefer to make extensions ignored by default, as they are in TLS. That puts it to extension designers to make sure their extensions can be safely ignored. If an extension can't be safely ignored, we should consider merging it into the core protocol. See URL for additional discussion.
To be clear NAME the spec is not clear here yet, hence :)
I apologize for splitting the discussion. I tend to side with NAME conclusions. If nothing else, the aggregators can publish what they accept ahead of time, along with keys.
I think my only concern with the third option is this: Suppose that, in some DAP deployment, the Client and Leader used some extension that is irrelevant to the Helper. For instance, imagine the Client produces some attestation () that the report was generated on a Leader-trusted device. The Helper may not be able to do this check. I guess you could argue that both Aggregators should be doing this check, in which case option 3 is actually what we want? This would probably depend on the threat model for the extension.
So are these extensions universal in the sense that the same extension applies to the data that is sent to all helpers (including the leader) or, as I believe is the case, the extensions are targeted and they are sent separately to each of the helpers? In the latter case, the client can condition what it sends on what it knows about the specific helper that it is sending to. Yes, that makes it more complicated for a client, but that is the nature of the trade-off.
The latter is what I'm imagining, yes. Hmm that's a pretty good idea, though (1) it would require some protocol changes (currently the Leader "re-transmits" all of the extensions to each Helper) and (2) we would have to unbind the extensions from the HPKE ciphertext (currently the extensions are included in the AAD for encryption). (1) seems workable to me; (2) doesn't seem desirable from a security perspective. (Fiddling with extensions should result in the report being excluded, hence the current binding.)
Thinking about it more, we don't have to do (2). Perhaps all we have to do is mark each extension with the identity of the intended consumer? Some extensions might be intended for the Leader, others for the Helper, others for both.
Why not move the extensions to the encrypted payload? Sure, the client replicates extensions toward each helper, but you avoid a lot of this complexity.
That could work, but I think we should avoid copying extensions. (People are worried about communication cost of DAP.)
I'd be down for something like this: Treat Report.extensions as "critical", in the sense that any unrecognized extension triggers an abort. Add an extensions field to the plaintext for each Report.encryptedinputshare.payload (HPKE ciphertext). Alternatively, we could add a new field to Report.encryptedinputshare. Either way, the consumer treats these as critical. NAME WDYT?
The idea being to (1) enable a client to send different extensions to either aggregator and (2) keep the extensions intended for the helper private from the leader? Yeah, seems like a good idea!
It would be great to work this out in DAP-03. I'll try to put together a PR for NAME suggestion next week, if no one else gets there first.
PR is up, please have a look!
Noticed a typo, otherwise LGTM.

"time_precision" in order to ensure that that the timestamp cannot be used to link a report back to the Client that generated it.  <ins> To generate a report, the Client begins by sharding its measurement into input shares as specified by the VDAF.  It then wraps each input share in the following structure: </ins>  <ins> Field "extensions" is set to the list of extensions intended to be consumed by the given Aggregator.  (See upload-extensions.)  Field "payload" is set to the Aggregator's input share output by the VDAF sharding algorithm. Next, the Client encrypts each PlaintextInputShare "plaintext_input_share" as follows: </ins> where "pk" is the aggregator's public key; "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the  <ins> helper), "plaintext_input_share" is the Aggregator's PlaintextInputShare, and "input_share_aad" is an encoded message of type "InputShareAad", constructed from the same values as the corresponding fields in the report.  The "SealBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. </ins> The order of the encrypted input shares appear MUST match the order of the task's "aggregator_endpoints".  That is, the first share rejects a report for this reason, it SHOULD abort the upload protocol and alert the client with error "reportTooEarly". <ins> If the Report contains an unrecognized extension, then the Leader MAY abort the upload request with error "unrecognizedMessage".  Note that this behavior is not mandatory because it requires the Leader to decrypt its input share. </ins> 4.3.3.  <ins> Each PlaintextInputShare carries a list of extensions that Clients use to convey additional information to the Aggregator.  Some extensions might be intended for all Aggregators; others may only be intended for a specific Aggregator.  (For example, a DAP deployment might use some out-of-band mechanism for an Aggregator to verify that Reports come from authenticated Clients.  It will likely be useful to bind the extension to the input share via HPKE encryption.) </ins>  <ins> Each extension is a tag-length encoded value of the following form: Field "extension_type" indicates the type of extension, and </ins> "extension_data" contains information specific to the extension. <ins> Extensions are mandatory-to-implement: If an Aggregator receives a Report containing an extension it does not recognize, then it MUST reject the Report.  (See early-input-share-validation for details.) </ins> 4.3.4. The contents of each input share must be kept confidential from 4.4.1.3. Each report share has a corresponding task ID, report metadata  <ins> (report ID and, timestamp), the public share sent to each Aggregator, and the recipient's encrypted input share.  Let "task_id", "metadata", "public_share", and "encrypted_input_share" denote these values, respectively.  Given these values, an aggregator decrypts the input share as follows.  First, the aggregator looks up the HPKE config and corresponding secret key indicated by </ins> "encrypted_input_share.config_id".  If not found, then it marks the report share as invalid with the error "hpke_unknown_config_id". Otherwise, it constructs an "InputShareAad" message from "task_id", the aggregator ("0x02" for the leader and "0x03" for the helper).  If decryption fails, the aggregator marks the report share as invalid with the error "hpke_decrypt_error".  Otherwise, it outputs the  <ins> resulting PlaintextInputShare "plaintext_input_share".  The "OpenBase()" function is as specified in HPKE for the </ins> 4.4.1.4. "task_expiration" time, if so the Aggregator MAY mark the input share as invalid with the error "task_expired". <ins> Check if the PlaintextInputShare contains unrecognized extensions. If so, then the Aggregator MUST mark the input share as invalid with error "unrecognized_message". </ins> Finally, if an Aggregator cannot determine if an input share is valid. it MUST mark the input share as invalid with error "report_dropped".  This situation arises if, for example, the "0x01" for the helper); "agg_param" is the opaque aggregation parameter distributed to the aggregators by the collector; "public_share" is the public share generated by the client and  <ins> distributed to each aggregator; and "plaintext_input_share" is the aggregator's PlaintextInputShare. </ins> If either step fails, the aggregator marks the report as invalid with error "vdaf_prep_error".  Otherwise, the value "out" is interpreted


where "pk" is the aggregator's public key; "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper), "task_id" is the task ID, "metadata" is the report metadata, <del> and "input_share" is the Aggregator's input share. </del>  The order of the encrypted input shares appear MUST match the order of the task's "aggregator_endpoints".  That is, the first share the aggregator ("0x02" for the leader and "0x03" for the helper).  If decryption fails, the aggregator marks the report share as invalid with the error "hpke_decrypt_error".  Otherwise, it outputs the <del> resulting "input_share". </del>  4.4.1.4. where "pk" is the HPKE public key encoded by the collector's HPKE key, "server_role" is "0x02" for the leader and "0x03" for a helper.  The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted For fixed_size tasks, the batch selector is the batch ID assigned sent in the response.  4.5.5. Collect sub-protocol messages must be confidential and mutually

I only tried searching for it, but all I found were places that it was used. This makes it difficult to determine which inputs are which. Which is the AAD, which is the plaintext, and which is the nonce? Please also describe the inputs in text. Note that at least one of the code blocks that use this is too wide at 76 characters.
SealBase() and SealOpen() refer to the "single-shot APIs" from URL I notice that we don't reference this explicitly. Would a reference suffice?

where "pk" is the aggregator's public key; "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper), "task_id" is the task ID, "metadata" is the report metadata,  <ins> and "input_share" is the Aggregator's input share.  The "SealBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. </ins> The order of the encrypted input shares appear MUST match the order of the task's "aggregator_endpoints".  That is, the first share the aggregator ("0x02" for the leader and "0x03" for the helper).  If decryption fails, the aggregator marks the report share as invalid with the error "hpke_decrypt_error".  Otherwise, it outputs the  <ins> resulting "input_share".  The "OpenBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. </ins> 4.4.1.4. where "pk" is the HPKE public key encoded by the collector's HPKE key, "server_role" is "0x02" for the leader and "0x03" for a helper. <ins> The "SealBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. </ins> The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted For fixed_size tasks, the batch selector is the batch ID assigned sent in the response. <ins> The "OpenBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration. </ins> 4.5.5. Collect sub-protocol messages must be confidential and mutually


rejects a report for this reason, it SHOULD abort the upload protocol and alert the client with error "reportTooEarly". <del> If the Report contains an unrecognized extension, then the Leader MAY abort the upload request with error "unrecognizedMessage".  Note that this behavior is not mandatory because it requires the Leader to decrypt its input share. </del>  4.3.3. If so, then the Aggregator MUST mark the input share as invalid with error "unrecognized_message".  Finally, if an Aggregator cannot determine if an input share is <del> valid. it MUST mark the input share as invalid with error </del>  "report_dropped".  This situation arises if, for example, the Aggregator has evicted from long-term storage the state required to perform the check.  (See reducing-storage-requirements for

Based on (merge that first).
As noted by NAME in URL, we need to decide how to handle duplicate extensions. One simple idea is to treat duplicate extensions as decoding failures, i.e., abort with "unrecognizedMessage".

rejects a report for this reason, it SHOULD abort the upload protocol and alert the client with error "reportTooEarly".  <ins> If the Report contains an unrecognized extension, or of two extensions have the same ExtensionType, then the Leader MAY abort the upload request with error "unrecognizedMessage".  Note that this behavior is not mandatory because it requires the Leader to decrypt its input share. </ins> 4.3.3. If so, then the Aggregator MUST mark the input share as invalid with error "unrecognized_message". <ins> Check if the ExtensionType of any two extensions in PlaintextInputShare is the same.  If so, then the Aggregator MUST mark the input share as invalid with error "unrecognized_message". </ins> Finally, if an Aggregator cannot determine if an input share is  <ins> valid, it MUST mark the input share as invalid with error </ins> "report_dropped".  This situation arises if, for example, the Aggregator has evicted from long-term storage the state required to perform the check.  (See reducing-storage-requirements for