report "metadata".  The metadata consists of the following fields: A report ID used by the Aggregators to ensure the report appears <del> in at most one batch.  (See anti-replay.)  The Client MUST generate this by generating 16 random bytes using a </del>  cryptographically secure random number generator. A timestamp representing the time at which the report was not succeed, clients MUST abort and discontinue retrying. The Leader MUST ignore any report pertaining to a batch that has <del> already been collected.  (See early-input-share-validation for details.)  Otherwise, comparing the aggregate result to the previous aggregate result may result in a privacy violation.  (Note that the Helpers enforce this as well.)  The Leader MAY ignore any reports whose timestamp is past the task's "task_expiration".  When it does so, the leader SHOULD abort the upload protocol and alert the client with error "reportTooLate".  Client MAY choose to opt out of the task if its own clock has passed "task_expiration". </del>  Leaders can buffer reports while waiting to aggregate them.  The leader SHOULD NOT accept reports whose timestamps are too far in the Extensions are mandatory-to-implement: If an Aggregator receives a Report containing an extension it does not recognize, then it MUST <del> reject the Report.  (See early-input-share-validation for details.) </del>  4.3.4. input-share-decryption. Check that the resulting input share is valid as described in <del> early-input-share-validation. </del>  Initialize VDAF preparation as described in input-share-prep. input-share-decryption. Check that the resulting input share is valid as described in <del> early-input-share-validation. </del>  Initialize VDAF preparation and initial outputs as described in input-share-prep. <del> [[OPEN ISSUE: consider moving the helper report ID check into early- input-share-validation]] </del>  Once the helper has processed each valid report share in "AggregateInitializeReq.report_shares", the helper then creates an Before beginning the preparation step, Aggregators are required to perform the following generic checks. <del> Check if the report has never been aggregated but is contained by a batch that has been collected.  If this check fails, the input share MUST be marked as invalid with the error "batch_collected". This prevents additional reports from being aggregated after its batch has already been collected. </del> Check if the report has already been aggregated with this aggregation parameter.  If this check fails, the input share MUST be marked as invalid with the error "report_replayed".  This is the case if the report was used in a previous aggregate request and is therefore a replay.  Depending on the query type for the task, additional checks may be applicable: Aggregator checks that "len(X) >= min_batch_size" and "len(X) <= max_batch_size", where "X" is the set of reports in the batch. <del> 4.5.7. Using a client-provided report multiple times within a single batch, or using the same report in multiple batches, may allow a server to learn information about the client's measurement, violating the privacy goal of DAP.  To prevent such replay attacks, this specification requires the aggregators to detect and filter out replayed reports. To detect replay attacks, each aggregator keeps track of the set of report IDs pertaining to reports that were previously aggregated for a given task.  If the leader receives a report from a client whose report ID is in this set, it either ignores it or aborts the upload sub-protocol as described in upload-flow.  A Helper who receives an encrypted input share whose report ID is in this set rejects the report as described in early-input-share-validation. [OPEN ISSUE: This has the potential to require aggregators to store report ID sets indefinitely.  See issue#180.] A malicious aggregator may attempt to force a replay by replacing the report ID generated by the client with a report ID its peer has not yet seen.  To prevent this, clients incorporate the report ID into the AAD for HPKE encryption, ensuring that the output share is only recovered if the aggregator is given the correct report ID.  (See upload-request.) Aggregators prevent the same report from being used in multiple batches (except as required by the protocol) by only responding to valid collect requests, as described in batch-validation. </del> 5. The DAP protocol has inherent constraints derived from the tradeoff and intra-batch replay mitigation storage, which includes some way of tracking batch report size.  Some of this state may be used for replay attack mitigation.  The replay mitigation strategy is <del> described in anti-replay. </del>  Beyond the minimal capabilities required of helpers, leaders are generally required to: In addition, for each DAP task, leaders are required to: Implement and store state for the form of inter- and intra-batch <del> replay mitigation in anti-replay. </del>  5.1.3. report upload rate, this can result in high storage costs.  To alleviate this burden, DAP allows Aggregators to drop this state as needed, so long as reports are dropped properly as described in <del> early-input-share-validation.  Aggregators SHOULD take steps to mitigate the risk of dropping reports (e.g., by evicting the oldest data first). </del>  Furthermore, the aggregators must store data related to a task as long as the current time has not passed this task's

Having to track all report IDs is a burden for sharding anti-replay storage across time. This change clarifies that, while tracking report IDs is sufficient, it is also sufficient to track any report metadata that contains the reprot ID.
cc/ NAME
LGTM
Squashed
Anti-replay is enforced in . Both DAP-01 and DAP-02 read: What's not fully specified, at least in this section, is what it means for a report to "have been aggregated (with this aggregation parameter)". For that we have been referring to the anti-replay section. reads: The report ID is defined as On the other hand, reads: The nonce is defined as Thus in DAP-01 we are required to check for collisions of the timestamp+random value, whereas in DAP-02 we are required to check for collisions of just a random value (report ID). The latter is stricter, since it disallows repeated random values. After discussing some implementation stuff with NAME I realized that this may have been an unintended regression. From a security standpoint, I think the DAP-02 requirement is better, since it forces the client implementation to generate a fresh report ID each time it generates a report. Otherwise it is easier to fingerprint reports from the same client. Does this violate assumptions in anyone's implementation? Where I can imagine this being problematic is if your implementation currently shards nonce sets into time windows. does this, but we have to amend this behavior to implement fixed-size queries anyway. Reply to this thread if this causes an issue for you and you think we should reconsider this requirement.
From Janus' perspective, I think we are OK with this change. Janus also breaks report sets into time windows -- I don't think this change will affect Janus' implementation of that functionality, though as you note this sharding behavior will change with tasks.
I'm nearly done with the implemenation ... once the code is reviewed and landed I'll try to remember to write up how I ended up organizing data.
Seems fine to me. I can't think of a reason why the client would want to reuse a reportID with a different time for the same task.
I am in favor of recommending that clients generate a good random id for every report AND for allowing the timestamp to be considered as part of the anti-replay strategy. While an unbounded memory of seen report ids would indeed be excellent protection, in practice this requires unbounded storage of report ids, which is a practical problem for implementations. Replay of a client report can leak information, so we want to assure that a report is aggregated at most once. A report has a report id and a timestamp, both generated by the reporting client. Both are included in the authenticated additional data, and cannot be altered without the tampering being evident. For the rest of this note, we will assume that the client is doing its part for its own privacy by choosing a new random report id for every report, and by having a clock is not hugely inaccurate. Even if the clients are doing their part, we must still be concerned with the actions of other entities, such as a compromised leader, helper, or collector. We also need to prevent third party manipulation, e.g. by replaying captured reports. The use of the timestamp can help us here, because the timestamp cannot be altered by any of these parties without detection. If at least one aggregator is honest and imposes the policy below, then we can use the timestamp to ease our storage burdens. Let’s consider the timestamp in more detail. For reporting accuracy we don’t want to permit just any timestamp to be aggregated. At any given time T by the server’s clock, there are bounds we want to apply based on the timestamp. Reports earlier than some time MinTime = T - MinDelta should be rejected as too old, and reports farther in the future than MaxTime = T + MaxDelta should be rejected for being so confused about the time. We allow reports slightly in the future as we want to allow for clock-skew. Call the closed interval [MinTime, MaxTime] for time T Allowed(T). We will discard reports outside of this interval when we receive them, and also when we process them. We impose the “on reception” policy to avoid queuing work that we’d never do, and we impose on processing to prevent backlog and to allow the definition of LeastRetainedTime (below). Assume a monotonically non-decreasing system clock. Whenever an aggregation begins, we durably record the current MinTime and associate it with the active state for the aggregation job, using appropriate locking in concurrent implementations. Define the LeastRetainedTime as the minimum of the current MinTime and the recorded MinTime over all active aggregation jobs (if any). It is impossible to aggregate any report before this time, as it will be rejected in any new job and isn’t present in any existing job. Garbage collection may safely remove any recorded report metadata with a timestamp before the LeastRetainedTime. We now only have to remember report ids that are greater than or equal to the LeastRetainedTime. If clients choose random ids with every report submitted, and there is at least one honest aggregator, then any given report cannot be replayed. Note that in cases where a server is so flooded with reports that it cannot process or remember them, then the DAP spec still has the "report_dropped" error available to ensure that reports are not aggregated when anti-replay cannot be assured.
Thanks NAME Full disclosure: Bob is helping out with Daphne development. When he showed me this analysis, I was pretty convinced that relaxing this requirement would be safe and worth it to reduce burden on implementations. Although I'm not strongly opposed to keeping the stronger requirement if folks think it is useful. I think the only change that would need to be made is to :
I also agree with Bob's analysis. I think we should make the change described by cjpatton in DAP-03.
This argument is good. One point: while it certainly justifies including the timestamp in the overall anti-replay strategy, it does not necessarily justify checking rather than only for the ID-based check. That is, the overall strategy could be: Check the timestamp; if it's out of the acceptable window, fail due to too old/too new. Check if [or ] is in the set of received reports; if it is, fail due to replayed-report. I would have a preference for doing the ID-based check on only, since the report ID is the primary identifier of the report, but plausibly implementations may want to check timestamps as well (for example, if they bucket report IDs by timestamp). Perhaps we could phrase things such that either implementation is allowed? In practice, as long as the report IDs are properly generated as random values, the timestamp will be irrelevant to the ID-based check. If the leader/helper disagree on whether to check or , the more restrictive check will be the one that is effectively in place. edit: another way of looking at this argument -- for the ID-based replay check, what justification is there for checking over, say, or ?
I think if we assume that honest clients always choose their report ID at random, then any anti-replay check that includes the report ID ought to be sufficient. I agree we could write this in a way that makes this implementation-specific perhaps suggesting that (report_id, timestamp) is one particularly useful relaxation.

report "metadata".  The metadata consists of the following fields: A report ID used by the Aggregators to ensure the report appears  <ins> in at most one batch.  (See input-share-validation.)  The Client MUST generate this by generating 16 random bytes using a </ins> cryptographically secure random number generator. A timestamp representing the time at which the report was not succeed, clients MUST abort and discontinue retrying. The Leader MUST ignore any report pertaining to a batch that has  <ins> already been collected.  (See input-share-validation for details.) Otherwise, comparing the aggregate result to the previous aggregate result may result in a privacy violation.  (Note that the Helpers enforce this as well.)  The Leader MAY ignore any reports whose timestamp is past the task's "task_expiration".  When it does so, the leader SHOULD abort the upload protocol and alert the client with error "reportTooLate".  Client MAY choose to opt out of the task if its own clock has passed "task_expiration". </ins> Leaders can buffer reports while waiting to aggregate them.  The leader SHOULD NOT accept reports whose timestamps are too far in the Extensions are mandatory-to-implement: If an Aggregator receives a Report containing an extension it does not recognize, then it MUST  <ins> reject the Report.  (See input-share-validation for details.) </ins> 4.3.4. input-share-decryption. Check that the resulting input share is valid as described in  <ins> input-share-validation. </ins> Initialize VDAF preparation as described in input-share-prep. input-share-decryption. Check that the resulting input share is valid as described in  <ins> input-share-validation. </ins> Initialize VDAF preparation and initial outputs as described in input-share-prep.  <ins> [[OPEN ISSUE: consider moving the helper report ID check into input- share-validation]] </ins> Once the helper has processed each valid report share in "AggregateInitializeReq.report_shares", the helper then creates an Before beginning the preparation step, Aggregators are required to perform the following generic checks.  Check if the report has already been aggregated with this aggregation parameter.  If this check fails, the input share MUST be marked as invalid with the error "report_replayed".  This is the case if the report was used in a previous aggregate request and is therefore a replay. <ins> Implementation note: To detect replay attacks, each Aggregator is required to keep track of the set of reports it has processed for a given task.  Because honest Clients choose the report ID at random, it is sufficient to store the set of IDs of processed reports.  However, implementations may find it helpful to track additional information, like the timestamp, so that the storage used for anti-replay can be sharded efficiently. Check if the report has never been aggregated but is contained by a batch that has been collected.  If this check fails, the input share MUST be marked as invalid with the error "batch_collected". This prevents additional reports from being aggregated after its batch has already been collected. </ins> Depending on the query type for the task, additional checks may be applicable: Aggregator checks that "len(X) >= min_batch_size" and "len(X) <= max_batch_size", where "X" is the set of reports in the batch.  5. The DAP protocol has inherent constraints derived from the tradeoff and intra-batch replay mitigation storage, which includes some way of tracking batch report size.  Some of this state may be used for replay attack mitigation.  The replay mitigation strategy is  <ins> described in input-share-validation. </ins> Beyond the minimal capabilities required of helpers, leaders are generally required to: In addition, for each DAP task, leaders are required to: Implement and store state for the form of inter- and intra-batch  <ins> replay mitigation in input-share-validation. </ins> 5.1.3. report upload rate, this can result in high storage costs.  To alleviate this burden, DAP allows Aggregators to drop this state as needed, so long as reports are dropped properly as described in  <ins> input-share-validation.  Aggregators SHOULD take steps to mitigate the risk of dropping reports (e.g., by evicting the oldest data first). </ins> Furthermore, the aggregators must store data related to a task as long as the current time has not passed this task's


the aggregator ("0x02" for the leader and "0x03" for the helper). The "OpenBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration.  If the leader supports multiple <del> HPKE configurations, it can use trial decryption with each configuration.  If decryption fails, the aggregator marks the report share as invalid with the error "hpke_decrypt_error".  Otherwise, the aggregator outputs the resulting PlaintextInputShare "plaintext_input_share". </del>  4.4.1.4.

The server only supplies one config, so what happens if the server would support X22519 and P-256? We can fix this by having the server supply multiple configs.
This is probably a good idea. We could just change the payload of the response to to a sequence of messages rather than one.
As mentioned in URL, it might be useful to allow the client to advertise which algorithms it supports. This would increase the likelihood of successfully negotiating an a HPKE suite that both parties support.
NAME should we try to tackle this in DAP-03?
I'm game. One minor editorial nit.

the aggregator ("0x02" for the leader and "0x03" for the helper). The "OpenBase()" function is as specified in HPKE for the ciphersuite indicated by the HPKE configuration.  If the leader supports multiple  <ins> HPKE configurations with non-distinct configuration identifiers, it can use trial decryption with each configuration.  If decryption fails, the aggregator marks the report share as invalid with the error "hpke_decrypt_error".  Otherwise, the aggregator outputs the resulting PlaintextInputShare "plaintext_input_share". </ins> 4.4.1.4.


helper, and so on. The leader responds to well-formed requests to "[leader]/upload" with <del> HTTP status code 200 OK and an empty body.  Malformed requests are handled as described in errors.  Clients SHOULD NOT upload the same measurement value in more than one report if the leader responds with HTTP status code 200 OK and an empty body. </del>  If the leader does not recognize the task ID, then it MUST abort with error "unrecognizedTask".

Section 4.2.2: "Clients SHOULD NOT upload the same measurement value in more than one report if the leader responds with status 200 and an empty body.”" Is a non-empty body understood to signal an error condition based on some other context? What if the client got status 200 from the leader but a non-empty body? Should the client retry as this text suggests? I don't know enough about HTTP semantics to know for sure, but it seems like specifying an empty body here is unnecessary. It seems fine to specify that the leader should send an empty body for 200, but maybe be less strict on the consumption side and drop the empty body requirement?

helper, and so on. The leader responds to well-formed requests to "[leader]/upload" with  <ins> HTTP status code 200 OK.  Malformed requests are handled as described in errors.  Clients SHOULD NOT upload the same measurement value in more than one report if the leader responds with HTTP status code 200 OK. </ins> If the leader does not recognize the task ID, then it MUST abort with error "unrecognizedTask".


This list is not exhaustive.  The server MAY return errors set to a URI other than those defined above.  Servers MUST NOT use the DAP URN namespace for errors not listed in the appropriate IANA registry (see <del> urn-space).  Clients SHOULD display the "detail" field of all errors. The "instance" value MUST be the endpoint to which the request was targeted.  The problem document MUST also include a "taskid" member which contains the associated DAP task ID (this value is always known, see task-configuration), encoded in Base 64 using the URL and filename safe alphabet with no padding defined in sections 5 and 3.2 of RFC4648. </del>  In the remainder of this document, the tokens in the table above are used to refer to error types, rather than the full URNs.  For

Section 3.2 discusses how to construct HTTP problem documents. In particular, states: >The problem document MUST also include a "taskid" member which contains the associated DAP task ID (this value is always known, see ), encoded in Base 64 using the URL and filename safe alphabet with no padding defined in sections 5 and 3.2 of ]. It's not true that the task ID is always known. allows implementations to respond to requests to that don't have a query parameter with an error, and in that scenario I don't think there is any sensible task ID value for the server to use. We should change this MUST to allow some problem documents to omit .
We might also consider making the key for consistency with the query parameter on .
I think this is text-ready.
Thinking about this further, I think this is actually a dupe of . Currently, we require that problem documents contain the and an value that "MUST be the endpoint to which the request was targeted". The examples of values in RFC 7807 look like this: Here, the value of is a relative path pointing to the resource that caused the error. Our requirement that be the "endpoint" seems unhelpful. If I hit some API like , and it fails, and I get back a problem document with , then that's not very helpful, because I already knew what URI I accessed when I made the request. It's also kinda annoying for implementations to have to know what public domain they're serving traffic from. For this reason, Janus always sets since RFC 7807 allows relative URLs in . Once we do , then I think we won't need a dedicated (or ) field in problem documents anymore, because if the task ID is relevant to the request, then it'll appear in the relative path that we can put in . For example, a failed upload request might yield a problem document with , and a client can parse the task ID out of that. So my plan here is to address , and in doing so, update the problem document text. If I don't get around to that for draft-03 for some reason, then the smaller change described here will suffice.
Since we're punting past draft-03, I'm going to update the tag.
Some of the requirements and recommendations in section 3.1 conflict with the spirit of RFC 7807, which it references. There is presently a requirement that: The "instance" key is defined as one of the standard members in RFC 7807, and it says: As a consequence of the current text, PPM aggregators would always provide the same URI in the "instance" field. I would recommend that we remove the MUST requirement here. Alternately, we could define a new extension member (say, "endpoint") and put the aggregator's endpoint in that member, if we still wish to have it as part of the problem details. (For what it's worth, RFC 7807 allows the "type" and "instance" members to be relative URIs, so there is already a baked-in assumption that problem detail consumers retain the request URL associated with a JSON document) Second, there's a SHOULD recommendation that the "detail" field be populated, and no mention of the "title" field. These are both standard members, defined as follows: I think we should add a recommendation that, when one of the error types from Table 1 is used, the accompanying description from the table (or a localization thereof) should be provided in the "title" member. The "detail" member could then provide more particularized information, for example, a message mentioning the old and new configuration IDs involved in an error. Third, there is currently the following requirement for a PPM-specific extension member: Assuming that an aggregator services multiple PPM tasks at one HTTP endpoint, there are edge cases where we may want to use , but not know the task ID, for example, if a request body is too short to encode a task ID. With the present text, I think aggregators would have to respond with a non-PPM error type, or just a 400 Bad Request response with an empty body. As "taskid" is the only extension member we define on our problem types, I think we should clarify the MUST requirement here, and only require it when the "type" is one of the error types in the PPM URN namespace. For the edge cases where "taskid" is not known we could relax the MUST requirement, and only require a "taskid" member when the PPM Task ID is known. Alternately, we could suggest that the error in such error cases be of type (the default type, indicating no additional semantics beyond that of the HTTP status code) and with an appropriate status line, like "400 Bad Request".
+1, I think this should be MAY. This sound fine. +1 to clarifying the edge case you point out, however I don't think this field should be required. MAY or SHOULD would be sufficient.

This list is not exhaustive.  The server MAY return errors set to a URI other than those defined above.  Servers MUST NOT use the DAP URN namespace for errors not listed in the appropriate IANA registry (see  <ins> urn-space).  The "detail" member of the Problem Details document includes additional diagnostic information. When the task ID is known (see task-configuration), the problem document SHOULD include an additional "taskid" member containing the ID encoded in Base 64 using the URL and filename safe alphabet with no padding defined in sections 5 and 3.2 of RFC4648. </ins> In the remainder of this document, the tokens in the table above are used to refer to error types, rather than the full URNs.  For


Before beginning the preparation step, Aggregators are required to perform the following generic checks.  Check if the report has already been aggregated with this aggregation parameter.  If this check fails, the input share MUST be marked as invalid with the error "report_replayed".  This is collect sub-protocol.  (See batch-validation.)  If both checks succeed, the input share is not marked as invalid.  Check if the report's timestamp has passed its task's "task_expiration" time, if so the Aggregator MAY mark the input share as invalid with the error "task_expired". The leader begins each round of continuation for a report share based on its locally computed prepare message and the previous PrepareStep from the helper.  If PrepareStep is of type "failed", then the leader <del> marks the report as failed and removes it from the candidate report set and does not process it further.  If the type is "finished", then the leader aborts with "unrecognizedMessage".  [[OPEN ISSUE: This behavior is not specified.]] If the type is "continued", then the leader proceeds as follows. </del>  Let "leader_outbound" denote the leader's prepare message and "helper_outbound" denote the helper's.  The leader computes the next

Allows helpers to reject reports from too far in the future, matching the leader check detailed in "Upload Requests". We add for this case, and allow the leader to try the report again later. Requires helpers to mark undecodable decrypted input shares as invalid with ().
The DAP specification defines a number of error modes, which ultimately end up mapped to error codes defined in section 3.1. Currently, the specified behavior does not describe how to treat message-deserialization failures, in any of client-aggregator, aggregator-aggregator, or collector-aggregator communications. We should consider specifying this. Initial thoughts: Moving to (with a new error code?) or makes sense to me, for deserialization errors during the aggregation process. It'd be very nice if one "bad" client report didn't fail the entire aggregation job, e.g. we should probably specify behavior that limits the "blast radius" of a badly-serialized message. We could leave this unspecified; I suppose the risk is that different implementations might treat this error mode with different semantics (e.g. some will fail a single report, some will fail the entirety of the aggregation job).
(This fell out of discussion on URL; FWIW, Janus will currently transition a report aggregation to if it fails to decode a message, though of course that's just one implementation's choice.)

Before beginning the preparation step, Aggregators are required to perform the following generic checks. <ins> Check that the decrypted input share can be decoded.  If not, the input share MUST be marked as invalid with the error "unrecognized_message". </ins> Check if the report has already been aggregated with this aggregation parameter.  If this check fails, the input share MUST be marked as invalid with the error "report_replayed".  This is collect sub-protocol.  (See batch-validation.)  If both checks succeed, the input share is not marked as invalid. <ins> Check if the report is too far into the future.  Implementors can provide for some small leeway, usually no more than a few minutes, to account for clock skew.  If a report is rejected for this reason, the Aggregator SHOULD mark the input share as invalid with the error "report_too_early". </ins> Check if the report's timestamp has passed its task's "task_expiration" time, if so the Aggregator MAY mark the input share as invalid with the error "task_expired". The leader begins each round of continuation for a report share based on its locally computed prepare message and the previous PrepareStep from the helper.  If PrepareStep is of type "failed", then the leader  <ins> acts based on the value of the "ReportShareError": If the error is "ReportShareError.report_too_early", then the leader MAY try to re-send the report in a later "AggregateInitializeReq". For any other error, the leader marks the report as failed, removes it from the candidate report set and does not process it further. If the type is "finished", then the leader aborts with "unrecognizedMessage".  If the type is "continued", then the leader proceeds as follows. </ins> Let "leader_outbound" denote the leader's prepare message and "helper_outbound" denote the helper's.  The leader computes the next


The client MUST abort if any of the following happen for any HPKE config request: <del> the GET request failed or did not return a valid HPKE configuration; or </del>  <del> the HPKE configuration specifies a KEM, KDF, or AEAD algorithm the client does not recognize. </del>  Aggregators SHOULD use HTTP caching to permit client-side caching of this resource RFC5861.  Aggregators SHOULD favor long cache lifetimes

The current text is out-dated. Update it to account for multiple HPKE configs returned by the Aggregator.

The client MUST abort if any of the following happen for any HPKE config request:  <ins> the GET request failed or did not return a valid HPKE config list; </ins>  <ins> the HPKE config list is empty; or no HPKE config advertised by the aggregator specifies a supported a KEM, KDF, or AEAD algorithm triple. </ins> Aggregators SHOULD use HTTP caching to permit client-side caching of this resource RFC5861.  Aggregators SHOULD favor long cache lifetimes


be used to link a report back to the Client that generated it. To generate a report, the Client begins by sharding its measurement <del> into input shares as specified by the VDAF.  It then wraps each input share in the following structure: </del>  Field "extensions" is set to the list of extensions intended to be consumed by the given Aggregator.  (See upload-extensions.)  Field the reports in sub-batches of 10.  Each sub-batch takes two round- trips to process. <del> [OPEN ISSUE: Should there be an indication of whether a given aggregate request is a continuation of a previous sub-batch?] </del> The aggregation flow can be thought of as having three phases for transforming each valid input report share into an output share: removes it from the candidate report set and does not process it further. <del> If the type is "finished", then the leader aborts with "unrecognizedMessage".  If the type is "continued", then the leader </del>  proceeds as follows. Let "leader_outbound" denote the leader's prepare message and

This PR includes several text cleanups previously folded into : The upload section explicitly describes the call to An obsolete TODO in the aggregation section is removed Text that incorrectly instructed leaders to abort if the helper finishes aggregation is corrected.

be used to link a report back to the Client that generated it. To generate a report, the Client begins by sharding its measurement  <ins> into input shares and the public share using the VDAF's sharding algorithm (VDAF): [TODO: in VDAF-04, "measurement_to_input_shares" will take a nonce argument, for which we will use the report ID #394] The Client then wraps each input share in the following structure: </ins> Field "extensions" is set to the list of extensions intended to be consumed by the given Aggregator.  (See upload-extensions.)  Field the reports in sub-batches of 10.  Each sub-batch takes two round- trips to process.  The aggregation flow can be thought of as having three phases for transforming each valid input report share into an output share: removes it from the candidate report set and does not process it further.  <ins> If the type is "finished" and the Leader's preparation of this report share is also finished, then the report share is aggregated and can now be collected (see collect-flow).  If the Leader is not finished, then the report cannot be processed further and MUST be removed from the candidate set. If the Helper's "PrepareStep" is of type "continued", then the Leader </ins> proceeds as follows. Let "leader_outbound" denote the leader's prepare message and


The number of reports included in the batch.  The vector of encrypted aggregate shares.  They MUST appear in the same order as the aggregator endpoints list of the task parameters.

The Leader now informs the Collector of what time interval is spanned by the reports aggregated into a collection, regardless of the query type. This is particularly useful for fixed size tasks as otherwise a Collector can only guess at the report times based on the timing of collection requests, but also useful in time interval tasks since the aggregated reports could span a smaller interval that what was requested (e.g., if the collector requested a one hour time interval, but reports only arrived during minutes 30-40 of that hour).
, the results of an aggregation are presented to the collector as: where is the encrypted form of . In the case of a time interval task, the collector will have explicitly requested collection over some interval of time, but in the case of a fixed size task, that aggregate result will have been compiled from a set of reports chosen by the leader. So the collector can only infer that the reports are from no later than when the collect request came in and no sooner than when the collector got the previous batch. DAP should explicitly tell the collector what time interval is covered by a collected batch.
One way to do this would be to amend . , it is defined as: We could add an to the case. However that is awkward because is also used in , so we have to make sure that there's something cogent the leader could put there. Another, simpler choice is to simply add an field to , but the downside there is that this information isn't useful in the time interval case, because the collector already knows what time interval it queried over.
I'm cool with adding an with the minimum and maximum timestamp of all reports in the batch. I think overloading is not the right approach, however, since this is meant to "partially determine" a batch of reports, where as the interval in our case is strictly informational. Unrelated follow question: if we define to be "the minimum and maximum timestamp of reports in the batch" then it's possible that . I think this is OK, and might even be useful.
+1 to not including an in ; instead, I think it should be placed directly in .
Chris' point that the interval in could be tighter than what was in is compelling. I also now think we should put the in (or, well, ). I'll draft this change and circulate it to the WG this week, as I would like to include it in draft-04, which already includes a bunch of message definition changes.

The number of reports included in the batch. <ins> The smallest interval of time that contains the timestamps of all reports included in the batch, such that the interval's start and duration are both multiples of the task's "time_precision" parameter.  Note that in the case of a "time_interval" type query (see query), this interval can be smaller than the one in the corresponding "CollectionReq.query". </ins> The vector of encrypted aggregate shares.  They MUST appear in the same order as the aggregator endpoints list of the task parameters.


"vdaf_verify_key": The VDAF verification key shared by the aggregators.  This key is used in the aggregation sub-protocol <del> (aggregate-flow).  [OPEN ISSUE: The manner in which this key is distributed may be relevant to the VDAF's security.  See issue#161.] </del>  Finally, the collector is configured with the HPKE secret key corresponding to "collector_hpke_config". 7.7.  Prio deployments should ensure that aggregators do not have common dependencies that would enable a single vendor to reassemble inputs. For example, if all participating aggregators stored unencrypted cloud vendor would be able to reassemble all the input shares and defeat privacy. <del> 7.8. </del>  <del> 7.8.1. </del>  8.

The analysis posted describes the security consideration for DAP: Assuming we agree this is an issue that needs to be addressed, what should we do about it?
One suggestion, due to NAME is to derive the task ID from the verification key by applying a one-way function. That way there is no possibility of the task ID being picked before the key. Being a bit more careful, we might use HKDF to derive both strings from a shared secret. Perhaps something like:
Should this be an ? I think we agreed that using a ROish derivation function including the client nonce would be sufficient, right?
Actually I intended "and" :) This security consideration has to do with privacy. For Prio3, we need the "query randomness" generated by the Aggregators to be "fresh". (By "fresh" I mean it needs to be random and the adversary should not be able to influence this value.) Our security model for privacy allows the adversary to pick the verifykey: In order to ensure freshness of the query randomness used for a given report, it is sufficient to (i) pick verifykey prior to generating the report and (ii) choose a random nonce. If we didn't have (i), then attacker controls the query randomness by picking a favorable verify_key to prepare the report with: if we didn't have (ii), then query randomness is predictable to the attacker.
But if the query randomness depends on the report nonce (as in ) an attacker who controls verifkey can not pick a favorable verifkey because of the RO properties. Right?
Good question. So the attacker knows and , and it can pick before the honest aggregator computes . Thus, it can query the RO with different s until it finds one for which the is "favorable". Does this make sense? In other words, "queryrand" is not uniformly distributed, even though it was output by the RO! Its distribution is biased towards whatever property is favorable to the attacker. For instance, if for some reason is favorable if its first bit is , then the attacker can easily arrange things so that the honest Aggregator computes that begins with .
If this is a problem, the problem also exists with 50% probability when using true randomness, so this shouldn't be an issue.
I'm just trying to give an example of how the attacker can bias the randomness. With more random oracle queries, it can fix the first 2 bits, 3 bits, ..., N bits, etc. It doesn't really matter. The point is: This possibility of this attack prevents a reduction to honest-verifier ZK of the FLP. (This is a standard assumption and is what is proven for the FLP we use in Prio3: See .) To avoid this, we need all inputs to the random oracle, including , to be fixed before honest Aggregators compute . Hello CFRG, This message is to share with you all some recent security analysis of draft-irtf-cfrg-vdaf-03 [1]. This is joint work with the draft editors (Phillipp and myself), Hannah Davis (UCSD), and Mike Rosulek (OSU): URL The primary goal of this paper is to formulate a provable security framework for studying candidate VDAFs. We give two definitions that formalize "privacy" and, separately, "robustness", as described in [1, Section 9]. We study two schemes in our paper: Prio3 [1, Section 7]; and a variant of Poplar1 [1, Section 8] that improves round complexity. We call this variant "Doplar". (More on this below.) Apart from some minor changes to the draft, the most important takeaways are additional security considerations for VDAF users, e.g., DAP [2], will need to attend to. I'll start with a quick overview of the security model in (1). I'll then describe how we think this model translates into security considerations in (2). In (3) I will enumerate the changes to algorithms in the draft that we are making in light of the results in this paper. I'll end in (4) with a couple of random observations. We (the authors of the paper) welcome and appreciate any feedback folks might have on the paper. We're also happy to answer questions, here or in GH issues :) On behalf of the authors: Thanks for reading! Chris P. (1) Security model In the following, we refer to the set of input shares generated by a Client (via the algorithm) as the Client's "report". In our security definitions, each report has an associated "nonce" generated by the Client as well. (1.1) Privacy The main goal for VDAFs is to ensure that, as long as some fraction of Aggregators is honest, the attacker learns nothing about the Clients' measurements beyond the aggregate result. Our privacy game (Figure 4 in the paper) gives the attacker complete control over VDAF execution, except for transmission of the input shares from honest Clients to honest Aggregators. The game models a kind of "chosen-batch attack" (analogous to a "chosen-plaintext attack" for encryption) where the attacker provides each Client with a pair of measurements, and the game picks which one is processed based on a coin toss at the start of the game. The adversary wins if, at the end of the game, it guesses correctly which batch of measurements was processed. We say the VDAF is private if no reasonable adversary wins with probability better than 1/2 (Definition 3). There are a couple features of our security game that are worth calling out. (1.1.1) First, the attacker gets to pick the VDAF verification key used for processing the reports. This models settings in which the key is generated by one (possibly malicious) Aggregator and distributed to the others. (For instance, in DAP we would like it if we could have the Leader pick this value unilaterally: See issue URL) This turns out to be fine, but we need to be somewhat careful here. The current privacy proof for Prio3 (Theorem 2) assumes the underlying FLP system ("Fully Linear Proof"; see [1, Section 7.1]) is "honest-verifier" zero-knowledge. An honest verifier picks its challenge at random, so in order to push through the reduction to VDAF privacy, we would need to emulate this random challenge in Prio3. The challenge (called the "query randomness" in the spec) is derived from the verification key and the nonce picked by the Client. When modeling the function used to derive the challenge as a random oracle, it is sufficient to ensure that (i) the VDAF verification is fixed prior to the report being generated and (ii) the nonce is chosen at random by the Client. Remark: Other restrictions on the model are possible. For instance, it may be sufficient to ensure, somehow, that the verification key is chosen randomly via some secure coin flipping protocol. (See discussion in URL) However, this would shift more complexity to DAP. Requiring (i) and (ii) seemed to be the minimally invasive change. (1.1.2) Our model involves a notion of "allowed initial states''. The "initial state" basically means "aggregation parameter", e.g., a set of candidate prefixes for Poplar1. In our analysis of Doplar (Theorem 4) we found it necessary to restrict the attacker to aggregating a given report at most one time per level of the IDPF tree. Otherwise, there are subtle attacks that arise when the validity proof for a given level is allowed to be reused. Remark: Although we did not study Poplar1 in detail, we believe there are similar attacks if the "authenticator value" for a given level of the IDPF tree is reused. Thus we have to restrict the initial states in the same way. (1.2) Robustness Our secondary goal for VDAFs is that, if all Aggregators execute the protocol correctly, then the Collector correctly computes the aggregate result of reports uploaded by honest Clients. (Reports generated by malicious Clients are filtered out during preparation.) Here we consider the attacker to be a cohort of malicious Clients who can eavesdrop on the preparation phase (i.e., it gets to see the messages exchanged by the Aggregators). The attacker's goal is to cause an Aggregator to accept a share of an invalid report. (Validity is defined by the VDAF itself; see "refinement consistency" in Section 3.1.) The game requires the VDAF verification key is unknown to the attacker. It also requires that nonces are non-repeating. We say that the VDAF is robust if the probability of the attacker winning this is small. Remark: There are important differences between the threat model for privacy and robustness. (One allows the attacker to control aggregators, while the other does not; one allows the attacker to control the network, while the other does not; one allows the verification key to be picked by the attacker, while the other requires the key to be random; one requires the nonce to be random, while the nonce to be simply non-repeating.) It's worthwhile asking whether both properties can be achieved under the same threat model. However, it is not possible for the current crop of VDAFs. (2) Security considerations Based on the security model and analysis of Prio3 and Doplar (which is related to Poplar1; see below), we have the following recommendations for additional security considerations. (2.1) Verification key requirements Some care is required in choosing the VDAF verification key. The Aggregators may use any strategy they wish to exchange this key, but the application MUST ensure that the key that is picked independently of the reports. For DAP in particular, this means it is safe for the Leader to pick the key and distribute it, but the Aggregators need to "commit" to the verification key used for a given task and use the same key for the duration of the task. (Rotation requires rolling the task configuration.) Unfortunately, this contradicts a current recommendation. From [1, Section 9]: "It is also possible to consider a stronger form of robustness, where the attacker also controls a subset of Aggregators (see [BBCGGI19], Section ). To satisfy this stronger notion of robustness, it is necessary to prevent the attacker from sharing the verification key with the Clients. It is therefore RECOMMENDED that the Aggregators generate verifykey only after a set of Client inputs has been collected for verification, and re-generate them for each such set of inputs." Note that preventing leakage of the verification key to the clients is necessary for robustness in this stronger threat model, but not sufficient. Thus this recommendation boils down to defense-in-depth for robustness. In our opinion, privacy takes precedence here. (2.1) Nonce requirements Clients MUST choose the report nonce at random. This ensures the nonce is unpredictable to a privacy attacker; and it reduces the chance of collisions, which is required for robustness. (2.2) Requirements for report processing Poplar1: Applications MUST ensure that reports are aggregated at most once per-level. Repeated usage of the authenticator at a given level may lead to subtle attacks on privacy. In particular, the current "maxbatchquery_count" mechanism of DAP [2, Section 4.5.6] is not sufficient. Prio3: Applications SHOULD ensure that each report is processed at most once. It's probably safe to allow re-processing, but (1) this is not captured in our model; (2) this gets hairy in terms of differential privacy (see issue URL); and (3) there doesn't seem to be a good reason to allow this. (It is explicitly disallowed in DAP.) (2.3) Security requirements for the PRG The object defined in [1, Section 6.2] is used to map a seed to a pseudorandom value of a suitable type, e.g., another seed, or a vector of field elements. Our security analysis models each such mapping as an independent, random oracle. As such, MUST be instantiated with a primitive that is well-suited for this purpose, like SHA-3. Relatedly, inputs to random oracles need to be "decodable". Essentially this is because the current proofs use an RO extractor for proving robustness. (3) Changes to Prio3 and Poplar1 To prove Prio3 secure in our model, we needed to make some minor adjustments to the scheme. We also have some suggestions for Poplar1 based on our analysis of Doplar, which is quite similar. We summarize our suggestions below. (Note that some of these changes have already been merged.) (3.1) Prio3: Improve domain separation of calls to . (Similarly for Poplar1.) PR: URL (3.2) Prio3: Add nonce to transcript for joint randomness derivation. This leads to an improved robustness bound. Issue: URL Closed by PR: URL (3.3) Replace with something that is more suitable for instantiating a random oracle, like SHA-3. Issue: URL Candidate PR (uses cSHAKE128): URL (3.4) Specify the nonce length. Right now the nonce is allowed to be variable length, but since privacy requires this to be randomly generated, and not all nonce lengths are safe, it makes sense to be more prescriptive here. This is also useful for making it easier to decode RO queries (see (2.3)). Issue: URL (4) Other things (4.1) What is this "Doplar" thing and what should we do with it? Doplar is like Poplar1 in that it solves the same use case (heavy hitters). It is different in that, instead of two rounds of communication for the preparation phase, only one round is required. The cost of this improvement is an increase in overall communication overhead (i.e., there are fewer messages, but they are larger in size). See Section 5.3 of the paper for a comparison. Before considering replacing Poplar1, it is worth investigating whether the communication overhead can be reduced. We are optimistic that it can be. In any case, the draft editors think that it would be best to keep Poplar1 in the current draft; if we were to specify Doplar, it would go in a new document. (4.2) Consider specializing VDAF syntax to one round preparation Given that heavy hitters is achievable with one-round preparation, we might consider specializing the syntax in [1, Section 5.2] to one round instead of many. This would make specs like DAP [2] vastly simpler. (DAP is currently suffering from a degree of complexity resulting from generality.) Acknowledgements Thanks to Simon Friedberger and Tim Geoghegan for illuminating conversations about security considerations. References [1] VDAF draft: URL [2] DAP draft: URL

"vdaf_verify_key": The VDAF verification key shared by the aggregators.  This key is used in the aggregation sub-protocol  <ins> (aggregate-flow).  The security requirements are described in verification-key. </ins> Finally, the collector is configured with the HPKE secret key corresponding to "collector_hpke_config". 7.7. <ins> The verification key for a task SHOULD be chosen before any reports are generated.  It SHOULD be fixed for the lifetime of the task and not be rotated.  One way to ensure this is to include the verification key in a derivation of the task ID. This consideration comes from current security analysis for existing VDAFs.  For example, to ensure that the security proofs for Prio3 hold, the verification key MUST be chosen independently of the generated reports.  This can be achieved as recommended above. 7.8. </ins> Prio deployments should ensure that aggregators do not have common dependencies that would enable a single vendor to reassemble inputs. For example, if all participating aggregators stored unencrypted cloud vendor would be able to reassemble all the input shares and defeat privacy.  <ins> 7.9. </ins>  <ins> 7.9.1. </ins> 8.


The following are some basic type definitions used in other messages: DAP uses the 16 byte "ReportID" as the nonce parameter for the VDAF <del> "measurement_to_input_shares" and "prep_init" methods (see VDAF. </del>  Thus for a VDAF to be compatible with DAP, it MUST specify a "NONCE_SIZE" of 16 bytes.

Catches us up with recent VDAF changes ([1], [2], [3]). We also require that VDAFs used in DAP have . [1]: URL [2]: URL [3]: URL
In the upcoming VDAF-04, . The intention is that clients will use the report ID they generate as the nonce, corresponding to . Once VDAF-04 is out, we will need to align DAP's discussion of report construction with it.
I probably missed a discussion elsewhere but why are we adding this?
URL explains why VDAF changed to take a random nonce during input sharding.
More information here: URL
It seems this didn't make it into vdaf-4. Should we also push it to dap-5 then?
VDAF-04 has the so we're good here.
Ah, okay, I searched from the beginning and it's missing in 4.1. (Which is for DAFs)
Yeah so far we don't see a need for providing a nonce for DAF sharding. Hello CFRG, This message is to share with you all some recent security analysis of draft-irtf-cfrg-vdaf-03 [1]. This is joint work with the draft editors (Phillipp and myself), Hannah Davis (UCSD), and Mike Rosulek (OSU): URL The primary goal of this paper is to formulate a provable security framework for studying candidate VDAFs. We give two definitions that formalize "privacy" and, separately, "robustness", as described in [1, Section 9]. We study two schemes in our paper: Prio3 [1, Section 7]; and a variant of Poplar1 [1, Section 8] that improves round complexity. We call this variant "Doplar". (More on this below.) Apart from some minor changes to the draft, the most important takeaways are additional security considerations for VDAF users, e.g., DAP [2], will need to attend to. I'll start with a quick overview of the security model in (1). I'll then describe how we think this model translates into security considerations in (2). In (3) I will enumerate the changes to algorithms in the draft that we are making in light of the results in this paper. I'll end in (4) with a couple of random observations. We (the authors of the paper) welcome and appreciate any feedback folks might have on the paper. We're also happy to answer questions, here or in GH issues :) On behalf of the authors: Thanks for reading! Chris P. (1) Security model In the following, we refer to the set of input shares generated by a Client (via the algorithm) as the Client's "report". In our security definitions, each report has an associated "nonce" generated by the Client as well. (1.1) Privacy The main goal for VDAFs is to ensure that, as long as some fraction of Aggregators is honest, the attacker learns nothing about the Clients' measurements beyond the aggregate result. Our privacy game (Figure 4 in the paper) gives the attacker complete control over VDAF execution, except for transmission of the input shares from honest Clients to honest Aggregators. The game models a kind of "chosen-batch attack" (analogous to a "chosen-plaintext attack" for encryption) where the attacker provides each Client with a pair of measurements, and the game picks which one is processed based on a coin toss at the start of the game. The adversary wins if, at the end of the game, it guesses correctly which batch of measurements was processed. We say the VDAF is private if no reasonable adversary wins with probability better than 1/2 (Definition 3). There are a couple features of our security game that are worth calling out. (1.1.1) First, the attacker gets to pick the VDAF verification key used for processing the reports. This models settings in which the key is generated by one (possibly malicious) Aggregator and distributed to the others. (For instance, in DAP we would like it if we could have the Leader pick this value unilaterally: See issue URL) This turns out to be fine, but we need to be somewhat careful here. The current privacy proof for Prio3 (Theorem 2) assumes the underlying FLP system ("Fully Linear Proof"; see [1, Section 7.1]) is "honest-verifier" zero-knowledge. An honest verifier picks its challenge at random, so in order to push through the reduction to VDAF privacy, we would need to emulate this random challenge in Prio3. The challenge (called the "query randomness" in the spec) is derived from the verification key and the nonce picked by the Client. When modeling the function used to derive the challenge as a random oracle, it is sufficient to ensure that (i) the VDAF verification is fixed prior to the report being generated and (ii) the nonce is chosen at random by the Client. Remark: Other restrictions on the model are possible. For instance, it may be sufficient to ensure, somehow, that the verification key is chosen randomly via some secure coin flipping protocol. (See discussion in URL) However, this would shift more complexity to DAP. Requiring (i) and (ii) seemed to be the minimally invasive change. (1.1.2) Our model involves a notion of "allowed initial states''. The "initial state" basically means "aggregation parameter", e.g., a set of candidate prefixes for Poplar1. In our analysis of Doplar (Theorem 4) we found it necessary to restrict the attacker to aggregating a given report at most one time per level of the IDPF tree. Otherwise, there are subtle attacks that arise when the validity proof for a given level is allowed to be reused. Remark: Although we did not study Poplar1 in detail, we believe there are similar attacks if the "authenticator value" for a given level of the IDPF tree is reused. Thus we have to restrict the initial states in the same way. (1.2) Robustness Our secondary goal for VDAFs is that, if all Aggregators execute the protocol correctly, then the Collector correctly computes the aggregate result of reports uploaded by honest Clients. (Reports generated by malicious Clients are filtered out during preparation.) Here we consider the attacker to be a cohort of malicious Clients who can eavesdrop on the preparation phase (i.e., it gets to see the messages exchanged by the Aggregators). The attacker's goal is to cause an Aggregator to accept a share of an invalid report. (Validity is defined by the VDAF itself; see "refinement consistency" in Section 3.1.) The game requires the VDAF verification key is unknown to the attacker. It also requires that nonces are non-repeating. We say that the VDAF is robust if the probability of the attacker winning this is small. Remark: There are important differences between the threat model for privacy and robustness. (One allows the attacker to control aggregators, while the other does not; one allows the attacker to control the network, while the other does not; one allows the verification key to be picked by the attacker, while the other requires the key to be random; one requires the nonce to be random, while the nonce to be simply non-repeating.) It's worthwhile asking whether both properties can be achieved under the same threat model. However, it is not possible for the current crop of VDAFs. (2) Security considerations Based on the security model and analysis of Prio3 and Doplar (which is related to Poplar1; see below), we have the following recommendations for additional security considerations. (2.1) Verification key requirements Some care is required in choosing the VDAF verification key. The Aggregators may use any strategy they wish to exchange this key, but the application MUST ensure that the key that is picked independently of the reports. For DAP in particular, this means it is safe for the Leader to pick the key and distribute it, but the Aggregators need to "commit" to the verification key used for a given task and use the same key for the duration of the task. (Rotation requires rolling the task configuration.) Unfortunately, this contradicts a current recommendation. From [1, Section 9]: "It is also possible to consider a stronger form of robustness, where the attacker also controls a subset of Aggregators (see [BBCGGI19], Section ). To satisfy this stronger notion of robustness, it is necessary to prevent the attacker from sharing the verification key with the Clients. It is therefore RECOMMENDED that the Aggregators generate verifykey only after a set of Client inputs has been collected for verification, and re-generate them for each such set of inputs." Note that preventing leakage of the verification key to the clients is necessary for robustness in this stronger threat model, but not sufficient. Thus this recommendation boils down to defense-in-depth for robustness. In our opinion, privacy takes precedence here. (2.1) Nonce requirements Clients MUST choose the report nonce at random. This ensures the nonce is unpredictable to a privacy attacker; and it reduces the chance of collisions, which is required for robustness. (2.2) Requirements for report processing Poplar1: Applications MUST ensure that reports are aggregated at most once per-level. Repeated usage of the authenticator at a given level may lead to subtle attacks on privacy. In particular, the current "maxbatchquery_count" mechanism of DAP [2, Section 4.5.6] is not sufficient. Prio3: Applications SHOULD ensure that each report is processed at most once. It's probably safe to allow re-processing, but (1) this is not captured in our model; (2) this gets hairy in terms of differential privacy (see issue URL); and (3) there doesn't seem to be a good reason to allow this. (It is explicitly disallowed in DAP.) (2.3) Security requirements for the PRG The object defined in [1, Section 6.2] is used to map a seed to a pseudorandom value of a suitable type, e.g., another seed, or a vector of field elements. Our security analysis models each such mapping as an independent, random oracle. As such, MUST be instantiated with a primitive that is well-suited for this purpose, like SHA-3. Relatedly, inputs to random oracles need to be "decodable". Essentially this is because the current proofs use an RO extractor for proving robustness. (3) Changes to Prio3 and Poplar1 To prove Prio3 secure in our model, we needed to make some minor adjustments to the scheme. We also have some suggestions for Poplar1 based on our analysis of Doplar, which is quite similar. We summarize our suggestions below. (Note that some of these changes have already been merged.) (3.1) Prio3: Improve domain separation of calls to . (Similarly for Poplar1.) PR: URL (3.2) Prio3: Add nonce to transcript for joint randomness derivation. This leads to an improved robustness bound. Issue: URL Closed by PR: URL (3.3) Replace with something that is more suitable for instantiating a random oracle, like SHA-3. Issue: URL Candidate PR (uses cSHAKE128): URL (3.4) Specify the nonce length. Right now the nonce is allowed to be variable length, but since privacy requires this to be randomly generated, and not all nonce lengths are safe, it makes sense to be more prescriptive here. This is also useful for making it easier to decode RO queries (see (2.3)). Issue: URL (4) Other things (4.1) What is this "Doplar" thing and what should we do with it? Doplar is like Poplar1 in that it solves the same use case (heavy hitters). It is different in that, instead of two rounds of communication for the preparation phase, only one round is required. The cost of this improvement is an increase in overall communication overhead (i.e., there are fewer messages, but they are larger in size). See Section 5.3 of the paper for a comparison. Before considering replacing Poplar1, it is worth investigating whether the communication overhead can be reduced. We are optimistic that it can be. In any case, the draft editors think that it would be best to keep Poplar1 in the current draft; if we were to specify Doplar, it would go in a new document. (4.2) Consider specializing VDAF syntax to one round preparation Given that heavy hitters is achievable with one-round preparation, we might consider specializing the syntax in [1, Section 5.2] to one round instead of many. This would make specs like DAP [2] vastly simpler. (DAP is currently suffering from a degree of complexity resulting from generality.) Acknowledgements Thanks to Simon Friedberger and Tim Geoghegan for illuminating conversations about security considerations. References [1] VDAF draft: URL [2] DAP draft: URL

The following are some basic type definitions used in other messages: DAP uses the 16 byte "ReportID" as the nonce parameter for the VDAF  <ins> "measurement_to_input_shares" and "prep_init" methods (see VDAF). </ins> Thus for a VDAF to be compatible with DAP, it MUST specify a "NONCE_SIZE" of 16 bytes.


BCP 14 RFC2119 RFC8174 when, and only when, they appear in all capitals, as shown here. <del> The following terms are used: </del> This document uses the presentation language of RFC8446 to define messages in the DAP protocol.  Encoding and decoding of these messages as byte strings also follows RFC8446.

Based on . Update the definitions to align better with terminology in the VDAF draft. Also, add "aggregation paramter" and "public share".

BCP 14 RFC2119 RFC8174 when, and only when, they appear in all capitals, as shown here.  This document uses the presentation language of RFC8446 to define messages in the DAP protocol.  Encoding and decoding of these messages as byte strings also follows RFC8446.


2. <del> The protocol is executed by a large set of clients and a small set of servers.  Servers are referred to as . Each client's input to the protocol is a set of measurements (e.g., counts of some user behavior).  Given the input set of measurements "x_1, ..., x_n" held by "n" users, the goal of a protocol for privacy preserving measurement is to compute "y = F(p, x_1, ..., x_n)" for some function "F" while revealing nothing else about the measurements. </del>  This protocol is extensible and allows for the addition of new cryptographic schemes that implement the VDAF interface specified in VDAF.  Candidates include: <del> Prio3, which allows for aggregate statistics such as sum, mean, histograms, etc.  This class of VDAFs is based on Prio CGB17 and includes improvements described in BBCGGI19. </del>  <del> Poplar1, which allows for finding the most popular strings among a collection of clients (e.g., the URL of their home page) as well as counting the number of clients that hold a given string.  This VDAF is the basis of the Poplar protocol of BBCGGI21, which is designed to solve the heavy hitters problem in a privacy </del>  preserving manner. <del> This protocol is designed to work with schemes that use secret sharing.  Rather than sending its input in the clear, each client shards its measurements into a sequence of and sends an input share to each of the aggregators.  This provides two important properties: </del>  <del> It is impossible to deduce the measurement without knowing </del>  <del> of the shares. It allows the aggregators to compute the final output by first aggregating up their measurements shares locally, then combining the results to obtain the final output. </del>  2.1. <del> The overall system architecture is shown in pa-topology. </del>  [[OPEN ISSUE: This shows two helpers, but the document only allows one for now.  https://github.com/ietf-wg-ppm/draft-ietf-ppm-dap/ The main participants in the protocol are as follows: The basic unit of DAP is the "task" which represents a single <del> measurement (though potentially taken over multiple time windows). The definition of a task includes the following parameters: </del>  The type of each measurement. The aggregation function to compute (e.g., sum, mean, etc.). <del> The set of aggregators and necessary cryptographic keying material </del>  to use. The VDAF to execute, which to some extent is dictated by the The minimum "batch size" of reports which can be aggregated. The rate at which measurements can be taken, i.e., the "minimum <del> batch window". </del>  <del> These parameters are distributed out of band to the clients and to the aggregators.  They are distributed by the collecting entity in some authenticated form.  Each task is identified by a unique 32-byte ID which is used to refer to it in protocol messages. </del>  <del> During the duration of the measurement, each client records its own value(s), packages them up into a report, and sends them to the leader.  Each share is separately encrypted for each aggregator so that even though they pass through the leader, the leader is unable to see or modify them.  Depending on the measurement, the client may only send one report or may send many reports over time. </del>  <del> The leader distributes the shares to the helpers and orchestrates the </del>  process of verifying them (see validating-inputs) and assembling them <del> into a final measurement for the collector.  Depending on the VDAF, it may be possible to incrementally process each report as it comes in, or may be necessary to wait until the entire batch of reports is received. </del>  2.2. An essential task of any data collection pipeline is ensuring that the data being aggregated is "valid".  In DAP, input validation is complicated by the fact that none of the entities other than the <del> client ever sees the values for individual clients. </del>  In order to address this problem, the aggregators engage in a secure, multi-party computation specified by the chosen VDAF VDAF in order to prepare a report for aggregation.  At the beginning of this <del> computation, each aggregator is in possession of an input share </del>  uploaded by the client.  At the end of the computation, each <del> aggregator is in possession of either an "output share" that is ready </del>  to be aggregated or an indication that a valid output share could not be computed. client include information used by the aggregators during aggregation in order to validate their corresponding output shares.  For example, Prio3 includes a distributed zero-knowledge proof of the input's <del> validity BBCGGI19 which the aggregators can jointly verify and reject </del>  the report if it cannot be verified.  However, they do not learn anything about the individual report other than that it is valid. the measurement being taken.  For instance, to measure the time the user took performing a given task the proof might demonstrate that the value reported was within a certain range (e.g., 0-60 seconds). <del> By contrast, to report which of a set of N options the user select, the report might contain N integers and the proof would demonstrate that N-1 were 0 and the other was 1. </del>  It is important to recognize that "validity" is distinct from "correctness".  For instance, the user might have spent 30s on a task

Based on .
NAME can you please rebase? Some of the changes we improved on in now appear here.
Rebased.

2.  <ins> The protocol is executed by a large set of Clients and a small set of servers.  Servers are referred to as "Aggregators".  Each Client's input to the protocol is its measurement (or set of measurements, e.g., counts of some user behavior).  Given the input set of measurements "x_1, ..., x_n" held by "n" users, the goal of DAP is to compute "y = F(p, x_1, ..., x_n)" for some function "F" while revealing nothing else about the measurements.  We call "F" the "aggregation function". </ins> This protocol is extensible and allows for the addition of new cryptographic schemes that implement the VDAF interface specified in VDAF.  Candidates include:  <ins> Prio3 (VDAF), which allows for aggregate statistics such as sum, mean, histograms, etc. </ins>  <ins> Poplar1 (VDAF), which allows for finding the most popular strings uploaded by a set of Clients (e.g., the URL of their home page) as well as counting the number of Clients that hold a given string. This VDAF is the basis of the Poplar protocol of BBCGGI21, which is designed to solve the heavy hitters problem in a privacy </ins> preserving manner.  <ins> VDAFs rely on secret sharing to protect the privacy of the measurements.  Rather than sending its input in the clear, each client shards its measurements into a sequence of "input shares" and sends an input share to each of the Aggregators.  This provides two important properties: </ins>  <ins> Given only a subset of the input shares, it is impossible to deduce the plaintext measurement from which they were generated. </ins>  <ins> It allows the Aggregators to compute the aggregation function by first aggregating up their input shares locally into "aggregate shares", then combining the aggregate shares into the aggregate result. </ins> 2.1.  <ins> The overall system architecture is shown in dap-topology. </ins> [[OPEN ISSUE: This shows two helpers, but the document only allows one for now.  https://github.com/ietf-wg-ppm/draft-ietf-ppm-dap/ The main participants in the protocol are as follows: The basic unit of DAP is the "task" which represents a single  <ins> measurement process (though potentially taken over multiple time windows).  The definition of a task includes the following parameters: </ins> The type of each measurement. The aggregation function to compute (e.g., sum, mean, etc.).  <ins> The set of Aggregators and necessary cryptographic keying material </ins> to use. The VDAF to execute, which to some extent is dictated by the The minimum "batch size" of reports which can be aggregated. The rate at which measurements can be taken, i.e., the "minimum  <ins> batch duration". </ins>  <ins> These parameters are distributed out of band to the Clients and to the Aggregators.  They are distributed by the Collector in some authenticated form.  Each task is identified by a unique 32-byte ID which is used to refer to it in protocol messages. </ins>  <ins> During the duration of the task, each Client records its own measurement value(s), packages them up into a report, and sends them to the leader.  Each share is separately encrypted for each Aggregator so that even though they pass through the Leader, the Leader is unable to see or modify them.  Depending on the task, the Client may only send one report or may send many reports over time. </ins>  <ins> The Leader distributes the shares to the Helpers and orchestrates the </ins> process of verifying them (see validating-inputs) and assembling them  <ins> into a final aggregate result for the Collector.  Depending on the VDAF, it may be possible to incrementally process each report as it comes in, or may be necessary to wait until the entire batch of reports is received. </ins> 2.2. An essential task of any data collection pipeline is ensuring that the data being aggregated is "valid".  In DAP, input validation is complicated by the fact that none of the entities other than the  <ins> Client ever sees that client's plaintext measurement. </ins> In order to address this problem, the aggregators engage in a secure, multi-party computation specified by the chosen VDAF VDAF in order to prepare a report for aggregation.  At the beginning of this  <ins> computation, each Aggregator is in possession of an input share </ins> uploaded by the client.  At the end of the computation, each  <ins> Aggregator is in possession of either an "output share" that is ready </ins> to be aggregated or an indication that a valid output share could not be computed. client include information used by the aggregators during aggregation in order to validate their corresponding output shares.  For example, Prio3 includes a distributed zero-knowledge proof of the input's  <ins> validity BBCGGI19 which the Aggregators can jointly verify and reject </ins> the report if it cannot be verified.  However, they do not learn anything about the individual report other than that it is valid. the measurement being taken.  For instance, to measure the time the user took performing a given task the proof might demonstrate that the value reported was within a certain range (e.g., 0-60 seconds).  <ins> By contrast, to report which of a set of "N" options the user select, the report might contain "N" integers and the proof would demonstrate that "N-1" were "0" and the other was "1". </ins> It is important to recognize that "validity" is distinct from "correctness".  For instance, the user might have spent 30s on a task


After stepping each state, the Helper advances its aggregation job to the Leader's "AggregationJobContinueReq.round".  If the "round" in the Leader's request is equal to the Helper's current round (i.e., this is not the first time the Leader has sent this request), then the Helper responds with the current round's

During implementation of round skew recovery in Janus ([1]), we noticed that the specification is unclear about what the helper should do if . This commit clarifies that this value is illegal and should be rejected. [1]: URL After handling , the helper's aggregation job will be on round 0, because the helper is waiting for the leader to send it the first-round broadcast prepare message, with which helper will advance to round 1. So before this commit, the applicable statement in the text was: Which means the helper would have to retransmit its first-round prepare shares, which it had previously sent in response to . This is implementable but weird: if the leader dropped the response to , then it should recover by re-sending it (I just wasted months of my life making sure that PUT to an aggregation job is idempotent). It seems simpler to me to just forbid round to be 0.

After stepping each state, the Helper advances its aggregation job to the Leader's "AggregationJobContinueReq.round". <ins> If the "round" in the Leader's request is 0, then the Helper MUST abort with an error of type "unrecognizedMessage". </ins> If the "round" in the Leader's request is equal to the Helper's current round (i.e., this is not the first time the Leader has sent this request), then the Helper responds with the current round's