the HTTP server responds with an HTTP error status as described in errors. <del> If a fatal error is encountered while processing an HTTP request, </del>  the HTTP client... [TODO: Figure out what behavior is called for here.  An HTTP request should be sent that conveys the error.] FAILED without sending a message to the helper.  Otherwise it interprets "out" as follows.  If this is the last round of VDAF preparation phase, then "out" is the leader's output share, in which <del> case the leader finishes.  Otherwise, it interpresets "out" as the </del>  tuple "(new_state, outbound)", where "new_state" is its new preparation state and "outbound" is its next VDAF messaage, and continues with "outbound" as its next VDAF message.  Either way, it the tag.  If this check fails, it MUST abort with error "invalidHmac". <del> Next, it first checks that the sequence of Transition messages corresponds to the ReportShare sequence of the AggregateInitReq.  If any message appears out of order or is missing, or a transition has an unrecognized nonce, then the leader MUST abort with error </del>  "unrecognizedMessage". Next, for each Transition, the leader proceeds as described in prep-

While implementing upload today, I also noticed some inconsistencies between the handling of reports in the leader during the upload protocol and the helper

the HTTP server responds with an HTTP error status as described in errors.  <ins> If a fatal error is encountered while processing an HTTP response, </ins> the HTTP client... [TODO: Figure out what behavior is called for here.  An HTTP request should be sent that conveys the error.] FAILED without sending a message to the helper.  Otherwise it interprets "out" as follows.  If this is the last round of VDAF preparation phase, then "out" is the leader's output share, in which  <ins> case the leader finishes.  Otherwise, it interprets "out" as the </ins> tuple "(new_state, outbound)", where "new_state" is its new preparation state and "outbound" is its next VDAF messaage, and continues with "outbound" as its next VDAF message.  Either way, it the tag.  If this check fails, it MUST abort with error "invalidHmac".  <ins> Next, it checks that the sequence of Transition messages corresponds to the ReportShare sequence of the AggregateInitReq.  If any message appears out of order or is missing, or a transition has an unrecognized nonce, then the leader MUST abort with error </ins> "unrecognizedMessage". Next, for each Transition, the leader proceeds as described in prep-


leader's, the second share should be for the first helper, and so on). <del> Encrypted input shares are structured as follows: "aggregator_config_id" is equal to "HpkeConfig.id", where "HpkeConfig" is the key config of the aggregator receiving the input share. "enc" is the HPKE encapsulated key, used by the aggregator to decrypt its input share. "payload" is the encrypted input share. </del> To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To encrypt an input share, the client first generates an HPKE I-D.irtf- cfrg-hpke context for the aggregator by running <del> where "pk" is the aggregator's public key, "task_id" is "Report.task_id" and "server_role" is a byte whose value is "0x01" if the aggregator is the leader and "0x00" if the aggregator is the helper. "enc" is the HPKE encapsulated key and "context" is the HPKE context used by the client for encryption.  The payload is encrypted as </del>  where "input_share" is the aggregator's input share and "nonce" and <del> "extensions" are the corresponding fields of "Report". </del>  The leader responds to well-formed requests to "[leader]/upload" with status 200 and an empty body.  Malformed requests are handled as value in more than one report if the leader responds with status 200 and an empty body. <del> The leader responds to requests with out-of-date "HpkeConfig.id" values, indicated by "EncryptedInputShare.config_id", with status 400 and an error of type 'outdatedConfig'.  Clients SHOULD invalidate any cached aggregator "HpkeConfig" and retry with a freshly generated Report.  If this retried report does not succeed, clients MUST abort and discontinue retrying. </del>  The leader MUST ignore any report whose nonce contains a timestamp that falls in a batch interval for which it has received at least one The "nonce" and "extensions" fields have the same value as those in the report uploaded by the client.  Similarly, the "helper_share" <del> field is the "EncryptedInputShare" from the "Report" whose index in </del>  "Report.encrypted_input_shares" is equal to the index of "[aggregator]" in the task's aggregator endpoints.  [OPEN ISSUE: We usually only need to send this in the first aggregate request.  Shall where "sk" is the HPKE secret key, "task_id" is "AggregateReq.task_id" and "server_role" is the role of the server <del> ("0x01" for the leader and "0x00" for the helper). "nonce" and "extensions" are obtained from the corresponding fields in "AggregateSubReq".  If decryption fails, then the sub-response consists of a "decryption error" alert.  [See issue#57.]  Otherwise, the helper handles the request for its plaintext input share "input_share" and updates its state as specified by the PPM protocol. </del>  After processing all of the sub-requests, the helper encrypts its updated state and constructs its response to the aggregate request. collector's public key as follows: where "pk" is the HPKE public key encoded by the collector's HPKE key <del> configuration, "task_id" is "AggregateShareReq.task_id" and "server_role" is the role of the server ("0x01" for the leader and "0x00" for the helper).  "agg_share" is the serialized "AggregateShare", and "batch_interval" is obtained from the "AggregateShareReq". </del>  This encryption prevents the leader from learning the actual result, as it only has its own share and not the helper's share, which is <del> encrypted for the collector.  The helper responds to the collector with HTTP status 200 OK and a body consisting of the following structure: "collector_hpke_config_id" is "collector_config.id" from the task parameters corresponding to "CollectReq.task_id". "enc" is the HPKE encapsulated key, used by the collector to decrypt the aggregate share. "payload" is an encrypted "AggregateShare". </del>  The leader uses the helper's aggregate share response to respond to the collector's collect request (see pa-collect). the helper's encrypted aggregate share for the batch interval by sending an "AggregateShareReq" to the helper as described in aggregate-share-request.  The leader then computes its own aggregate <del> share by aggregating all of the prepared output shares that fall within the batch interval. </del>  When both aggregators' shares are successfully obtained, the leader responds to subsequent HTTP GET requests to the collect job's URI with HTTP status 200 OK and a body consisting of a "CollectResp": <del> "shares" is a vector of "EncryptedAggregateShare"s, as described in aggregate-share-request, except that for the leader's share, the "task_id" and "batch_interval" used to encrypt the "AggregateShare" are obtained from the "CollectReq". </del>  If obtaining aggregate shares fails, then the leader responds to subsequent HTTP GET requests to the collect job URI with an HTTP where "sk" is the HPKE secret key, "task_id" is "AggregateReq.task_id" and "server_role" is the role of the server <del> that send the aggregate share ("0x01" for the leader and "0x00" for </del>  the helper). "batch_interval" is the "Interval" from the "CollectReq".  The collector then unshards the aggregate shares into an aggregate result using the VDAF's "agg_shares_to_result"

When constructing HPKE contexts, we now include both sender and recipient roles, and the application label also includes the PPM version. I used here because we recently submitted draft-01 of and so the next submitted draft, which would include this change, will be . I also backported the definition of struct HpkeCiphertext from the interop PR because it makes everything tidier and should be non-controversial we can get a head-start on bringing those changes into main.
Fixed, thank you! I also rewrote the relevant paragraph to acknowledge that there's two encrypted input shares and that the leader should examine the HPKE config ID of the one intended for itself. Further, I recall NAME pointing out to me that should this draft get adopted by the WG, we'll reset to something like , so is the more appropriate application label.
There's one more instance of that needs to be changed to . Otherwise, LGTM.

leader's, the second share should be for the first helper, and so on).  To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To encrypt an input share, the client first generates an HPKE I-D.irtf- cfrg-hpke context for the aggregator by running  <ins> where "pk" is the aggregator's public key and "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper).  In general, the info string for computing the HPKE context is suffixed by two bytes, the first of which identifies the role of the sender and the second of which identifies the role of the intended recipient. "enc" is the HPKE encapsulated key and "context" is the HPKE context used by the client for encryption.  The payload is encrypted as </ins> where "input_share" is the aggregator's input share and "nonce" and  <ins> "extensions" are the corresponding fields of "Report".  Clients MUST NOT use the same "enc" for multiple reports. </ins> The leader responds to well-formed requests to "[leader]/upload" with status 200 and an empty body.  Malformed requests are handled as value in more than one report if the leader responds with status 200 and an empty body.  <ins> The leader responds to requests whose leader encrypted input share uses an out-of-date "HpkeConfig.id" value, indicated by "HpkeCiphertext.config_id", with status 400 and an error of type 'outdatedConfig'.  Clients SHOULD invalidate any cached aggregator "HpkeConfig" and retry with a freshly generated Report.  If this retried report does not succeed, clients MUST abort and discontinue retrying. </ins> The leader MUST ignore any report whose nonce contains a timestamp that falls in a batch interval for which it has received at least one The "nonce" and "extensions" fields have the same value as those in the report uploaded by the client.  Similarly, the "helper_share"  <ins> field is the "HpkeCiphertext" from the "Report" whose index in </ins> "Report.encrypted_input_shares" is equal to the index of "[aggregator]" in the task's aggregator endpoints.  [OPEN ISSUE: We usually only need to send this in the first aggregate request.  Shall where "sk" is the HPKE secret key, "task_id" is "AggregateReq.task_id" and "server_role" is the role of the server  <ins> ("0x02" for the leader and "0x03" for the helper; "0x01", the client role, is used as the sender). "nonce" and "extensions" are obtained from the corresponding fields in "AggregateSubReq".  If decryption fails, then the sub-response consists of a "decryption error" alert. [See issue#57.]  Otherwise, the helper handles the request for its plaintext input share "input_share" and updates its state as specified by the PPM protocol. </ins> After processing all of the sub-requests, the helper encrypts its updated state and constructs its response to the aggregate request. collector's public key as follows: where "pk" is the HPKE public key encoded by the collector's HPKE key  <ins> configuration and "task_id" is "AggregateShareReq.task_id". "agg_share" is the serialized "AggregateShare", and "batch_interval" is obtained from the "AggregateShareReq". </ins> This encryption prevents the leader from learning the actual result, as it only has its own share and not the helper's share, which is  <ins> encrypted for the collector.  The helper responds to the leader with HTTP status 200 OK and a body consisting of an "HpkeCiphertext" where "config_id" is set to the collector's HPKE config ID, "enc" is set to the encapsulated HPKE context "enc" and "ciphertext" is "encrypted_agg_share". </ins> The leader uses the helper's aggregate share response to respond to the collector's collect request (see pa-collect). the helper's encrypted aggregate share for the batch interval by sending an "AggregateShareReq" to the helper as described in aggregate-share-request.  The leader then computes its own aggregate  <ins> share "agg_share" by aggregating all of the prepared output shares that fall within the batch interval.  Finally, it encrypts it under the collector's HPKE public key as follows: where "pk" is the collector's public key. </ins> When both aggregators' shares are successfully obtained, the leader responds to subsequent HTTP GET requests to the collect job's URI with HTTP status 200 OK and a body consisting of a "CollectResp":  <ins> The "encrypted_agg_shares" field is the vector of encrypted aggregate shares.  They MUST appear in the same order as the aggregator endpoints list of the task parameters. </ins> If obtaining aggregate shares fails, then the leader responds to subsequent HTTP GET requests to the collect job URI with an HTTP where "sk" is the HPKE secret key, "task_id" is "AggregateReq.task_id" and "server_role" is the role of the server  <ins> that sent the aggregate share ("0x02" for the leader and "0x03" for </ins> the helper). "batch_interval" is the "Interval" from the "CollectReq".  The collector then unshards the aggregate shares into an aggregate result using the VDAF's "agg_shares_to_result"


The following messages are defined in this section and are used to aggregate a number of reports simultaneously: <del> [CP: Since these parameters are required for all variants, I think we should consider moving task_id and job_id to precede the message type.] </del> As discussed in aggregation-flow, the leader may choose to shard the task of aggregating a large number of reports into any number of sub- batches.  The process of aggregating the reports in a leader-defined order to initialize the process of preparing a set of reports for aggregation.  It is structured as follows: <del> The message contains the PPM task ID, the aggregation job ID assigned by the leader, an opaque, VDAF-specific aggregation parameter, and the helper's report shares. </del>  To construct an AggregateInitReq, the leader begins by picking a sequence of reports, all of which MUST pertain to the same PPM task Next, the leader extracts its own report shares and computes the initial preparation state for each as described in prep-start.  It then removes from the report sequence any report for which the <del> transition state is FAILED. </del>  Let "[aggregator]" denote the helper's API endpoint.  The leader extracts the sequence of report shares for "[aggregator]" and sends payload. After verifying the authentication tag as described in aggregate- <del> message-auth, the helper handles the AggregateInitReq by computing the initial preparation state for each ReportShare in </del>  "AggregateInitReq.seq" as described in prep-start.  It then constructs an AggregateResp: Next, it checks that the sequence of Transition messages corresponds to the ReportShare sequence of the AggregateInitReq.  If any message <del> appears out of order or is missing, or a transition has an unrecognized nonce, then the leader MUST abort with error "unrecognizedMessage". </del>  Next, for each Transition, the leader proceeds as described in prep- leader.  If any state transition results in INVALID, this indicates Finally, the leader removes from the report sequence any report for which the corresponding state is FAILED and proceeds as described in <del> the next section. </del>  4.4.4.2. advance the preparation state of each report being aggregated.  This message is defined as follows: <del> The contents are the PPM task ID, the aggregation job ID, and the sequence of Transition messages corresponding to the report sequence being aggregated.  The sequences MUST have the same order. </del>  Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with an Aggregate message of type "agg_cont_req" with payload AggregateContinueReq. After verifying the authentication tag as described in aggregate- <del> message-auth, the helper processes the AggregateContinueReq as follows.  It first scans "AggregateContinueReq.seq" to check for any reports that may have been dropped by the leader in the previous step. </del>  Next, the helper processes the Transition messages from the leader. <del> If any leader Transition contains an unrecognized nonce, the Helper sends a Transition with "tran_type = failed" and "TransitionError = unrecognized-nonce".  [CP: It may be better to go to abort rather than just fail, since this s tentamount to sending a malformed aggregate request.]  If any Transition appears out-of-order, the helper MUST abort with error "unrecognizedMessage". </del>  <del> Next, it constructs a transition as described in prep-helper.  If any state transition results in INVALID, the helper MUST abort with error "unrecognizedMessage". </del>  Next, the helper constructs an AggregateResp containing its next <del> flight of Transition messages and it updated state.  The messages </del>  MUST appear in the same order as "AggregateContinueReq.seq".  The helper's response is an HTTP 200 OK whose body is the AggregateResp.  4.4.4.3.

Minor changes/clarifications: Clarify how to handle empty Transition sequences Handle repeated nonces by aborting. While at it, this changes how the helper handles unrecognized nonces to be consistent with how the leader handles unrecognized nonces. Move task/job ID to top of AggregateReq

The following messages are defined in this section and are used to aggregate a number of reports simultaneously:  As discussed in aggregation-flow, the leader may choose to shard the task of aggregating a large number of reports into any number of sub- batches.  The process of aggregating the reports in a leader-defined order to initialize the process of preparing a set of reports for aggregation.  It is structured as follows:  <ins> The message contains an opaque, VDAF-specific aggregation parameter and the helper's report shares. </ins> To construct an AggregateInitReq, the leader begins by picking a sequence of reports, all of which MUST pertain to the same PPM task Next, the leader extracts its own report shares and computes the initial preparation state for each as described in prep-start.  It then removes from the report sequence any report for which the  <ins> transition state is FAILED.  If the resulting sequence is empty, then the leader halts without issuing a request. </ins> Let "[aggregator]" denote the helper's API endpoint.  The leader extracts the sequence of report shares for "[aggregator]" and sends payload. After verifying the authentication tag as described in aggregate-  <ins> message-auth, the helper first checks that the nonces in "AggregateInitReq.seq" are all distinct.  If two ReportShare messages have the same nonce, then the helper MUST abort with error "unrecognizedMessage".  Next, the helper handles the AggregateInitReq by computing the initial preparation state for each ReportShare in </ins> "AggregateInitReq.seq" as described in prep-start.  It then constructs an AggregateResp: Next, it checks that the sequence of Transition messages corresponds to the ReportShare sequence of the AggregateInitReq.  If any message  <ins> appears out of order, is missing, has an unrecognized nonce, or if two messages have the same nonce, then the leader MUST abort with error "unrecognizedMessage". </ins> Next, for each Transition, the leader proceeds as described in prep- leader.  If any state transition results in INVALID, this indicates Finally, the leader removes from the report sequence any report for which the corresponding state is FAILED and proceeds as described in  <ins> the next section, even if the sequence is empty. </ins> 4.4.4.2. advance the preparation state of each report being aggregated.  This message is defined as follows:  <ins> The message consists of a sequence of Transition messages corresponding to the report sequence being aggregated.  The sequences MUST have the same order. </ins> Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with an Aggregate message of type "agg_cont_req" with payload AggregateContinueReq. After verifying the authentication tag as described in aggregate-  <ins> message-auth, the helper begins by scanning "AggregateContinueReq.seq" to check for any reports that may have been dropped by the leader in the previous step. </ins> Next, the helper processes the Transition messages from the leader.  <ins> If any message appears out of order or has an unrecognized nonce, or if any two messages have the same nonce, then the helper MUST abort with error "unrecognizedMessage". </ins>  <ins> Next, the helper processes each Transition as described in prep- helper.  If any state transition results in INVALID, the helper MUST abort with error "unrecognizedMessage". </ins> Next, the helper constructs an AggregateResp containing its next  <ins> flight of Transition messages and its updated state.  The messages </ins> MUST appear in the same order as "AggregateContinueReq.seq".  The helper's response is an HTTP 200 OK whose body is the AggregateResp. <ins> The leader processes this message as described in aggregate-init- request. </ins> 4.4.4.3.


with which the batch interval of each collect request must be aligned.  (See batch-parameter-validation.) <del> A unique identifier for the VDAF being used for the task. </del>  In addition, in order to facilitate the aggregation and collect protocols, each of the aggregators is configured with following

Extracted and slightly modified from .
(Build failure appears to be some GitHub API issue.)
Ah, this looks like we're hitting the patch for URL I'll check if the upstream template repository has a fix.
Thanks NAME
!

with which the batch interval of each collect request must be aligned.  (See batch-parameter-validation.)  <ins> A unique identifier for the VDAF instance used for the task, including the type of measurement associated with the task. </ins> In addition, in order to facilitate the aggregation and collect protocols, each of the aggregators is configured with following


The aggregator keeps track of the number of times each report was aggregated.  If any input share in "X" was added to at least "max_batch_lifetime" previous batches, then the aggregator aborts <del> and alerts the peer with error "privacyBudgetExceeded". </del>  4.7.

Adds definitions of error codes to section 3.1 for validation of batch intervals in or . Also adds an error code for leader to use when rejecting a report from too far in the future.
NAME I can't request review from you until you accept the invite to the ietf-wg-ppm organization but you should see this PR too!

The aggregator keeps track of the number of times each report was aggregated.  If any input share in "X" was added to at least "max_batch_lifetime" previous batches, then the aggregator aborts  <ins> and alerts the peer with error "batchLifetimeExceeded". </ins> 4.7.


Once a set of clients have uploaded their reports to the leader, the leader can send them to the helpers to be verified and aggregated. In order to enable the system to handle very large batches of <del> reports, this process can be performed incrementally.  To aggregate a set of reports, the leader sends a message to each helper containing those report shares.  The helper then processes them (verifying the proofs and incorporating their values into the ongoing aggregate) and replies to the leader. The exact structure of the aggregation flow depends on the VDAF. </del>  Specifically: Some VDAFs (e.g., "prio3") allow the leader to start aggregating 4.3.1. <del> The leader begins aggregation by choosing a set of candidate reports that pertain to the same PPM task.  The leader can run this process for many candidate reports in parallel as needed.  After choosing the set of candidates, the leader begins aggregation by splitting each report into "report shares", one for each aggregator.  The leader and helpers then run the aggregate initialization flow to accomplish two tasks: </del>  Recover and determine which input report shares are invalid. The leader begins the aggregate initialization phase with the set of candidate report shares as follows:  Decrypt the input share for each report share as described in input-share-decryption. "Report.encrypted_input_shares" is equal to the index of the aggregator in the task's "aggregator_endpoints" to which the AggregateInitReq is being sent.  The "agg_param" field is an opaque, <del> VDAF-specific aggregation parameter.  The "helper_state" parameter contains the helper's state.  This is an optional parameter of an aggregate request that the helper can use to carry state across requests and across aggregate flows. </del>  Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its AggregateInitResp message to complete its initialization.  This message is structured as follows: <del> The "helper_state" parameter contains the helper's initial state. This is an optional parameter of an aggregate request that the helper can use to carry state across requests.  At least part of the state will usually need to be encrypted in order to protect user privacy. However, the details of precisely how the state is encrypted and the information that it carries is up to the helper implementation. [[OPEN ISSUE: we may end up removing helper_state.  See #185]] The rest of the message is a sequence of PrepareStep values, the order of which matches that of the ReportShare values in </del>  "AggregateInitReq.report_shares".  Each report that was marked as invalid is assigned the PrepareStepResult "failed".  Otherwise, the PrepareStep is either marked as continued with the output "prep_msg", "checksum" is the checksum computed over the set of client reports. <del> "helper_state" is the helper's state, which is carried across requests from the leader. </del>  To handle the leader's request, the helper first ensures that the request meets the requirements for batch parameters following the

Since aggregators already need to store nonce sets for the purpose of replay prevention, it doesn't make much sense to also offload part of their state to the leader. Instead, aggregators might benefit from a unique index into some local storage for keeping track of per-aggregation job state. This change drops the helper_state in favor of such an index.
A few points: This seems strictly less flexible. The helpers could already store an index if they wanted, but now they are required to do so. Can't you store nonce sets in the helper state?
Yeah, they could store an index in helperstate. However, helperstate is a "per-aggregation" storage mechanism, not a "per-task" mechanism, and (I think) the replay state needs to cover the entire task. So we could (a) change helper_state to be a per-task thing, in which case it's not clear 2^16 is enough space, or if we want to be paying that bandwidth for the duration of the task, or (b) go with the layer of indirection as in here. On balance, I think (b) is probably better?
This can get really hairy. Think of this way: Storing nonce sets is required for report replay protection, which is needed for privacy. Depending on what state the helper has locally for managing the "version" of the state blob, it may be possible for the leader to "replay" an old state and bypass replay protection for reports. There are other ways that helper state gets hairy. For example, we haven't been precise yet about how handling concurrent aggregation flows: If the helper state is "per-task", then concurrent flows would be a problem. Overall the goal of this change is to remove this complexity altogether. We can always add it back later if we need to.
I don't agree that this change reduces complexity. AFAICT, this is two changes: It has the leader assign the ID. It restricts the ID space to uint32. ISTM that it's just as straightforward for the helper to assign the id at the start of the job, and then use any format it chooses, which the current system allows. I get that you think that certain ways of assigning and using that ID are complex, but that's not our problem because the helper can figure it out for themselves.
You mean 32-byte strings, right? uint32 is just 8 bytes, whereas the AggregationJobID is 32 bytes. Anther important thing it does is remove the state blob from the AggregateShareReq. Here's the bit I think you're missing: Right now we require the leader to send the same it received in the helper's previous response. This means that all aggregate requests, i.e., AggregateInitReq, AggregateContReq, and AggregateShareReq, need to be issued one after the other. In particular, it's not possible to run multiple aggregation flows concurrently. You're forced to do before you do (Note that the "aggregation flow" excludes the AggregateShareReq, which is used to combine the results across multiple aggregation flows.) It's going to be important for scalability that we can run multiple aggregation flows simultaneously. That means that we need, at a minimum, the following things: Don't carry state across aggregation flows Make sure there's a way for a helper to "link" aggregate requests pertaining to the same aggregation flow This PR accomplishes both of these, but as you point out, is more strict than necessary. Perhaps we could keep the helper state blob as-is, but remove it from the AggregateShareReq and mandate only that the helper state is maintained for the duration of a single aggregation flow.
I'm undecided about whether this is a good or bad change, but it should be decided on its own merits, not buried in a PR entitled "Replace helper_state with index into per-aggregator storage"
I believe this change does yield a significant reduction in complexity, because we no longer have to spell out how the helper needs to protect its state from the leader. Aggregation job IDs do not need to be kept secret from the leader, nor does the helper need to need to implement any anti-replay protections for them (because all the sensitive state is now assumed to be in the helper's database).
NAME The reasoning for this was originally discussed on issue . This issue didn't get much attention, so we didn't expect the change to be controversial. We certainly didn't intend to bury it. We tried implementing both and thought this approach solved several problems compared to the status quo. Would you prefer that we take this too the list?
Let's just move this discussion to the list
I've just read that issue several times and I don't really see much discussion of issuing aggregation requests in parallel. The question here is not primarily about the list versus not the list. It's about the process we are going to follow for landing changes. In general, substantive changes should be getting the consensus of the WG prior to being landed. At this relatively early phase, IETF sometimes allows editors some latitude to make design decisions ahead of time that will then be reviewed by the group when the next draft is published. However, typically the way to do this would be to email the list with a clear explanation of what the changes were and a notification that you were going to land a set of PRs unless someone objected by (say) a week away.
NAME mind rebasing to fix conflicts?
NAME done.
NAME and I were debating the . Now that helpers have to store all the report nonces and are also likely to be expected to store their own report shares, I argue that we should get rid of it: we don't need it and it entails a bunch of complicated security considerations. Helpers need to encrypt their state so leaders can't see it, need to protect themselves against leaders replaying old state blobs and they need to version it so that a helper can deal with a blob constructed by an older version of itself. Chris' rebuttal is that even if you don't use the blob to store something like a set of nonces or a counter or a partial aggregate share, it's still useful as a key into the storage where the helper keeps its state. This is helpful for a helper running on something like a Cloudflare Worker, where ephemeral servers are launched in response to incoming messages, and so a different helper instance might handle the sequence of and requests. We agreed that we don't need an arbitrarily large opaque blob shipped back and forth between leader and helper to solve that problem. All you need is a unique value like a UUID so that a stateless helper can recover context from storage. This will let us delete a whole bunch of SHOULD and MAY discussion from the spec.
I think we also owe a definition of an "aggregation job", which is evolving in
Some discussion of the implications of helper state to parallelism is in .
The changes LGTM, but of course we should take this to the ppm list before merging.

Once a set of clients have uploaded their reports to the leader, the leader can send them to the helpers to be verified and aggregated. In order to enable the system to handle very large batches of  <ins> reports, this process can be performed incrementally.  Verification of a set of reports is referred to as an aggregation job.  Each aggregation job is associated with exactly one PPM task, and a PPM task can have many aggregation jobs.  Each job is associated with an ID that is unique within the context of a PPM task in order to distinguish different jobs from one another.  Each aggregator uses this ID as an index into per-job storage, e.g., to keep track of report shares that belong to a given aggregation job. To run an aggregation job, the leader sends a message to each helper containing the report shares in the job.  The helper then processes them (verifying the proofs and incorporating their values into the ongoing aggregate) and replies to the leader. The exact structure of the aggregation job flow depends on the VDAF. </ins> Specifically: Some VDAFs (e.g., "prio3") allow the leader to start aggregating 4.3.1.  <ins> The leader begins an aggregation job by choosing a set of candidate reports that pertain to the same PPM task and a unique job ID.  The job ID is a 32-byte value, structured as follows: The leader can run this process for many candidate reports in parallel as needed.  After choosing the set of candidates, the leader begins aggregation by splitting each report into "report shares", one for each aggregator.  The leader and helpers then run the aggregate initialization flow to accomplish two tasks: </ins> Recover and determine which input report shares are invalid. The leader begins the aggregate initialization phase with the set of candidate report shares as follows: <ins> Generate a fresh AggregationJobID.  This ID MUST be unique within the context of the corresponding PPM task.  It is RECOMMENDED that this be set to a random string output by a cryptographically secure pseudorandom number generator. </ins> Decrypt the input share for each report share as described in input-share-decryption. "Report.encrypted_input_shares" is equal to the index of the aggregator in the task's "aggregator_endpoints" to which the AggregateInitReq is being sent.  The "agg_param" field is an opaque,  <ins> VDAF-specific aggregation parameter.  The "job_id" parameter contains the leader's chosen AggregationJobID. </ins> Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its AggregateInitResp message to complete its initialization.  This message is structured as follows:  <ins> The "job_id" parameter is equal to AggregateInitReq.job_id.  The rest of the message is a sequence of PrepareStep values, the order of which matches that of the ReportShare values in </ins> "AggregateInitReq.report_shares".  Each report that was marked as invalid is assigned the PrepareStepResult "failed".  Otherwise, the PrepareStep is either marked as continued with the output "prep_msg", "checksum" is the checksum computed over the set of client reports.  <ins> "job_id" is the ID of the aggregation job. </ins> To handle the leader's request, the helper first ensures that the request meets the requirements for batch parameters following the


ppm-urn-space).  Clients SHOULD display the "detail" field of all errors.  The "instance" value MUST be the endpoint to which the request was targeted.  The problem document MUST also include a <del> "taskid" member which contains the associated PPM task ID, encoded in Base 64 using the URL and filename safe alphabet with no padding as defined in sections 5 and 3.2 of RFC4648 (this value is always known, see task-configuration). </del>  In the remainder of this document, we use the tokens in the table above to refer to error types, rather than the full URNs.  For comparing the aggregate result to the previous aggregate result may result in a privacy violation.  (Note that the helpers enforce this as well; see collect-flow.)  In addition, the leader SHOULD abort the <del> upload protocol and alert the client with error "staleReport". </del>  The leader MUST buffer reports while waiting to aggregate them.  The leader SHOULD NOT accept reports whose timestamps are too far in the future.  Implementors MAY provide for some small leeway, usually no more than a few minutes, to account for clock skew.  If the leader rejects a report for this reason, it SHOULD abort the upload protocol <del> and alert the client with error "reportFromTheFuture". </del>  4.2.3.

Defines more error types to represent various failure modes, updates protocol text to indicate when the new error types should be used, and affirms that the task ID should be encoded as base64url with no padding (since task ID has a fixed size).
NAME observes that we have an awkward MUST requirement on leaders buffering received reports: URL
We have some error codes like that aren't enumerated in the table in . We should bring that table up to date. While we're in there, I think and aren't the right types, since outdatedness or staleness aren't the only reasons that an HPKE config or report, respectively, could be rejected.
The error table was brought up to date by .
Looks great!

ppm-urn-space).  Clients SHOULD display the "detail" field of all errors.  The "instance" value MUST be the endpoint to which the request was targeted.  The problem document MUST also include a  <ins> "taskid" member which contains the associated PPM task ID (this value is always known, see task-configuration), encoded in Base 64 using the URL and filename safe alphabet with no padding defined in sections 5 and 3.2 of RFC4648. </ins> In the remainder of this document, we use the tokens in the table above to refer to error types, rather than the full URNs.  For comparing the aggregate result to the previous aggregate result may result in a privacy violation.  (Note that the helpers enforce this as well; see collect-flow.)  In addition, the leader SHOULD abort the  <ins> upload protocol and alert the client with error "reportTooLate". </ins> The leader MUST buffer reports while waiting to aggregate them.  The leader SHOULD NOT accept reports whose timestamps are too far in the future.  Implementors MAY provide for some small leeway, usually no more than a few minutes, to account for clock skew.  If the leader rejects a report for this reason, it SHOULD abort the upload protocol  <ins> and alert the client with error "reportTooEarly". </ins> 4.2.3.


Communications between PPM entities are carried over HTTPS RFC2818. HTTPS provides server authentication and confidentiality.  In addition, report shares are encrypted directly to the aggregators <del> using HPKE I-D.irtf-cfrg-hpke. </del>  3.1. parameters: "collector_config": The HPKE configuration of the collector <del> (described in key-config). </del>  "vdaf_verify_param": The aggregator's VDAF verification parameter output by the setup algorithm computed jointly by the aggregators Before the client can upload its report to the leader, it must know the public key of each of the aggregators.  These are retrieved from <del> each aggregator by sending a request to "[aggregator]/key_config", where "[aggregator]" is the aggregator's endpoint URL, obtained from the task parameters.  The aggregator responds to well-formed requests with status 200 and an "HpkeConfig" value: </del>  [OPEN ISSUE: Decide whether to expand the width of the id, or support multiple cipher suites (a la OHTTP/ECH).] <del> The client MUST abort if any of the following happen for any "key_config" request: </del>  the client and aggregator failed to establish a secure, aggregator-authenticated channel; <del> the GET request failed or didn't return a valid key config; or </del>  <del> the key config specifies a KEM, KDF, or AEAD algorithm the client doesn't recognize. </del>  Aggregators SHOULD use HTTP caching to permit client-side caching of this resource RFC5861.  Aggregators SHOULD favor long cache lifetimes To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To <del> encrypt an input share, the client first generates an HPKE I-D.irtf- cfrg-hpke context for the aggregator by running </del>  where "pk" is the aggregator's public key and "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for This specification defines the following protocol messages, along with their corresponding media types types: <del> HpkeConfig task-configuration: "application/ppm-hpke-config" </del>  Report upload-request: "message/ppm-report"

When obtaining an aggregator's HPKE configuration, clients now specify the task ID they are interested in. Aggregators are not required to use a distinct HPKE configuration for each task, but now it's possible for them to do so. The endpoint is also renamed to from for clarity.
Aggregators are expected to have an endpoint from which they vend the HPKE public key and parameters so that clients may encrypt the contents of reports. There are no parameters to the request, so there is no opportunity for an aggregator to provide a different config based on the task in play. Now, the endpoint is relative to the aggregator's base endpoint URL, obtained from the per-task parameters, so an aggregator could simply provide base URLs that have the task ID in their path and use different key material on that basis. But all the other protocol messages like or explicitly include task ID, suggesting that aggregators are supposed to be able to service multiple tasks from a single endpoint. Something feels inconsistent there.
I think we could change the endpoint to . We would probably have to redefine task ID to make sure it's safe to use in URL construction (URL encoded byte strings look terrible).
Good catch, and I like your suggested fix.
I could swear we talked about this a long time ago. Anyway, I am putting this in for now because I don't think any progress is blocked on it, and it strikes me as the kind of thing that we will go back and forth on a lot during the IETF process.
I think the real issue here is that when handling a message, collectors, leaders and helpers need to know which task it pertains to in order to select the appropriate HPKE context (among other parameters). We should decide whether the task ID appears in protocol messages or whether it can be inferred from the endpoint and be consistent about it. Retitling accordingly.
nit: IMO it would be more consistent to put the task ID in a request into the request body -- all other request/response pairs include the task ID in the request body, rather than including it in the request path. (There's no technical downside to putting the task ID in the path, this is only a "consistency" concern.)
I think the issue with putting the task ID for a /hpke_config request in the body is that we have to switch from GET to POST, and lose cache-ability.
That's a good point -- it's still pretty unsatisfying that this one request/response pair will have to be structured in this way, while all other request/response pairs (Upload, Aggregate, AggregateShare, Collect) use a different structuring. However, given that some clients will literally be web browsers, where HTTP caching is "free", I think it's worthwhile to keep HTTP caching semantics working.
(FWIW, this would be a great place to use which is "GET but with a body", except that (a) it's still not standardized and (b) even if it were, it's still new enough that many HTTP libraries won't have good support for it.)
One final thought for now: if we do stick to a GET request, we should consider putting the task ID for an request in a query parameter (e.g. ) rather than including it as part of the request path. Otherwise, every other request type will duplicate the task ID in both the request path & the request body.
The endpoint has been changed to take the task ID as a query parameter, and all other messages include a task ID in the body. Should we close this out?
Nice

Communications between PPM entities are carried over HTTPS RFC2818. HTTPS provides server authentication and confidentiality.  In addition, report shares are encrypted directly to the aggregators  <ins> using HPKE RFC9180. </ins> 3.1. parameters: "collector_config": The HPKE configuration of the collector  <ins> (described in hpke-config). </ins> "vdaf_verify_param": The aggregator's VDAF verification parameter output by the setup algorithm computed jointly by the aggregators Before the client can upload its report to the leader, it must know the public key of each of the aggregators.  These are retrieved from  <ins> each aggregator by sending a request to "[aggregator]/hpke_config?task_id=[task-id]", where "[aggregator]" is the aggregator's endpoint URL, obtained from the task parameters, and "[task-id]" is the task ID obtained from the task parameters, encoded in Base 64 with URL and filename safe alphabet with no padding, as specified in sections 5 and 3.2 of RFC4648.  If the aggregator does not recognize the task ID, then it responds with HTTP error status 404 Not Found and an error of type "unrecognizedTask".  The aggregator responds to well-formed requests with status 200 and an "HpkeConfig" value: [TODO: Allow aggregators to return HTTP 403 Forbidden in deployments that use authentication to avoid leaking information about which tasks exist.] </ins> [OPEN ISSUE: Decide whether to expand the width of the id, or support multiple cipher suites (a la OHTTP/ECH).]  <ins> The client MUST abort if any of the following happen for any HPKE config request: </ins> the client and aggregator failed to establish a secure, aggregator-authenticated channel;  <ins> the GET request failed or didn't return a valid HPKE configuration; or </ins>  <ins> the HPKE configuration specifies a KEM, KDF, or AEAD algorithm the client doesn't recognize. </ins> Aggregators SHOULD use HTTP caching to permit client-side caching of this resource RFC5861.  Aggregators SHOULD favor long cache lifetimes To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To  <ins> encrypt an input share, the client first generates an HPKE RFC9180 context for the aggregator by running </ins> where "pk" is the aggregator's public key and "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for This specification defines the following protocol messages, along with their corresponding media types types:  <ins> HpkeConfig hpke-config: "application/ppm-hpke-config" </ins> Report upload-request: "message/ppm-report"


additional information using a problem document RFC7807.  To facilitate automatic response to errors, this document defines the following standard tokens for use in the "type" field (within the DAP <del> URN namespace "urn:ietf:params:ppm:error:"): </del>  This list is not exhaustive.  The server MAY return errors set to a URI other than those defined above.  Servers MUST NOT use the DAP URN above to refer to error types, rather than the full URNs.  For example, an "error of type 'unrecognizedMessage'" refers to an error document with "type" value <del> "urn:ietf:params:ppm:error:unrecognizedMessage". </del>  This document uses the verbs "abort" and "alert with "[some error message]"" to describe how protocol participants react to various Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its AggregateInitializeReq message as the payload.  The media type is <del> "message/ppm-aggregate-initialize-req".  In addition, this request </del>  MUST be authenticated as described in https-sender-auth. 4.3.1.2. is finished for the report share. The helper's response to the leader is an HTTP 200 OK whose body is <del> the AggregateInitializeResp and media type is "message/ppm-aggregate- </del>  initialize-resp". Upon receipt of a helper's AggregateInitializeResp message, the "AggregateContinueReq.task_id"'s parameters except its own, the leader sends a POST request to "[aggregator]/aggregate" with AggregateContinueReq as the payload and the media type set to <del> "message/ppm-aggregate-continue-req".  In addition, this request MUST </del>  be authenticated as described in https-sender-auth. 4.3.2.2. The order of AggregateContinueResp.prepare_shares MUST match that of the PrepareStep values in "AggregateContinueReq.prepare_shares".  The helper's response to the leader is an HTTP 200 OK whose body is the <del> AggregateContinueResp and media type is "message/ppm-aggregate- </del>  continue-resp".  The helper then awaits the next message from the leader. This specification defines the following protocol messages, along with their corresponding media types types: <del> HpkeConfig hpke-config: "application/ppm-hpke-config" </del>  <del> Report upload-request: "message/ppm-report" </del>  <del> AggregateInitializeReq collect-flow: "message/ppm-aggregate- </del>  initialize-req" <del> AggregateInitializeResp collect-flow: "message/ppm-aggregate- </del>  initialize-resp" <del> AggregateContinueReq collect-flow: "message/ppm-aggregate- </del>  continue-req" <del> AggregateContinueResp collect-flow: "message/ppm-aggregate- </del>  continue-resp" <del> AggregateShareReq collect-flow: "message/ppm-aggregate-share-req" </del>  <del> AggregateShareResp collect-flow: "message/ppm-aggregate-share- </del>  resp" <del> CollectReq collect-flow: "message/ppm-collect-req" </del>  <del> CollectResp collect-flow: "message/ppm-collect-resp" </del>  The definition for each media type is in the following subsections.

This change is basically a search-and-replace for /ppm/dap/. However, I have also re-aligned the context strings to match the next draft that will be cut (-01). I think the change is mostly correct, my main concern is the "URN Sub-namespace for PPM (urn:ietf:params:dap)" section. : Does it make sense for all drafts in the PPM WG to share the namespace, or do we want to define one specifically for DAP?
Before merging I would appreciate an answer on the question I raised above. I want to make sure the URN bits are correct.
Separate namespaces seems better, especially if we're thinking of DAP and STAR. Names are cheap, after all.
Separate namespaces, because otherwise I think we'd have to write a further document enumerating the error type URNs that both this document and other PPM drafts like STAR could reference. We should avoid spreading the specification across multiple documents more than necessary. Maybe including the WG in the URN path like helps?
Here's the RFC defining URL It's pretty loosely structured, and the existing sub-namespaces from published RFCs appear to just be named after protocols and formats, not working groups. URL
I took NAME suggestion of prefixing the dap space with ppm.

additional information using a problem document RFC7807.  To facilitate automatic response to errors, this document defines the following standard tokens for use in the "type" field (within the DAP  <ins> URN namespace "urn:ietf:params:ppm:dap:error:"): </ins> This list is not exhaustive.  The server MAY return errors set to a URI other than those defined above.  Servers MUST NOT use the DAP URN above to refer to error types, rather than the full URNs.  For example, an "error of type 'unrecognizedMessage'" refers to an error document with "type" value  <ins> "urn:ietf:params:ppm:dap:error:unrecognizedMessage". </ins> This document uses the verbs "abort" and "alert with "[some error message]"" to describe how protocol participants react to various Let "[aggregator]" denote the helper's API endpoint.  The leader sends a POST request to "[aggregator]/aggregate" with its AggregateInitializeReq message as the payload.  The media type is  <ins> "message/dap-aggregate-initialize-req".  In addition, this request </ins> MUST be authenticated as described in https-sender-auth. 4.3.1.2. is finished for the report share. The helper's response to the leader is an HTTP 200 OK whose body is  <ins> the AggregateInitializeResp and media type is "message/dap-aggregate- </ins> initialize-resp". Upon receipt of a helper's AggregateInitializeResp message, the "AggregateContinueReq.task_id"'s parameters except its own, the leader sends a POST request to "[aggregator]/aggregate" with AggregateContinueReq as the payload and the media type set to  <ins> "message/dap-aggregate-continue-req".  In addition, this request MUST </ins> be authenticated as described in https-sender-auth. 4.3.2.2. The order of AggregateContinueResp.prepare_shares MUST match that of the PrepareStep values in "AggregateContinueReq.prepare_shares".  The helper's response to the leader is an HTTP 200 OK whose body is the  <ins> AggregateContinueResp and media type is "message/dap-aggregate- </ins> continue-resp".  The helper then awaits the next message from the leader. This specification defines the following protocol messages, along with their corresponding media types types:  <ins> HpkeConfig hpke-config: "application/dap-hpke-config" </ins>  <ins> Report upload-request: "message/dap-report" </ins>  <ins> AggregateInitializeReq collect-flow: "message/dap-aggregate- </ins> initialize-req"  <ins> AggregateInitializeResp collect-flow: "message/dap-aggregate- </ins> initialize-resp"  <ins> AggregateContinueReq collect-flow: "message/dap-aggregate- </ins> continue-req"  <ins> AggregateContinueResp collect-flow: "message/dap-aggregate- </ins> continue-resp"  <ins> AggregateShareReq collect-flow: "message/dap-aggregate-share-req" </ins>  <ins> AggregateShareResp collect-flow: "message/dap-aggregate-share- </ins> resp"  <ins> CollectReq collect-flow: "message/dap-collect-req" </ins>  <ins> CollectResp collect-flow: "message/dap-collect-resp" </ins> The definition for each media type is in the following subsections.


as well; see collect-flow.)  In addition, the leader SHOULD abort the upload protocol and alert the client with error "reportTooLate". <del> The leader MUST buffer reports while waiting to aggregate them.  The </del>  leader SHOULD NOT accept reports whose timestamps are too far in the future.  Implementors MAY provide for some small leeway, usually no more than a few minutes, to account for clock skew.  If the leader

Per NAME
NAME observes that we have an awkward MUST requirement on leaders buffering received reports: URL

as well; see collect-flow.)  In addition, the leader SHOULD abort the upload protocol and alert the client with error "reportTooLate".  <ins> Leaders can buffer reports while waiting to aggregate them.  The </ins> leader SHOULD NOT accept reports whose timestamps are too far in the future.  Implementors MAY provide for some small leeway, usually no more than a few minutes, to account for clock skew.  If the leader


in Base 64 with URL and filename safe alphabet with no padding, as specified in sections 5 and 3.2 of RFC4648.  If the aggregator does not recognize the task ID, then it responds with HTTP status code 404 <del> and an error of type "unrecognizedTask".  The aggregator responds to well-formed requests with HTTP status code 200 OK and an "HpkeConfig" value: </del>  [TODO: Allow aggregators to return HTTP status code 403 Forbidden in deployments that use authentication to avoid leaking information

All of the following are currently used to refer to 200 responses: status 200 HTTP 200 OK HTTP status 200 HTTP status 200 OK Would be nice to pick one and use it consistently.
I didn't check if this change is complete, in the sense of making all the language in the doc consistent, but it's at least self-consistent. !

in Base 64 with URL and filename safe alphabet with no padding, as specified in sections 5 and 3.2 of RFC4648.  If the aggregator does not recognize the task ID, then it responds with HTTP status code 404  <ins> Not Found and an error of type "unrecognizedTask".  The aggregator responds to well-formed requests with HTTP status code 200 OK and an "HpkeConfig" value: </ins> [TODO: Allow aggregators to return HTTP status code 403 Forbidden in deployments that use authentication to avoid leaking information


To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To <del> encrypt an input share, the client first generates an HPKE RFC9180 context for the aggregator by running </del>  <del> where "pk" is the aggregator's public key and "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper).  In general, the info string for computing the HPKE context is suffixed by two bytes, the first of which identifies the role of the sender and the second of which identifies the role of the intended recipient. "enc" is the HPKE encapsulated key and "context" is the HPKE context used by the client for encryption.  The payload is encrypted as where "input_share" is the aggregator's input share and "nonce" and "extensions" are the corresponding fields of "Report".  Clients MUST NOT use the same "enc" for multiple reports. </del>  The leader responds to well-formed requests to "[leader]/upload" with HTTP status code 200 OK and an empty body.  Malformed requests are report share as invalid with the error "hpke-unknown-config-id". Otherwise, it decrypts the payload with the following procedure: <del> where "sk" is the HPKE secret key, "task_id" is the task ID, "nonce" and "extensions" are the nonce and extensions of the report share respectively, and "server_role" is 0x02 if the aggregator is the leader and 0x03 otherwise.  If decryption fails, the aggregator marks the report share as invalid with the error "hpke-decrypt-error". Otherwise, it outputs the resulting "input_share". </del>  4.3.1.4. "AggregateShareReq" is done as follows: where "pk" is the HPKE public key encoded by the collector's HPKE <del> key, and server_role is is "0x02" for the leader and "0x03" for a helper. </del>  The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted "enc_share", for a given batch interval, denoted "batch_interval", decryption works as follows: <del> where "sk" is the HPKE secret key, "task_id" is the task ID for a given collect request, and "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper). </del>  4.4.5.

Specifically: Specify use of the "one-shot" HPKE APIs defined in RFC9180, replacing equivalent use of the "multi-shot" APIs. (This change is intended to be a clarification, not a functional change.) Move the task_id parameter from the "info" parameter to the "aad" parameter. This is intended to protect against key-commitment-related attacks[1].) This is a functional change. Clarify that the URL field is an encapsulated key, rather than an encryption context. (This is a clarification, not a functional change.) This is a second copy of a previous PR () which was merged too hastily, and then reverted. There are no changes from that PR. [1] See discussion in .
Per URL section 8.1: PPM effectively uses the single-shot APIs; we should consider refactoring PPM's usage of HPKE to use only one of applicationinfo & AAD. Following the advice of the HPKE RFC, we would want to use applicationinfo. This would permit slightly simpler implementations. I am not sure how much it would simplify the HPKE computations.
Should we then formally use the single shot HPKE APIs from ?
That would be my recommendation (though, somewhat confusingly given the advice quoted above, those APIs still allow specifying both application info & AAD). Implementations can & do already use the single-shot APIs, so I strongly suspect specifying the single-shot APIs will be feasible. & .]
+1 to using single-shot APIs, though I'm not sure I agree that using either info or add (but not both) is simpler. What's the reasoning?
The reasoning is: a) it follows the advice of the HPKE spec (quoted above) b) AFAICT, in a single-shot setting, application info & AAD serve exactly the same purpose, so using both is arbitrary/confusing/a potential source of mistakes. (in a multi-shot setting, application info is bound to all encryptions/decryptions done by the same context; AAD is bound to a single encryption/decryption)
TBH I'm not sure I agree with the quoted guidance here. There is a concept in protocol design known as "domain separation", the goal of which is to provide some kind of binding of long-term secret key operations to the context in which they were used. Imagine, for example, that someone used an HPKE secret key for PPM and some other protocol. Suppose further that both are using the empty string for . Then the security of our PPM deployment depends on how the other protocol uses the derived AEAD key, since in both protocols may end up deriving the same AEAD key. One way to avoid creating this attack surface is to pick an string that no other application is likely to choose so that derived keys are guaranteed to not collide (except with some negligible probability). That said, I'd go for changing how we use and . Something like this might be simpler: Let be a fixed string that identifies the protocol. It would be good if this string were relatively long, say the SHA-256 hash of "ppm-00". Let encode the task ID, sender/receiver roles, etc.
I definitely agree that application info should include information allowing domain separation, for the reasons you mention. (Indeed, the current PPM spec includes the strings "ppm input share" or "ppm aggregate share" in its application infos, presumably for this purpose.) But I think that's orthogonal to the question of whether we should also use AAD, since the application info could also include information other than the "domain separation parameter". Picking one example to make things concrete, a client upload request sets its application info to and its AAD to . But it could just as easily set the application info to (i.e. the concatenation of the two previous strings) and not use AAD. (Or alternatively, it could place that string in the AAD and not use application info.) My argument is that it's simpler to use only one of the parameters, since in the single-shot setting that PPM is using HPKE, the two parameters serve exactly the same purpose (i.e. including additionally-authenticated data).
As I see it, our goal is to make sure that decryption only succeeds if the sender and receiver agree on the "context", i.e., the task ID, sender/receiver role, nonce, extensions, and so on. This way a MITM cannot force the aggregator to interpret an input share incorrectly. Does this sound reasonable to you? From the point of view of this threat model, I don't think it's immediately clear that sticking the context in or is equivalent. For example, if we stuff the context in , then it influences the derivation of the AEAD key; but depending on the AEAD, it might be feasible for an attacker to find a find two keys that decrypt the same ciphertext. (See URL) More analysis will be needed to say for sure whether this weakness in the AEAD amounts to an attack against our protocol. In the meantime, sticking the "context" in seems like a more conservative choice to me.
Given your concern, your approach SGTM. It sounds like this may be a concern worth raising with the folks working on the HPKE spec, given the advice to "use the Setup info parameter for specifying auxiliary authenticated information", but I'm not going to chase this down currently.
So I think there's two tasks for this issue: Move task ID from to Use single-shot APIs for HPKE Anything else?
Yep, I think that's it.
(reopening for now as the fix PR has been reverted temporarily)
See .
Please note that (regular, not necessarily incremental) DPFs can also be used for aggregation, and are described in a series of papers: Proof of concept implementations of DPFs can be found or .
Strong +1 to this idea. I had a longer write-up motiving a use case and outlining a high level solution which leverages this technology for a web API use-case. Then I found this existing issue. Here's what I had: The existing Prio protocol enables distributed computation of histograms which have buckets determined by a set of aggregation ids. This construction requires communication overhead per contribution per client that scales linearly with the size of the domain of aggregation (i.e. the size of the histogram vector). This solution works for modest domains (e.g. ~1000s of entries), but starts to incur significant communication overhead as domains get larger (~1M entries or more). With the effort, we are looking at aggregation use-cases where domain sizes for aggregation are extremely large and sparse e.g. on the order of ~1B possible aggregation buckets, with ~1k - 1M non-zero buckets. This is analogous to the “heavy hitters” problem. Additionally, we are looking at protecting the output with central differential privacy noise. In trying to solve for these use-cases, we are exploring a few optimizations: Improving communication overhead via compression techniques that grow only logarithmically in the size of the output domain Improving server-side performance by allowing for dynamic / hierarchical query models and partial evaluations of the output domain These techniques are inspired by . Note: we are also exploring MPC protocols which don’t grow proportionally to the output domain at all, but they are at an early stage. A new cryptographic primitive can help us achieve these goals. Given a function f(x): X -> Y, a function secret sharing is a primitive defined by two algorithms GenKey(f) outputs two keys K0 and K1 Eval(Kb, x) outputs value zb for b=0, 1 and all x∈X such that z0 and z1 are two random shares of f(x) in Zp, i.e z0 + z1 = f(x) mod p. A point function fa,b(x) : X -> Y is a function such that f(a) = b and f(a) = 0 for all x ≠ a. A distributed point function is a function secret sharing for a point function fa,b(x). There exist such that the size of the DPF keys K0 and K1 is logarithmic in the function input domain . By themselves, DPFs can be drop-in replacements for vector secret shares (via calling Eval on the full domain), and there are secure ways of verifying input consistency, like in Prio, with interaction between the servers. This trades off a cpu increase for a large communication overhead decrease. Additionally, Boneh et. al. have shown that these techniques can be extended to allow for evaluation on prefixes of the input point (“incremental” DPFs or "IDPFs"). In other words: Eval(Kb, xt) outputs value zb for b=0, 1 and all x∈X such that z0 and z1 are two random shares of f(x) in Zp, i.e z0 + z1 = f(x) mod p, for xt a t-bit prefix of x. Prefix evaluation critically allows for evaluating input in a hierarchical manner (i.e. counts at a fixed prefix). By leveraging IDPFs, we can allow for dynamic queries on large, sparse domain data that traverses a hierarchy. For example, imagine a domain of size 2^25 where only a small fraction of the entries are expected to be non-zero. Both vector secret sharing and naive DPF expansion would struggle with this problem. With IDPFs though, a client could: Evaluate the data at a 12-bit prefix Find that only 2^5 prefixes hold meaningful amounts of data Partially evaluate the data again (unprefixed), but only for entries which match the 2^5 prefixes that have data. This will end up evaluating 2^(5 + 25 - 12) = 2^18 entries, which is more manageable (~250 kb expansion per record) than expanding the whole domain from the beginning Timings from preliminary C++ benchmarks: 526ms for a full domain evaluation (size 2^25) 12ms for incremental / sparse evaluation. This single pruning step is a simple example, but this technique can be leveraged in much more complex ways to solve real world private metrics problems. Of course, if we are applying DP noise in the computation we may need to split a privacy budget across each of these dynamic queries. Internally at Google we are working on a production C++ implementation of IDPFs (for fast performance), to be open sourced very soon. We are seeing ~10ms to fully expand a ~4 mb vector from a ~350 byte key, and ~1ms to expand a 500kb vector from a ~300 byte key. There is a (non-production) of IDPFs by Henry Corrigan-Gibbs NAME referenced a golang and C++ implementation of DPFs above
NAME we'd welcome your thoughts in this thread :-)
Thanks for looping me in here! It would definitely be nice if Prio v3 could support heavy-hitters queries. There are a number of applications that need this, so I like the idea of building it into the design. At the same time, the machinery needed to support these heavy-hitter applications involves a quite a few moving parts that are not needed for Prio v1/v2 functionality. In particular, Prio v1/v2 uses: Two or more servers (where privacy holds if at least one is honest) Simple additive secret sharing with finite-field arithmetic ZK proofs on secret-shared data (called "SNIPs" in the Prio paper) The simplest version of the heavy-hitters scheme from the paper needs: Two servers (where privacy holds if at least one is honest) Simple additive secret sharing with finite-field arithmetic Distributed point functions Sketch-verification schemes (Section 4 in the BBCGI21 paper) Do we want to expand the scope of this document to include the more recent and more advanced techniques? Or, is it better to stick with the Prio v1/v2 technology, which is quite stable and which we understand pretty well at this point—even if it does not support all of the functionality that we want?
I think it's worth spinning up a new doc, though presumably there are pieces shared by both (i.e., ZK proofs on secret shared data). These could live in a third doc.
The C++ implementation of (I)DPFs mentioned by NAME above is now public: URL
FYI I published an example query model for how the flexible, non-interactive heavy hitters could work via some configuration here: URL It basically requires either the collector or leader to give some configuration outlining which bit-prefixes to expand.
I think we're well passed the relevant point on the decision tree here :) Any reason to keep this issue open? I suppose the links are useful, but it's easy enough to rummage around in the closed issues to find them.
Let's close it out.

To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To  <ins> encrypt an input share, the client generates an HPKE RFC9180 ciphertext and encapsulated key for the aggregator by running </ins>  <ins> where "pk" is the aggregator's public key; "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper); "task_id", "nonce", and "extensions" are the corresponding fields of "Report"; and "input_share" is the aggregator's input share. </ins> The leader responds to well-formed requests to "[leader]/upload" with HTTP status code 200 OK and an empty body.  Malformed requests are report share as invalid with the error "hpke-unknown-config-id". Otherwise, it decrypts the payload with the following procedure:  <ins> where "sk" is the HPKE secret key, and "server_role" is the role of the aggregator ("0x02" for the leader and "0x03" for the helper).  If decryption fails, the aggregator marks the report share as invalid with the error "hpke-decrypt-error".  Otherwise, it outputs the resulting "input_share". </ins> 4.3.1.4. "AggregateShareReq" is done as follows: where "pk" is the HPKE public key encoded by the collector's HPKE  <ins> key, "server_role" is "0x02" for the leader and "0x03" for a helper. </ins> The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted "enc_share", for a given batch interval, denoted "batch_interval", decryption works as follows:  <ins> where "sk" is the HPKE secret key, "task_id" is the task ID for the collect request, and "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper). </ins> 4.4.5.


the reports in sub-batches of 10.  Each sub-batch takes two round- trips to process. <del> In order to allow the helpers to retain minimal state, the helper can attach a state parameter to its response, with the leader returning the state value in the next request, thus offloading the state to the leader.  This state value MUST be cryptographically protected as described in agg-init. </del> [OPEN ISSUE: Should there be an indication of whether a given aggregate request is a continuation of a previous sub-batch?] Implement some form of batch-to-report index, as well as inter- and intra-batch replay mitigation storage, which includes some way <del> of tracking batch report size with optional support for state offloading.  Some of this state may be used for replay attack mitigation.  The replay mitigation strategy is described in anti- replay. </del>  Beyond the minimal capabilities required of helpers, leaders are generally required to: Implement and store state for the form of inter- and intra-batch replay mitigation in anti-replay; and <del> Store helper state. </del> 5.1.3. Collectors statefully interact with aggregators to produce an schemes like Prio in which the input-validation protocol is only run once per report, each aggregator only needs to store its aggregate share for each possible batch interval, along with the number of <del> times the aggregate share was used in a batch.  (The helper may store its aggregate shares in its encrypted state, thereby offloading this state to the leader.)  This is due to the requirement that the batch interval respect the boundaries defined by the DAP parameters.  (See batch-parameter-validation.) </del>  6.

This is a follow-up to , removing a few dangling references to the helper state blob.

the reports in sub-batches of 10.  Each sub-batch takes two round- trips to process.  [OPEN ISSUE: Should there be an indication of whether a given aggregate request is a continuation of a previous sub-batch?] Implement some form of batch-to-report index, as well as inter- and intra-batch replay mitigation storage, which includes some way  <ins> of tracking batch report size.  Some of this state may be used for replay attack mitigation.  The replay mitigation strategy is described in anti-replay. </ins> Beyond the minimal capabilities required of helpers, leaders are generally required to: Implement and store state for the form of inter- and intra-batch replay mitigation in anti-replay; and  5.1.3. Collectors statefully interact with aggregators to produce an schemes like Prio in which the input-validation protocol is only run once per report, each aggregator only needs to store its aggregate share for each possible batch interval, along with the number of  <ins> times the aggregate share was used in a batch.  This is due to the requirement that the batch interval respect the boundaries defined by the DAP parameters.  (See batch-parameter-validation.) </ins> 6.


given task.  If the leader receives a report from a client whose nonce is in this set, it simply ignores it.  A helper who receives an encrypted input share whose nonce is in this set replies to the <del> leader with an error as described in collect-flow. </del>  [OPEN ISSUE: This has the potential to require aggreagtors to store nonce sests indefinitely.  See issue#180.]

This fixes a reference that was pointing at the wrong section. The behavior relating to is under "Input Share Validation".

given task.  If the leader receives a report from a client whose nonce is in this set, it simply ignores it.  A helper who receives an encrypted input share whose nonce is in this set replies to the  <ins> leader with an error as described in input-share-batch-validation. </ins> [OPEN ISSUE: This has the potential to require aggreagtors to store nonce sests indefinitely.  See issue#180.]


AggregateInitializeResp message to complete its initialization.  This message is structured as follows: <del> The "job_id" parameter is equal to AggregateInitializeReq.job_id. The rest of the message is a sequence of PrepareStep values, the order of which matches that of the ReportShare values in </del>  "AggregateInitializeReq.report_shares".  Each report that was marked as invalid is assigned the PrepareStepResult "failed".  Otherwise, the PrepareStep is either marked as continued with the output

(see that issue for justification of this change)
both include the relevant aggregation job ID; but in both cases, the includes the aggregation job ID, the response will always reflect the aggregation job ID that was included in the request, and a request is always paired with a response by means of being part of the same HTTP request/response. This looks to be unnecessary communication overhead--if that's accurate, I think we can just drop these fields.
I wonder if a better design is to pull the aggregation job ID out of the DAP structure and make it a query parameter for the URL, e.g., , for two reasons: (1) the URL isn't echoed in the response, achieving the desired property of this change, and (2) this avoids any caching headaches, since different each aggregate init request would have a different URL. Thoughts?
I think caching is a non-issue, because requests to are POSTs, and POSTs are not required to be idempotent (hence not cacheable). That said I kind of like the idea of hoisting the aggregation job ID into the request URL. However, I think it'd be appropriate to put the aggregation job ID in the path (i.e., ), since the job ID is part of the resource being acted on. At that point, I think we might want to revisit all the DAP endpoints. Currently, we have , and , which are all verbs. HTTP urges us to think about , which ought to be nouns, and we should let the HTTP method be the verb. So instead of POST to , maybe we want POST to , and instead of , we want and so on. However all of that is far beyond the scope of this issue.
I'm not so sure about the proposed benefit to caching -- the relevant requests are s, so they shouldn't be cached. And if we ever encounter an HTTP client that does cache requests, I don't including the IDs as query params will help: the endpoint is used for all aggregation messages (initialization and continue), so we'd still run into potential caching headaches on aggregate continue requests since they'd be using identical URLs to the aggregate initialization requests. That said, perhaps this change is desirable from the perspective of fitting HTTP semantics more closely. A few thoughts: Almost all messages include all relevant data in the HTTP body, rather than using the query parameter. We'd probably want to decide on a rule for what goes into query parameters vs the body, and apply it to all messages for consistency. The one place in the DAP spec that doesn't place all request information in the body is the request, which places the task ID as a query parameter. This is done because we want to use HTTP caching for these requests, which implies using , and can't reliably transmit a body. (IMO, if/when the HTTP verb is standardized+practically usable, it would be preferable to move the parameters to the request body, for consistency.) For aggregate initialization/continue messages specifically, IMO we'd want to move the task ID to the query parameter, too, since the task ID is effectively another part of the aggregation job identifier. (...depending on how the discussion in goes.) Placing this ID in the body allows it to be transmitted raw/unencoded. Placing the ID in the query parameters would require some URL-safe encoding, such as base64url, which would cost a small but non-negligible number of bytes. All that said, IMO I kinda like keeping all of the request parameters together in one place, and IMO the body is a slightly nicer place to put the request parameters than the URL query parameter (since we can transmit with whatever encoding we want, saving a few bytes).
Yeah, sorry, I forgot these were POSTs.
I like Tim's idea of making the IDs part of the URL path if we move the IDs out of the request body, but also agree that a larger refactoring of the URL layout is beyond the scope of this issue.
Agreed that these are redundant with the current design, so I'm OK merging this as-is. Let's split out how we convey task and job parameters (via URL or application message or whatever) to a separate issue.

AggregateInitializeResp message to complete its initialization.  This message is structured as follows:  <ins> The message is a sequence of PrepareStep values, the order of which matches that of the ReportShare values in </ins> "AggregateInitializeReq.report_shares".  Each report that was marked as invalid is assigned the PrepareStepResult "failed".  Otherwise, the PrepareStep is either marked as continued with the output


HpkeConfig hpke-config: "application/dap-hpke-config" <del> Report upload-request: "message/dap-report" </del>  <del> AggregateInitializeReq collect-flow: "message/dap-aggregate- </del>  initialize-req" <del> AggregateInitializeResp collect-flow: "message/dap-aggregate- </del>  initialize-resp" <del> AggregateContinueReq collect-flow: "message/dap-aggregate- </del>  continue-req" <del> AggregateContinueResp collect-flow: "message/dap-aggregate- </del>  continue-resp" <del> AggregateShareReq collect-flow: "message/dap-aggregate-share-req" </del>  <del> AggregateShareResp collect-flow: "message/dap-aggregate-share- </del>  resp" <del> CollectReq collect-flow: "message/dap-collect-req" </del>  <del> CollectResp collect-flow: "message/dap-collect-resp" </del>  The definition for each media type is in the following subsections.

(opinionatedly: please see discussion on that issue)
Currently, DAP uses the following media types (defined in section 7.1): Note that the first uses the top-level type, while the remainder use the top-level type. For consistency, I suggest we use the same top-level type for all messages. Which type is more appropriate? Per : is for "either uninterpreted binary data or information to be processed by an application" is for "an encapsulated message"; all of the motivating examples (see section 5.2) are around encaspulating one message format inside another, or fragmenting a message into multiple chunks. provides further guidance that "multipart and message are composite types; that is, they provide a means of encapsulating zero or more objects, each one a separate media type." Given that DAP messages are not a method of encapsulating a message of a different type, I suggest that is the more appropriate top-level type to use. This is also supported by the rather large number of media types vs the relatively tiny number of media types in . (this issue fell out of discussion on URL)
Agreed, seems better.
The types were chosen in PR on a tentative basis, and marked with "OPEN ISSUE: Solicit review of these allocations from domain experts." Based on the encapsulation reasoning, I agree that be most appropriate for our media types.
As a non-HTTP expert, I agree with the rationale. Thanks, NAME based on the text quoted by NAME in the linked issue. It would be great to get feedback from folks who have more experience working on HTTP-based applications.

HpkeConfig hpke-config: "application/dap-hpke-config"  <ins> Report upload-request: "application/dap-report" </ins>  <ins> AggregateInitializeReq collect-flow: "application/dap-aggregate- </ins> initialize-req"  <ins> AggregateInitializeResp collect-flow: "application/dap-aggregate- </ins> initialize-resp"  <ins> AggregateContinueReq collect-flow: "application/dap-aggregate- </ins> continue-req"  <ins> AggregateContinueResp collect-flow: "application/dap-aggregate- </ins> continue-resp"  <ins> AggregateShareReq collect-flow: "application/dap-aggregate-share- req" </ins>  <ins> AggregateShareResp collect-flow: "application/dap-aggregate-share- </ins> resp"  <ins> CollectReq collect-flow: "application/dap-collect-req" </ins>  <ins> CollectResp collect-flow: "application/dap-collect-resp" </ins> The definition for each media type is in the following subsections.


These parameters are distributed out of band to the clients and to the aggregators.  They are distributed by the collecting entity in <del> some authenticated form.  The reason being, that the collecting entity (e.g. a company or group of companies providing some client software it wants to collect measurements from) is in a privileged position anyway, because it could simply chose not to use DAP at all. Each task is identified by a unique 32-byte ID which is used to refer to it in protocol messages. </del>  During the duration of the measurement, each client records its own value(s), packages them up into a report, and sends them to the

Proposal for URL
as part of task parameters is distributed out of band by aggregators. At the moment there's no protocol level support to guarantee is not maliciously reduced by a dishonest leader. For use cases that work with central differential privacy, the is chosen based on an epsilon-dp guarantee over a certain batch. A dishonest leader can send a smaller to helper to break a strong dp guarantee. One possible solution is to include parameters like (and other privacy parameters, for eg. epsilons in differential privacy) in , which will become part of AAD therefore avoid tempering, and enforcing aggregators to check the out-of-band matches the ones included in every field for the same task. Aggregator can also implement a one-off calculation on the task parameters to verify the is reasonable with other provided privacy parameters. For example, given a local differential privacy epsilon e0, target central differential privacy epsilon ec, and a minbatchsize N, we can calculate whether ec can be achieved by aggregating e0 over N reports, by using various shuffling amplification analysis.
is distributed out-of-band, which means that the helper and leader are considered to both have the value configured separately (i.e. without a chance for one of the aggregators modify the value that is configured for the other aggregator). is also evaluated separately and independently by each aggregator for each collect operation. The leader's check is specified in & the helper's check is specified in . (In both cases, the relevant section refers to the section which specifies the check directly.) Given the above, I believe a malicious leader cannot reduce , either at time of task initialization or at time of collection. (A malicious leader can certainly skip its own check, but as long as the helper does not collude, a small batch will still be caught by the helper's check.) This is an interesting idea & worthy of consideration IMO -- AFAIK, the DAP spec currently does not speak to what values of are reasonable.
I think the problem is DAP doesn't specify how task parameters are distributed out-of-band. If the task parameters is sent to helper from a malicious leader before task initialisation, then the helper would check against a wrong later. Having said that, my above suggestion of checking against a in AAD also assumes client have the right task parameters. On the other hand, I see there is an open issue in : These DP parameters will face the same problem: how do we guarantee they are configured honestly. I agree we should consider adding calculation from epsilon-dp parameters. There are well researched paper about calculating based on local DP from each client and a desired central DP on the batch. This may need to be VDAF specific though.
Regarding the values of , there is an explicit statement in "": I think there also used to be a somewhat obvious statement that if is 1 there is no privacy but I cannot find it anymore. I assume specification is avoided because there might be corner cases where low values make sense, e.g. some tasks might have a batch size of 2 because plausible deniability is enough. I would also like to see some guidance/examples on this but probably in the VDAF spec.
As NAME points out, the DAP draft doesn't try to define a set of configuration parameters that are sufficient for all deployments. Indeed, this is something we expect to be deployment specific, particularly when DP is desired. NAME it sounds like what you're after here is a means of opting out of a task if its configuration does not provide a sufficient level of privacy (for some definition of "privacy"). I think some part of this mechanism is inherently deployment-specific, but it might be useful to specify some protocol logic that is likely to be common to multiple deployments. One question is whether this mechanism should be implemented during the "online" phase of the protocol, as you suggest (i.e., during upload, aggregation, or collection) or "offline" at configuration time. Offline seems preferable to me, given that this is always treated as a fatal error. However since we haven't yet specified how tasks are configured, it's not clear that we can define any behavior here. Suppose each Aggregator implemented an endpoint, e.g., , to which the Collector posted task configurations. We could allow the aggregator to "abort" this request with a specific error, e.g., . Would this be sufficient?
I don't think that it makes sense to have the set by the client and covered by the HPKE AAD. There's no way for the client to verify or enforce that either aggregator actually honored the value covered by the report AAD, so I don't think it would mitigate any of the currently possible attacks. Further, I believe that in order to undermine the privacy parameter, you need both/all aggregators to collude -- it doesn't do leader any good to emit aggregate shares over a single report unless helper computes an aggregate share over that same report. If you admit colluding aggregators, then you're outside of DAP's threat model: if they're already colluding, then they might as well just send each other decrypted input shares. I think the same applies to aggregators using the wrong epsilon on an aggregate share -- it won't help unless both aggregators subvert the protocol together. >Suppose each Aggregator implemented an endpoint, e.g., , to which the Collector posted task configurations. We could allow the aggregator to "abort" this request with a specific error, e.g., . Would this be sufficient? I don't think we can specify an endpoint in DAP without opening the can of worms of bringing task parameter negotiation into the scope of the protocol.
Yes. I agree complete mitigation will likely be deployment-specific. NAME that's only the case after task initialization. But the parameters can be corrupted at task initialization stage, I imagine it's very common for the leader's organisation to drive the task, if helpers just receive task parameters from leader, then they are unable to verify if the parameters from leader can guarantee any privacy for the task.
This issue essentially captures the transparency and parameter enforcement part of issue URL
NAME I think the threat model here necessarily assumes that at least one aggregator is honest. I think you're right that if both were dishonest, including in Report has no benefit. But if at least one is honest, then doesn't the client have assurance that at least one aggregator checked it and acted accordingly?
I think there are three issues being conflated here: A malicious leader reducing which, it was assumed that leader, helpers and collector agreed on this value out of band so this should be impossible. Aggregators actually enforcing (what the title says) which they always had to do. Clients actually knowing what will be enforced which is the transparency part. I think 3. is a great idea, 1. should probably be discussed in and 2. is a misunderstanding.

These parameters are distributed out of band to the clients and to the aggregators.  They are distributed by the collecting entity in  <ins> some authenticated form.  Each task is identified by a unique 32-byte ID which is used to refer to it in protocol messages. </ins> During the duration of the measurement, each client records its own value(s), packages them up into a report, and sends them to the


the aggregator MUST abort and alert the peer with "batchLifetimeExceeded".  4.4.6. Using a client-provided report multiple times within a single batch,

Issue discusses a problem with the current collect request validation requirements. We don't yet have a solution that permits safe query flexibility. This change fixes the privacy problem at the cost of flexibility, leaving an OPEN ISSUE for us to address this regression in a followup change. It does not close the corresponding issue.
Some applications might not need as much protection against malicious users and might be okay with a malicious user changing the values by a larger amount. They might be using Prio purely for privacy and might have other ways to guard against malicious inputs, for example by verify the user account or by attesting the application. It might be useful for the Validity proof to be optional depending on the application. This would reduce overheads perhaps of communication between servers to validate the proofs and also in communication of the proofs from the client -> server. These applications could still benefit from: the multi-server deployment model AFE encoding specifications
This seems reasonable to me. Right now we have several pending design decisions floating around, so I'd like to suggest we wait on text until more of the protocol is settled.
2021/6/16 design call: Something to note here: if we have have a bit in the PA parameters that disables verification, we'll need to make sure this bit is explicitly authenticated.
NAME can we close this out and track under the VDAF draft? In particular, Prio without verification can just be a different VDAF from the perspective of PPM. I don't think anything else really needs to change.
Sounds good to me. Here is the issue for reference: URL
We currently have an open issue for tracking the consensus protocol needed for this system to work. In working through this text, we should make note of what sort of synchronization we assume between different participants of the system.
In the 3/17 design call we agreed that for purposes of the design doc and probably the initial deployments of Prio, we would assume that shared parameters (e.g., arithmetic circuits, primes, aggregation identifiers) would be exchanged between all parties (i.e., clients, aggregators, collector, leader) before the start of the protocol over some unspecified secure channel. However, eventually, it should be possible for a client that wishes to use Prio to programmatically: discover participating servers (i.e., a directory of aggregators) discover the capabilities of participating servers (i.e., what algorithms do they support for secret sharing or validity proofs or aggregations? What optional protocol features do they support?) configure a Prio aggregation (i.e., select some number of aggregation servers, getting a unique handle, configuring field size, etc.)
One thing that's still TBD is how to choose the joint randomness. It could be as simple as having the leader choose a seed for a PRG and distributing it to the aggregators over secure channels. It could be as complicated as querying at some point after the shares are received by the aggregators.
To close the loop on this issue: the PDA parameters are distributed to all entities out-of-band.

the aggregator MUST abort and alert the peer with "batchLifetimeExceeded". <ins> Finally, the aggregator checks that the batch interval defined by the collect request satisfies one of the conditions: The batch interval does not overlap with the batch interval of any prior completed collect requests. The batch interval, including its start and duration values, match a prior completed collect request. [[OPEN ISSUE: #195 tracks how we might relax this constraint to allow for more collect query flexibility.  As of now, this is quite rigid and doesn't give the collector much room for mistakes.]] </ins> 4.4.6. Using a client-provided report multiple times within a single batch,