consider different data providers holding vertically partitioned data .
each data provider holds different information about the same set of individuals and there is a common identifier ( social security number , or any other sort of i d ) that allows one to cross - reference individuals across data providers .
for example , the data providers contain a data of a set of individuals : gene `` banks '' hold genetic information , police departments hold criminal records , financial institutions keep record of credit history , hospitals record health history of patience for future diagnosis , etc .
those data providers might be geographically spread , belong to different organizations and have their specific privacy and security requirements . for instance , a data provider of genetic information might not allow a public access to its data while allowing access for medical doctors to their patients data , or police departments that allow data access for international law enforcements , but deny access to anyone outside the departments . in such cases , it is usually impossible to gather all the data in one place , either due to the size of the data or due to privacy restrictions on data sharing . a client issuing a query to a different data providers in public cloud settings ,
might be required to pay for utilized cpu time and network traffic . in many cases ,
such client will be willing to trade performance ( in other words , cost ) for accuracy of the answer .
especially , if the trade off is controlled by the client , which can define the accepted error in the received answer .    in this paper
we propose a method of trading off performance for accuracy using sampling .
we suggest a specific way to perform sampling in vertically split datasets and calculate sample size given acceptable error .
performance improvements of the method are shown on a calculation of intersection set cardinality , together with a proposed heuristic algorithm .
we also discuss a case of non - cooperative data providers and adaptation of the proposed algorithm for differential privacy calculations .    * our goal
* : given a set of @xmath0 data providers @xmath1 which data records share a common identifier @xmath2 and a set of predicates @xmath3 , find the size of intersection @xmath4 where @xmath5 is a set of all records in @xmath6 that satisfy predicate @xmath7 .
we assume that the number of records that are not present in all datasets is very small compared to the total number of records and thus , for the sake of simplicity , the data providers in @xmath8 are assumed to contain the same records with size denoted by @xmath9 , i.e. vertical split of the data .
the size of the intersection might not be calculated precisely , but up to a given accuracy @xmath10 provided by the client querying the data providers .
for example , the client that performs a query is interested in an integer percentage of people from the entire population having both criminal record and specific genetic mutation , i.e. the results can be rounded to a closest percentage .
intuitively , this relaxed requirement should result in more computationally and network efficient algorithms .
the optimization of computation time and network traffic are important , as mentioned above , in the commonly used public - cloud deployment ; the user is charged for computation time ( also called cpu time ) and network traffic .
thus , minimization of those parameters is a very attractive algorithm property and present a trade - off with accuracy of the intersection set size .    * related work and our contribution : * conjunctive queries over distributed databases
were extensively researched ( see as an example  @xcite ) .
previous research has references both the aggregation techniques and performance issues of such queries .
however , those works are based on an assumption that databases freely share the data , which does not hold in a settings of non - cooperative data providers .
there were a number of works describing privacy preserving calculations of intersection size , see  @xcite .
most of those works described a protocol that performs an exact calculation of the intersection set using secure multiparty computations ( mpc ) , first investigated by  @xcite and later generalized to multiparty computation .
performing mpc protocols allows efficient calculation of the intersection size , however , when used by itself , it is also posssible to leak information about specific items in the datasets .
for instance , the user can issue a query knowing with a single possible answer and check whether the intersection set is empty or not .
such technique allow the user to identify the presence or absence of a specific item in the datasets .
a technique that can be used to hide a presence of individual items in the dataset is differential privacy ( @xcite ) .
while some of the above works ( @xcite ) performed inexact calculations of the intersection set , the source of approximation was to preserve differential privacy of the results .
for instance , @xcite applied oblivious transfer ( ot ) protocol to approximate the size of the intersection of two databases , where ot has lead to the inexact result .
our approach is to utilize the relaxed requirement of providing inexact result to improve the computational complexity of queries
.    sampling as a way to cope with huge volumes of data or to increase computation performance were considered in a number of different contexts .
( @xcite ) and section 4.2 of  @xcite discussed sampling as a means to cope with massive datasets by creation of a histogram - based estimators .
once the estimator is created , when possible , the database performed estimation calculations on the histogram .
reservoir stream sampling algorithms ( @xcite ) were developed to provide a sample of online data .
the techniques fill in the sample ( `` reservoir '' ) by knocking out existing sample items from the sample with reducing probability as the sample begins to fill .
( @xcite ) initiated usage of document sampling for document similarity comparison on internet scale .
ideas were later developed into min - wise sampling techniques ( @xcite ) , which appear to be more suited for horizontal split datasets .
in addition , a considerable corpus of work exists on concentration inequalities in scope of machine learning .
concentration inequalities investigate the relation between the size of the sample and its statistical similarity to the entire dataset .
pac - bayes bounds on hypothesis error as a function of sample size were provided in  @xcite .    in this paper
, we consider the case of approximate calculations in distributed databases where the data is split vertically .
we suggest an algorithm that takes advantage of a lack of accuracy in a distributed answer to considerably speed up the queries .
we show a use - case of an algorithm on intersection set cardinality calculations both with and without privacy considerations .
the algorithm adopts sampling techniques from ( @xcite ) and uses techniques similar to pac - bayes bounds from ( @xcite ) to decide on a mimimal , representative sample size .
performance improvement might be especially significant in privacy preserving setting , where the calculations take considerable time .
specifically our contributions are :    * suggested efficient method for approximate , distributed calculations with vertical - split datasets .
* proposed a method of choosing sampling size given a required error level and provided simulation results of comparing the error level of different sample sizes .
* suggested a heuristic algorithm of speeding up the intersection set size calculation based on bounds convergence and showed its adoptation for privacy preserving intersection set size calculation .
the rest of the article is structured in the following way .
section  [ sec : naive algorithm ] defines a nave algorithm for calculation of exact intersection set size , section  [ sec : sampling ] relaxes the requirement to calculate the exact size of intersection set and defines theoretical bounds for _ inexact _ intersection .
section  [ sec : experiments ] presents results of experiments that validate the proposed sampling method .
section  [ sec : bounded estimation ] describes a heuristic algorithm for calculation of intersection based on the bounds of intersection size .
finally , section  [ sec : privacy ] discusses privacy issues in the described algorithms .
the paper is concluded in section  [ sec : concluding remarks ] .
the idea of the nave algorithm is simple : iterate over data providers @xmath11 and exchange a monotonously decreasing set of keys that satisfy all predicates up to the current iteration .
the data provider then checks which of the received keys answer the corresponding predicate and returns a new list to the client .
the client continues the process until all data providers were queried and the intersection set is found . in this way
, the client calculates the intersection set iteratively .
denote this set by @xmath12 where @xmath13 is the iteration number .
in other words , @xmath14 where @xmath5 is a set of records in @xmath6 that satisfy predicate @xmath7 , and @xmath15 .    the nave algorithm can also be performed in a parallel way , where the client receives a set of records that satisfy the corresponding predicate from each data provider .
the client then calculates the intersection set .
comparing the sequential and parallel nave algorithms , sequential algorithm optimizes the network load and also improves cpu time , but not a wall - clock time of the algorithm ( i.e. , the time between the beginning of the operation and the time the client gets the final answer ) . unless stated otherwise , in the rest of the section we consider only the sequential version of the algorithm .    despite its virtue of being simple
, the nave algorithm has a few drawbacks .    *
the amount of information transferred between the client and the data providers is relatively large , as the entire set of keys in the intersection set is sent .
* the first data providers evaluate the given predicate over their entire dataset , which is expensive . *
the wall - clock time of the sequential algorithm is linear in a number of data providers and the size of the data . *
the algorithm completely exposes information both between data providers and between data providers to the client .
as mentioned above , those drawbacks are even more extreme in a current common practice , where a public cloud infrastructure is used for calculations , as in a public cloud infrastructure the client is charged for computation time ( cpu time ) and for network traffic . thus , there is a monetary incentive to minimize those two values .
both the sequential and the parallel nave algorithms described above calculate the intersection set exactly , thus having a relatively high network traffic and cpu time demands .
the next section discusses a way to reduce computation and network complexity by calculation of an estimation of the intersection size .
relaxing the requirement for exact calculation of the intersection , this section discusses a way to calculate a smaller intersection set whose size can be extrapolated to the intersection size of the entire population . instead of calculating the intersection of the entire datasets ,
perform a sample and calculate the intersection of the sample . then , scale up the size of the sample intersection to the size of the intersection for the entire population .
the results of scale up will be an estimate for the intersection size .
the requirements from the sample are clear : the sample should be as small as possible while truthfully representing the entire population ( given the definition and the error of representation ) .
those two requirements present a trade - off between sample size and the accuracy of algorithm results .
a number of different sampling techniques were developed over the years : on - line ( reservoir ) stream sampling algorithms ( @xcite ) , histogram - based estimators ( @xcite ) and min - wise sampling techniques ( @xcite ) which are more suited for horizontal split datasets . in our settings
, it seems natural to choose hash function based technique , which in a sense a randomized version of histogram .    to provide sufficient improvements in performance
, the size of the sample ( denoted by @xmath16 ) should be much smaller than the average size of the datasets ( @xmath17 ) : @xmath18 .
therefore , if each data provider performs its own sampling , the overlap between the samples will be very small .
in other words , the probability of the data record ( person in our example ) appearing in the sample is low . for every data provider @xmath13 @xmath19 where @xmath20 is a sample from the dataset of the data provider @xmath13 .
leading to the probability of the item appearing in all samples being ( assuming that all data providers contain the same records ) : @xmath21 where @xmath22 and @xmath23 are a sample size and a dataset size of a data provider @xmath24 .
assuming , for simplicity , that the sample size and dataset size are equal for all data providers , the probability becomes : @xmath25 as this probability is low , the size of the intersection will be essentially zero for most of the samples .
the solution is to make all data providers create the same sample . in order to do that , we will use a sampling technique based on hashing ( @xcite ) .
the technique works in the following way :    1 .
the client defines an accuracy threshold for the calculation .
2 .   using techniques described below ,
the size of the sample is determined : @xmath16 .
the number of hash buckets is @xmath26 .
4 .   pick a hash function @xmath27 that will distribute datasets of data providers @xmath6 into @xmath28 buckets .
@xmath27 hashes only the i d of the records and not the entire data , as the i d is the shared information across different data providers . 5 .
the client sends each data provider 3 parameters : @xmath27 , the number of the bucket that was chosen randomly and predicate @xmath7 .
this will ensure that all data providers use the same sample .
each data provider evaluates its predicate on the records in a given bucket ( @xmath5 ) and sends the results to the client .
the client performs the intersection calculation on the received results .
as client acceptable error defines sample size , which is the bucket size , it is possible to pre - calculate a number of buckets for different error value .
this will eliminate the need for sequential scan of the entire database on each query .
however , it will also mean that the client will get approximately the required error .
notice that just like the nave algorithm sampling can be done both in parallel and sequential ways . following the similar idea of the nave algorithm , in the parallel version of sampling algorithm
each data provider evaluates the predicate over the entire given bucket , where in the sequential version , the client sends the current intersection set ( which is a subset of the given bucket ) to the next data provider , which evaluates the given predicate only over the received subset .
however , in contrast to the nave algorithm , where the sequential variant significantly reduces the network load and cpu time , in sampling algorithm the reduction is much smaller , as the core underlying idea of the sampling algorithm is minimization of the amount of records participating in the intersection calculation .
importantly , the accuracy of the intersection size estimation is the same in both sequential and parallel versions of the sampling algorithm .
sampling improves both the computation time and network traffic over the nave algorithm by the factor of @xmath29 .
thus , one important question still remains unanswered : what should be the size of the sample given an accuracy threshold ?
the following sections define bounds on the sample size such that with high probability it will represent behavior similar to the entire dataset .
two different cases are considered : sample size is small compared to the dataset size ( section  [ subsec : small sample ] ) and sample size is comparable to dataset size ( section  [ subsec : big sample ] ) . notice that in both cases the sampling is performed in the same way as explained above .
however the bound used to calculate the sample size is different for those two cases .
selection of the sample size is driven by the trade - off between the accuracy of the sample and the performance of calculations .
the bigger the sample , the more accurate it is , but also the larger the computation time required to calculate the intersection . using a bound on the error as a function of the sample size , it is possible , given a limit on an acceptable error , to choose the size of the sample set .
when a sample size is very small compared to the dataset size , the sampling can be approximated by independent sampling of the same size with replacement .
the approximation can be done , as the probability of any record being a part of the sample goes from @xmath30 for the first pick to @xmath31 for the last pick .
if @xmath32 then @xmath33 , and therefore , the probability is approximately ( @xmath30 ) , which is the same probability for a record to be picked when sampling with replacement .
the reason to perform such approximation is due to a fact that it is simpler to provide a bound for independent samples with replacement rather than for sampling without replacement .
the size of the sample should be big enough that the sample will be a good representative of the entire dataset for the intersection calculations .
how can the size be estimated ?
let us consider a sampling from the dataset @xmath6 according to the uniform distribution .
notice that even though the sampling is done according to uniform distribution , as described in the sampling algorithm ( section  [ sec : sampling ] ) , the datasets themselves might be drawn from other distributions .
the provided bounds still hold , as the sample `` similarity '' to the entire dataset depends only on its size . if a different , non - uniform sampling technique is used , the bound might be changed to use more general bernstein inequalities ( @xcite ) .
define @xmath34 to be a random variable that the sampled item @xmath35 satisfies condition @xmath36 .
then , the average value of the sample , denoted by @xmath37 , should be close to the mean value , @xmath38 , of the entire dataset .
notice that in the binary case , the average is simply the number of data records that satisfy a given predicate divided by the data size . in order to bound the difference between the average value of the sample and the mean value of the dataset , we can use concentration inequalities .
namely using results by hoeffding ( @xcite ) if @xmath39 are independent and @xmath40 , then : @xmath41    for simplicity , for now we will consider only the right - side of the equation , i.e. @xmath42 where @xmath16 is the size of the sample and @xmath10 measures the `` resemblance '' of the sample to the mean of the dataset . notice that the equation does not depend on the size of the dataset , as it is considered much larger than the sample .
the bound in equation  [ eq : short hoeffding ] is used by the client when it issues queries to the data providers by defining both the acceptable error ( @xmath10 ) and the target probability of exceeding the acceptable error .      in cases where sampling size is comparable to the dataset size , it is possible to develop bounds directly for sampling without replacement . in the rest of the section , we show two bounds on sampling without replacement , one is based on a reduction of `` randomness '' of the data and the second one is based on a direct counting technique .
those bounds can be expected to be tighter than those based on reduction to independence or bounds for sampling with replacement .
the reason for this as follows .
assume that @xmath0 points were sampled out of @xmath43 points without replacement .
the next point is to be sampled from a set of @xmath44 rather than @xmath43 points , which would be the case in sampling with replacement .
the successive reduction in the size of the sampled set reduces the `` randomness '' of the newly sampled point as compared to the independent case , and also introduces dependency between samples .
whereas , the bound provided in equation  [ eq : short hoeffding ] does not depend on the dataset size and does not take the reduction in population size into consideration .
this intuition is at the heart of serfling s improved bound ( @xcite ) which is stated next .
the result holds for general bounded loss functions and is established by a careful utilization of martingale techniques combined with chernoff s bounding method .
[ theorem : serfling ] ( serfling ) let @xmath45 be a finite set of non - negative bounded real numbers , @xmath46 .
let @xmath47 be a random variables obtaining their values by sampling @xmath48 uniformly at random * without * replacement .
set @xmath49 .
then , @xmath50 similar bounds hold for @xmath51 .
compared to the bound in equation  [ eq : short hoeffding ] , the above bound is always tighter when @xmath52 , i.e. when @xmath53 .    in our case @xmath54
s are binary variables and the bound could be improved further by using a proof based on a counting argument ( @xcite ) .
figure  [ fig : bounds comparison ] presents a single example of comparison between the above bounds ( [ eq : short hoeffding ] and [ eq : serfling ] ) . as expected , the serfling bound is tighter than the hoeffding bound and thus , using this bound for calculation of sample size will result in a smaller sample set .
, @xmath55 .
the error value is preset and then the sample size that fits this value is picked . in tighter bounds , the sample size will be smaller.,scaledwidth=50.0% ]    now we can briefly describe the process of using those bounds .
we assume that the client attempts to minimize the size of the sampling sets , as this also minimizes the network load and cpu time , both of which are chargeable in public cloud environments .
the client defines a value of an acceptable error for the combined distributed query ( @xmath56 ) and a `` confidence '' of the error .
the bounds defined in ( [ eq : short hoeffding ] ) or ( [ eq : serfling ] ) provide a `` confidence '' that the error will be smaller than @xmath10 given a sample size .
now the client uses the bound from ( [ eq : short hoeffding ] ) or ( [ eq : serfling ] ) to determine the minimum sampling size that allow the required error with `` high enough '' confidence . using a predefined confidence and a chosen bound , the client can find the minimum value of sample size that will provide the required error with the required confidence .
for example , consider figure  [ fig : bounds comparison ] , where for dataset size of @xmath57 the client choose @xmath58 and confidence of @xmath59 .
thus , using the simplified hoeffding bound ( [ eq : short hoeffding ] ) the minimal sample size is @xmath60 and using the serfling bound ( [ eq : serfling ] ) the minimal sample size is @xmath61 . of a sample size . ]    after the intersection of sample sets is calculated , there is a need to estimate the size of the intersection of the entire datasets . as mentioned above , the mean of the binary random variable is also the number of records that belong to a set defined by the predicate divided by the size of the sample .
thus , the ratio of the records in a sample that satisfy a given predicate is close to the ratio of the records that satisfy the predicate in the entire dataset of a given data provider .
this leads to the conclusion that the ratio of the sample intersection set size to the sample size should be the same as the ratio of the intersection set size to the size of entire dataset .
let @xmath62 be a sample of @xmath6 of size @xmath16 , then @xmath63    notice that even though the absolute error in the intersection estimation of the entire dataset is proportionally larger than the error in estimation of a sample , the relative error will remain the same .
the reason for the error to remain the same in a process of intersection calculation is due to the sampling method .
even though the sample from each data provider has the same ( potential ) relative error , the error is unique for the sample and thus , it does not accumulates when the intersection is calculated .
as @xmath10 in the above bounds ( equation  [ eq : short hoeffding ] and  [ eq : serfling ] ) is a bound on difference in estimation , it is a relative error . therefore , if the client defines an acceptable error ( @xmath56 ) as an absolute error , it can be easily translated : @xmath64 .
we have performed a number of experiments to show the utility and error resulted from sampling .
the experiments were performed on simulated data and validated on the adult dataset ( @xcite ) .
the described sampling technique is targeted at large datasets with many records , such that calculation of predicates over the entire dataset is wasteful . for such datasets , it is much more practical to test the algorithm on simulated data , which can be generated on any required scale .      for experiments on simulated data ,
a number of datasets were generated with random values of the predicates and then the intersection size was calculated according to each algorithm .
the methodology works as follows .    *
generate a dataset of a given size .
all data providers assumed to share the same set of records with possibly a different data per record .
* for each data provider generate a set of predicate values randomly , such that the frequency of `` true '' values is as defined .
* calculate the intersection iteratively , i.e. by addition of a single datasets at a time .
this is a both a simpler way of implementation ( as opposite to parallel calculation ) and also allows observation of convergence rate of the intersection size .
all experiments were averaged over 10 runs , standard deviation was calculated and drawn on the resulting graphs .    figure  [ fig : algorithms comparison ] shows the intersection size as calculated by the sequential nave algorithm and estimation by sampling of various sizes .
the graph provides a high - level , visual practical validity of the approach .
even a relatively small sample sizes ( @xmath65 of the dataset size ) estimate intersection size close to the real value .    in order to focus on the error caused by sampling and
show the difference in sample size more clearly , we have performed a number of experiments showing relative error between estimation and the real intersection size .
the nave algorithm calculated was taken as a baseline for error calculations .
the sampling algorithm was executed with a number of sample sizes , each one showing the relative error from the nave algorithm results .
figures  [ fig : algorithms error comparison ] , [ fig : algorithms error comparison 1 m ] , [ fig : algorithms error comparison 100k r05 ] , and  [ fig : algorithms error comparison 1 m r05 ] show the relative error of the sampling algorithm with various sample sizes .
the y axis shows the error of the sampling , i.e. |sampled estimation - intersection size| / ( intersection size ) , while the x axis is the number of datasets in the intersection .
the graphs compare datasets sizes of 100,000 ( figures  [ fig : algorithms error comparison ] and [ fig : algorithms error comparison 100k r05 ] ) and 1,000,000 ( figures  [ fig : algorithms error comparison 1 m ] and [ fig : algorithms error comparison 1 m r05 ] ) records with predicate satisfaction frequency of @xmath66 and @xmath67 , i.e. in each data provider , @xmath68 or @xmath69 of the records satisfy the corresponding predicate .
as mentioned above , each graph is an average of 10 different runs and standard deviation is depicted by error bars on the graph .
the graphs show that , as expected , larger samples result both in smaller error and smaller standard deviation . however , it also can be seen , that the error quickly converges as a function of sample size . in some cases , even for sample of @xmath65 from entire dataset ,
it is possible to achieve reasonable error .
overall , sampling @xmath70 or @xmath71 of the dataset resulted in errors of approximately @xmath59 from the intersection size , which is an order of a single percent .
thus , queries of a type : `` what is the approximate percentage of people ... ? '' can be answered using only tenth of a data .    , number of datasets is @xmath72 , ratio of predicate satisfaction in each dataset is @xmath66 .
the y axis shows the size the intersection estimation .
the x axis show the number of the data sets that participated in the intersection .
error bars show standard deviation over 10 different executions.,scaledwidth=50.0% ]    , number of datasets is @xmath72 , ratio of predicate satisfaction in each dataset is @xmath66 .
error bars show standard deviation over 10 different executions.,scaledwidth=50.0% ]    , number of datasets is @xmath72 , ratio of predicate satisfaction in each dataset is @xmath66 .
error bars show standard deviation over 10 different executions.,scaledwidth=50.0% ]    , number of datasets is @xmath72 , ratio of predicate satisfaction in each dataset is @xmath67 .
error bars show standard deviation over 10 different executions.,scaledwidth=50.0% ]    , number of datasets is @xmath72 , ratio of predicate satisfaction in each dataset is @xmath67 .
error bars show standard deviation over 10 different executions.,scaledwidth=50.0% ]      in addition to experiments on simulated datasets , we have tested the methodology on adult dataset from uci machine learning repository ( @xcite ) .
the dataset contains data from census bureau , where each record has data about different person .
this fits a description of the setup we have described .
the dataset contains 32,562 instances .
as a test case we have used an intersection of the following predicates : age @xmath73 30 , marital status : never - married , sex : female or male in two different tests and income @xmath74 50k .
the exact intersection set size is 252 for males and 139 for females .
since the dataset was not used for classification task , it allowed us to use income as one of the predicates .
in addition , the intersection was of 4 predicates , as addition of more predicates resulted in a small or empty intersection set due to a limited dataset size .
samples of different sizes were drawn with sample size ratio going from 0.1 to 0.5 .
the intersection set size was calculated from the drawn sample .
figure  [ fig : adult sampling ] depicts the results of the testing .
while the accuracy of the estimation does not necessary improves by taking larger samples , the standard deviation becomes smaller .
the accuracy improvement is most probably caused by the small size of the intersection , while improvement in standard deviation fits the results shown in simulated datasets .
overall , the accuracy of the intersection set size calculation verify the validity of the approach .
previous sections described exact and approximate ways of calculating the size of the intersection .
this section presents a heuristic algorithm that attempts to optimize the calculation of the intersection by not performing the calculations for all datasets .
the simplest case of heuristic is when the intersection is calculated iteratively and at some point the intersection set is empty .
clearly , the calculation can be stopped at this stage .
following is a heuristic algorithm for the intersection calculation that attempts to stop at the earliest possible point .
the size of the intersection depends on the sizes of the sets from each data provider that answer the corresponding predicate and on the correlation between those sizes .
below we suggest a heuristic algorithm for estimation of the intersection set size .
the algorithm starts with an accuracy parameter , the ratio of records that satisfy the relevant predicate in each data set , and the lower and the upper bounds on the intersection size .
the algorithm iteratively tightens the bounds until the difference between the bounds is smaller or equal to the accuracy parameter .
then the algorithm stops and returns the middle value of the range between the lower and the upper bound .
let @xmath7 be the predicate that is associated with a data provider @xmath6 .
then , @xmath75 is the set of members in @xmath6 that satisfy @xmath7 .
also , @xmath76 will denote the fraction of the members that satisfy @xmath7 in @xmath6 , i.e. @xmath77 .
now , let us define the bounds on the size of the intersection set @xmath78 for the sake of the algorithm iterations . for simplicity , we assume that data provider datasets @xmath79 contain _ exactly _ the same records .
the bounds will hold when the difference in members between datasets is small .    as the intersection size is at most the size of the smallest predicate set ,
the size of the intersection is bounded from above by the following bound : @xmath80    the lower bound on the intersection of two sets @xmath81 and @xmath82 is ( @xcite ) @xmath83 in case of 3 sets , the lower bound becomes : @xmath84 which in general is @xmath85 where @xmath86 is the average size of the datasets ( as they contain the same set of records ) .
as described above , the iterative step of the algorithm is to tighten the bounds on the intersection size .
the algorithm will then stop once the difference between bounds is smaller than the required accuracy . as the upper bound ( equation  [ eq : upper bound ] ) is the minimal predicate set , an iteration step will attempt to make the minimum value smaller by calculating the intersection between the two smallest predicate sets .
the intersection of two sets will be smaller or of the same size as minimal predicate set and will replace those two sets in the bound .
calculating the intersection of two minimal predicate sets is also a good technique for increasing the lower bound ( equation  [ eq : lower bound ] ) as well .. in this case the lower bound of their intersection is 0 . ] notice that each iteration step of the algorithm also decreases the value of @xmath0 and thus removes the subtractive member of the lower bound .
the algorithm steps are as follows .    1 .
define a required accuracy threshold : @xmath87 as @xmath88 is a relative error , the iterations continue until an absolute error is smaller than @xmath89 .
when the iterations stop , the middle of the bounds range is returned , therefore , the iterations stop when the distance between the bounds is less or equal to @xmath90 2 .
calculate @xmath5 for every data provider @xmath13 .
3 .   calculate the lower ( @xmath91 ) and the upper bounds ( @xmath92 ) , given @xmath7 .
[ step : iteration ] while @xmath93 and the number of sets is larger than 1 : 1 .
let @xmath94 and @xmath95 .
in other words , pick two minimal predicate sets : @xmath96 and @xmath97 .
[ step : intersect ] calculate the intersection between those two sets .
3 .   replace @xmath98 and @xmath99 with the new predicate set : @xmath100 .
4 .   update new values of @xmath92 and @xmath91 .
if only one set remains @xmath101 ( @xmath102 ) , then the size of its predicate set , @xmath103 , is the size of the intersection set .
6 .   if there is more than one set and @xmath104 , then the size of the intersection set to the middle value of @xmath105 $ ] range : @xmath106 .
the algorithm clearly converges , as the iterations stop when the intersection size is calculated exactly . at this point
the lower bound is equal to the upper bound and thus , the accuracy requirement will be satisfied .
notice that if step  [ step : intersect ] results in an empty set , the algorithms also stops , as the lower and the upper bounds will be equal and zero .
* example : * [ example : algorithm example ] as an example of the algorithm execution , assume @xmath107 , equal size of all data sets @xmath43 and the required accuracy of @xmath108
. let the respective ratios be @xmath109 and @xmath110 .
in this case , the upper bound on the intersection set size will be @xmath111 , whereas the lower bound will be @xmath112 , as the sum of all ratios is @xmath113 less than @xmath114 .    the first iteration of the algorithm will be to find the intersection of @xmath115 and @xmath116 .
let us assume that their intersection results in @xmath117 .
now , the upper bound of the intersection becomes @xmath118 , whereas the lower bound increases to @xmath119 .
the iterations continue until the bounds converge to within @xmath120 .
in general , the heuristic algorithm will work fine in cases where the ratio of records that satisfy the predicate is high across most data providers , but the intersection size might decrease relatively fast . see figure  [ fig : heuristic algorithm ] for example of convergence of the algorithm bounds on simulated data .
, number of datasets is @xmath72 , ratio of predicate satisfaction in each dataset is @xmath121 ( the high ratio is required for the lower bound to be larger than zero for @xmath72 ) . notice that in simulated case , the intersection size is co - located with the upper bound of the intersection.,scaledwidth=50.0% ]      clearly , both the computation time and network traffic depend heavily on the calculation of the intersection between the two sets in step  [ step : intersect ] of the heuristic algorithm . to improve the performance of this step it is possible to use the sampling technique from section  [ sec : sampling ] . instead of performing intersection between two sets @xmath13 and @xmath96
, the algorithm will calculate the intersection of two samples of those sets .
however , sampling introduces an error into calculation of the intersection , which has to be related to the accuracy threshold @xmath88 of the heuristic algorithm .    assuming that the error in sampling distributes proportionally among datasets and noting that each iteration `` eliminates '' one dataset
, we can define the following changes to the algorithm .
every sample introduces error of @xmath122 . to accommodate this error ,
the algorithm will decrease the required accuracy threshold by this amount on each iteration .
this will ensure that required by user accuracy threshold will be honored .
two steps are changed in the algorithm :    1 .
step  [ step : iteration ] then becomes : + _ while @xmath123 where @xmath13 is the iteration number , and the number of sets is larger than 1 .
step  [ step : intersect ] becomes : + _ calculate the intersection between those two sets : @xmath96 and @xmath24 by sampling with error threshold @xmath124 .
_    the rest of the algorithm remains the same , including required accuracy threshold value .
in the described setting there are 2 different types of actors : data providers and client .
there are different privacy concerns between those actors .
one privacy concern is preserving the privacy of data providers .
the client querying the data providers should not learn the identity of the records that are part of the intersection set or the number of records that satisfy a specific predicate in any single data provider .
notice that the client should be limited to a reasonable number of queries ( sub - linear in a data size ) , as it is not possible to answer a large number of arbitrary queries while preserving privacy ( @xcite ) .
another privacy concern is in keeping a privacy of records in a specific data provider from other data providers .
for instance , a data provider might be interested in hiding the presence of specific record from other data providers .
first , it is imperative to note that the sampling ( see section  [ sec : sampling ] ) provides a very nave form of privacy . as every data
set record has a @xmath125 chance of not being included in sample , the presence of the specific record in the data set is not immediately observable . yet , the fact that the client gets record identifiers of each data provider is undesirable .
data providers can encrypt the sampling indices to hide the identity of the records that are sent to the client .
even though an additional encryption will obscure the identity of those items from the client , it still will expose the presence of individual records to other data providers .
in addition , using an additional knowledge , the client can easily learn the presence of a specific record in the data providers .
one way to preserve privacy is to use secure multiparty computations to calculate the intersection set ( @xcite ) a simple method that uses commutative cryptography ( another approach is to use secret sharing ) is described in  @xcite .
each data provider encrypts record identifiers using its own commutative key and passes the key to other data providers .
once all data providers have encrypted all keys , it is possible to find the intersection using encrypted identifiers utilizing commutativity of the encryption and calculate its size .
there are a few downsides to using secure multi party computations .
first , performance is heavily impacted by performing secure computations using these methods .
it is possible to aleviate the performance issues by using the sampling to reduce a number of records in the calculated intersection , as described in section  [ sec : sampling ] .
thus , the idea is to perform the sampling algorithm and then calculate the intersection in a secure way using the mpc computations between the data providers .
this method however , will still require calculation of the sampled intersection exactly , without the ability to halt when required accuracy was reached , as it is possible in heuristic algorithm .
the second drawback is specific for the described use - case where the data is kept in different organizations .
secure multi party computations require direct communication between data providers , which in some cases is very challenging in a real - world deployments due to both security and technological reasons .
while it is still possible for the client to act as an intermediary between different data providers , such setup also doubles the network traffic .
a different approach is only to allow communication between client and data provider with adopted algorithms .
however , the most significant drawback of mpc is that the exact intersection size is calculated .
this might allow the client to use additional information to infer a presence of a specific record in the dataset .
the current de - facto privacy standard is differential privacy ( @xcite ) .
informally , the idea of differential privacy is to protect the privacy of individual records in the dataset without any assumption on the additional knowledge .
differential privacy is preserved if it is practically impossible to identify whether the record is present or absent in the dataset from the result of database queries .
the most common methods of ensuring differential privacy rely either on addition of a carefully chosen amount of noise to the result or by using an exponential mechanism that chooses the output according to some specific probability distribution .
the protocols that are based on secure multiparty computations perform the exact calculation of the set , which is impossible in differential - privacy settings .
this led to several works describing differential - privacy - preserving calculations of the intersection size , see  @xcite .
even though those algorithms preserve differential privacy , the requirement to provide an approximate answer in our case allows our scheme to optimize the algorithm for performance by performing secure computations on as little number of the records as possible due to sampling .    the most common method of ensuring differential privacy is an addition of a specifically crafted random noise to the released data ( @xcite ) . moreover , there are two different approaches for the private data release : interactive ( @xcite)and non - interactive ( @xcite ) .
interactive data release is when the client sends data query to a data provider .
the data provider then releases the data to the client while ensuring differential privacy of records in its database .
the data provider might decide not to answer a specific queries or to stop answering queries from a given client , as this might impact the privacy . in non - interactive data release
, the noise is applied only once and then the data is released to the client .
the client can perform any number and any type of queries on the data .
intuitively , non - interactive release requires to add more noise to the released data , as it has to be ready for any query , while interactive release can adopt added noise to the results of each query ( @xcite ) .      in the described above sampling algorithm ( section  [ sec : sampling ] ) , there are two intuitive locations to add random noise .
first , it is possible to add noise in hashing function that assigns records to buckets .
such noise will preserve privacy of individual records , as it will be impossible to distinguish whether a specific record is within the bucket or not present at all in the dataset . a second location to add the noise is during a query processing .
since the data provider exposes only the number of records that satisfy a given predicate , the noise can be added to the output after the predicate was evaluated or to the predicate itself [ [ ehud : this is not clear , the noise should be fake ids ] ] .
the difference between those two locations is exactly the difference between interactive and non - interactive data release . adding noise to the hashing function
is a one - time operation and has no relationship to the result of the specific predicate .
thus , this noise addition can be seen as non - interactive data release . on the other hand ,
if the noise is added after the predicate is evaluated over records in a bucket , then the noise value can be adapted to the results of predicate evaluation .
thus , the amount of noise can be directly related to the results of a specific query , like in interactive data release settings .    due to the above reasoning
, the next section describes an addition of a noise to the predicate function results .
differentially private calculation of a counting function ( referred in the paper as `` noisy sum '' ) , i.e. counting a number of records that satisfy a given predicate , was considered previously in ( @xcite ) .
( @xcite ) showed that adding a laplacian noise @xmath126 to the sum query output : @xmath127 , ensures @xmath128differential private function .
notice that the sensitivity of counting function is @xmath129 and thus , the distribution standard deviation is @xmath130 .    the fast algorithm for privacy - preserving intersection calculation is described below .
heuristic algorithm from section  [ sec : bounded estimation ] is used as a basis for privacy preserving algorithm .    1 .   use a hash function @xmath27 to divide data into buckets . in case only a single bucket is used for all queries , only the agreed bucket is kept .
2 .   when a predicate is received from the client , evaluate the predicate over the chosen bucket .
3 .   add random noise according to laplace distribution to the size of the predicate set .
the noisy set - size is shared with the client .
the client gathers results from all data providers and calculate upper and lower bounds .
[ step : chosen data providers ] the client continues to execute heuristic algorithm and picks two data providers , @xmath96 and @xmath0 , to perform intersection according to the algorithm step  [ step : intersect ] .
perform privacy - preserving , secure intersection between two chosen providers using the below algorithm . 7 .
update the bounds and continue until the bounds are close enough .
the algorithm used for privacy - preserving , secure intersection set calculation is based on a work of ( @xcite ) , which allows a differentially private calculation of an intersection over multiple databases .
the algorithm provides computational differential privacy , as it relies on homomorphic encryption and makes some assumptions on computational hardness .
this section describes how the algorithm from ( @xcite ) can be used in our heuristic algorithm from section  [ sec : bounded estimation ] for inexact computations .
the algorithm describes a basic operation _ private set - intersection cardinality ( psi - ca ) _ and then adds noise to ensure differential privacy , thus resulting in bn - psi - ca .
psi - ca operation is based on usage of homomorphic cryptosystem that allows addition and multiplication by constant ( for instance , paillier s cryptosystem ( @xcite ) ) .
when two data providers , @xmath13 and @xmath96 , attempt to calculate intersection , @xmath13 defines a polynomial whose roots are the members of its set , the polynomial coefficients are then encrypted and transfered to @xmath96 data provider , which calculates @xmath131 . in other words
, it evaluates the polynomial on its set members , multiplies by a constant number @xmath132 and adds a special string of zeros @xmath133 .
when transfered back to the first data provider ( @xmath13 ) , it calculates the number of zero string . in order to add differential privacy , a random number of dummy values
are added to the transfered set by both parties ( for details see ( @xcite ) rounding the set size to the bucket size in our case . in the end
, data provider @xmath13 learns the noised cardinality and data provider @xmath96 knows the amount of noise it added to the intersection set size .    adopting this algorithm for our case , the client performs the following actions .
assume that two data providers are chosen for intersection in step  [ step : chosen data providers ] : @xmath96 and @xmath0 .    1 .
perform bn - psi - ca algorithm to calculate a noisy intersection set .
assume that without loss of generality , @xmath96 will hold the size of a noisy set , where @xmath0 will hold the amount of added noise .
2 .   @xmath96 sends the size of a noisy set to the client .    in the following iterations of the heuristic algorithm , bn - psi - ca
might be invoked to find intersection size of @xmath96 , @xmath0 and @xmath132 , and more additional data providers . as due to
privacy issues , no single data provider holds the intersection dataset , it is necessary to perform intersection size calculations over again for any additional data provider .
notice that in cases where data providers do not communicate directly , the client acts as an intermediary for the protocol . in the end
, the client has a value of intersection size which is still preserving differential privacy .
the client also keeps record that @xmath0 holds the amount of noise of the last intersection . as described in ( @xcite ) , the noise added in one intersection , might be removed if there is a need to calculate another intersection of @xmath96 and @xmath0 with additional data provider .
( @xcite ) performance results show approximately linear dependency of an algorithm run - time in the number of records in the participating datasets .
using results from section  [ sec : experiments ] , with a reasonable error , it is possible to reduce the number of records by a factor of 5 - 10 ( according to acceptable error ) , thus tens of minutes to single minutes .
such a reduction might make a difference between practical and impractical system .
the algorithm accelerate approximate intersection calculation by :    * computationally intensive calculations of intersection sizes are performed on samples .
those calculations are demanding computationally , thus , any reduce in the size of datasets is important .
[ [ ehud : we should try to find a third - year student to implement this ] ] * the intersection is not calculate exactly , but rather the algorithm stops when the difference between bounds is close enough .
a steady and rapid increase in the amount of data has resulted in an abundance of various data providers .
more data is kept for longer and in more places . in parallel ,
the awareness of data privacy and security is also on a rise both from government regulation and personal perspective .
this trend requires new algorithms and protocols for dealing with distributed data .
those new methods should be both efficient and privacy preserving .
the main practical downside of the current privacy preserving computations is the run - time complexity .
both mpc , oblivious - transfer and private - information - retrieval methods that are used to perform distributed calculations in privacy - preserving manner significantly increase the running time of the query . while some advances in improving performance were made , the running time still remains a limiting factor .
our proposed method uses the ability to provide an approximate answer to the supplied query and thus to reduce considerably the dataset that is used for computations .
this decouples the use of sampling technique for reducing the size of the dataset used for calculations from the protocols designed to preserve privacy of individual records or data providers . as such ,
most of the above protocols can be used in conjunction with sampling techniques to perform approximate calculations with improved performance .
as showed in this work , when acceptable , calculating an approximate answer can significantly improve computation time . in those cases
we have suggested a way to use this possibility for approximate answering by using sampling of the datasets .
the sampling leads to a dramatic reduction in the size of data required for calculation : @xmath134 of the data as observed in simulations
. such reduction of the data size is much more significant when calculations are done in a secure and privacy - preserving way .
r.  agrawal , a.  evfimievski , and r.  srikant .
information sharing across private databases . in _ proceedings of the 2003 acm
sigmod international conference on management of data _ , sigmod 03 , pages 8697 , new york , ny , usa , 2003 .
acm .                                c.  dwork , f.  mcsherry , k.  nissim , and a.  smith . calibrating noise to sensitivity in private data analysis . in _
theory of cryptography , third theory of cryptography conference , tcc 2006 , new york , ny , usa , march 4 - 7 , 2006 , proceedings _ , pages 265284 , 2006 .
p.  b. gibbons .
distinct sampling for highly - accurate answers to distinct values queries and event reports . in _ in proceedings of the 27th international conference on very large data bases _ , pages 541550 .
n.  mohammed , r.  chen , b.  c. fung , and p.  s. yu .
differentially private data release for data mining . in _ proceedings of the 17th acm
sigkdd international conference on knowledge discovery and data mining _ , kdd 11 , pages 493501 , new york , ny , usa , 2011 .
a.  narayan and a.  haeberlen .
djoin : differentially private join queries over distributed databases . in _ in proceedings of the 10th usenix symposium on operating systems design and implementation _ , 2012 .
k.  nissim , s.  raskhodnikova , and a.  smith .
smooth sensitivity and sampling in private data analysis . in _ proceedings of the thirty - ninth annual acm symposium on theory of computing _ , stoc 07 , pages 7584 , new york , ny , usa , 2007 .
r.  pagh , m.  stckel , and d.  p. woodruff . is min - wise hashing optimal for summarizing set intersection ? in _ proceedings of the 33rd acm sigmod - sigact - sigart symposium on principles of database systems , pods14 , snowbird , ut , usa , june 22 - 27 , 2014 _ , pages 109120 , 2014 .
d.  woodruff and q.  zhang .
when distributed computation is communication expensive . in y.
afek , editor , _ distributed computing _ , volume 8205 of _ lecture notes in computer science _ ,
pages 1630 .
springer berlin heidelberg , 2013 .