mobile devices such as smartphones and wearable devices are increasingly gaining popularity as platforms for collecting and sharing sensor data .
these devices , which are often equipped with sensors such as accelerometer , orientation sensor , magnetometer , camera , microphone , gps and so on , are being used by mobile sensing systems to make sophisticated inferences about users .
these inferences have in turn enabled an entire ecosystem of context - aware apps such as behavior - based authentication @xcite , fitness monitoring based on activity tracking  @xcite , speech translation  @xcite and traffic / environmental monitoring  @xcite .
while shared data has enabled context - aware apps to provide utility to the users , the same data can also be used by an adversary to make inferences that are possibly sensitive to the user such as speaker identity  @xcite , keystroke detection  @xcite , location tracking  @xcite , device placement  @xcite , onscreen taps recognition  @xcite , onset of stress  @xcite and detection of emotional state  @xcite .
therefore , there exist fundamentally conflicting requirements between protecting privacy of the sensitive information contained in mobile sensor data and preserving utility of the same data for authorized inferences .
we consider the following case studies that further illustrate this privacy - utility tradeoff :    1 .
case study 1 : a user shares accelerometer data with an authentication app that performs keyless validation of user identity ( _ useful inference _ )  @xcite .
but the same data can also be used to infer the activity mode of the user  if she is walking , or standing still , or moving up or down the stairs ( _ sensitive inferences _ )  @xcite  which in turn may lead to more serious inferences regarding living habits and health conditions .
case study 2 : a user shares location and accelerometer data with a real - time life - logging app that helps with travel planning ( _ useful inferences _ )  @xcite .
however , the same data can also be maliciously used to extract sequences of the user s entered text ( e.g. , passwords or pins ) ( _ sensitive inference _ )  @xcite .
case study 3 : a user shares speech data with a voice - based search app that translates speech to text for web searches ( _ useful inference _ )  @xcite .
but the same data can also be used to recognize the user s identity ( _ sensitive inference _ )  @xcite , compromising her privacy .
recently , several sensor privacy protection mechanisms ( sppms ) have been proposed  @xcite .
however , most existing sppms only provide binary access control over the sensor data that requires users to choose between sharing or blocking a sensor  limiting their applicability and adoption  @xcite .
a limited set of sppms that do focus on obfuscating sensor data , however , lack provable privacy guarantees  @xcite .    in this paper
, we propose a new framework called deeprotect that mediates privacy - preserving access to mobile sensor data .
deeprotect enables a new permission model where instead of blocking or allowing sensors , users specify their utility and privacy preferences in terms of inferences that can be derived from mobile sensor data .
the overall information flow of deeprotect is summarized in figure  [ fig : motivation ] .    in deeprotect
, users can specify as input a set of useful / authorized inferences denoted by @xmath1 ( e.g. , behavior - based authentication , speech - to - text translation ) .
all other inferences ( possibly unknown ) are considered sensitive by default .
alternatively , a subset of sensitive inferences can be specified by the users , which is denoted by @xmath2 ( e.g. , detection of the text entered in keyboard , speaker identity recognition ) .
these useful and sensitive inferences represent functions that map the mobile sensor data to their corresponding inference results ( i.e. , labels ) , which is the information we want to share ( for useful inferences ) or protect ( for sensitive inferences ) .
outgoing sensor data is initially intercepted by deeprotect , which obfuscates the data before sharing such that @xmath3 can be derived accurately by the app and @xmath4 is kept private .    * approach overview *
: we now illustrate the working of deeprotect by using case study 3 as an example .
figure  [ venn_diagram ] shows the venn diagram of information measures in deeprotect . to protect users privacy
, it is required that we should not directly share the raw sensor data @xmath5 ( the entire white square shown in figure  [ venn_diagram]1 ) with the untrusted third - party apps . in case
study 3 , @xmath5 corresponds to the microphone data .
our objective is to enable @xmath3 ( speech - to - text translation ) with high accuracy while protecting against @xmath4 ( speaker identity recognition ) . the key idea in deeprotect is to first ignore the information that is orthogonal to authorized ( useful ) inference information . in this case , we only retain information that enable @xmath3 and ignore the remaining information @xmath6 ( shown in yellow in figure  [ venn_diagram]2 ) . in the next step ,
we perturb those information that are correlated to both the useful and sensitive inferences , @xmath7 ( shown in blue in figure  [ venn_diagram]3 ) , to satisfy provable privacy guarantees .
this helps in selectively hindering the sensitive inferences without severely degrading the useful inferences .    for arbitrary useful and sensitive inferences , the three - stage data obfuscation pipeline used in deeprotect
is illustrated in figure  [ fig : pipeline ] . in the first stage
, we extract features from raw sensor data and perform _ data minimization _
@xcite to guarantee that only features that are relevant to the authorized ( useful ) inferences are retained for further processing .
specifically , deeprotect modifies an autoencoder network  @xcite to _ automatically _ explore the inherent structural characteristics of the sensor data , and learn a sparse feature representation of the data specific to the useful inferences . at this point ,
some of the extracted features , although highly specific to the useful inferences , might still have some correlation with the sensitive inferences . in the next step of the pipeline , to provide provable privacy guarantees for a single user s mobile sensor stream , we exploit _ local differential privacy _
@xcite to obfuscate the extracted features .
note that differential privacy  @xcite is typically applied in multi - user settings , and local differential privacy can be applied in single - user settings .
specifically , we develop a computationally efficient mechanism that perturbs the sparse features , using either the conventional local differential privacy framework  @xcite or our relaxed notion of local differential privacy framework , depending on users specification of sensitive inferences .
finally , we reconstruct obfuscated sensor data from the perturbed features such that the existing interfaces for mobile apps can continue to work without any change .
* usage mode : * deeprotect supports two usage modes , depending on how sensitive inferences are specified .    _ usage mode 1 : _ only the useful inference set @xmath3 is provided and the sensitive inference set @xmath8 is the default set of ` all possible inferences ' . under this mode ,
the user is interested in rigorous privacy guarantees with respect to the entire set ( possibly unknown ) of inferences .
deeprotect first implements deep learning based feature extraction method to raw sensor data and then applies local differential privacy to these extracted features .
_ usage mode 2 : _ both the useful and the sensitive inference sets are provided . under this scenario ,
the user is interested in protecting only a specific set of sensitive inferences ( and not all possible inferences ) . to provide provable privacy guarantees
, we develop a relaxed variant of local differential privacy and use it to perturb the extracted features in a fine - grained manner .    our theoretical analysis ( section  [ sec_thory ] ) and
experimental results on real - world datasets ( section  [ sec_evaluation ] ) show that both usage modes of our approach significantly outperform previous state - of - the - art with up to 8x improvement .
* contributions : * we now summarize our contributions under three main thrusts :    1 .   _ provable privacy guarantees in single - user settings : _ we support rigorous local differential privacy guarantees for a single user s mobile sensor data
however , this rigorous privacy guarantee can require significant addition of noise impacting application utility .
thus , we propose a novel relaxed variant of local differential privacy for providing inference - based access control over the mobile sensor data .
our proposed privacy metric satisfies composition properties , and can achieve improved utility by focusing on protecting a specific set of inferences .
_ novel perturbation mechanisms for enhanced privacy - utility tradeoffs : _ we propose an effective perturbation mechanism which consists of two key techniques : deep learning based data minimization and feature obfuscation based data perturbation .
our first technique automatically extracts features relevant to the useful / authorized inferences by extending the deep learning models for dimensionality reduction , which to our best knowledge is the first such attempt .
for the second technique , we propose effective feature perturbation methods , that achieve both conventional and relaxed notions of local differential privacy guarantees . through rigorous theoretical analysis , we demonstrate the advantage of our approach over the state - of - the - art obfuscation mechanisms .
implementation and evaluation : _ we implement our system using real - world datasets and mobile apps ( for all the case studies discussed previously ) . for our first feature learning step , we demonstrate that our modified deep learning network outperforms the state - of - the - art feature extraction approaches with up to @xmath9 improvement . for our end - to - end approach combining both feature extraction and perturbation ,
we demonstrate the advantage of our technique over the state - of - the - art obfuscation mechanisms with up to @xmath0 improvement .
we also implement experiments to show the effectiveness of our method in defending against inference attacks leveraging mobile sensor data .
we will provide an open source software framework for all the experiments carried out in our study for independent verification and extensions of our results .
mockdroid  @xcite is a modified version of the android operating system ( os ) with ability to ` mock ' a given resource requested by an app .
this approach allows users to revoke access to particular resources at run - time , encouraging users to consider the trade - off between functionality and privacy .
appfence  @xcite offers two approaches for protecting sensitive data from untrusted apps : shadowing and blocking sensitive data from being exfiltrated off the device .
identidroid  @xcite is also a customized android os providing different identifications and privileges , to limit the uniqueness of device and user information .
the above systems still rely on the user to determine the sensitive sensors and block , mock , or shadow them .
ipshield  @xcite , provides users the ability to specify their privacy preferences in terms of inferences , however , they only generate binary privacy policies of _ allow _ and _ deny _ for individual sensors
. such binary access control over the sensor data may seriously affect the functionality of various apps .
deeprotect on the other hand provides the much needed automatic translation from higher - level privacy specifications in terms of inferences to obfuscation of sensor data .
another important limitation of the state - of - the - art sppms is that they are often heuristic in nature and fail to provide rigorous privacy guarantees for inference - based access control over sensors in mobile devices .
for instance , ipshield  @xcite does provide users with ability to configure obfuscation policies for raw sensor data ( through addition of random noise ) , but does not quantify any privacy guarantees for those policies .
boxify  @xcite provides app sandboxing for untrusted apps and can be used to run an untrusted app in an isolated environment with minimum access privileges .
easedroid  @xcite uses semi - supervised learning to perform automatic policy analysis and refinement .
this could be used to compliment deeprotect by automatically learning the set of inferences that need to be protected and ones that need to be released from access patterns and audit logs .
privacy - preserving mobile data aggregation has been studied in  @xcite .
however , they assume the presence of a multi - user database whereas we aim to protect inference - based privacy for a single user s sensor stream .
we surveyed more than @xmath10 research papers published in relevant conferences and journals over the past @xmath11 years to form a knowledge repository of inferences ( see table  [ tableknowledge ] in the appendix ) that can be made using a combination of sensors accessed by mobile apps . this table forms the universe of possible inferences over which the set of useful and sensitive inferences in deeprotect are defined .      to the best of our knowledge
, deeprotect is the first system that provides a novel inference - based access control in mobile sensing applications .
furthermore , deeprotect 1 ) provides provable privacy guarantees including both conventional local differential privacy and our relaxed variant of local differential privacy for sensitive sensor data ; 2 ) presents an effective mechanism consisting of two key techniques : deep learning based data minimization and feature obfuscation based data perturbation ; 3 ) outperforms the state - of - the - art methods in multiple mobile sensing datasets and applications .
[ sec_threat ] we assume that deeprotect together with the underlying mobile os , sensors , and system services form the trusted domain . our adversary is an untrusted app provider , who publishes apps in the marketplace for advertised useful inferences .
the app accesses sensors including ones to which the user has provided explicit permission , and also others for which no permission is required .
the app would then send out the collected sensor data to the adversary .
our goal is to ensure that the data shared with the app can only be used for deriving the authorized ( useful ) inferences and not any sensitive inferences .
we aim to limit / bound the adversary s inference of the user s sensitive information via access to the obfuscated sensor data while achieving two rigorous privacy properties : ( a ) first , we aim to provide rigorous local differential privacy guarantees  @xcite , that limit an adversary against all possible inferences ( corresponding to usage mode 1 in section  [ sec : intro ] ) ; and ( b ) second , we aim to provide a novel relaxed variant of local differential privacy that limits an adversary against a specified set of sensitive inferences ( corresponding to usage mode 2 in section  [ sec : intro ] ) .
furthermore , we adapt the popular laplace perturbation mechanism  @xcite to construct effective perturbation approaches , in order to provide both conventional and relaxed notions of local differential privacy guarantees for protecting mobile sensor data ( in sections  [ sec : usage1 ] ,  [ sec : usage2 ] ) .
it is also interesting to note that a threat model similar to usage mode 2 ( i.e. , protecting against specified inferences ) has been explored in pufferfish privacy  @xcite and blowfish privacy  @xcite .
for example , the set of sensitive inferences computed over sensor data in deeprotect is the set of potential secrets defined in pufferfish and blowfish privacy .
blowfish privacy has recently been adopted by the u.s .
census bureau demonstrating its usefulness as an enabler for practical deployment .
differential privacy ( dp )  @xcite is a rigorous mathematical framework that prevents an attacker from inferring the presence or absence of a particular record in a statistical database .
dp randomizes the query results , computed over the _ multi - user _ database , to ensure that the risk to an individual record s privacy does not increase substantially ( bounded by a function of the privacy budget @xmath12 ) as a result of participating in the database . local differential privacy ( ldp )  @xcite , as an adaption of dp ,
is defined under the setting where the user does not trust anyone ( not even the central data collector ) .
local privacy dates back to warner  @xcite , who proposed the randomized response method to provide plausible deniability for individuals responding to sensitive surveys . in our setting
, we therefore apply ldp to protect a _ single _
user s mobile sensor data while satisfying rigorous privacy guarantees .
similar applications have also been explored in google s rappor system  @xcite .
specifically , @xmath12-ldp is defined as follows :    [ dpdef ] _ ( @xmath12-ldp )  @xcite _ a randomized algorithm @xmath13 provides @xmath12-ldp if for any two databases @xmath14 and for any output set @xmath15 , @xmath16 where @xmath17 ( resp.@xmath18 ) is the output of @xmath13 on input @xmath19 ( resp.@xmath20 ) and @xmath12 is the privacy budget .
smaller value of @xmath12 corresponds to a higher privacy level .    )
.,title="fig:",width=259,height=124 ] .      for a mobile sensing system , consisting of @xmath21 sensors across @xmath22 timestamps ,
the temporal mobile sensor matrix @xmath23 is a real matrix where @xmath24 records the sensing data of the @xmath25-th sensor at the @xmath26-th timestamp . to fully explore the temporal characteristics of the data , we follow common practice  @xcite and partition it into a series of segments according to a user - specified time window size @xmath27 as shown in figure  [ twodata ] . for the @xmath28-th window
, we stack the corresponding columns within it to form a column vector which is denoted by @xmath29 . the temporal mobile sensor matrix can thus be reformulated as @xmath30 $ ] , where @xmath31 .
we therefore apply ldp to a single user s mobile sensor data which is segmented according to a user - specified window size . in this way
, we can take the temporal dynamics of users mobile sensor data into consideration , while providing rigorous privacy guarantees .    from definition  [ dpdef ]
, we observe that @xmath12-ldp has the same mathematical formulation as @xmath12-dp , except that the neighboring databases in ldp are _ any two possible databases without the constraint of one tuple s difference as in dp  @xcite_. thus , when apply ldp to mobile sensing applications , the neighboring databases represent any two segmented sensor data that differ in _ any possible sensor recording at any timestamp within the same window_. a mechanism that provides rigorous ldp can defend against any inference attacks over the obfuscated data , according to the post - processing invariant property  @xcite .
therefore , we can provide rigorous ldp for users sensor data to defend against all possible inferences over the data ( corresponding to usage mode 1 in section  [ sec : intro ] ) .      * the need for a relaxed variant : * although we can provide rigorous ldp and defend against all inferences computed over the obfuscated sensor data ( in usage mode 1 ) , in practice , it may suffice to protect a specific subset of sensitive inferences instead of focusing on all possible inferences .
therefore , we further consider a scenario where the user aims to defend against a _ specific subset of sensitive inferences _ over the sensor data ( corresponding to usage mode 2 ) . under this threat model , there are novel opportunities to strategically add noise and enhance utility .
we thus relax the definition of ldp to provide provable privacy guarantees for a specific subset of sensitive inferences but significantly improve the utility performance ( will discuss in sections  [ sec_thory ] , [ sec_evaluation ] ) .
pufferfish  @xcite and blowfish  @xcite privacy frameworks have similarly motivated the threat model of protecting against a specific set of sensitive computations over the data and have been deployed in real - world settings .
inspired by these frameworks and definition  [ dpdef ] , we define our relaxed neighboring databases as follows :    [ neigh_data ] let us represent a set of sensitive inferences as @xmath32 , and let us represent an orthogonal function of @xmath33 as @xmath34 .
thus , @xmath35 , where @xmath36 and @xmath37 is the orthogonal component of @xmath32 consisting of all the functions that are orthogonal to @xmath32 .
two databases @xmath38 are relaxed neighboring databases , iff .
@xmath39 , @xmath40 for any function @xmath34 .
we illustrate the relaxed neighboring databases above in figure  [ nei ] . for concrete interpretation of the definition
, we refer back to case study 3 in section  [ sec : intro ] , where the useful inference is _ speech - to - text translation _ and the sensitive inference is _ speaker identity recognition_. intuitively , both the useful and sensitive inferences rely on a limited number of features extracted from the sensor data .
the commonly used features for speech - to - text translation are mel - frequency cepstral coefficients ( mfcc ) , spectrogram and n - gram  @xcite , while the popular features for speaker identity recognition are mfcc , spectrogram and the voice biometrics of the user ( such as her pitch , accent , etc . )
according to definition  [ neigh_data ] , the relaxed neighboring databases there refer to two sensor data streams that correspond to different speakers ( i.e. , different values of mfcc , spectrogram and the voice biometrics ) and have the same values of n - gram ( the orthogonal component ) .
therefore , the relaxed neighboring databases in definition  [ neigh_data ] focuses on protecting the privacy of sensitive inferences only ( whose values differ in the two databases ) , and not the component of the data that is orthogonal to sensitive inferences ( whose values remain the same in the two databases ) .    , which have different sensitive inference values whilst the same values for the orthogonal inferences.,title="fig:",width=259,height=67 ] .    *
remark on orthogonal functions : * we further illustrate the relaxed neighboring databases in a three - dimensional space ( our definition can be generally applied to any space ) in figure  [ illu ] , where there are two functions @xmath41 that are orthogonal with the sensitive inference function @xmath33 , i.e. , @xmath42 . according to definition  [ neigh_data ] , we have the following properties for the two relaxed neighboring databases @xmath43 : 1 ) they have the same projection on the direction corresponding to the orthogonal functions , i.e. , @xmath44 ; 2 ) they have different projection on the direction corresponding to the sensitive inference function , i.e. , @xmath45 .    inspired by the relaxed neighboring databases in definition  [ neigh_data ] ,
we now formulate a relaxed variant of local differential privacy ( rldp ) which is also an instantiation of the pufferfish and blowfish privacy frameworks .
in fact , we can observe that our set of sensitive inferences computed over sensor data is the set of potential secrets defined in pufferfish privacy ( definition 3.4 in @xcite ) and blowfish privacy ( definition 4.2 in @xcite ) .
[ ipdef]_(@xmath12-rldp ) _ for a set of sensitive inference functions @xmath46 and a privacy budget @xmath12 , a randomized algorithm @xmath13 satisfies @xmath12-rldp for protecting @xmath46 , if for any two relaxed neighboring databases @xmath38 defined in definition  [ neigh_data ] and any output set @xmath47 , @xmath48 smaller values of @xmath12 correspond to higher privacy level .
to achieve @xmath12-rldp for protecting sensitive information , it is required that the shared data @xmath49 conditioned on any two different sensitive information @xmath50 ( where @xmath39 in definition  [ neigh_data ] ) , is statistically indistinguishable from each other . this in turn guarantees that an adversary gains negligible information about the true sensitive information upon observing the shared data .
we re - iterate that unlike traditional ldp frameworks , its relaxed variant focuses only on the privacy of a specific subset of sensitive inferences .
a perturbation mechanism developed to satisfy rldp guarantees would thus result in better utility performance than the ldp mechanisms ( will discuss in sections  [ sec_thory ] , [ sec_evaluation ] ) .     in definition  [ neigh_data ] .
@xmath41 are orthogonal functions of the sensitive inference function @xmath33.,title="fig:",width=240,height=115 ] .     + .    *
( sequential composition theorem)*[seq ] let randomized algorithm @xmath51 ( @xmath52 ) each provide @xmath53-rldp under the sensitive inference functions @xmath33 .
the sequence of these algorithms @xmath54 provides @xmath55-rldp .    *
( parallel composition theorem)*[paral ] let randomized algorithms @xmath51 ( @xmath52 ) provide @xmath53-rldp under the sensitive inference functions @xmath33 and @xmath56 be arbitrary disjoint data set .
the sequence of these randomized algorithm @xmath57 provides @xmath58-rldp .
in this section , we design effective perturbation mechanisms to achieve @xmath12-ldp and @xmath12-rldp ( corresponding to the two usage modes in section  [ sec : intro ] ) for protecting mobile sensor data in a single - user setting .
our key insight is to exploit the structural characteristics of mobile sensor data to enhance privacy - utility tradeoffs . before proposing our privacy mechanisms
, we first introduce a baseline approach which directly generalizes the traditional dp perturbation mechanisms to achieve @xmath12-ldp .
to achieve @xmath12-dp , the laplace perturbation mechanism ( lpm )  @xcite applies noise drawn from a suitable laplace distribution to perturb the query results .
more formally , for a query function @xmath59 , lpm computes and outputs @xmath60 , where @xmath61 is the parameter of the laplacian noise and @xmath62 is the global sensitivity of @xmath59  @xcite .
when applying ldp ( recall definition  [ dpdef ] ) on segmented mobile sensor data @xmath63 , the neighboring databases @xmath64 may differ in all their possible tuples ( i.e. , all sensor recordings across all timestamps within the same window ) , while the neighboring databases in traditional dp frameworks only differ in one tuple . according to the _ composition theorem _ of dp  @xcite
, we propose a baseline approach to achieve @xmath12-ldp , which inserts a laplacian noise to each temporal sensor data point with the same parameter of @xmath65   is estimated from our collected data serving as a local sensitivity .
] , where @xmath66 is the dimension of each segmented sensor data @xmath63 ( we refer interested readers to @xcite for similar perturbation mechanisms ) .
the baseline approach achieves @xmath12-ldp guarantees and the corresponding proof is deferred to the appendix to improve readability .
note that the baseline approach introduces noise that is linear in the number of temporal sensor recordings within the time window .
this increases the magnitude of noise that needs to be added and thus degrades the usability of the data .
therefore , a better alternative to protect sensor data is to build a compact , privacy - preserving synopsis from the data by exploiting its structural characteristics .
data minimization  @xcite is a fundamental legal instrument that protects privacy by limiting the collection of personal data to the minimum extent necessary for attaining legitimate goals . in our work , we enforce the principle of data minimization and retain only the minimum number of features necessary to enable the authorized inferences .
these extracted features may still be correlated with sensitive inferences ; thus we incorporate local differential privacy to obfuscate the extracted features for protecting against sensitive inferences .    in this section ,
we propose deeprotect ( recall its pipeline in figure  [ fig : pipeline ] ) which consists of two key steps of 1 ) first extracting features based on the data minimization principle and 2 ) then perturbing these features to achieve both conventional and relaxed notions of ldp guarantees ( corresponding to the two usage modes in section  [ sec : intro ] respectively ) .
we will demonstrate the significant advantages of our mechanism over the existing privacy methods theoretically as well as using multiple real - world case studies on real datasets .
mobile sensor data is usually high - dimensional in nature , which typically exhibits both structure and redundancy , allowing minimization  @xcite .
this lays the foundation for the first technique in our deeprotect system : deep learning based data minimization .
* existing autoencoder models : * deep learning models  @xcite learn multi - layer transformations from the input data to the output representations , which is more powerful for feature extraction than hand - crafted shallow models . among the building blocks of these models ,
autoencoders @xcite automatically extract features in an unsupervised manner by minimizing the reconstruction error between the input and its reconstructed output .
a single - layer autoencoder is shown in figure  [ auto ] .
the encoder function @xmath67 maps the input data @xmath63 to the hidden units ( features ) according to @xmath68 , where @xmath69 is typically a sigmoid function , @xmath70 is a weight matrix and @xmath71 is a bias vector .
the decoder function @xmath72 maps these features back to the original input space according to @xmath73 , where @xmath74 is usually the same form as that in the encoder , @xmath75 is a weight matrix and @xmath76 a bias vector . considering the inherent structure of the mobile sensor data  @xcite , we aim to learn an effective feature space on which the mobile sensor data would have a succinct representation , and the corresponding objective function is as follows .     .
@xmath77    where @xmath78 is the reconstruction error between the input data @xmath63 and its reconstructed output @xmath79 ( detailed mathematical formulation for @xmath80 is deferred to the appendix to improve readability ) .
existing autoencoders thus aim to minimize @xmath81 with respect to @xmath82  @xcite .
* constructing data minimization model : * using a nonlinear encoding function an autoencoder can typically extract better features than previous linear transformation methods  @xcite .
but , the features learnt are not specific to the useful ( authorized ) inferences . to handle this
, we explicitly modify the autoencoder models in eq .
[ cost ] by incorporating the useful inferences and other associated constraints as follows .    * incorporating useful inferences : * the objective for our data minimization mechanism is to maximize the utility performance with the minimum amount of information , therefore it is important to combine the useful inferences with the autoencoder models to automatically extract features in a supervised manner .
_ to the best of our knowledge , this is the first work to modify the deep learning models through incorporating the authorized inferences . _
specifically , we incorporate the minimization of cost function corresponding to the useful inferences to the objective function of existing autoencoders in eq .
+ we analyze the cost function for each inference using case study 1 in section  [ sec : intro ] , where behavior - based authentication was considered as a useful inference and activity mode detection was deemed sensitive .
both the useful and sensitive inferences can be mathematically transformed into a classification problem , which can be addressed by machine learning techniques .
for instance , by leveraging the popular _ ridge regression _
technique  @xcite , we can learn an optimal classifier as follows .
@xmath83 where @xmath84 represents the cost function for the useful and sensitive inferences respectively . for behavior - based authentication ( _ useful inference _ ) , the label @xmath85 where @xmath86 represents the legitimate user and @xmath87 represents the adversary .
similarly , for activity mode detection ( _ sensitive inference _ ) , @xmath88 where the labels @xmath89 and @xmath90 represent _ walking _ , _ standing still _ and _ moving up or down the stairs _ respectively .
the optimal classifier @xmath91 learned in eq .
[ krr_obj ] can be utilized to label the newly - coming mobile sensor data for behavior - based authentication or activity mode detection .
the cost function in eq .
[ krr_obj ] has been popularly used in machine learning community whilst other general models in  @xcite can also be explored as potential cost function to reflect the prediction accuracy of the inferences .
note that our analysis are not restricted to these settings and can be utilized for arbitrary inference - based mobile applications .    * incorporating orthogonality : * in addition
, we also aim to learn orthogonal features so that we can deal with each feature independently in our feature perturbation mechanism for achieving enhanced utility performance .
therefore , we add another constraint as @xmath92 to the objective function to guarantee the _ orthogonality _ of the features ( similar intuition has also been utilized in  @xcite ) .    * our new autoencoder model : * incorporating the two constraints ( useful inferences and orthogonality ) into eq .
[ cost ] and combining with eq .
[ krr_obj ] , we construct the objective function for our data minimization method as @xmath93 where @xmath94 controls the trade - off between the reconstruction loss and the utility penalty , and @xmath92 represents the orthogonality constraint .
note that the last two terms in eq .
[ final_obj ] correspond to the cost function in ridge regression ( recall eq .
[ krr_obj ] ) .
although we only consider the cost function of ridge regression to optimize the data minimization process , our algorithm can be generalized to multiple machine learning techniques such as support vector machine  @xcite , naive bayesian  @xcite and random forests  @xcite in a straightforward manner .
* model learning and stacking : * to minimize @xmath95 in eq .
[ final_obj ] with respect to @xmath96 , we explore the stochastic gradient descent ( sgd ) technique , which has been shown to perform fairly well in practice  @xcite .
furthermore , we explore multiple hidden layers to stack multiple model units , in order to generate even more compact and higher - level semantic features resulting in better data minimization . to improve readability , we defer all the details about model learning and stacking into the appendix , based on which we summarize our deep learning based data minimization mechanism in algorithm  [ alg1 ] .
ldp considers the worst - case adversary which can rigorously protect _ against all possible inferences _ computed over the data ( recall definition  [ dpdef ] ) . in the absence of a user - specified set of sensitive inferences , or
otherwise if the user chooses to operate under the ldp guarantees ( corresponding to usage mode 1 in section  [ sec : intro ] ) , we develop our perturbation mechanism through perturbing the features learnt from the deep learning based data minimization step in section  [ sec_feature ] . formally , to achieve @xmath12-ldp , deeprotect under usage mode 1 inserts an laplacian noise with parameter @xmath97 to each previously learned feature , where @xmath98 represent the dimension of the segmented sensor data @xmath63 and the features @xmath99 extracted from @xmath63 , respectively .
, the number of features , i.e. , @xmath100 , needed to be perturbed is much smaller than the number of raw sampling points , i.e. , @xmath66 , in the baseline approach . ]
deeprotect mechanism under usage mode 1 is summarized in algorithm  [ alg3 ] , which satisfies rigorous @xmath12-ldp as will be discussed in theorem  [ proofprivacy ] .
our mechanism is different from the baseline approach in section  [ sec_baseline ] because we add laplacian noise after the application of the deep learning based data minimization step , while the baseline approach directly adds laplacian noise to the raw sensor data without the deep learning mechanism .
after perturbing these features , we reconstruct the perturbed sensor data according to the decoder function @xmath72 in autoencoder ( recall eq .  [ cost ] ) .
we will show that our mechanism significantly outperforms the baseline approach ( in sections  [ sec_thory ] , [ sec_evaluation ] ) .
note that our privacy objective is also different from that in  @xcite since they aim to protect each user s training data in the deep learning training stage under the multi - user settings while in contrast we aim to protect the privacy of mobile sensor data stream in single - user settings .     + perturbed sensor data @xmath101 ; + * for each * @xmath102 +  1 .
extract features @xmath99 from @xmath63 by data minimiza- +   -tion mechanism in algorithm  [ alg1 ] ; +  2 .
obtain perturbed features @xmath103 by inserting noise +  of @xmath104 ; +  3 .
reconstruct perturbed sensor data @xmath105 ;      if the user has specified a subset of sensitive inferences ( corresponding to usage mode 2 in section  [ sec : intro ] ) , we develop our perturbation mechanism ( summarized in algorithm  [ alg2 ] ) by first computing the relaxed sensitivity @xmath106 , and then inserting an laplacian noise with parameter @xmath107 to each previously learned feature , to satisfy @xmath12-rldp .
the detailed process to compute the relaxed sensitivity is as follows .
* computing relaxed sensitivity : * similar to _ sensitivity _ in dp  @xcite , our _ relaxed sensitivity _ can be computed as @xmath108 where @xmath109 are relaxed neighboring databases in definition  [ neigh_data ] , and the denominator is set to make _ relaxed sensitivity _ comparable with traditional _
sensitivity _ computed over neighboring databases that differ in only one tuple .
the sensitive inferences can be mathematically transformed into a classification problem , which can be addressed by machine learning techniques .
the ridge regression classifier in eq .
[ krr_obj ] is a linear function computed over the segmented sensor data @xmath63 , i.e. , @xmath110 .
we explain the computation of relaxed sensitivity using this formulation , though our analysis can be generally applied to non - linear situations using kernel - based techniques  @xcite .
we apply the _ gram - schmidt orthogonalization _ technique @xcite to a matrix @xmath111 $ ] ( where @xmath112 is an _ identity _ matrix and thus @xmath111 $ ] is full - rank ) , in order to obtain orthogonal vectors of @xmath113 as @xmath114 .
based on that , we form an orthogonal matrix @xmath115 $ ] .
any function @xmath34 that is orthogonal of the sensitive inference function @xmath32 ( recall definition  [ neigh_data ] ) can be represented by a linear combination of @xmath114 .
for the two relaxed neighboring databases @xmath64 , we have @xmath116 , according to definition  [ neigh_data ] .
therefore , we know that @xmath117^t=[\gamma , 0,0,\cdots]^t$ ] , i.e. , @xmath118^t$ ] , and the value of @xmath119 is restricted by the range of @xmath120 . since we consider the _ identity _ query ( as we publish the sanitized sensor data to mobile apps ) , we obtain the constraint of @xmath119 as @xmath121^t\|_1=\|{\bm{x}}_{t1}-{\bm{x}}_{t2}\|_1\le\dim({\bm{x}}_t)\delta q$ ] .
therefore , we can compute the relaxed sensitivity as follows .
@xmath122^t\|_1\le \dim({\bm{x}}_t)\delta q}}\frac{\|{\bm{s}}^{-1}\cdot[\gamma,0,0,\cdots]^t\|_1}{\dim({\bm{x}}_t)}\ ] ] it is worth noting that @xmath123 due to the constraint of relaxed neighboring databases for achieving the same orthogonal inference values ( recall definition  [ neigh_data ] ) .
therefore , comparing to usage mode 1 , deeprotect under usage mode 2 would add less noise to the features extracted in the data minimization step thus achieving better utility .
+ perturbed sensor data @xmath101 ; + * for each * @xmath102 +  1 . extract features @xmath99 from @xmath63 by data minimiza- +   -tion mechanism in algorithm  [ alg1 ] ; +  2 .
compute relaxed sensitivity @xmath106 in eq .
[ relaxed_sensitivity0 ] ; +  3 .
obtain perturbed features @xmath103 by inserting noise +  of @xmath124 ; +  4 .
reconstruct perturbed sensor data @xmath105 ;    our perturbation mechanisms summarized in algorithms  [ alg3 ] ,  [ alg2 ] satisfy @xmath12-ldp , @xmath12-rldp respectively , according to the following theorems . to improve readability , we defer the corresponding proofs to the appendix .
[ proofprivacy ] deeprotect under usage mode 1 ( corresponding to algorithm  [ alg3 ] ) satisfies @xmath12-ldp .
[ proofprivacy2 ] deeprotect under usage mode 2 ( corresponding to algorithm  [ alg2 ] ) satisfies @xmath12-rldp .      for a perturbation algorithm @xmath125 ,
let us denote @xmath126 $ ] as the expected error in the release of data @xmath5 , where @xmath127 $ ] is the expectation taken over the randomness of @xmath128 .
we quantify the utility advantage of deeprotect over the baseline approach in the following theorems  [ the_error ] , [ the_error2 ] ( corresponding to the two usage modes ) , and the detailed proofs are deferred to the appendix .
[ the_error ] for deeprotect under usage mode 1 ( corresponding to algorithm  [ alg3 ] ) , the expected error @xmath129 is lower than that of the baseline approach in section  [ sec_baseline ] by a factor of @xmath130 , where @xmath99 is the feature set extracted from the segmented sensor data @xmath63 by using our deep learning based data minimization approach .
[ the_error2 ] for deeprotect under usage mode 2 ( corresponding to algorithm  [ alg2 ] ) , the expected error @xmath129 is lower than that of the baseline approach in section  [ sec_baseline ] by a factor of @xmath131 , where @xmath132 are the sensitivity and the relaxed sensitivity corresponding to the query function @xmath59 , respectively .    therefore , we can see that the utility advantage of our deep learning based data minimization step in section  [ sec_feature ] is @xmath130 and of our relaxed variant of local differential privacy in section  [ sec : usage2 ] is @xmath133 .
in this section , we describe our methodology for collecting sensor data from mobile devices , and the experimental setup ( including system parameters ) for evaluation .      in our experiments
, we collected data using a google nexus 5 ( with 2.3ghz , krait 400 processor , 16 gb internal storage and 2 gb ram on android 4.4 ) and a moto360 smartwatch ( with omap 3 processor , 4 gb internal storage , 512 mb ram on android wear os ) . on the smartphone
, data was captured from the _ accelerometer _ , _ gyroscope _ , _ orientation _ , _ ambient light _ , _ proximity _ , and _
sensors . on the smartwatch , the _ accelerometer _ and _ gyroscope _ sensors were recorded ( recall figure  [ twodata ] ) .
the sampling rate was fixed at @xmath134 hz .
@xmath135 graduate students ( @xmath136 males and @xmath137 females ) in our university were invited to take our smartphone and smartwatch for two weeks and use them in the same way that they would use their personal devices in their daily lives .    to obtain the ground - truth information for performance evaluation
, we ask the users to record labels for both the _ useful _ and _ sensitive inferences_. the labelled training data is grouped under two different categories : mode - detection data and identity - recognition data .
users perform tasks such as walking , enunciating digits or specific alphabets , and the corresponding data segments are then labelled as per the tasks . the mode - detection data , correspond to labelled user activities ( e.g. , accelerometer data segments are marked with labels such as `` walking '' ) , and speech - to - text translation labels ( where audio segments are labelled with the corresponding spoken digit or alphabet ) .
the identity - recognition data is used for authentication and speaker identity recognition experiments .
the labelled data is generated by associating the identity of a user as label to a mobile device , on first use .
we provide provable privacy guarantees for temporal mobile sensor data ( discussed above ) , which is segmented according to the parameter of time window size @xmath138 . for the deep learning based data minimization step ( in section  [ sec_feature ] )
, we use 10-fold cross validation to generate the training data and testing data , where @xmath139 of our collected data is used as training data and the remaining @xmath140 is used as testing data .
we repeated this process for @xmath141 iterations and reported the averaged results . in our experiments , we used stacked autoencoders ( @xmath142 in algorithm  [ alg1 ] ) with two hidden layers comprising of 15 and 7 units respectively . we will show that an autoencoder with only two - hidden layers was able to extract better features than the state - of - the - art techniques .
the reduced number of layers ( and units ) in the autoencoder allowed us to train the model using small amount ( 2 weeks ) of labelled data from the user ( note that we are protecting the sensitive inferences for a single user ) .
we implemented all the three case studies ( tradeoff between authentication and activity recognition , tradeoff between transportation detection and text recognition , tradeoff between speech translation and speaker identification ) discussed in section  [ sec : intro ] on our real - world dataset using the system parameters discussed above .
in this section , we experimentally demonstrate the effectiveness of deeprotect using multiple real - world datasets and applications .
we first show the advantage of our deep learning based data minimization step ( in section  [ sec_feature ] ) over existing feature extraction approaches with up to 2x improvement .
next , we show the advantage of our end - to - end approaches combining both feature extraction and perturbation ( in sections  [ sec : usage1 ] ,  [ sec : usage2 ] ) over the baseline approach with up to 8x improvement .      we experimentally evaluate our deep learning based data minimization mechanism in section  [ sec_feature ] , using the dataset collected for case study 1 ( recall section  [ sec : intro ] ) , where the useful inference is the _ behavior - based authentication_. to show the advantage of our method , we further compare it with the state - of - the - art feature extraction approaches .
the discrete _ fourier _ transform ( dft ) and discrete cosine transform ( dct ) are two basic transformation techniques in the signal processing community , and the haar basis forms the fundamental wavelet transformation for time - frequency signal analysis
. the principal component analysis ( pca ) technique @xcite can also be utilized to reveal hidden structure in the mobile sensor data .
furthermore , we also consider blind compressive sensing ( bcs ) as a typical dictionary learning method for comparison  @xcite .    * higher accuracy
* : figure  [ feature_utility ] shows the utility - preserving performance under different feature extraction methods .
note that @xmath143-axis is in log scale and @xmath144-axis is the accuracy for behavior - based authentication in case study 1 representing the ratio of correctly authenticated users .
note that we use three - dimensional _
accelerometer _ measurement for behavior - based authentication and set the window size as @xmath138 ( recall section  [ para ] ) , therefore the dimension of each segmented sensor data is @xmath145 .    from figure  [ feature_utility ]
, we can see that 1 ) more features would be beneficial for improving the utility performance since the combination of multiple features would be more accurate and expressive to represent the input data ; 2 ) our method achieves higher accuracy than the state - of - the - art approaches with up to @xmath9 improvement .
@xmath146 features are enough for our method to provide good utility performance with @xmath147 accuracy , while the accuracy by using all the @xmath148 features is @xmath149 . in comparison ,
the maximum accuracy for the baseline approaches is only @xmath150 by using @xmath146 features ; 3 ) our data minimization mechanism significantly benefits from the automatic learning process which explicitly incorporates the useful inference information into deep learning models .
based on our analysis above , we constrain the dimension of features extracted by our deep learning based data minimization method to @xmath146 , whose corresponding accuracy is higher than @xmath151 ( @xmath152 ) of the accuracy achieved by using all the features .
therefore , we represent each of the @xmath148-dimensional input sensor data @xmath153 with a feature set @xmath99 consisting of only @xmath146 features .    * higher informativeness * : we further leverage _ informativeness _ in  @xcite as an important metric to evaluate the information - theoretic relationship between each individual feature and the useful inference information
. the _ informativeness _
@xmath154 of the @xmath25-th feature @xmath155 , measures the relative mutual information between the feature and the label of the utility function @xmath156 , and is computed as @xmath157 is the random variable taking the @xmath25-th value in each @xmath158 . ] . under the setting of case study 1
, @xmath159 represents the legitimate user and the adversary , respectively .
@xmath160 is the entropy of the variable @xmath156 and @xmath161 is the mutual information between the random variables of @xmath155 and @xmath156 . for each feature
@xmath155 , this measure of _ informativeness _ takes a value between @xmath162 and @xmath86 , where @xmath162 means that the feature contains no information about the utility label @xmath156 and @xmath86 means that the feature can completely determine the utility label @xmath156 .
recall that we set the dimension of features extracted by our method to @xmath146 ( see figure  [ feature_utility ] ) .
we further evaluate their corresponding _ informativeness _ in figure  [ informativeness ] .
we can observe that the features learned by our method have much higher _ informativeness _ with up to @xmath9 improvement over previous works .
therefore , our proposed method captures more expressive , higher - quality features than the existing state - of - the - art approaches .          to demonstrate the effectiveness of our end - to - end perturbation mechanisms combining both feature extraction and perturbation
, we again consider the _ behavior - based authentication _ as useful inference and the _ activity mode detection _ as sensitive inference ( recall case study 1 in section  [ sec : intro ] ) .
we first compare the baseline method with our mechanism under usage mode 1 ( corresponding to algorithm  [ alg3 ] ) , since they achieve the same level of privacy guarantees for preventing all possible sensitive inferences .
then , we compare the baseline method with our mechanism under usage mode 2 ( corresponding to algorithm  [ alg2 ] ) , to verify the effectiveness of our rldp for protecting a specific subset of sensitive inferences .    * utility advantage under both usage modes : * figure  [ utiltity_epsilon ] shows the utility performance computed over the obfuscated sensor data generated by deeprotect .
we can make the following important observations using figure  [ acc_epsilon ] : 1 ) deeprotect achieves considerable advantage over the baseline approach with up to @xmath0 improvement in utility .
this validates the effectiveness of deeprotect that not only provides rigorous privacy guarantees for protecting sensitive inferences , but also retains the utility of the perturbed data ; 2 ) deeprotect achieves better utility performance under usage mode 2 ( which only considers specific sensitive inferences ) than that under usage mode 1 ( which considers the entire set of sensitive inferences ) ; 3 ) as expected , at higher values of @xmath12 , there is an improvement in utility but at the cost of degradation in privacy ; 4 ) even at moderate value of @xmath163 which is a typical privacy budget in ldp ( similar values can also be found in google s rappor system  @xcite ) , the authentication accuracy using deeprotect ( under usage mode 2 ) is close to the noise - free level ( @xmath149 in figure  [ feature_utility ] ) .
this observation further validates the effectiveness of our mechanisms .
note that the neighboring databases in ldp / rldp may differ in all their possible tuples ( instead of differing in only one tuple as in dp ) .
thus , a proper privacy budget in ldp / rldp for balancing utility and privacy is usually higher than that of dp .
+    to investigate the effectiveness of deeprotect on real - world mobile applications , we plot the trade - off between the accuracy of making useful inferences versus the accuracy of making sensitive inferences for all the three case studies ( corresponding to usage mode 2 and thus algorithm  [ alg2 ] is applied ) , as shown in figure  [ balance123 ] . * utility - privacy tradeoffs for inference - based access control : * in figure  [ balance1 ]
, we observe that 1 ) deeprotect achieves good inference performance for behavior - based authentication ( useful inference ) , while significantly deteriorating the activity mode detection ( sensitive inference ) . for privacy budget @xmath163 which is a typical privacy budget in ldp ( similar values have also been used in google s rappor system  @xcite ) , the accuracy for inferring the useful information ( authentication ) is larger than @xmath164 while the accuracy for inferring sensitive information ( activity modes ) drops to roughly @xmath165 which is equivalent to random guessing users data in our experiments , the accuracy for random guessing is @xmath166 . ] .
therefore , deeprotect can achieve practical privacy while only degrading utility performance by @xmath167 .
this provides an effective guide for users to choose a proper value of @xmath12 for real world applications ; 2 ) higher levels of perturbation would degrade the inference performance for both the useful and sensitive information ; 3 ) deeprotect is effective for defending against sensitive inference attacks computed over sensor data .
these observations demonstrate that deeprotect works well in practice and returns an acceptable utility performance while satisfying provable privacy guarantees .
similarly , for the other two case studies , the accuracy of the sensitive inferences degrades at a much faster rate than that of the useful inferences when more noise is added ( corresponding to a smaller privacy budget @xmath12 ) , which further validates the effectiveness of deeprotect . for case study 2 , in figure  [ balance2 ] , when @xmath168 , deeprotect achieves good performance for transportation detection ( useful inference ) , while significantly degrading the identification of entered text ( sensitive inference ) .
for case study 3 , in figure  [ balance3 ] , when @xmath169 , deeprotect achieves good performance for speech translation ( useful inference ) , while preventing the recognition of speaker identity ( sensitive inference ) .
depending on the mode the user chooses to operate in deeprotect can provide either the rigorous ldp guarantee that protects all sensitive inferences ( usage mode 1 ) or our rldp that protects a user - chosen subset of inferences ( usage mode 2 ) . while rldp provides weaker privacy guarantees than ldp , it comes with the advantage of stronger utility properties . when applying ldp / rldp on mobile sensor data , the neighboring databases may differ in all their possible tuples ( i.e. , all sensor recordings across all timestamps within the same window ) , while the neighboring databases in traditional dp only differ in one tuple .
therefore , a proper privacy budget in ldp / rldp for balancing privacy and utility is usually higher than that of dp  @xcite .
previous work in @xcite proposed to ignore several records in the database to bound sensitivity in dp for providing better privacy - utility tradeoffs .
our approach of using deep learning techniques to extract authorized features is a conceptual improvement over these methods .
while deeprotect has been presented in the context of privacy - aware data sharing on mobile phones , the techniques developed are not restricted to the specific setting , and can be applied to other scenarios beyond mobile phones .
we will make deeprotect tool available to public as open source software .
although our privacy guarantee is limited to segmented sensor data , we do take the temporal dynamics existing in mobile sensor stream into consideration according to a user - specified parameter of window size .
similar to any dp - oriented metrics , our privacy guarantees also compose securely , i.e. , retain privacy guarantees even in the presence of multiple independent releases ( recall theorems  [ seq ] ,  [ paral ] ) . while there is a non - trivial utility cost incurred by our mechanisms , deeprotect ( under both usage modes )
significantly outperforms the baseline approach .
in addition , our utility performance can be further improved by 1 ) exploiting fine - grained correlation among sensors ( e.g. , among the three dimensions of accelerometer in case study 1 ) according to @xcite ; 2 ) training customized models specific to each individual user that installs deeprotect in phone ; and 3 ) training models based on a larger amount of users data as in @xcite .
our perturbation mechanism can also be easily generalized to consider correlation across time windows , according to the composition properties of our privacy metrics ( refer to @xcite and theorems  [ seq],[paral ] ) .
an important component of our perturbation mechanism is the computation of the feature sensitivity in algorithms  [ alg3],[alg2 ] .
ways to accurately compute the sensitivity for arbitrary mobile applications , and deal with its possible underestimation would be a challenge we would like to address in the future .
we are working towards an implementation of the auto - encoder on the android platform using one of the deep learning packages on android  @xcite .
the training itself could be done offline and then transferred on the phone to perform data minimization in real time  @xcite .
our off - line training process requires a small amount of labeled data , for example , in our experiments , we only required 20 users data collected for 2 weeks .    any dp - oriented mechanism that adds noise to the data ( or corresponding query ) can risk the problem of breaking the integrity constraints of these sensors . in practice , since our framework targets simple sensors such as accelerometer , gyroscope , etc .
, our decoding process is unlikely to break the integrity constraints for these sensors .
therefore , apps can continue to use the perturbed sensor data generated by deeprotect without any change to their existing interface .
in this paper , we propose deeprotect , a general privacy - preserving framework for inference - based access control of time - series sensor data on mobile devices .
deeprotect not only supports conventional ldp guarantees , but also provides a novel relaxed variant of ldp .
we further propose effective perturbation mechanisms consisting of two key steps : 1 ) we uniquely explore deep learning techniques to realize data minimization and only retain features relevant to the useful ( authorized ) inferences .
this prevents the leakage of any information that is orthogonal to the useful inferences ; and 2 ) to further enhance the privacy of sensitive inferences , we perturb the extracted features , using an effective obfuscation mechanism that ensures both conventional and relaxed ldp guarantees while simultaneously maintaining the utility of the data . finally , for reasons of compatibility with existing third - party apps , we reconstruct the sensor data , from the noisy features before sharing . through theoretical analysis and extensive experiments over multiple real - world datasets ,
we demonstrate that compared to the state - of - the - art research , deeprotect significantly improves the accuracy for supporting mobile sensing applications while providing provable privacy guarantees .
this work has been partly supported by faculty awards from google , intel and nsf .
changchang liu is partly supported by ibm phd fellowship .
we are very thankful to brendan mcmahan , keith bonawitz , daniel ramage , nina taft from google , and richard chow from intel for useful feedback and discussions .
* sequential composition theorem : * for @xmath170 and @xmath171 , let @xmath172 and @xmath173 . for any sequence @xmath174 of outcomes @xmath175 ,
the probability of output @xmath174 from the sequence of @xmath176 is @xmath177 . applying the definition of rldp for each @xmath51
, we have @xmath178 .    * parallel composition theorem : * for @xmath170 and @xmath171 , let @xmath179 and @xmath180 . for any sequence @xmath174 of outcomes @xmath175 , the probability of output @xmath174 from the sequence of @xmath176 is @xmath177 . applying the definition of rldp for each @xmath51 , we have @xmath181 .
* proof for theorem  [ the_error ] : * first , we derive the variance of a randomized algorithm @xmath13 for our deeprotect under usage mode 1 as @xmath189 .
then , we compute the expected error as @xmath190 \le \mathbb{e}[\|{a}({\bm{x}}_t)-\mathbb{e}[{a}({\bm{x}}_t)]\|_1]+\mathbb{e}[\|{a}({\bm{x}}_t)-{\bm{x}}_t\|_1 ] = re\_{error}({a}({\bm{x}}_t))+\sqrt{\mathbb{e}[\|{a}({\bm{x}}_t)-{\bm{x}}_t\|_2 ^ 2 ] } = re\_{error}({a}({\bm{x}}_t))+\sqrt{var({a}({\bm{x}}_t))}$ ] .
note that our deep learning based data minimization mechanism would result in a negligible reconstruction error , which makes it fairly applicable in many practical scenarios .
it is likely that the reconstruction error @xmath191 is much lower than the perturbation error caused by adding noise .
therefore , we approximate the utility performance of deeprotect as @xmath192 ( similar results can also be found in  @xcite ) . similarly , we evaluate the expected error for the baseline approach as @xmath193=\mathbb{e}[\|(lap(\delta q/\epsilon))\|_1]=\delta q/\epsilon$ ] . comparing the utility performance for both methods , we know that deeprotect under usage mode 1 reduces the expected error of the baseline approach with a factor of @xmath194 .
* proof for theorem  [ the_error2 ] : * first , we derive the variance of a randomized algorithm @xmath13 for our deeprotect under usage mode 2 as @xmath190 \le \le \mathbb{e}[\|{a}({\bm{x}}_t)-\mathbb{e}[{a}({\bm{x}}_t)]\|_1]+\mathbb{e}[\|{a}({\bm{x}}_t)-{\bm{x}}_t\|_1 ] = re\_{error}({a}({\bm{x}}_t))+\sqrt{\mathbb{e}[\|{a}({\bm{x}}_t)-{\bm{x}}_t\|_2 ^ 2 ] } = re\_{error}({a}({\bm{x}}_t))+\sqrt{var({a}({\bm{x}}_t))}$ ] .
similarly , we have @xmath195 . comparing the utility performance for both methods , we know that deeprotect under usage mode 2 reduces the expected error of the baseline approach with a factor of @xmath196 .
[ [ model - learning - and - stacking - in - deep - learning - based - data - minimization ] ] model learning and stacking in deep learning based data minimization ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~    * existing autoencoder models : * the mathematical formulations of existing autoencoder models  @xcite ( expanding eq .
[ cost ] ) is @xmath197 where the encoder function @xmath67 maps the input data @xmath198 to the hidden units ( features ) @xmath199 according to @xmath68 , and the decoder function @xmath72 maps the outputs of the hidden units ( features ) back to the original input space according to @xmath73 .
similar to most existing deep learning methods  @xcite , we choose the sigmoid function for both the encoding activation function @xmath69 and the decoding activation function @xmath74 , and we focus on the tied weights case in which @xmath200 ( where @xmath201 is the transpose of @xmath202 ) .
@xmath203 is a loss function , typically the square loss @xmath204 .
the second term in eq .
[ cost2 ] is a weight decay term that helps prevent overfitting  @xcite . the third term in eq .
[ cost2 ] is a sparsity constraint @xmath205 with a pre - determined sparsity parameter @xmath206 , where @xmath207 is the average activation of hidden unit @xmath25 and @xmath208 is the kl divergence between two bernoulli random variables with mean @xmath206 and @xmath209 respectively .
* model learning : * the objective for our data minimization mechanism is to minimize @xmath95 in eq .  [ final_obj ] with respect to @xmath96 .
we explore the stochastic gradient descent ( sgd ) technique to solve this optimization problem , which has been shown to perform fairly well in practice  @xcite .
note that we use sgd to only solve the convex optimization problem in eq .
[ final_obj ] ( without considering the non - convex constraint of @xmath213 ) . to train our neural network ,
we first initialize each parameter @xmath96 to a small random value near zero , and then apply sgd for iterative optimization . before computing the exact gradient for the objective function in eq .
[ final_obj ] with respect to each variable , we first perform the feedforward pass and evaluate @xmath214 .
next , we compute the error terms @xmath215 as @xmath216 . finally , the gradient descents as partial derivatives can be computed as @xmath217 .
_ update @xmath70 to satisfy orthogonality constraint : _ after obtaining @xmath70 from sgd , we need to modify @xmath70 to satisfy the orthogonality constraint of @xmath213 in eq .
[ final_obj ] .
this constraint is difficult to solve since it is non - convex and of high computational complexity .
our method to overcome this constraint follows the rigorous analysis in @xcite by adopting a simple operation , @xmath218 , which sets the singular values of @xmath70 to be all ones .
the above operation is conducted after every sgd step .
the overall process for our deep learning based data minimization method is shown in algorithm  [ alg1 ] .    * model stacking : * although algorithm  [ alg1 ] is effective for solving eq .
[ final_obj ] , the learnt result heavily relies on the seeds , used to initialize the optimization process .
therefore , we use multiple hidden layers to stack the model in order to achieve more stable performance . in other words ,
our data minimization mechanism can also be used to build a deep network through model stacking . for the first layer in the deep learning model
, we find the optimal layer by minimizing the objective function in eq .
[ final_obj ] .
the representations learned by the first layer are then used as the input of the second layer , and so on so forth . using a stacked deep auto - encoder
, we can learn stable and finer - grained features to better represent the raw sensor data .