wyner and ziv @xcite , we study a setting where a sequence generated by a source is to be described succinctly to a reconstructor ( `` decoder '' ) with access to some side information .
wyner and ziv showed that , although the side information is not available at the describing terminal ( `` encoder '' ) , it can be beneficial in improving the trade - off between the rate of description and the reconstruction distortion .
they fully characterized this trade - off for memoryless sources with single - letter distortion measures . unlike the case without side information
 since the side information is used in the reconstruction process , and since the side information is not available at the describing terminal  the describing terminal can not tell how the source sequence it observes will be reconstructed . in some settings , this is unacceptable .
steinberg @xcite therefore studied the common - reconstruction problem where an additional restriction is imposed that the reconstruction sequence be computable with probability nearly one at the describing terminal .
this greatly limits the extent by which the reconstruction can depend on the side information .
more generally , there is a tension between the degree by which the reconstructing terminal utilizes the side information and the precision with which the describing terminal can compute the reconstruction sequence .
it is this tension that we study in this paper .
( 7.7,3.8)(-4.5,-2.4 )    ( -5,-0.75)(1,0)1 ( -4.8,-0.5)@xmath0    ( 2.5,1)(0,-1)1 ( 2.7,0.5)@xmath1    ( -1.5,-0.5)@xmath2 ( -2,-0.75)(1,0)3.5    ( 2.5,-1.5)(0,-1)1 ( 2.75,-2.25)@xmath3    ( -3,-1.5)(0,-1)1 ( -2.8,-2.25)@xmath4    ( -0.3,-3.5)@xmath5 \leq \dd$ ] ( -6,-3.5)@xmath6 \leq \de;$ ]    ( -3.75,-0.85)encoder ( 1.8,-0.85)decoder    ( -4.0,-1.5)(0,1)1.5 ( -4.0,0)(1,0)2 ( -2,-1.5)(0,1)1.5 ( -2,-1.5)(-1,0)2 ( 1.5,-1.5)(0,1)1.5 ( 1.5,0)(1,0)2 ( 3.5,-1.5)(0,1)1.5 ( 3.5,-1.5)(-1,0)2    to quantify this tension , we require that the describing terminal generate an estimate of the sequence that will be produced at the reconstructing terminal ( figure  [ fig : problem - diagram ] ) .
we then study the distortions that can be simultaneously achieved at the describing terminal (  the encoder distortion  ) and at the reconstructing terminal (  the decoder distortion  ) as a function of the description rate .
if the encoder s distortion function is the hamming distance and if the allowed distortion is zero , then our problem reduces in essence to steinberg s common - reconstruction problem . in section  [ sec : related ] ahead . ] and if the allowed encoder distortion is infinite , our problem reduces to wyner and ziv s problem
. we can thus view our problem as a generalization of the wyner - ziv problem and steinberg s common reconstruction problem .    for discrete memoryless sources and finite single - letter distortion functions , we provide a single - letter characterization of the trade - off between the description rate and the distortions at the encoder and decoder sides .
we also calculate this trade - off for a memoryless gaussian source and squared - error distortion functions . finally , in section  [
sec : kdistortions ] , we generalize the results to account for more than two constraints and to allow each distortion function to depend on three arguments : the source symbol , the encoder s reconstruction symbol , and the decoder s reconstruction symbol .
steinberg s work was also extended in other ways .
kittichokechai , oechtering , and skoglund @xcite determined the rate - distortion function under a common - reconstruction constraint for a modified wyner - ziv setup where the encoder can influence the decoder s side information via an action - generator .
timo , grant , and kramer @xcite , @xcite and ahmadi , tandon , simeone , and poor @xcite , @xcite derived the rate - distortions function under a common - reconstruction constraint for two special cases of the heegard - berger / kaspi problem ( the wyner - ziv problem with two decoders ) : @xcite , @xcite for physically degraded side informations , and @xcite , @xcite for complementary side informations .
ahmadi , tandon , simeone , and poor @xcite , @xcite also presented the rates - distortions function under a common - reconstruction constraint for a cascade source - coding problem when the side informations are physically degraded .
finally , already in  @xcite , steinberg studied the implications of the common - reconstruction constraint on the simultaneous transmission of data and state and on joint source - channel coding for the degraded broadcast channel .
the paper is organized as follows . in the rest of this section
we introduce our notation . in section  [ sec : setup ] we treat discrete sources and general distortions , and in section  [ sec : gauss ] gaussian sources with quadratic distortions . in section
[ sec : kdistortions ] we revisit discrete sources but this time with more and more general distortion constraints .
random variables are denoted by upper - case letters and their realizations by lower - case letters .
vectors are denoted by bold - face letters : random vectors by upper - case bold - face letters , and deterministic vectors by lower - case bold - face letters .
sets and events are denoted by calligraphic letters , i.e. , @xmath7 .
an @xmath8-tuple @xmath9 is denoted @xmath10 , and the @xmath8-fold cartesian product of the set @xmath7 is denoted @xmath11 .
the convex hull of a set @xmath7 is denoted by @xmath12 . to indicate that the random variables @xmath13 and @xmath14 and conditionally independent given @xmath15 we write @xmath16    the transpose of a vector  @xmath17 is denoted by @xmath18 ; its euclidean norm by @xmath19 ; and the euclidean inner product between the vectors @xmath17 and @xmath20 by @xmath21 .
the set of real numbers is denoted @xmath22 and its @xmath23-fold cartesian product @xmath24 .
the nonnegative reals are denoted @xmath25 , and the positive reals @xmath26 .
the respective @xmath23-fold cartesean products are denoted @xmath27 and @xmath28 .
we use @xmath29 to denote the indicator function : @xmath30 is equal to one if the statement is true and is equal to zero if it is false . throughout
the paper @xmath31 denotes base-2 logarithm , and @xmath32 .
the abbreviation iid stands for independently and identically distributed .
our setting is illustrated in  figure  [ fig : problem - diagram ] and is specified by a tuple @xmath33 where @xmath34 are finite sets , @xmath35 is a probability distribution on @xmath36 ; @xmath37 and @xmath38 are nonnegative functions @xmath39 and @xmath40 and @xmath41 are nonnegative real numbers .    the sets @xmath42 , @xmath43 , and @xmath44 model the source , side information , and reconstruction alphabets .
a source sequence @xmath45 is observed at the encoder ( but not at the decoder ) and a side - information sequence @xmath46 at the decoder ( but not at the encoder ) . the sequence of pairs @xmath47 is assumed to be drawn iid according to the joint law @xmath35 .
the encoder describes the source sequence @xmath48 to the decoder by an index    rcl[eq : enc ] & = & ( x^n )    where @xmath49 is the encoding function and @xmath50 based on the index @xmath51 and its side information  @xmath52 , the decoder forms a reconstruction sequence    rcl[eq : decd ] _
^n & = & ( , y^n )    where @xmath53 is the decoder s reconstruction function .
the encoder s estimate of the decoder s reconstruction sequence is    rcl[eq : dece ] _
e^n & = & ( x^n )    for some @xmath54    the goal of the communication is that the decoder s reconstruction @xmath3 matches the source sequence @xmath0 up to a distortion no larger than @xmath40 and the encoder s estimate @xmath4 matches the decoder s reconstruction @xmath3 up to a distortion no larger than @xmath41 .
the distortions are measured by the bounded , nonnegative , single - letter distortion functions @xmath37 and @xmath38 .
we say that a nonnegative triple @xmath55 is _ achievable _ if for every @xmath56 and sufficiently large @xmath8 there exists a message set of size @xmath57 as above such that the _ decoder - side reconstruction constraint _ @xmath58
\leq   \dd + \epsilon\ ] ] and the _ encoder - side reconstruction constraint _ @xmath59 \leq   \de+\epsilon\ ] ] are both met .
our problem is not very interesting if the distortion constraints can not be met even when the source sequence is revealed losslessly to the reconstructor .
consequently , we shall make the following assumption throughout :    [ assumption ] the distortion functions @xmath60 and @xmath61 are such that for each @xmath62 there exist @xmath63 satisfying @xmath64 and @xmath65 .
as we shall see , this assumption ensures that the triple @xmath66 is achievable whenever @xmath67 .
we are interested in finding the smallest rate @xmath68 such that a given distortion pair @xmath69 is achievable . for given @xmath70 ,
let @xmath71 denote the set of rates @xmath72 such that the tuple @xmath73 is achievable : @xmath74 notice that by the assumption above , the set @xmath75 contains all rates @xmath67 and is thus nonempty
. we can now define _ rate - distortions function _ as @xmath76 where the minimum exists because the set @xmath77 is nonempty , closed , and bounded from below by 0 .
wyner and ziv s classic lossy source - coding problem with side information  @xcite is similar to our problem except that wyner and ziv do not impose the encoder - side reconstruction constraint  .
informally , our problem thus reduces to the wyner - ziv problem if we set @xmath41 to infinity .
wyner and ziv s result can be summarized as follows :    [ thm : wz ] the rate - distortion function @xmath78 in the wyner - ziv setup is given by @xmath79 where @xmath80 , and where the minimization is over all functions @xmath81 and discrete random variable @xmath82 for which : @xmath82 takes values in an auxiliary alphabet @xmath83 of size at most @xmath84 ; @xmath85 forms a markov chain ; and    rcl & & .    since imposing
the encoder - side reconstruction constraint   can not increase the set of achievable rates , @xmath86 equality holds whenever the encoder - side reconstruction constraint   does not pinch . for example , when @xmath87 ; @xmath88 ; and @xmath89 indeed , in this case the encoder can set @xmath90 to be @xmath91 .
this results in   being identical to and thus superfluous .
steinberg s setup in @xcite is obtained from ours by replacing the encoder - side distortion constraint   by the more stringent perfect - reconstruction constraint @xmath92    [ thm : steinberg ] the rate - distortion function @xmath93 in steinberg s setup is given by @xmath94 where the minimization is over all @xmath95 taking value in @xmath44 and satisfying @xmath96 and @xmath97    [ rem : steinberg ] constraint   is equivalent to the block - distortion constraint @xmath98 thus , when in our setup @xmath99 is the hamming distortion and @xmath100 , then steinberg s setup differs from ours only in that is a block - distortion constraint whereas is an average - per - symbol distortion constraint .      to describe the rate - distortions function for the setup of section  [ subsec : setup ] , we introduce the function @xmath101 .
the expression for @xmath101 in is similar to the expression for @xmath78 in   except that in the expression for @xmath101 we have the additional constraint ; see   ahead .    given the joint law @xmath35 of the source and side information , and given the distortion functions
@xmath102 , this function is defined as @xmath103 where the minimization is over all discrete random variables  @xmath82 taking value in some finite auxiliary alphabet @xmath83 and forming the markov chain @xmath104 and over the functions @xmath105 and @xmath106 satisfying    rcl & & [ eq : distdr0 ] + & & . [ eq : dister0 ]    note that , thanks to assumption  [ assumption ] , the feasible set in   is not empty : we can choose @xmath82 as @xmath107 and @xmath108 , @xmath109 as the functions whose existence is guaranteed by the assumption .
this choice demonstrates that @xmath110    using the convex cover method @xcite it can be shown that :    allowing for sets @xmath83 of cardinality greater than @xmath111 does not decrease the value of the optimization problem .
a consequence of this remark is that the minimum in is achieved : indeed , we may choose @xmath112 as the set latexmath:[$\{1 , \ldots ,    functions @xmath108 , @xmath109 , and the problem is reduced to minimizing a continuous function over a compact set .
the key properties of @xmath101 are summarized in the following proposition :    [ prop : key_properties ] the function @xmath114 is bounded from above by @xmath115 and is nondecreasing in the distortions @xmath116 moreover , it is convex and continuous .
see appendix  [ app : key_properties ] .
our main result can be now stated as :    [ thm : finite ] the rate - distortions function for the setup in section  [ subsec : setup ] is equal to @xmath101 @xmath117    the coding scheme that establishes achievability is a variation on the coding scheme of wyner and ziv  @xcite and is thus only sketched .
its analysis is omitted .
fix @xmath118 satisfying   and , and fix also a blocklength  @xmath8 and some ( small ) @xmath56 .
let @xmath119 be a random blocklength-@xmath8 codebook with @xmath120 bins , each containing approximately @xmath121 codewords with the total number of codewords thus being @xmath122 .
generate the codewords independently with the components of each codeword being drawn iid @xmath123 .
number the bins @xmath124 through @xmath125 .    upon observing the source sequence @xmath0 ,
the encoder seeks a codeword @xmath126 in @xmath127 that is jointly typical with @xmath0 .
if successful , it sends the number of the bin containing @xmath126 as the message  @xmath2 .
it also produces the reconstruction sequence  @xmath128 by applying the function @xmath129 componentwise to @xmath126 and @xmath0 .
the decoder seeks a codeword @xmath130 in bin  @xmath2 that is jointly typical with its side - information @xmath1 and applies the reconstruction function @xmath131 componentwise to @xmath130 and @xmath1 to produce @xmath132 .
the converse is proved in subsection  [ sec : th1 ] .
though not identical , steinberg s setup is very similar to our setup when @xmath133 is the hamming distortion and @xmath134 is zero ( remark  [ rem : steinberg ] ) .
it is therefore not surprising that , as the following corollary shows , the two setups lead to identical rates :    [ cor : steinberg ] let @xmath135 be arbitrary , and let @xmath99 be the hamming distortion function @xmath136 then @xmath137    see appendix  [ app : rdcr ] .
our results can be extended to a scenario where the encoder observes not only the source sequence @xmath138 but also some sequence @xmath139 which is correlated with the decoder s side - information sequence @xmath140 .
this additional sequence @xmath139 makes it easier for the encoder to estimate the decoder s reconstruction sequence and thus allows the decoder to rely more heavily on its side information @xmath140 . to see how this seemingly more general scenario reduces to our scenario assume that @xmath141 are iid random triples of law @xmath142 and that @xmath143 takes value in the finite set @xmath144
consider now a new iid source @xmath145 taking value in the set @xmath146 according to the law @xmath147 with @xmath148 .
the encoder now observes the source sequence @xmath145 only and no additional sequences .
the decoder side information is still @xmath140 , and the joint law of @xmath149 is @xmath142 . finally define the new decoder distortion function @xmath150 as @xmath151 i.e. , the distortion function @xmath152 does not depend on the @xmath153-component .
solving the original scenario for this new source and new decoder distortion function is equivalent to solving the seemingly more general problem we described .      to establish the converse
, we show that if a triple @xmath154 is achievable , then for every @xmath155 @xmath156 since @xmath101 is continuous ( proposition  [ prop : key_properties ] ) , and since @xmath157 can be arbitrarily small , this implies that @xmath158 whenever @xmath55 is achievable , and consequently that @xmath159 .
the first part of our proof identifying the auxiliary random variable @xmath160 and the function @xmath161 is similar to the proof of the wyner - ziv result @xcite . for a given blocklength-@xmath8 code @xmath162 , @xmath163 , @xmath164 satisfying  , we have    rcl + & & h ( ) [ eq:25 ] + & & i(x^n;|y^n ) + & & _ i = 1^n i(x_i;|y^n , x^i-1 ) + & = & _ i = 1^n h(x_i|y^n , x^i-1 ) - h(x_i|,y^n , x^i-1 ) + & & _ i = 1^n h(x_i|y_i ) - h(x_i|,y^n , x^i-1 ) + & & _ i = 1^n h(x_i|y_i ) - h(x_i|,y^n ) +
& & _ i = 1^n h(x_i|y_i ) - h(x_i|z_i , y_i ) + & = & _ i = 1^n i(x_i;z_i|y_i ) + & & _
i = 1^n h(z_i|y_i ) - h(z_i|x_i ) + & = & _ i = 1^n
i(x_i;z_i ) - i(y_i;z_i ) , [ equ : cv - part1-last ]    where follows by ; follows because conditioning can not increase entropy and because @xmath165 ; follows from the chain rule for mutual information ; follows because the pair @xmath166 is independent of the tuple @xmath167 ; follows from the fact that conditioning can not increase entropy ; follows by defining @xmath168 and follows because with the definition above @xmath169    denote by @xmath170 the function that maps @xmath171 to the @xmath172-th component of the @xmath8-tuple @xmath173 , and denote by  @xmath174 the function that maps @xmath0 to the @xmath172-th component of the @xmath8-tuple @xmath175 .
since there is a one - to - one correspondence between the pairs @xmath176 and @xmath177 , we can define a function @xmath178 that maps @xmath176 to @xmath179 @xmath180 we now define @xmath181 where @xmath182 $ ] is with respect to @xmath183 . by definitions and , @xmath184 where @xmath182 $ ] is with respect to @xmath185 .
we next turn to the encoder - side distortion .
we will show that there exists a deterministic function @xmath186 that achieves a distortion no larger than @xmath187 , where @xmath187 is the distortion achieved by @xmath188 , namely , @xmath189    to this end , we express @xmath187 as    rcl + & = & + & = & _ x^n , z_i + & = & _ x^n , z_i , [ equ : cv - exp - de - xnminusi ]    where @xmath190 . for every @xmath191
, we define @xmath192 ( or for short @xmath193 ) as : is not unique , @xmath194 is defined as the first in lexicographical order . ]
@xmath195{\de \bigl (      \deci(y_i , z_i),\ercni(x_i,{{x}_{\!\backslash \!i } } ) \bigr ) } \end{gathered}\ ] ]    or in any other way that guarantees @xmath196 { \de\bigl(\deci(y_i , z_i),\ercni(x_i,{{x}_{\ !
\backslash\!i}})\bigr ) } \geq \\
\bige[y_i|x_i = x_i,{{x}_{\ ! \backslash\!i}}={{x}_{\!\backslash \!i}^*},z_i = z_i]{\de\bigl(\deci(y_i , z_i),\ercni(x_i,{{x}_{\!\backslash \!i}^*})\bigr)}.\end{gathered}\ ] ] we can now define the function @xmath197 as    [ block : amos_gym ]    rcl & & + ( x_i , z_i ) & & ( x_i , x_\i^*(x_i , z_i ) ) .    for every @xmath198
, we have    rcl + & & + & & + & & , [ equ : cv - cond - exp - de - erci ]    where follows from the definition of @xmath193 ; follows because @xmath199 and follows from the definition of  @xmath197 .
it now follows from   and   that @xmath200{\de\bigl(\deci(y_i , z_i),\erci(x_i , z_i)\bigr ) } \leq \dei.\ ] ]    continuing from  ( [ equ : cv - part1-last ] ) we thus obtain    rcl n(r+ ) & & _ i = 1^n i(x_i;z_i ) - i(y_i;z_i ) + & & _ i = 1^n ( , d_,i ) [ equ : cv - part2-first ] + & & n _ i = 1^n ( , d_,i ) + & & n ( _ i = 1^n , _
i = 1^n d_,i ) + & & n [ eq : amos300 ]    where follows from the definition of @xmath101 and from , , and ; follows by multiplying by @xmath124 ; follows from the convexity of @xmath101 ( proposition  [ prop : key_properties ] ) ; and follows from the monotonicity of @xmath101 ( proposition  [ prop : key_properties ] ) and the fact that @xmath201 and @xmath202 .
this establishes   and thus concludes the proof of the converse .
we next consider the case where the source , side information , and reconstruction alphabets @xmath34 are the reals @xmath203 ; the distortion functions @xmath60 and @xmath61 are quadratic    rcl d_(x , _ ) & = & ( x-_)^2,[eq : dd ] + d _ ( _ , _ ) & = & ( _ -_)^2;[eq : de ]    and the source and side - information pair @xmath204 is a centered bivariate gaussian , where @xmath107 is of variance @xmath205 @xmath206 and @xmath207 for some centered gaussian @xmath208 that is independent of  @xmath107 and that is of variance @xmath209 and where @xmath210 is a nonzero constant .
is zero , because in this case the side information is independent of the source and is thus irrelevant . ]
the rate - distortions function depends on @xmath210 only through the ratio @xmath211 , because the receiver can premultiply its side information by @xmath212 without affecting the rate - distortions function . in the following
we thus assume that @xmath213 , i.e. , @xmath214 we denote the rate - distortions function for this setup by @xmath215 .
when @xmath216 is zero the problem is not interesting , because in this case the source sequence is determined by the side information , and @xmath215 is thus zero for all nonnegative values of @xmath217 and @xmath218 .
we shall henceforth thus assume @xmath219 in this case , no finite rate can allow @xmath217 to be zero ( even if we ignore the encoder - side reconstruction constraint ) .
thus , we shall also assume @xmath220      as we have seen in section  [ sec : related ] , the wyner - ziv setup is obtained from ours if the encoder - side reconstruction constraint   is omitted , and steinberg s common reconstruction setup is obtained if   is replaced by  .
for a gaussian source and quadratic distortion measures , steinberg s common reconstruction rate - distortion function is @xcite @xmath221    the wyner - ziv rate - distortion function is @xcite @xmath222 this is the rate - distortion function even if the side information is revealed not only to the decoder but also to the encoder .
[ thm : gaussian ] for a gaussian source and quadratic distortion measures , the rate - distortions function @xmath215 can be expressed as follows :    if @xmath223 , then @xmath224    if @xmath225 , then    lcl [ eq : ropga2 ] r^g(d _ , d _ ) & = & ^+ ( ) .
the direct part is proved in section  [ sec : achgauss ] and the converse in section  [ sec : convgauss ] .    if @xmath100 , then our rate - distortions function @xmath226 coincides with steinberg s common - reconstruction rate - distortion function @xmath227 of  : @xmath228    [ rem : wzg ] if @xmath229 and @xmath134 are such that @xmath230 or @xmath231 then @xmath232 coincides with wyner and ziv s rate - distortion function @xmath233 in  .
thus , if or holds , then relaxing constraint   and/or revealing the side information also to the encoder does not decrease the rate - distortions function .      in the two cases that we shall describe in   and   ahead ,
no encoding is necessary because the encoder and the decoder can produce sufficiently good reconstructions @xmath128 and @xmath132 based solely on their observed sequences @xmath0 and @xmath1 . in these cases @xmath232 is thus zero .    1 .
if + [ eq : d3 ] @xmath234 and @xmath235 + then the encoder and decoder can produce the sequences + rcl _ ^n & = & x^n + _ ^n & = & y^n + which satisfy the distortion constraints .
2 .   if + [ eq : d4 ] @xmath236 and @xmath237 + then the encoder and decoder can produce the sequences + rcl[eq : xe1 ] _ ^n & = & x^n + _ ^n & = & y^n[eq : xd1 ] + which satisfy the distortion constraints .
the achievability of theorem  [ thm : gaussian ] in the remaining cases will be established using the following proposition with a judicious choice of the parameters .
[ prop : gach ] for the setup in section  [ sec : gausssetup ] of a gaussian source and quadratic distortion measures , the tuple @xmath238 is achievable whenever    rcl r [ equ : gaussian - ac - diff - i - log ]    for some parameters @xmath239 and @xmath240 satisfying    [ block : amos_lunch ] @xmath241 and @xmath242    thus , @xmath243 where the minimization is over all @xmath239 and @xmath240 satisfying  .
see appendix  [ sec : gscheme ] .
we can now prove the achievability part of theorem  [ thm : gaussian ] for the remaining cases .    1 .
if + [ eq : d1 ] @xmath234 and @xmath244 + then the choice + [ eq : c1 ] @xmath245 ( which is positive by  ) and @xmath246 + satisfies because @xmath247 and @xmath248 moreover , for this choice , + rcl[eq : rates2 ] + & = & .
+ thus , by   and by proposition  [ prop : gach ] , we conclude that when @xmath217 and @xmath218 satisfy  , @xmath249 2 .   if + [ eq : d2 ] @xmath250 and @xmath251 + then we consider the choice + [ eq : c2 ] @xmath252 + to see that the rhs of is positive note that   implies that the denominator is positive , and   implies that the numerator is positive because @xmath253 ( since @xmath254 is smaller than one , the lhs of implies that @xmath255 .
this , and the fact that the lhs of   also implies that @xmath256 demonstrates that the lhs of   also implies that @xmath257 . )
+ this choice satisfies   because @xmath258 and @xmath259 moreover , for this choice , + rcl [ eq : requal ] + & = & .
+ thus , by   and by proposition  [ prop : gach ] , we conclude that when   holds , @xmath260    the expressions in proposition  [ prop : gach ] and their relation to   become more transparent when we define    [ eq : choice ]    rcl z&=&a(x+w ) + _ & = & by+z + _ & = & bx+z    for @xmath261 , @xmath262 , and @xmath263 a centered gaussian of positive variance @xmath264 independent of the pair @xmath204 .
with these definitions    [ eq : equivgpar ]    rcl i(x;z|y ) & = & + & = & ( 1-a - b)^2 _ x^2 + a^2 _
w^2 + b^2 _
u^2 + & = & b^2 _ u^2 .    since @xmath265 for all choices of the parameters @xmath261 , @xmath262 , @xmath266
, we can also rewrite   as : @xmath267 where the minimum is over all @xmath268 that are of the form in   and satisfy the distortion constraints    rcl & & , + & & .
if @xmath269 then the converse follows by relaxing the constraint ; see remark  [ rem : wzg ] .
we thus focus on the case where @xmath270    we define the function @xmath271 like @xmath272 except that its first argument ( @xmath217 ) is strictly positive ; the minimum is replaced by an infimum ; and the size of the auxiliary alphabet  @xmath83 can be unbounded .
thus , @xmath273 where the infimum is over all choices may take value , and we must restrict the functions @xmath131 and @xmath129 to be measurable . in the converse  @xmath82 will correspond to the tuple @xmath274 , and we can therefore restrict @xmath82 here to be the space where such tuples take value . ] of the random variable  @xmath82 and functions @xmath275 satisfying    [ eq : amos600 ]    rcl & & , [ equ : gaussian - cv - max - h - x - dec - constraint ] + & & , [ equ : gaussian - cv - max - h - x - enc - constraint ] + z & x & y , [ eq : markov ]    where    rcl _ & & ( y , z),[eq : phi ] + _ & & ( x , z).[eq : psi ]    in analogy to proposition  [ prop : key_properties ] we have :    [ lem : gaus_mon_convex ] over @xmath276 the function @xmath277 is finite ; monotonic in each of its arguments ; and convex .
the function is bounded by the rate - distortion function of the gaussian source without side information .
the proof of monotonicity is identical to the proof of monotonicity in proposition  [ prop : key_properties ] .
the proof of convexity is also very similar ; only a minor change is needed to account for the fact that , _ prima facie _ , the infimum need not be achieved .
the following lemma provides an explicit expression for @xmath277 when   holds .
[ lemma : amos_aldi100 ] if @xmath278 and @xmath279 satisfy  , then @xmath280    we first prove @xmath281 to this end , we present a choice for @xmath82 , @xmath282 , @xmath283 that satisfies the constraints   and is such that the objective function @xmath284 in   evaluates to the rhs of  .
our choice depends on whether @xmath285 or @xmath286 in the first case   the rhs of   evaluates to 0 , whereas in the second case   it is positive .
when @xmath217 and @xmath218 satisfy  , a suitable choice is  as in and in the proof of the direct part @xmath287    when @xmath217 and @xmath218 satisfy  , a suitable choice is  as in   and in the direct part    rcl z&=&a(x+w),_=bx+z , _
= by+z ,    where @xmath263 is a centered gaussian of variance @xmath288 and independent of the pair @xmath204 and where @xmath289 and @xmath290 . that this choice has the desired properties follows by   and  .
having established  , we now complete the proof of the lemma by proving the reverse inequality @xmath291 since rates are nonnegative , it suffices to prove @xmath292 where @xmath293 has been replaced by @xmath294 .
since the joint law of @xmath204 is fixed and is a bivariate gaussian law    rcl i(x;z|y ) & = & h(x|y ) - h(x|y , z ) + & = & ( 2 e ) - h(x|y , z ) .
[ equ : gaussian - cv - i - as - diff - log ]    consequently , is equivalent to @xmath295 where @xmath296 is defined as @xmath297 under the same constraints   that define @xmath277 in  .    to prove   we first note that , since @xmath282 is a deterministic function of @xmath298 ,    rcl h(x|y , z ) & = & h(x - _ |y , z , _ ) + & = & h(x - _ |x - _ + u , z , _ ) + & & h(x - _ |x - _ + u ) [ equ : gaussian - cv - h - x - upper - bounded - by - h - xxd - cond ] [ equ : gaussian - cv - h - x - upper - bounded - by - log - var ]    where in the second line we recalled that @xmath299  , and where the last line follows because conditioning can not increase differential entropy .    the markov condition @xmath300 and the fact that @xmath299   imply that @xmath301 this , combined with the assumption that @xmath208 is independent of  @xmath107 , implies that @xmath208 is independent of @xmath302 . and since @xmath283 is a function of @xmath302 , @xmath303 this independence implies that @xmath208 is independent of @xmath304 .
this latter independence and the fact that @xmath305 can be expressed as @xmath306 implies that @xmath307 from , , the fact that the variance of a random variable can not exceed its second moment , and the fact that the magnitude of a correlation coefficient can not exceed @xmath124 , it follows that @xmath308 where @xmath309 is defined as @xmath310 subject to the relaxed constraints    [ eq : amos650 ] @xmath311    we now proceed to study @xmath309 .
define @xmath312 so @xmath313 subject to    [ eq : aamos650 ] @xmath314    by the conditional max - entropy theorem  @xcite , the supremum in is achieved when @xmath315 are jointly gaussian , as we henceforth assume . as we next argue , the lemma s hypothesis that   holds implies that the choice of @xmath13 as @xmath316 is not in the feasible set .
indeed , with this choice @xmath317 is equal to  @xmath318 , which violates because   and   imply @xmath319 we thus assume in the following that @xmath13 is jointly gaussian with @xmath208 and that @xmath320 . consequently ,    rcl + & = & ( 2 e ( _ u^2 - ) ) [ eq : diffentropy ] + & = & ( 2 e )    where @xmath321 and @xmath322 .
we can thus rewrite the optimization problem in as @xmath323 subject to @xmath324 ( we have to add the last constraint because the magnitude of a correlation coefficient can not exceed one . ) for fixed @xmath325 , the objective function in is monotonically increasing in @xmath326 ( see also  ) , and so is the rhs of constraint  .
therefore , it is optimal to choose in   @xmath327 substituting this choice in and yields @xmath328 subject to and @xmath329 notice that , whenever   holds , the rhs of   is upper - bounded by the square of @xmath330 .
consequently , @xmath331 since the rhs of   implies , @xmath332 and constraint   is redundant .
we therefore ignore constraint   and study the maximization in   subject to only .    to this end , we compute the derivative of the objective function in with respect to @xmath325 :    rcl + & = & .[eq : dev ]    by  , the derivative in is negative for all feasible @xmath325 .
hence , the objective function in is decreasing on the ( symmetric ) interval of interest  , and it is optimal to choose @xmath333 the optimality of this choice allows us to evaluate @xmath309 via and hence to upper - bound @xmath296 via  .
this yields the desired bound  , which establishes the lemma .    using lemma  [ lem : gaus_mon_convex ] and lemma  [ lemma : amos_aldi100 ] we can follow the steps of the proof in section  [ sec : th1 ] of the converse part of theorem  [ thm : finite ] .
the remaining technicality is continuity .
continuity in the interior , i.e. , on @xmath334 follows from convexity .
it thus only remains to establish continuity when @xmath335 , holds , and @xmath134 is zero .
this can be done by inspecting  .
so far we have only studied settings with two distortion functions , one of which  the decoder - side distortion function @xmath336depends on the source symbol and the decoder s reconstruction , and the other  the encoder - side distortion function @xmath337depends on the decoder s and the encoder s reconstruction symbols . in this section
we extend our setting to allow for more than two distortion functions and to allow for distortions that depend on all three symbols : the source symbol @xmath338 , the decoder s reconstruction symbol @xmath339 , and the encoder s reconstruction symbol @xmath340
. we shall also allow the reconstruction alphabets to differ .
but all alphabets are assumed finite .
the new setup differs from the setup in section  [ sec : setup ] in two ways .    * the encoder - side reconstruction @xmath128 and the decoder - side reconstruction @xmath132 take value in the finite alphabets @xmath341 and @xmath342 which can be different .
* there are @xmath343 ( possibly larger than @xmath344 ) distortion constraints specified by the @xmath343 distortion functions @xmath345 and the corresponding @xmath343 maximal - allowed distortions @xmath346 ( all of which are assumed to be nonnegative ) .
we say that the tuple @xmath347 is _ achievable _ if for every @xmath56 and sufficiently large @xmath8 there exist a message set @xmath348 of size @xmath349 and functions    [ eq : funk ]    rcl f^(n ) & & ^n + ^(n ) & & ^n _
^n + ^(n ) & & ^n ^n _    such that the message @xmath350 and the reconstruction sequences @xmath351 and @xmath352 satisfy :    rcl [ eq : distck ] _ i=1^n d_k+ , k\{1 ,  , k}. +    in analogy to assumption  [ assumption ]
, we shall assume :    [ ass : extended ] to each @xmath353 corresponds some @xmath354 and some @xmath355 satisfying @xmath356    we seek the smallest rate @xmath68 for which the tuple @xmath357 is achievable .
this is defined as follows .
given a maximal - allowed - distortion tuple @xmath358 , let @xmath359 assumption  [ ass : extended ] implies that the set @xmath360 contains all rates exceeding @xmath115 and is thus nonempty .
the rate - distortions function @xmath361 can now be defined as @xmath362 where the minimum exists because the region @xmath363 is nonempty , closed , and bounded from below by 0 .      to describe the rate - distortions function for the extended setup of section  [ sec : problem_extended ] ,
we next introduce the function @xmath364 .    given the joint law @xmath35 of the source and side information , and given the distortion functions @xmath365 , this function is defined as @xmath366 where the minimization is over all discrete auxiliary random variables @xmath82 and @xmath208 satisfying @xmath367 and over all functions @xmath368 and @xmath369 that simultaneously satisfy the @xmath343 distortion constraints    rcl d_k , k\{1 ,  , k}. + [ eq : disterk ]    the following proposition provides cardinality bounds on the support sets of the auxiliary random variables .    [ prop : amoscardbound ] the minimum defining @xmath364 is not increased if we restrict the cardinality of the support set  @xmath83 of @xmath82 to @xmath370 and the cardinality of the support set @xmath371 of @xmath208 to @xmath372    the cardinality bound on @xmath83 can be justified using the convex cover method @xcite .
the cardinality bound on @xmath371 is proved in appendix  [ app : amos ] .
the cardinality bound on @xmath371 can be strengthened : @xmath373 need not exceed the number of distortion constraints in   that depend on @xmath90 .
the latter number equals @xmath124 in the original setup of section  [ sec : setup ] thus allowing us to recover theorem  [ thm : finite ] .
[ prop : key_propertiesk ] the function @xmath374 is bounded from above by @xmath115 ; it is nondecreasing in the distortions @xmath375 and it is convex and continuous .    the proof is similar to the proof of proposition  [ prop : key_properties ] in appendix  [ app : key_properties ] and is omitted .
[ thm : extension ] the rate - distortions function for the setup in section  [ sec : problem_extended ] is equal to @xmath376 : @xmath377    the achievability , i.e. , that @xmath378 can be proved using a scheme that is similar to the one that was sketched in the proof of theorem  [ thm : finite ] .
the only difference is that , to produce the reconstruction sequence @xmath128 , the encoder applies the function  @xmath129 component - wise to the tuple @xmath379 , where , conditional on @xmath380 , the components of the sequence @xmath381 are generated independently according to the conditional law @xmath382 .
the analysis of this scheme is omitted .
we next prove the converse , i.e. , that @xmath383 fix some positive @xmath157 , a blocklength @xmath8 , and a rate @xmath68 .
let @xmath348 be a message set of size @xmath384 , and let @xmath162 , @xmath385 , and @xmath386 be encoding and reconstruction functions as in that satisfy the @xmath343 distortion constraints in . for every @xmath387 , define @xmath388 in @xmath389 and define @xmath390 as @xmath391 notice that for every @xmath387 @xmath392 also , following the steps in   , we can conclude that @xmath393    we further define  as in section  [ sec : th1]@xmath170 to be the function that maps @xmath171 to the @xmath172-th symbol of @xmath173 and @xmath174 to be the function that maps @xmath0 to the @xmath172-th symbol of @xmath175 .
then , the symbol @xmath394 can be written as @xmath395 and @xmath396 can be written as @xmath397 for some functions @xmath161 and @xmath398 with arguments in the respective domains .
we finally define for each @xmath399 and @xmath387 @xmath400 where @xmath182 $ ] is with respect to @xmath183 .
notice that @xmath401 because the chosen encoding and reconstruction functions @xmath162 , @xmath385 , and @xmath386 satisfy  .
moreover , by definitions  , @xmath402 where @xmath182 $ ] is with respect to @xmath403 .
combining   and with the definition of @xmath404 , we obtain    rcl n(r+ ) & & _ i = 1^n i(x_i;z_i ) - i(y_i;z_i ) + & & _ i = 1^n _ ext(d_1,i ,  , d_k , i ) [ eq : rext1 ] + & & n _ ext ( _ i = 1^n d_1,i ,  , _ i = 1^n d_k , i ) + & & n _ ext ( d_1 + ,
, d_k+),[eq : fina ]    where the last two inequalities follow by the convexity and the monotonicity of @xmath404 and by  . by the continuity of @xmath404 and
because @xmath56 and the blocklength @xmath8 are arbitrary , the converse   follows immediately from  .
we acknowledge helpful discussions with prof .  g.  kramer .
when @xmath99 is the hamming distortion and @xmath100 , our average - per - symbol distortion constraint is less stringent than the block - distortion constraint   in steinberg s setup ( remark  [ rem : steinberg ] ) . consequently , @xmath405 it remains to prove the reverse inequality .
let @xmath82 , @xmath131 , and @xmath129 be minimizers of @xmath406 , so    @xmath407    @xmath408   \leq   \dd\ ] ]    @xmath409    @xmath410    to prove the reverse inequality we shall upper - bound @xmath93 by showing that @xmath411 is feasible in the minimization that defines it . from the definition of @xmath95 and from  , it follows that @xmath95 is computable ( almost surely ) from @xmath302 .
this combines with   to establish that @xmath412 and , _ a fortiori _ , that    [ block : amos_ayef30 ] @xmath413 and by   and  , @xmath414   \leq   \dd.\ ] ]    it follows from   that @xmath95 is feasible in the minimization defining @xmath93 and thus    rcl r_cr(d _ ) & & i(x;)- i(y ; ) + & = & i(x;|y ) [ eq : amos_ayef30b ] + & & i(x;z|y ) [ eq : amos_ayef30c ] + & = & i(x;y)-i(x;z ) [ eq : amos_ayef30d ] + & = & r(d_,0 ) [ eq : amos_ayef30e ]    where   follows from  ; where   follows , by the ( conditional ) data processing inequality , from @xmath415 ( which holds by  ) ; where   follows from  ; and   follows from
. inequalities   and   establish the corollary .
that @xmath101 is bounded by @xmath115 is just a restatement of  .
monotonicity holds because the feasible set in the minimization defining @xmath101 is enlarged ( or is unaltered ) when @xmath40 and/or @xmath41 are increased .    as to the convexity ,
let @xmath416 and @xmath417 be the random variables and functions that achieve the minima in the definitions of @xmath418 and @xmath419 .
let @xmath420 be independent of @xmath421 .
define @xmath422 and the functions @xmath423 @xmath424 then @xmath425 @xmath426}\quad   \\      &   =    & \lambda e[d_{\mathrm{d}}(x,\phi^{(1)}(y , z^{(1 ) } ) ) ] \nonumber \\      & &               + ( 1-\lambda ) e[d_{\mathrm{d}}(x,\phi^{(2)}(y , z^{(2 ) } ) ) ] \\      & \leq & \lambda d_{\mathrm{d}}^{(1 ) } + ( 1-\lambda ) d_{\mathrm{d}}^{(2 ) } ;               \label{equ : cx - dd - leq - lambda}\end{aligned}\ ] ] and @xmath427 } \quad \\      &   =    & \lambda               e[d_{\mathrm{e}}(\phi^{(1)}(y , z^{(1)}),\psi^{(1)}(x , z^{(1 ) } ) ) ]                \nonumber \\      &       & + ( 1-\lambda )               e[d_{\mathrm{e}}(\phi^{(2)}(y , z^{(2)}),\psi^{(2)}(x , z^{(2)}))]\ieeeeqnarraynumspace \\      & \leq & \lambda d_{\mathrm{e}}^{(1 ) } + ( 1-\lambda ) d_{\mathrm{e}}^{(2 ) } ;               \label{equ : cx - de - leq - lambda}\end{aligned}\ ] ] so @xmath428 are feasible for the distortions @xmath429    consequently ,    rcl + & & i(x;z ) - i(y;z ) + & = & h(x ) - h(x|z ) - h(y ) + h(y|z ) + & = & h(x ) - h(x|z^(q),q ) - h(y ) + h(y|z^(q),q ) + & = & h(x ) - h(x|z^(1 ) ) - ( 1- ) h(x|z^(2 ) ) + & & - h(y ) + h(y|z^(1 ) ) + ( 1- ) h(y|z^(2 ) ) + & = & ( i(x;z^(1 ) ) - i(y;z^(1 ) ) ) + & & + ( 1- ) ( i(x;z^(2 ) ) - i(y;z^(2 ) ) ) .
[ equ : cx - i - eq - lambda ] + & = & ( d_^(1),d_^(1 ) ) + ( 1- ) ( d_^(2),d_^(2 ) ) .    to conclude the proof it remains to prove that @xmath101 is continuous on @xmath430 .
( continuity on @xmath431 is a consequence of the convexity , but we also claim continuity in the closed set @xmath431 . ) since @xmath430 is locally simplicial ( as can be verified by the definition in ( * ? ? ?
* section 10 , p.  84 ) or using ( * ? ? ?
* theorem 20.5 , p  184 ) ) , the convexity of @xmath101 on @xmath430 implies its upper - semicontinuity relative to @xmath430 .
it thus remains to prove lower - semicontinuity relative to @xmath430 .
that is , we need to show that @xmath432 implies that there is a subsequence @xmath433 such that @xmath434 let @xmath435 , @xmath436 , @xmath437 achieve @xmath438 with @xmath439 . since there are only a finite number of functions from @xmath440 to @xmath44 and only a finite number of functions from @xmath441 to @xmath44 , we can choose a subsequence @xmath433 along which : the mappings @xmath442 do not depend on @xmath443 and can be thus denoted @xmath108 ; the mappings @xmath444 do not depend on  @xmath443 and can be thus denoted @xmath109 ; and the conditional laws @xmath445 converge to some conditional law that we denote @xmath446 . by the continuity of mutual information , @xmath447 converges to @xmath448 evaluated with respect to @xmath449 , and @xmath450 can not exceed this value because @xmath446 , @xmath109 , and @xmath108 are in the feasible set defining it .
we present and analyze a scheme that achieves the rate - distortions tuples in proposition  [ prop : gach ] . before describing the scheme , we introduce some notation and lemmas on @xmath8-dimensional spheres .
an _ @xmath8-sphere of radius @xmath451 centered at @xmath452 _ is the set of all vectors @xmath453 satisfying @xmath454 when the center of the sphere @xmath455 is the origin @xmath456 , we call it a _ centered _ sphere , and when the radius of the sphere is @xmath124 , we call it a _ unit _ sphere .
we denote the angle between two nonzero vectors @xmath457 by @xmath458 .
its cosine is @xmath459 given a nonzero vector @xmath460 on an @xmath8-sphere @xmath461 , the _ spherical cap of half - angle @xmath462 centered at @xmath460 _ is the set of all vectors @xmath463 on @xmath461 satisfying @xmath464 the surface area of such a spherical cap does not depend on the vector @xmath465 but only on the dimension @xmath8 , the radius of the sphere @xmath466 , and the angle @xmath462 .
if the radius @xmath467 , we denote this surface area by @xmath468 .
we say that a random @xmath8-vector is uniformly distributed over an @xmath8-sphere , if it is drawn according to a uniform probability measure over the surface of this sphere .
the proofs of the following four lemmas are based on results in @xcite and omitted .
[ lem : cnuniform ] let @xmath469 be uniformly distributed over the centered unit @xmath8-sphere , and let @xmath470 be a deterministic unit - length vector in @xmath471 .
then , @xmath472    [ lem : cnlim ] for @xmath473 : @xmath474    [ lem : ft ] let @xmath475 $ ] be such that the limit @xmath476 exists and @xmath477 .
then , @xmath478    [ lem : lim0 ] for @xmath479 @xmath480 whereas for @xmath481 @xmath482      our scheme has parameters @xmath483 that must satisfy conditions   and , which we repeat for convenience here : @xmath484 to describe and analyze the scheme we use vector notation .
let @xmath485 denote the @xmath8-dimensional column - vector that results when the source symbols are stacked on top of each other @xmath486 likewise define the side - information vector @xmath487 and the reconstruction vectors @xmath488 , and @xmath489 .
let    rcl _ z^2 & & a^2 ( _ w^2 + _ x^2 ) , + r & & ( ) , + r & & ( ) .    draw @xmath490 independent random @xmath8-vectors @xmath491 uniformly over the centered @xmath8-sphere of radius @xmath492 .
assign these vectors to @xmath493 bins : the first @xmath494 are assigned to bin  @xmath124 , the following @xmath494 vectors are assigned to bin  @xmath344 , etc .
more specifically , if @xmath495 denotes the set of vectors assigned to bin  @xmath496 , then @xmath497 for @xmath498 and @xmath499 the codebook @xmath500 .      given the source sequence @xmath501 , the encoder looks for the codeword @xmath502 that is closest to having the `` correct '' angle with @xmath503 : @xmath504 the encoder then sends @xmath505 , where @xmath506 denotes the index of the bin containing @xmath507 .
it also produces the reconstruction sequence @xmath508 .
given @xmath505 and the side - information vector @xmath509 , the decoder chooses @xmath510 and produces the reconstruction sequence @xmath511 .
with probability 1 the @xmath512s in and are unique .
we fix @xmath56 sufficiently small such that @xmath513 and define the following four events :    1 .
@xmath514 `` the source and side information are atypical '' , i.e. , + rcl & |^2 - _
x^2| > _ x^2 + & |^2 - _ y^2| > _ y^2 + & | ( , ) - _
xy| > _ xy[eq : rxy ] + where @xmath515 denotes the correlation coefficient between @xmath107 and @xmath516 : + rcl _ xy=. 2 .
@xmath517 `` no codeword has a good angle with the source sequence '' , i.e. , + rcl| ( , ) - | > .
@xmath518 `` the chosen codeword @xmath519 does not have the correct angle with the side - information sequence '' , i.e. , + rcl| ( , ) - | >
+ [ eq : dec1 ] 4 .
@xmath520 `` the decoder does not find the correct codeword '' , i.e. , @xmath521    also , we define the event @xmath522    [ lem : lime ] @xmath523    we note    rcl[eq : sume ] [ ] & & [ _ ] + [ _ | _ ^c ] + [ _ 1 | _
^c _ ^c ] + & & + [ _ 2 | _ ^c _ ^c ] .    in the following
we show that each term on the rhs of   tends to zero as the blocklength @xmath8 tends to infinity .
the first limit @xmath524 = 0\ ] ] follows directly from the weak law of large numbers . the second limit @xmath525=0\ ] ]
can be shown following the same steps as in the proof of limit  ( 134 ) in @xcite . the third limit @xmath526=0\ ] ] is proved as follows .
we have @xmath527 where @xmath528 and @xmath529 denote the components of @xmath530 and @xmath531 that are orthogonal to @xmath485 : @xmath532 and @xmath533 let @xmath534 satisfy @xmath535\ ] ] and let @xmath463 and @xmath536 be vectors in @xmath471 satisfying    rcl & |^2 - _ x^2| _ x^2 + & |^2 - _ y^2| _ y^2 _ y^2 + & | ( , ) - _ xy| _ xy .
[ eq : rhoxy ]    then , conditional on events @xmath537 by   and , we have    [ eq : product ]    rcl ( , ) ( , ^ * ) & & ( 1 + ) _ xy(1 + ) + & & ( 1 + 3 )    and    rcl ( , ) ( , ^ * ) & & ( 1- ) _
xy(1- ) + & & ( 1 - 3 ) , +    where inequalities  @xmath538 follow because @xmath539 and because @xmath540 .
moreover , conditional on the events in  , the vector @xmath529 is uniformly distributed over a centered @xmath541-dimensional sphere of radius @xmath542 , and thus limit   on top of the next page follows by lemmas  [ lem : cnuniform ] and [ lem : lim0 ] .
we can combine limit   and inequalities   to obtain the limit   on top of the next page .
@xmath543      = 1\ ] ]    @xmath544      = 1\ ] ]    ' '' ''    if in   we take the expectation with respect to @xmath545 , and @xmath546 ( but keep the conditioning on events @xmath547 and @xmath548 ) , we obtain the desired third limit  .
we finally prove the fourth limit @xmath549=0.\ ] ] to this end , we define event @xmath550 as @xmath551 recalling the decoding rule in   and the definition of event @xmath552 in  , we see that when @xmath553 and @xmath550 occur simultaneously , then by condition   the decoder finds the correct codeword @xmath554
. therefore ,    rcl[eq : sumu ] & & 1 - ,    and thus   and the limit @xmath555 establish  .
we now prove  . for each @xmath556 , we index the vectors in the @xmath557-th bin from @xmath124 to @xmath558 and we shall refer to the @xmath559-th vector in this @xmath557-th bin by @xmath560 .
let @xmath561 be the index of @xmath519 , i.e. , @xmath562 . by the symmetry of the code construction and the encoding rule ,
the probability @xmath563 does not depend on the values @xmath557 and @xmath559 .
we therefore , assume in the following that @xmath564 and @xmath565 .
if we additionally condition on @xmath566 and on @xmath567 , the vectors @xmath568 ( i.e. , the vectors in bin  @xmath124 that are not @xmath519 ) are independent and uniformly distributed over the centered @xmath8-sphere of radius @xmath569 without the spherical cap of half - angle @xmath570 centered at @xmath463 .
thus , @xmath571 is an upper bound on the conditional density of the normalized vectors @xmath572 on the centered unit @xmath8-sphere .
applying lemma  [ lem : cnuniform ] , we therefore obtain inequality   shown on top of the next page .
rcl[eq : in1 ] + & = & 1 - _ k=2^|(1)| ( 1 - ) + & & < 1 - ( 1 - ) ^|(1)|-1 [ equ : gaussian - ac - er - other - closer - cond - pr - others - close ] + & & 1 - ( 1- ) ^2^n ( r - r- ) [ equ : gaussian - ac - er - other - closer - pr - others - close - bound ]    ' '' ''    we note that for any @xmath573 $ ] @xmath574 and hence the mapping @xmath575 is decreasing in @xmath576 . therefore , since @xmath577 we further obtain  .
if now we take the expectation with respect to @xmath578 , @xmath2 , and @xmath561 ( but keep the conditioning on @xmath579 and @xmath580 ) , results in @xmath581 \nonumber\\      &   <      1 -      \left(1-\frac{2 \ , c_n\left(\arccos\sqrt{1 - 2^{-2 ( r ' - r-\frac{\delta}{2})}}\right)}{c_n(\pi ) } \right)^{2^{n ( r ' - r-\delta)}}\!\!.         \label{equ : gaussian - ac - er - other - closer - pr - others - close - bound2 }    \end{aligned}\ ] ] the desired limit  ( [ equ : gaussian - ac - er - other - closer - pr - others - close ] ) follows by and by lemma  [ lem : ft ] .
in fact , applying lemma  [ lem : ft ] to @xmath582 and to the function @xmath583 we obtain that the right - hand side of   tends to 1 as @xmath8 tends to infinity because    rcl[equ : gaussian - ac - er - other - closer - lim - log - cn ] _
1&&- _ n ( ) + & = & r - r-/2 + & > & _ 2 .    here , the equality holds by lemma  [ lem : cnlim ] and because the factor 2 in the logarithm does not change the limit , and the inequality holds by   and because @xmath584 .
this concludes the proof of limit  ( [ equ : gaussian - ac - er - other - closer - pr - others - close ] ) and thus of the fourth limit  .
combining finally   with  and establishes the proof of the lemma .
we can now bound the expected distortions of our scheme .
we have    rcl[equ : gaussian - ac - exp - dd ] & = & + & & + ,    and    rcl[equ : gaussian - ac - exp - de ] & = & [ ^c ] + & & + [ ] .
the decoder - side distortion satisfies    rcl d_^(n ) ( , _ ) & = & - ^ * - b ^2 + & & ^2 + ^ * ^2 + ^2,[eq : ddrough ]    where the inequality holds by the cauchy - schwarz inequality and because an arithmetic mean of two nonnegative numbers can not be smaller than it s geometric mean .
therefore ,    rcl + & & + & = & + & & - + & & 3(_x^2+_z^2 + b^2 ( _ x^2+_u^2 ) ) + & & - 3 ( _ x^2(1- ) + _ z^2 + b^2 ( _ x^2+_u^2)(1- ) ) + & & 3(_x^2+_z^2 + b^2 ( _ x^2+_u^2 ) ) ( 1- ( 1- ) ) .[eq : dde ]    in the event @xmath585 , we can derive a bound on the decoder - side distortion @xmath586 that is tighter than  : @xmath587 where the first inequality follows from the definition of the event @xmath585 , the second by throwing away some negative @xmath157-terms , and the third from condition   and because @xmath588 . since @xmath589 \leq 1 $ ] , we thus have :    rcl[eq : ddec ] + & & d_d + ( _ x^2 + b^2 ( _ x^2 + _
u^2 ) + 4 a _ x^2 + 8 b _
x^2 + 18 ab _
x^2 ) . +    combining , , and , we obtain    rcl + & & d_+ 3(_x^2 + _ z^2 + b^2_y^2)(1- ( 1 + ) ) + & & + ( _ x^2 + b^2 ( _ x^2 + _
u^2 ) + 4 a _ x^2 + 8 b
_ x^2 + 18 ab _
+ [ eq : concdd ]    similarly , we have for the encoder - side distortion :    rcl d_^(n ) ( , _ ) & = & -b ^2 + & & ^2 + b^2 ^2 , [ eq : derough ]    and thus ,    rcl + & & + & & - + & & 2(b^2 ( _ x^2+_u^2)+ b^2 _ x^2 ) ( 1- ( 1- ) ) .[eq : dee ]    moreover , in the event @xmath585 we can derive a bound on the encoder - side distortion @xmath590 that is tighter than  : @xmath591 where the last inequality follows by assumption   and because @xmath588 . since @xmath592\leq 1 $ ] , we thus have    rcl[eq : deec ] [ ^c ] & & d _ + b^2 ( 9_x^2 + _
u^2 ) .    combining finally  , , and , we obtain    rcl + & & d _
+ 2(b^2 _ y^2 + b^2 _
x^2 ) ( 1- ( 1- ) ) + & & + b^2 ( 9_x^2 + _
[ eq : concde ]    recall that the rate of our scheme is smaller than @xmath593 and that @xmath594 can be chosen arbitrarily close to 0 . therefore ,
from , , and lemma  [ lem : lime ] we conclude that when @xmath595 and @xmath596 satisfy   and , then our scheme can achieve the triple @xmath597 this establishes proposition  [ prop : gach ] .
to prove the cardinality bound   on @xmath371 , we shall need the following variation on carathodory s theorem .
let @xmath599 be a compact subset of @xmath24 , and let @xmath463 be a boundary point of its convex hull @xmath600 .
since @xmath463 is in the convex hull of @xmath599 , it follows from carathodory s theorem that there exist @xmath601 or fewer points @xmath602 and positive coefficients summing to @xmath124 @xmath603 such that @xmath604 we shall show that , in fact , of these @xmath443 points , we can find @xmath23 or fewer points whose convex combination is @xmath463 .          so @xmath609 we shall next show that the points @xmath610 are in @xmath605 . to that end
we note that by   @xmath611 where the second equality holds because the @xmath612 s sum to @xmath124 .
since the @xmath612 s are all positive , it follows from   that all the terms on the rhs are nonnegative .
since they sum to zero , they must all be zero . and
since the @xmath612 s are positive , we conclude that @xmath613 and the vectors @xmath614 are all in @xmath605 .
the vector @xmath463 can thus be written as a convex combination of the @xmath443 vectors in @xmath615 in @xmath605 .
since @xmath605 is @xmath616-dimensional , it follows from carathodory s theorem that @xmath463 is in fact a convex combination of @xmath23 or fewer of the vectors @xmath615 .
let the discrete random variables @xmath208 and @xmath82 over the alphabets @xmath371 and @xmath112 , the function @xmath617 , and the function @xmath618 satisfy   and .
we shall exhibit a random variable @xmath619 over the alphabet @xmath620 and a function @xmath621 satisfying @xmath622 and the @xmath343 distortion constraints @xmath623 since the markov conditions   and imply   @xmath624 this will allow us to replace @xmath208 and @xmath129 with @xmath619 and @xmath625 and thus conclude the proof .    to describe @xmath619 and @xmath625
, we need some definitions . for each pair
@xmath626 and each @xmath399 , define @xmath627 } \nonumber \\      & = \e { d_k \bigl ( x , \fdc(y , z),\psi(x , z , u ) \bigr ) } ,     \label{eq : dkxz}\end{aligned}\ ] ] where the expectation is , by  , with respect to @xmath628 . define also the vector - valued function        where the expectation is with respect to @xmath629 .
let @xmath630 denote the image of @xmath631 : @xmath632 by definitions   @xmath633 and , consequently , there exists a point @xmath634 on the boundary of conv@xmath635 with @xmath636 since @xmath630 is compact ( it contains at most @xmath637 points because @xmath638 depends on @xmath639 only via @xmath640 ) , lemma  [ lem : card ] implies that @xmath641 can be written as a convex combination of @xmath343 or fewer points in @xmath630 : @xmath642 where @xmath643 and the coefficients @xmath644 $ ] sum to 1 .
let @xmath645 be preimages of @xmath646 so @xmath647    we can now define the function @xmath625 as mapping every pair @xmath648 and every @xmath649 to @xmath650 and we define the random variable @xmath619 to be conditionally independent of @xmath516 given @xmath302 with the conditional law @xmath651 the markov condition   thus holds by definition . moreover , , , and  combine to prove that @xmath619 and @xmath625 also satisfy the @xmath343 distortion constraints in  : denoting the @xmath559-th component of the vector @xmath652 by @xmath653 , for @xmath654 ,                  k.  kittichokechai , t.  j. oechtering , and m.  skoglund , `` source coding with common reconstruction and action - dependent side information , '' in _ proceedings 2010 ieee information theory workshop _ , dublin , ireland , 30 aug.3 sep .
2010 , pp .
r.  timo , a.  grant , and g.  kramer , `` rate distortion functions for source coding with complementary side information , '' in _ proceedings ieee international symposium on information theory _ , st .
petersburg , russia , 31 july5 aug . 2011 .
b.  ahmadi , r.  tandon , o.  simeone , and h.  v. poor , `` on the heegard - berger problem with common reconstruction constraints , '' in _ proceedings ieee international symposium on information theory _ ,
mit , cambridge , ma , 2012 .
b.  ahmadi , t.  r. , s.  o. , and p.  h. v. , `` heegard - berger and cascade source coding problems with common reconstruction constraints , '' _ submitted to ieee transactions on information theory _
[ online ] .
available : http://arxiv:1112.1762v4