communication is a recently developed communication paradigm whose communication process involves `` transmission and reception of information encoded in molecules '' @xcite .
it is born in the study of nano - scale communication in nanonetworks @xcite where the applicability of classical electromagnetic communication is limited by several constraints and a novel solution is called for .
inspired by the biological communication process such as the intra- and inter - cellular communication by cells @xcite and the pheromone diffusion by insects @xcite , the researchers have created various message - carrying molecules and their corresponding receptors . since the molecules are themselves at the nano - scale , the difficulty of nano - scale communication is solved by building up communication media between the transmitter and the receptor .
moreover , since the major potential application of nanonetworks is implemented in the organisms body ( e.g. , organ monitoring in human body ) , the compatibility of the bioinspired communication process makes it a perfect choice to consider the encoding , transmission , and decoding of messages in molecules .
excited by the reasons above ( and beyond ) , the researchers have treated the molecular communication as one of the mainstream subjects of nanotechnology nowadays
.    among the various proposed system blue maps for molecular communication , a specific class of system called the `` diffusion - based molecular system '' draws lots of interest due to the universality of _ mass transport phenomena _ in molecular communication .
the mass transport phenomena refer to the propagation of encoded molecules in the communication media . in most cases ,
the communication media of molecular communication are modeled as fluid . as is well - known , the motion of molecules in fluid media is governed by the `` diffusion process '' .
the universality of mass transport phenomena not only make the study of diffusion - based system natural , but at the same time guarantee the widest possible generality of research results . to this end ,
most efforts in molecular communication are devoted to diffusion - based systems , both in practical system design and information - theoretic analysis point of view .
for example , for the practical system design , the physical phenomena of bacteria chemotaxis , pheromone diffusion @xcite , or calcium diffusion @xcite are adopted , and there are many existing systems operating on these phenomena such as @xcite , @xcite , and @xcite .
the information - theoretic analysis of diffusion - based molecular system are also conducted in @xcite , @xcite , and @xcite .    despite of this much effort for diffusion - based molecular communication
, two important things are still lacking . first , _ an unified channel model for diffusion - based molecular communication
is missing . _
the existing works on information - theoretic analysis tend to be system - dependent ; that is , they are treating different kinds of diffusion - based systems independently and giving their own capacity analysis .
so far , the most general result concerning the information rate capacity of diffusion - based molecular communication is probably @xcite where the authors have cleverly utilized the thermodynamics to derive the information rate capacity in closed form , but the result is only valid for receiver with poisson noise .
second , and the more important one , _ no mathematical foundations have been put down for information theory in diffusion - based molecular communication_. in particular , the channel coding theorem is not proven for diffusion - based systems . to state more directly
, no one has ever shown that , for the diffusion - based molecular channel , the information rate capacity ( maximum mutual information ) is equal to the code rate capacity ( supremum achievable code rate ) , where the former is a system statistic of mathematical importance and the latter is the real practical concern .
furthermore , the diffusion - based molecular channels exhibit non - classical randomness ( which is explained later in section iv.a ) and _ no existing channel model apply to diffusion - based molecular channel_. therefore , the lacking of proof for equivalence between the information rate capacity and code rate capacity renders the purpose of calculating information rate capacity ambiguous , or at least in a shaky situation .
this paper aims at solving the above two problems once and for all .
let us first put molecular communication aside for a moment and recall the history of information theory .
ever since the advent of the paper @xcite by shannon in 1948 , the `` channel capacity '' has been the primary subject of every communication systems . in particular , the way that shannon intervenes the information - theoretic and coding - theoretic quantities for discrete memoryless channels is the one that communication engineers imitate when treating the communication system under consideration . despite its great novelty and ingenuity , a complete and rigorous mathematical setting for information theory
is not laid down until a series of papers by mcmillan @xcite , feinstein @xcite , and finally khinchin @xcite , where the definition of `` finite memory channel without anticipation '' is made precise and for such channels the equivalence between the information rate capacity and code rate capacity is established .
a key step of @xcite is the _ measure - theoretic formulation for sources and channels_. khinchin noticed that except for some simple classes of channel ( e.g. , i.i.d . , markov , ... ) , the mathematical formulations of sources and channels are most conveniently made in the measure - theoretic form to provide rigorousness and applicability .
later , works in @xcite are generalized by many others ( e.g. , @xcite , @xcite , @xcite , ... ) to include classes of channels other than finite memory channel without anticipation , all under the same proof structure .
it is now a communication engineers consensus that the measure - theoretic formulations of source and channel have struck a perfect balance between the mathematical rigorousness of information theory and the applicability thereof to practical communication systems .
therefore , for any communication system , once its channel abstraction is made in the measure - theoretic form and the corresponding capacity theorems are derived , then a solid mathematical foundation is laid down and applicability of information theory to the system is guaranteed .    in this paper
, we shall follow the path of khinchin @xcite to provide the mathematical foundations for information theory in diffusion - based molecular communication . fortunately , there are already many building blocks from the works of predecessors @xcite , @xcite , and @xcite . from these works ,
the two crucial concepts of `` channels with asymptotically decreasing input memory and anticipation '' and `` @xmath0-continuous channels '' will be used heavily in our paper .
we will give a channel model for diffusion - based molecular communication in the most rigorous language , namely the measure - theoretic one . the channel model we create
captures the essential features of all existing diffusion - based molecular systems and all diffusion - based molecular systems to come , thus allowing our results the maximum possible generality . for our channel
, we first prove its @xmath0-continuity and then utilize the @xmath0-continuity to establish various equivalence relationships between the statistical and operational definitions of channel capacity , including the equivalence of information rate capacity and the code rate capacity . in particular , the channel coding theorems for diffusion - based molecular channel is given .
a solid foundation is therefore laid down and we can use the usual sense of `` channel capacity '' without worry thereafter .
the rest of paper is organized as follows .
we state the essential terminologies and notations in section ii .
section iii introduces `` channels with asymptotically decreasing input memory and anticipation '' and `` @xmath0-continuous channels '' as well as their properties for our use .
section iv is the main section in which we give the definition of diffusion - based molecular channel and prove capacity theorems , along with some discussions .
finally , we conclude this paper in section v.
we begin by setting up terminology and notation for sources .
if a set @xmath1 is finite , the symbol @xmath2 will denote the set of all doubly infinite sequences @xmath3 , @xmath4 for all @xmath5 .
the notation @xmath6 denotes the finite sequence @xmath7 and @xmath8 stands for the set of all such sequences .
given a sequence @xmath9 , there corresponds a cylinder set ( or rectangle set ) : @xmath10 the borel field generated by all cylinder sets in @xmath2 will be denoted by @xmath11 .
a source is a probability measure @xmath12 on the measurable space @xmath13 and @xmath1 is called the alphabet of the source .
the probability measure @xmath12 induces a sequence of random variables @xmath14 on the probability space @xmath15 by the formula @xmath16 for all @xmath17 . for
integers @xmath18 and @xmath19 define @xmath20 as the borel field generated by random variables @xmath21 and we allow @xmath22 and @xmath23 .
the induced sequence @xmath24 is viewed as the transmitted signal for each time @xmath19 , and @xmath20 represents all possible events in time @xmath18 to @xmath25 .
besides @xmath24 , @xmath12 also defines a probability measure @xmath26 on @xmath8 by the formula @xmath27 when @xmath28 , @xmath26 is abbreviated @xmath29 . whenever we need to emphasize the alphabet or the induced random variables , we will also use the notation @xmath30 $ ] or @xmath31 $ ] to denote a source .
the ( left ) shift operator @xmath32 on @xmath2 is defined by @xmath33 , where @xmath34 is the element at the @xmath19th position .
a source @xmath12 is said to be stationary if @xmath35 a source is @xmath19-stationary for some positive integer @xmath19 if @xmath36 for all @xmath37 .
in other words , @xmath12 is stationary if it is shift - invariant with respect to all events , and it is @xmath19-stationary if it is @xmath19-times shift - invariant with respect to all events .
obviously stationarity implies @xmath19-stationarity .
if @xmath12 is @xmath19-stationary for some @xmath19 , we say that @xmath12 is block stationary .
a source @xmath12 is ergodic if for every @xmath37 such that @xmath38 , @xmath39 is either @xmath40 or @xmath41 .
an important property of the ergodic sources is the following :    for a given source @xmath30 $ ] , denote the characteristic function of the set @xmath37 by @xmath42 .
then , we have for all @xmath37 @xmath43 if and only if for every @xmath44 such that @xmath38 , @xmath39 is either @xmath40 or @xmath41 .
the necessity is provided by the well - known `` ergodic theorem '' of birkhoff and the sufficiency is a direct verification of the definition . for complete proof ,
see , e.g. , @xcite .
the summation appeared in the theorem represents the `` ensemble mean '' .
* theorem 1 * therefore states that the ergodic sources are those whose features can be approximated well by ensemble statistics on observable samples .
a discrete channel is characterized by an input alphabet @xmath1 , an output alphabet @xmath45 , and a set of probability measures @xmath46 on @xmath47 .
the probability measure @xmath48 is determined for each @xmath17 such that for each @xmath49 the mapping @xmath50 is @xmath11-measurable .
each @xmath48 is recognized as the output distribution conditioning on the input @xmath51 .
we shall refer @xmath52 $ ] to such structure and say that @xmath52 $ ] is a channel .
the channel is stationary if for every @xmath53 and @xmath49 we have @xmath54    now , consider the case of connecting a source @xmath30 $ ] to a channel @xmath52 $ ] .
we are naturally led to consider the probability measure @xmath55 on the product space @xmath56 characterized by @xmath57 the event @xmath58 is interpreted as `` source emits event @xmath44 and the channel output receives @xmath59 '' . if we are interested in the output process , we shall consider the probability measure @xmath60 on the output space @xmath61 ) : @xmath62 the sequence of random variables induced by @xmath60 on @xmath61 ) will be denoted by @xmath63 .
elementary analysis shows that if @xmath30 $ ] and @xmath52 $ ] are both stationary , then so is @xmath55 and @xmath60 .    in this paper
we will encounter the `` cascade '' of channels .
suppose that we are given two channels @xmath64 $ ] and @xmath65 $ ] .
if we have these two channels in cascade , the overall channel @xmath52 $ ] is described by @xmath66    another important notion of channel is the ergodicity .
the channel @xmath67 is ergodic if for any input sequence @xmath51 , @xmath48 is ergodic .
the following is an useful criterion for examining the ergodicity of a channel :    a stationary channel @xmath52 $ ] is said to be strongly mixing if for any input sequence @xmath51 and any two cylinders @xmath68 , we have @xmath69 where @xmath32 is the time shift operator . if a channel is strongly mixing , then it is ergodic .
.    intuitively speaking , a channel is strongly mixing if for any two events separated by a long time at the output , the two events are `` almost ''
independent , regardless of the input .
we have seen that the source @xmath12 can be viewed as a sequence of random variables @xmath24 .
similarly , we can view @xmath60 as @xmath63 and @xmath55 as @xmath70 .
it is well - known that if the processes represented by these sequences of random variables are stationary then their entropy rates exist .
since we shall only consider stationary ergodic sources and stationary channels in this paper , we are free to write @xmath71 @xmath72 @xmath73 the mutual information rate between the input and output is defined as @xmath74 or , equivalently , @xmath75 in the rest of this paper we shall refer to these quantities interchangably in their measure forms and random variable forms .
+ there are many defintions of channel capacity . in the followings we state and
itemize them for ease of reference .
see @xcite , @xcite , and @xcite for detail descriptions of these channel capacities .    given a channel @xmath52 $ ] , there are several choices of source which we can extremizing the mutual information over .
the most common ones are the follows :    * the ergodic capacity @xmath76 $ } } i(\mu\nu),\ ] ]    * the stationary capacity @xmath77 $ } } i(\mu\nu),\ ] ]    and    * the block stationary capacity @xmath78 } i(\mu\nu).\ ] ]    we now state coding - theoretic terminologies .
a blocklength @xmath19 channel block code @xmath79 for a noisy channel @xmath52 $ ] is a collection of @xmath80 distinct codewords @xmath81 and @xmath82 disjoint decoding sets @xmath83 for each @xmath5 .
the rate @xmath84 of a code is @xmath85 .
a code @xmath86 has probability error @xmath87 if @xmath88 where @xmath89 is the complement of @xmath90 .
such a channel block code is called an @xmath91 channel code .
we say @xmath84 is a permissible transmission rate if there exist @xmath92 channel codes such that @xmath93 as @xmath94 .
the operational channel block coding capacity @xmath95 is defined to be the supreme of all permissible rate :    * the operational channel block coding capacity @xmath96    an alternate operational definition of capacity , the operational source / channel block coding capacity @xmath97 , is defined as follows .
we consider the transmission of an ergodic source @xmath98 $ ] over the channel @xmath52 $ ] .
define a blocklength @xmath19 source / channel block code as a pair of mappings @xmath99 and @xmath100 .
the @xmath101 is called the encoder and @xmath102 the decoder . the block encoder
@xmath101 implements a mapping on infinite sequences @xmath103 defined by @xmath104 .
all such encoded sequences can be viewed as another channel input process @xmath105 $ ] characterized by @xmath106 for all @xmath37 .
the block error probability for the code is @xmath107 where @xmath108 is the random variable induced at the channel output , or , presented in the measure from , @xmath109 a source / channel code of blocklength @xmath19 and of probability error @xmath110 is called an @xmath111 source / channel code .
a source @xmath112 $ ] is said to be admissible if there exist @xmath113 source / channel codes for the source such that @xmath114 as @xmath115 .
the operational source / channel block coding capacity @xmath97 is defined as the supreme of the entropy over all admissible stationary ergodic sources :    * the operational source / channel block coding capacity@xmath116 } h(\tau).\ ] ]    the last operational capacity definition concerns another function of samples defined over input / output alphabets . the @xmath19th order sample mutual information @xmath117
is defined by @xmath118 if @xmath55 is stationary then there exist a measurable shift - invariant nonnegative bounded function @xmath119 such that @xmath120 with convergence in @xmath121 and @xmath122 if , furthermore , @xmath55 is ergodic , then the shannon - mcmillan theorem states that @xmath123 @xmath55-almost everywhere .
therefore , for stationary ergodic sources and channels the @xmath19th order sample mutual information @xmath117 is a good approximation of the `` true '' mutual information @xmath124 provided @xmath19 is large enough .
define for a stationary channel @xmath67 and for @xmath125 the quantile @xmath126 where @xmath127 and the outer supreme is taken over all block stationary input process @xmath30 $ ] .
our last operational capacity definition , the information quantile capacity @xmath128 , is defined by    * the information quantile capacity @xmath129    the relation between all these capacities ( namely , @xmath130 , and @xmath128 ) will be given in section iii , where we need the notion of @xmath131-continuity to unify these concepts .
a channel @xmath67 is said to have asymptotically decreasing input memory if given @xmath132 there is an integer @xmath133 such that , for any @xmath19 and any output event @xmath134 , @xmath135whenever @xmath136 for @xmath137 .
a channel @xmath67 is said to have asymptotically decreasing input anticipation if given @xmath132 there is an integer @xmath138 such that for any @xmath19 and any output event @xmath139 , @xmath135 whenever @xmath136 for @xmath140 .
if a channel has both asymptotically decreasing input memory and input anticipation then we call such channels the adima ( asymptotically decreasing input memory and anticipation ) channels or channels with adima . therefore , a channel @xmath67 with adima is one for wich given @xmath141 , there exist integers @xmath133 and @xmath138 such that for all @xmath142 and all @xmath143 we have @xmath135 whenever @xmath136 , @xmath144
.    intuitively speaking , a channel has asymptotically decreasing input memory if for any two input sequences @xmath51 and @xmath145 , the conditional channel output distributions @xmath48 and @xmath146 in the interval @xmath147 are `` close '' provided the two input sequences coincide on the interval @xmath148
. similar interpretation works for channel with asymptotically decreasing input anticipation .
the measure of closeness can be made precise in the language of the _
variational distance_. given two probability measures @xmath149 and @xmath150 on @xmath151 , the variational distance between the measures is defined by @xmath152 if we consider stationary channels only ( so that the definition of adima can be made to require having only all @xmath153 with the desired property , not with all @xmath154 for all @xmath155 ) , then we can rephrase the definition of the adima channel by saying that a stationary channel is with adima if , given @xmath132 , there are integers @xmath133 and @xmath138 such that for all n , @xmath156 whenever @xmath136 , @xmath157
.      given two probability measures @xmath158 and @xmath159 on @xmath151 , let @xmath160 denote the set of all joint probability measures @xmath161 on @xmath162 satisfying the property that for all @xmath163 , @xmath164 and @xmath165 in other words , @xmath161 has @xmath149 and @xmath150 as marginals .
define the coordinate functions @xmath166 and @xmath167 , @xmath168 by @xmath169 @xmath170 let @xmath171 be the hamming distance and let @xmath172 to be the @xmath19th order normalized hamming distance @xmath173 the @xmath19th order @xmath0-distance between @xmath149 and @xmath150 is defined as @xmath174    a channel @xmath67 is @xmath0-continuous if given @xmath141 there is an @xmath175 such that for @xmath176 we have @xmath177 whenever @xmath136 , @xmath178 .
equivalently , @xmath67 is @xmath0-continuous if @xmath179      here we list several important properties of @xmath0-continuous channel .
we begin by establishing the relation between the adima channel and @xmath0-continuous channel .
a channel @xmath67 is strongly @xmath131-continuous if given @xmath141 there are positive integers @xmath133 and @xmath138 such that for all @xmath19 , @xmath177 whenever @xmath136 , @xmath180 .
an adima channel is strongly @xmath131-continuous .
a strongly @xmath131-continuous channel is @xmath0-continuous .
see @xcite .
next , we summarize information - theoretic results .
recall the following notions : @xmath181    \a ) for stationary channels we have @xmath182 hence , for stationary channels , we can simply call @xmath183 the information rate capacity ( or transmission rate capacity or shannon capacity ) .
+   + b ) if a channel is stationary and @xmath0-continuous then @xmath184 + c ) let @xmath67 be a discrete stationary @xmath0-continuous channel .
if @xmath185 and @xmath132 , then for sufficiently large @xmath175 there exist @xmath186 block codes for all @xmath176 .
if @xmath187 there exists an @xmath188 such that there do not exist @xmath189 block code for any @xmath190 .
+   + d ) let @xmath67 be a discrete stationary @xmath0-continuous channel and @xmath12 a discrete stationary ergodic source .
if @xmath191 , then @xmath12 is admissible .
if @xmath192 , then @xmath12 is not admissible .
+   + e ) if , in addition , @xmath67 is also ergodic in b ) , c ) , and d ) , then @xmath193 .    see @xcite .
* theorem 4 * lists the most important channel coding theorems for discrete stationary @xmath0-continuous channels . for a discrete stationary @xmath0-continuous channel *
theorem 4.c ) * says that if we transmit message at a rate @xmath194 , then @xmath84 can be achieved by some block codes with arbitrarily small block error probability ; conversely , if we transmit at a rate @xmath195 , then the block error probability is always bounded away from zero .
similarly , * theorem 4.d ) * says that , for a discrete stationary @xmath0-continuous channel and any discrete stationary ergodic source @xmath12 , if @xmath191 , then @xmath12 can be transmitted via block coding with arbitrarily small block error probability ; conversely , if @xmath192 , then no such block code exists .
the contents of * theorem 4.c ) * and * theorem 4.d ) * therefore collectively tell us that @xmath128 is the ultimate limitation of the system performance over the given channel , where the performance is measured in terms of the maximal code rate with vanishing block error probability . * theorem 4.e ) * says that if , in addition , @xmath67 is also ergodic , then the shannon capacity @xmath196 equals @xmath128 .
the last property we need for @xmath0-continuous channel is its preservation under the @xmath0-limit @xcite :    the @xmath0-limit of a sequence of @xmath0-continuous channels is @xmath0-continuous .
see @xcite .
this is the main section of our paper . in this section ,
we first state the general properties of diffusion - based molecular channel in section iv.a .
the abstract diffusion - based molecular channel model is given in section iv.b and capacity theorems and channel coding theorems are proven in section iv.c .
section iv.d discusses the implications and assumptions of our results .
the theory of molecular communication studies the process of `` transmission and reception of information encoded in molecules '' @xcite . in other words , in a molecular communication system , we utilize features of certain molecules as the information carrier , transmit the molecules through the communication media , and design a receiver which is capable of recognizing the features encoded over .
the diffusion - based systems refer to those systems whose communication media are described by the `` diffusion processes '' . in the following
we introduce the essential components of diffusion - based molecular communication systems :          in molecular communication , the transmitter has many choices of information carrier .
for example , we can embed the information in the molecule concentration magnitudes @xcite , the inter - transmission times @xcite , the relative transmission times @xcite , the molecule types @xcite , or hybrids of each other @xcite , @xcite .
after the messages are encoded , the transmitter conveys the messages by simply releasing the encoded molecules into the communication media .
+      on contrast to the variety of information carrier , to the present days the communication media of molecular communication are concentrated on the fluid media almost exclusively .
the fluid media represent the environment where the communication takes places .
a typical environment is the blood , where the communication process occurs at human vessels .
as is well - known , the motion of molecules in fluid media is described by the `` diffusion process '' .
the molecular communication systems whose communication media are described by the diffusion process are collectively called the `` free diffusion - based molecular systems '' or simply `` diffusion - based molecular systems '' . for systems operating under the diffusion process , there is a special type of randomness which distinguishes those systems from the classical communication : the disorder of message arrivals . consider an one - dimensional molecular communication system with transmitter and receiver separated by some distance ( fig .
1 ) . for each released molecule diffusing independently in the fluid medium , the time it takes to be detected by the receiver is a random variable . if we assume that the receiver perfectly detects every molecule at its first arrival at the location of the receiver , then the detection times of the transmitted messages are the _ first passage times _ of the corresponding molecules , which are random variables depending only on the fluid medium , the corresponding transmitted molecules , and the distance from transmitter to receiver .
denote the first passage time of the @xmath5th message by @xmath197 .
it is possible to see that @xmath198 with positive probability even if @xmath199 , i.e. , it can happen that _ the later transmitted message arrives earlier_. more generally , if we release a set of molecules in sequence , then it can happen that the receiver receives these molecules in _ any _ order .
the disorder of message arrivals is inevitable in diffusion - based molecular communication .
rather unfortunately , the classical tools dealing with disorders of message arrivals are targeting at the link - layer disorders ( e.g. @xcite ) and do not suit the case of diffusion - based disorder . specifically , there are no existing mathematical tools describing the behavior of random ordering of arbitrary set of infinite random variables . to cope with it , we must develop useful channel model that describes the phenomena of the random ordering of first passage times , and by useful we mean the capability of applying the existing communication theory to the channel .
the following is the strategy we take : we first find out the essential features of first passges times occurring in diffusion - based systems and define the notion of _ permutation channel induced by the random ordering of first passage times_. afterwards , we prove that the permutation channel is with adima , and apply the channel coding theorems to such channel .    despite the randomness described above , the first passage times in diffusion - based molecular systems do contain some regularities .
the followings are the essential properties of first passage times in diffusion - based systems :    * we require that the @xmath200 is a _ delayed version _ of @xmath197 ; that is , there are two independent random variables @xmath201 and @xmath202 having the same distribution , and a random variable @xmath203 independent of @xmath204 and @xmath205 , such that @xmath206 @xmath207 + the above condition actually says that we are releasing the encoded molecules _ in sequence _ ( see fig .
the intuitive meaning is that @xmath208 is the _ inter - transmission _ time of the @xmath5th and @xmath209th messages , and @xmath204 ( or @xmath205 ) is the first passage distribution of the @xmath5th message .
if we release the @xmath5th message , wait for a time @xmath208 , and release the @xmath209th message , then the time it takes for the @xmath5th and @xmath209th molecules to diffuse to the receiver is of the above form .
note that the above condition can be applied iteratively on @xmath5 to derive @xmath210 @xmath211 where @xmath204 and @xmath205 are independent and have the same distribution , and all @xmath208 s are independent .
note also that the @xmath197 s are pairwise independent . + let us give some examples . if the system is `` synchronous '' ( which is the case in @xcite , @xcite , and @xcite ) then @xmath212 for a real number @xmath213 and @xmath214 .
the work in @xcite also considers an asynchronous communication scheme for which @xmath208 s are i.i.d
. random variables . + * the second restriction we impose on the first passage times is a qualitative one : we shall require that the arrival order of each @xmath197 is _ `` effected '' `` most exclusively '' _ by the @xmath215 s for those whose indices are close ( or , equivalently , @xmath216 s are small ) .
the precise meaning of `` effected '' and `` most exclusively '' are made clear by the next section , where we need the notion of the _ permutation channel _ to state them properly .
+ our second restriction aims at capturing the nature of the diffusion processes occurring in molecular communication .
the common feature of these diffusion processes is that if the release times of two molecules are @xmath217 and @xmath218 , then the probability of the later released molecule advancing the first is very small provided @xmath219 is large , _ regardless of any feature of the transmitted molecules ( e.g. , weight , radius , ... ) _ . in other words , to infer the order of @xmath215 in @xmath220 , it suffices to observe the nearby @xmath221 and @xmath222 for some large @xmath19 .
+ let us give a concrete example to support the belief . consider the diffusion process governed by the _ brownian motion with drift _ , which is adopted in most diffusion - based molecular communication systems ( see , e.g. , @xcite , @xcite , and @xcite ) .
the first passage time of a released molecule is given by @xcite @xmath223 where @xmath67 is the diffusion coefficient , @xmath51 is the distance from the transmitter to the receiver , and @xmath224 is the drifting velocity .
suppose the inter - transmission times @xmath208 s are supported on @xmath225 and @xmath226 .
then , according to the first restriction , the probability of the @xmath5th released molecule advancing the first is @xmath227 by discarding all the irrelevant constants we see that @xmath228 behaves like @xmath229 for large @xmath230 .
therefore , for large @xmath230 , @xmath231 in fact , the above derivation shows that if @xmath232 , then @xmath233 in @xmath5 , which means that the probability of the @xmath5th release molecule advancing the first molecule converges to @xmath40 in @xmath234 ( that is , the probability decays super - exponentially ) .
similar reasonings apply to @xmath235 with all @xmath236 s .
in particular , if @xmath237 , then by union bound @xmath238 that is , the probability of the event `` any molecule released after the @xmath239th message advancing the @xmath5th released molecule is vanishing in @xmath239 '' .
this strong and universal property of diffusion processes is the one we shall capitalize upon when formulating the mathematical structure of diffusion - based molecular channels .
+      in molecular communication , the receivers are designed to match the information carrier chosen by the transmitter .
for example , if the information are carried on the molecule types , then the receiver is some nanomachines capable of recognizing these types .
if the transmitter emits different concentration levels of a compound , then the receiver is usually operating on a chemical reaction involving the transmitted compound . some examples for the first case
are @xcite , @xcite , and @xcite . for the later case ,
see , e.g. , @xcite or @xcite .    just as the white noise may occur in classical communication , the receiver might contain some sources of detection noise . again , in all cases of molecular communication , the detection noise of a message is governed by its nearby received messages `` most exclusively '' .
the mathematical formulation of a receiver is given in the next section .
for an integer @xmath19 , denote the set of symmetric group of order @xmath19 by @xmath240 ; that is , @xmath240 is the set of all bijections of integers @xmath241 onto itself .
an element @xmath242 is called a _
permutation _ and can be represented by the permutation matrix :    @xmath243{ccccc } 1 & 2 & 3 & \cdots & n \\
b_1 & b_2 & b_3 & \cdots & b_n                  \end{array } \right)\ ] ] where @xmath244 exhausts @xmath241 , indicating the position after the effect of @xmath245 ; that is , @xmath246 .
we also consider the _
infinite permutation _ defined as any bijection of the set of all integers @xmath247 onto itself .
the set of all infinite permutations are denoted by @xmath248 .
similar to @xmath240 , any @xmath249 has its permutation matrix representation : @xmath243{ccccc } \cdots & -1 & 0 & 1 & \cdots \\
\cdots & b_{-1 } & b_0 & b_1 & \cdots                  \end{array } \right)\ ] ] for some doubly infinite sequence @xmath250 as the position indicators .
let @xmath251 be any probability measure on @xmath248 ( with borel field being all subsets of @xmath248 in discrete topology ) .
the effect of @xmath243{ccccc } \cdots & -1 & 0 & 1 & \cdots \\                                               \cdots & b_{-1 } & b_0 & b_1 & \cdots                  \end{array } \right)\ ] ] on an input sequence @xmath252 is viewed as the permutation of index : @xmath253 and @xmath254 is , of course , the probability of such index permutation .    given a finite set of random variables @xmath255 , let @xmath256 denote the @xmath5th largest order statistic of @xmath255 ; that is , @xmath257 . the order statistics of @xmath258
s induce a probability measure @xmath251 on @xmath240 by considering the probability @xmath254 of @xmath243{ccccc } 1 & 2 & 3 & \cdots & n \\
b_1 & b_2 & b_3 & \cdots & b_n                  \end{array } \right)\ ] ] as @xmath259 .
we shall say that @xmath251 is induced by the ordering of @xmath197 s .
for the case of @xmath248 the ordering of an infinite sequence of random variables is ambiguous . to this end
, we need another way of identifying the bijections in @xmath248 . for a finite subindex set @xmath260 ( i.e. , @xmath261 s are integers and @xmath262 ) and a @xmath263,@xmath243{ccccc } \cdots & -1 & 0 & 1 & \cdots \\                                               \cdots & b_{-1 } & b_0 & b_1 & \cdots                  \end{array } \right),\ ] ] define the _ local @xmath260 version _ of @xmath245 as @xmath264{ccccc } k_1 & k_2 & k_3 & \cdots & k_n \\
b'_1 & b'_2 & b'_3 & \cdots & b'_n              \end{array } \right)\ ] ] where @xmath265 is the @xmath5th least integer of @xmath266 .
for example , the local @xmath267 version of @xmath264{ccccc } 1 & 2 & 3 & 4 & 5 \\                                               5 & 4 & 3 & 1 & 2              \end{array } \right)\ ] ] is @xmath264{ccc } 1 & 3 & 5 \\                                            3 & 2 & 1               \end{array } \right)\ ] ] since @xmath268 . obviously , for @xmath269 , @xmath270 if and only if every local version of @xmath245 and @xmath271 coincides .
therefore , if we view the local @xmath260 version as the permutation matrix induced by the order statistics on the random variables @xmath272 , then these partial orderings of the infinite sequence @xmath273 uniquely determine an infinite permutation @xmath249
. however , the explicit close - form solution of the distribution @xmath251 on @xmath248 induced by @xmath273 is a difficult and long standing problem for which all existing solutions require strong assumptions ( see , e.g. , the case for i.i.d . and exchangable random variables in @xcite ) .
fortunately , we can take a different route to consider the _ qualitative _ properties of @xmath251 and the qualitative features are enough for our application , as illustrated below .
we shall say that a probability measure @xmath251 on @xmath248 is induced by the orderings of @xmath273 if @xmath251 possesses the following property : for any finite subindex @xmath260 and any set of integers @xmath274 exhausting @xmath241 , we have @xmath275 where the summation is taken over all @xmath263 with local @xmath260 version @xmath264{ccccc } k_1 & k_2 & k_3 & \cdots & k_n \\                                               b'_1 & b'_2 & b'_3 & \cdots & b'_n              \end{array } \right).\ ] ] in other words , for any finite sub - collection @xmath276 of @xmath273 , we demand that the probability of the event @xmath277 equals the sum of probability of the infinite permutations for which their local @xmath260 versions are the permutation matrix generated by the ordering of @xmath276 .    for difficulty illustrated before ,
the explicit construction for @xmath251 satisfying the condition ( 2 ) is beyond the scope of this paper . in particular , the uniqueness of @xmath251 is not of our concern : any @xmath251 with property ( 2 ) is a good model for the channel under our consideration .
if @xmath251 is not unique we can simply choose any one as the representative .
we are now ready to define the notion of a _
let @xmath278 be any input sequence of a diffusion - based molecular communication system . as discussed in section iv.a.2
, @xmath51 corresponds a set of random variables @xmath279 where @xmath197 is the first passage time of the message @xmath280 .
the ordering of these first passage times then induces a probability measure @xmath281 on @xmath248 .
permutation channel @xmath46 _ on a same input and output alphabet , say @xmath1 , is defined by @xmath282 where @xmath249 and @xmath283 means ( 1 ) . with slight abuse of notation
we also say that @xmath284 or @xmath251 is a permutation channel .
+ with the help of the permutation channel , we now give the following characterizations of a new class of channel , which is the mathematical channel model capturing all essential features of diffusion - based molecular channels considered in section iv.a .
a diffusion - based molecular channel @xmath52 $ ] is a cascade of a permutation channel @xmath251 on @xmath1 and a stationary ergodic adima channel @xmath285 $ ] , where @xmath251 is induced by the first passage times of input messages .
$ ] is called the receiver channel .
the permutation channel has the following property : let @xmath51 and @xmath145 be two input sequences and induce @xmath281 and @xmath286 on @xmath248 , respectively .
for @xmath132 , there exists integers @xmath18 and @xmath287 with @xmath288 such that for all integer @xmath19 , if @xmath136 for @xmath289 , then the following holds :    ( i ) : :    for any @xmath249 with    @xmath290 and    @xmath291 , we have    @xmath292 ( ii ) : :    the `` outlier '' probability is small : @xmath293    in both @xmath281 and @xmath286 measure .    in other words ,    @xmath294    where the summation is taken over all @xmath245 with the    property in the probability bracket .
let us give the engineering interpretation to all the assumptions we made .
in section iv.a we have illustrated that the messages arrive at the receiver in disorder for diffusion - based molecular communication systems .
since such disorder comes from the random ordering of the first passage times , it does contain some implicit regularities .
suppose we aim at an interval @xmath295 at the output of the permutation channel .
then condition ( i ) says that , since @xmath136 for an interval @xmath296 much larger than @xmath297 , their corresponding first passage times are equally distributed within @xmath296 and consequently the output distributions of @xmath295 given @xmath51 and @xmath145 should be close .
moreover , since the probability of @xmath298 vanishes in @xmath299 in all diffusion processes in molecular communication , any model for a diffusion - based molecular channel must be able to capture the fact that the `` crossing probability @xmath298 '' is small in @xmath300 , which is the content of condition ( ii ) .    the total effect at the receiver is simply modeled as an adima channel .
the notion of adima already characterizes fairly well a receiver whose detection of a message is `` mostly '' effected by its nearby received messages .      in this section
we shall prove the following fundamental theorem of diffusion - based molecular communication :    let @xmath1 be a finite alphabet .
for a diffusion - based molecular channel @xmath52 $ ] with a stationary ergodic source @xmath30 $ ] ,    \a ) @xmath301 + b ) if @xmath302 and @xmath132 , then for sufficiently large @xmath175 there exist @xmath186 block codes for all @xmath176 . if @xmath303 there exists an @xmath188 such that there do not exist @xmath189 block codes for any @xmath190 .
+ c ) if @xmath304 , then @xmath12 is admissible . if @xmath305 , then @xmath12 is not admissible .
+    we divide our proof into the following steps :    ( i ) : :    the permutation channel in a diffusion - based molecular channel is    stationary .
( ii ) : :    the permutation channel in a diffusion - based molecular channel is with    adima .
( iii ) : :    the cascade of adima channels is an adima channel .
( iv ) : :    by * theorem 3 ) * , an adima channel is @xmath0-continuous ,    and therefore * theorem 4.a)@xmath306d ) * apply .
( v ) : :    the diffusion - based molecular channel is strongly mixing ( and    therefore by * theorem 2 * it is ergodic ) .
( vi ) : :    the cascade of two ergodic channels is ergodic . by * theorem 4.e ) * the    proof is complete .
( i ) : this is an immediate result of the fact that @xmath307 if and only if @xmath283 , where @xmath32 is the time - shift operator .
( ii ) : since by ( i ) the channel is stationary , it suffices to prove that for any @xmath141 and all integer @xmath19 , there exists an integer @xmath308 such that if @xmath136 for @xmath289 , then @xmath309 for any @xmath310 .
let @xmath132 and @xmath44 be given .
let @xmath18 be a large integer determined later and @xmath136 for @xmath289 .
for each integer @xmath287 , divide the elements of @xmath311 into three classes @xmath312 , @xmath313 , and @xmath314 where @xmath315 @xmath316 and @xmath314 is the complement of @xmath317 . if @xmath318 , then @xmath136 for @xmath319 and therefore @xmath312 actually equals @xmath313 .
now , by definition of a diffusion - based molecular channel , there is an @xmath18 such that if @xmath320 , then @xmath321 since there exists a @xmath249 with @xmath322 such that @xmath323 . on the other hand , if @xmath324 , then either @xmath325 . by definition of the diffusion - based molecular channel , @xmath326
since any set @xmath310 is a finite union of @xmath327 , the proof is complete .
( iii ) : we shall state the result formally and prove it in appendix a.    ( iv ) : it is a fact .
the * theorem 6 * provides the mathematical foundations for information theory in diffusion - based molecular communication .
in this section , we discuss several issues concerning the assumptions of the * theorem 6 * along with some important implications .
* byproducts of * theorem 6 * + the proof of * theorem 6 * contains many useful byproducts .
we list them formally as below : + a diffusion - based molecular channel is stationary .
+ this is an easy consequence of the fact that the permutation channel and the receiver channel are both stationary , and the cascade of two stationary channels are stationary . + the permutation channel in a diffusion - based molecular channel is with adima , strongly @xmath131-continuous , and @xmath0-continuous .
+ a diffusion - based molecular channel is ergodic . +
a diffusion - based molecular channel is @xmath0-continuous . + * why @xmath0-continuity ?
+ in the proof of * theorem 6 * we have shown that the diffusion - based molecular channel is an adima channel .
since adima channel has its own coding theorems , it seems better to directly view the diffusion - based molecular channel as adima channel and not bothering using the @xmath0-continuity .
this is , however , not appropriate due to the fact that the adima channel is very `` vulnerable '' to modeling errors .
this is best illustrated by the following example .
consider two i.i.d .
binary sources with parameter @xmath361 and @xmath331 . recall that an adima channel demands the variational distances between distributions under consideration to be close , which is _ not _ the case for these two i.i.d .
sources when @xmath362 since the variational distance @xmath363 between the two i.i.d .
sources tends to @xmath41 as @xmath94 , _ no matter how small the error @xmath364 is_. the example tells us that if our model differs slightly from the reality , then the theorem might not carry over .
unfortunately , the diffusion process is itself a mathematical assumption and not the physical reality . in particular , for the most widely adopted diffusion model , the brownian motion , various peculiar physical behaviors have been reported ( e.g. , almost - surely nowhere - differentiable path ) and now it is commonly accepted that the brownian motion is a good approximation of diffusion process but not the exact solution .
there do exist realistic models for diffusion processes ( e.g. , the ornstein - uhlenbeck process ) , but they usually involve the `` stochastic differential equations '' which render the derivation of analytical results hard , if not impossible . +
on contrast to the adima channel , the @xmath0-continuity is very `` robust '' to the modeling errors @xcite . in particular , by * theorem 5 * the @xmath0-limit of a sequence of @xmath0-continuous channels is @xmath0-continuous , while the limit of a sequence of adima channels is not necessarily an adima channel .
this tells us that suppose we can approximate the `` true '' channel by a sequence of @xmath0-continuous channels , then the results we derived are still valid for the `` true '' channel .
this is the reason we adopt the @xmath0-continuity in this paper . + * continuous alphabets are possible .
+ in molecular communication , it can happen that the input and output alphabets are continuous ( a classical example being the transmission of concentration waveforms over fluid media @xcite ) .
we point out that there do exist works on the @xmath0-continuous channel with continuous input and output alphabets @xcite .
the extension of our results to continuous alphabets is therefore possible . + * why ergodic sources ? +
the notion of ergodicity comes from the statistical physics , where the introduce of ergodic process is used to describe a closed physical system . since a large application area of molecular communication is to monitor the physical systems occurring in biological world , we are naturally lead to consider the ergodic sources .
besides , in most cases of communication a transmitter is itself a closed physical system , thus justifying the use of ergodicity .
last but not least , the ergodic sources have so many strong mathematical properties that we could barely derive anything without it .
we just ca nt help but using it ( and so do many great predecessors in information theory ) . + * why @xmath95 , @xmath128 , and @xmath97 ?
+ @xmath95 clearly has its importance in coding theory .
the importance of @xmath97 and @xmath128 is not so obvious , as we illustrated below .
+ a major application of molecular communication is the monitoring of biological processes ( e.g. , the output signal of a human organ ) , which is usually assumed to be ergodic . for such scenarios , our communication problem at hand is `` given an ergodic source @xmath30 $ ] and a diffusion - based molecular channel @xmath52 $ ] , what is the best we can do ? '' the question is answered in a very strong sense by the definition of @xmath97 and * theorem 6 * : a good source / channel code for @xmath30 $ ] exists if @xmath365 ; if @xmath366 , no good source / channel code exist .
+ the reason for taking @xmath128 into consideration is of practical concern .
suppose that we try to really find out the channel capacity of the permutation channel @xmath251 . to this end , the direct calculation of @xmath367 , @xmath368 , and @xmath369 is impractical , since the calculation of @xmath368 and @xmath369 involves all combinatorial terms for each @xmath370 ( i.e. , for each @xmath370 and a realization @xmath371 and @xmath372 , we must consider all @xmath249 for which @xmath373 ) and the passing of limit @xmath374 .
on contrast , since @xmath128 is defined through the _ sample _ mutual information @xmath117 , it suffices to choose a large @xmath370 , calculate the expectation of @xmath375 , and the convergence of @xmath117 in @xmath376 and the shannon - mcmillan theorem will guarantee the closeness of @xmath375 and @xmath377 .
as a promising paradigm for nano - communication , the molecular communication has been developed over the past decade . in this paper
, we consider a major subclass of molecular communication systems called the diffusion - based molecular systems .
solid mathematical foundations for information theory are laid down for diffusion - based molecular communication . in particular , we have created an abstract channel model capturing all the essential features of diffusion - based molecular systems , and for such channel , the capacity theorems and channel coding theorems are proven .
other useful notions concerning stationarity and ergodicity of diffusion - based molecular channel are also established .
although the generalization is possible , here we shall confine ourselves to a special case which suffices our purpose .    for channels with finite input and output alphabets ,
the cascade of adima channels is an adima channel .
it suffices to consider the cascade of two adima channels .
the case for cascade of arbitrary number follows from induction .
let @xmath64 $ ] and @xmath65 $ ] be two adima channels with finite input and output alphabets and let @xmath52 $ ] be their cascade .
fix @xmath141 .
since @xmath378 is with adima , for any @xmath19 there are integers @xmath287 and @xmath379 such that for any @xmath380 , @xmath381 if @xmath382 for @xmath383
. write @xmath384 by ( 3 ) , we have @xmath385 put @xmath386 .
then we can further write @xmath387 therefore , @xmath388 if @xmath51 and @xmath145 are chosen with integers @xmath18 and @xmath389 such that @xmath136 for @xmath390 implies @xmath391 is small , where the condition can be achieved since @xmath64 $ ] is with adima . the proof is completed by noting that all @xmath153 is a finite union of @xmath380 .
the authors would like to thank prof .
ian f. akyildiz for his inspring advices during a meeting in taiwan .
the authors would also like to thank prof .
kwang - cheng chen , prof .
chia - han lee , po - jen shih , and yen - chi lee for their contribution to this paper .