the aim of this paper is to investigate an optimal stopping problem under partial observation for piecewise - deterministic markov processes ( pdmp ) both from the theoretical and numerical points of view .
pdmp s have been introduced by davis @xcite as a general class of stochastic models .
they form a family of markov processes involving deterministic motion punctuated by random jumps .
the motion depends on three local characteristics , the flow @xmath1 , the jump rate @xmath2 and the transition measure @xmath3 , which selects the post - jump location .
starting from the point @xmath4 , the motion of the process @xmath5 follows the flow @xmath6 until the first jump time @xmath7 , which occurs either spontaneously in a poisson - like fashion with rate @xmath8 or when the flow hits the boundary of the state space . in either case , the location of the process at @xmath7 is selected by the transition measure @xmath9 and the motion restarts from @xmath10 .
we define similarly the time until the next jump and the next post - jump location and so on .
one important property of a pdmp , relevant for the approach developed in this paper , is that its distribution is completely characterized by the discrete time markov chain @xmath11 where @xmath12 is the @xmath13-th post - jump location and @xmath14 is the @xmath13-th inter - jump time .
a suitable choice of the state space and local characteristics provides stochastic models covering a large number of applications such as operations research @xcite , reliability @xcite , neurosciences @xcite , internet traffic @xcite , finance @xcite .
this list of examples and references is of course not exhaustive .    in this paper
, we consider an optimal stopping problem for a partially observed pdmp @xmath5 . roughly speaking ,
the observation process @xmath15 is a point process defined through the embedded discrete time markov chain @xmath11 .
the inter - arrival times are given by @xmath16 and the marks by a noisy function of @xmath17 .
for a given reward function @xmath18 and a computation horizon @xmath19 , we study the following optimal stopping problem @xmath20,\ ] ] where @xmath21 is the @xmath22-th jump time of the pdmp @xmath5 , @xmath23 is a stopping time with respect to the natural filtration @xmath24 generated by the observations @xmath15 . in some applications
, it may be more appropriate to consider a fixed optimization horizon @xmath25 rather than the random horizon @xmath21 .
this is a difficult problem with few references in the literature , see for instance @xcite where the underlying process is not piecewise deterministic . regarding pdmp s , this problem could be addressed using the same ideas as in @xcite .
it involves the time - augmented process @xmath26 .
although this process is still a pdmp , its local characteristics may not have the same good properties as those of the original process leading to several new technical difficulties .    a general methodology to solve such a problem is to split it into two sub - problems .
the first one consists in deriving the filter process given by the conditional expectation of @xmath27 with respect to the observed information @xmath28 .
its main objective is to transform the initial problem into a completely observed optimal stopping problem where the new state variable is the filter process .
the second step consists in solving this reformulated problem , the new difficulty being its infinite dimension .
indeed , the filter process takes values in a set of probability measures .
our work is inspired by  @xcite which deals with an optimal stopping problem under partial observation for a markov chain with finite state space .
the authors study the optimal filtering and convert their original problem into a standard optimal stopping problem for a continuous state space markov chain .
then they propose a discretization method based on a quantization technique to approximate the value function .
however , their method can not be directly applied to our problem for the following main reasons related to the specificities of pdmps .
firstly , pdmps are continuous time processes .
although the dynamics can be described by the discrete - time markov chain @xmath11 , this optimization problem remains intrinsically a _ continuous - time _ optimization problem .
indeed , the performance criterion is maximized over the set of stopping times defined with respect to the _ continuous - time _
filtration @xmath29 . consequently
, our problem can not be converted into a fully discrete time problem .
secondly , the distribution of a pdmp combines both absolutely continuous and singular components .
this is due to the existence of forced jumps when the process hits the boundary of the state space . as a consequence
the derivation of the filter process is not straightforward .
in particular , the absolute continuity hypothesis * ( h ) * of @xcite does not hold .
thirdly , in our context the reformulated optimization problem is not standard , unlike in @xcite .
as already explained , this reformulated optimization problem combines _ continuous - time _ and _ discrete - time _ features .
consequently , this problem does not correspond to the classical optimal stopping problem of a discrete - time markov chain .
moreover , it is different from the optimal stopping problem of a pdmp under complete observation mainly because the new state variables given by the markov chain @xmath30 are not the underlying markov chain of some pdmp . therefore the results of the literature @xcite can not be used .
finally , a natural way to proceed with the numerical approximation is then to follow the ideas developed in @xcite namely to replace the filter @xmath31 and the inter - jump time @xmath14 by some finite state space approximations in the dynamic programming equation .
however , a noticeable difference from @xcite lies in the fact that the dynamic programming operators therein were lipschitz continuous whereas our new operators are only lipschitz continuous between some points of discontinuity .
we overcome this drawback by splitting the operators into their restrictions onto their continuity sets .
this way , we obtain not only an approximation of the value function of the optimal stopping problem but also an @xmath0-optimal stopping time with respect to the filtration @xmath29 that can be computed in practice .
our approximation procedure for random variables is based on quantization .
there exists an extensive literature on this method .
the interested reader may for instance consult @xcite and the references within .
the quantization of a random variable @xmath32 consists in finding a finite grid such that the projection @xmath33 of @xmath32 on this grid minimizes some @xmath34 norm of the difference @xmath35 .
roughly speaking , such a grid will have more points in the areas of high density of @xmath32 . as explained for instance in @xcite , under some lipschitz - continuity conditions , bounds for the rate of convergence of functionals of the quantized process towards the original process are available , which makes this technique especially appealing .
quantization methods have been developed recently in numerical probability or optimal stochastic control with applications in finance , see e.g. @xcite .
the paper is organized as follows .
section [ section - def ] introduces the notation , recalls the definition of a pdmp , presents our assumptions and defines the optimal stopping problem we are interested in , especially the observation process .
the recursive formulation of the filter process is derived in section  [ section - filtre ] . in section [ section - dynamic ] , we reduce our partially observed problem for the pdmp @xmath5 to a completely observed one involving the process @xmath36 for which we provide the dynamic programming equation and construct a family of @xmath0-optimal stopping times . then , our numerical methods to compute the value function and an @xmath0-optimal stopping time are presented in section [ section - quantif ] where we also prove the convergence of our algorithms after having recalled the main features of quantization .
finally , an academic example is discussed in section [ section - example ] while technical results are postponed to the appendices .
in this first section , let us define a piecewise - deterministic markov process ( pdmp ) and introduce some general assumptions . for any metric space @xmath37 , we denote @xmath38 its borel @xmath23-field , @xmath39 the set of real - valued , bounded and measurable functions defined on  @xmath37 and @xmath40 the subset of functions of @xmath39 that are lipschitz continuous . for @xmath41 , denote @xmath42 and @xmath43 .
let @xmath37 be an open subset of @xmath44 .
let @xmath45 be its boundary and @xmath46 its closure and for any subset @xmath47 of @xmath37 , @xmath48 denotes its complement .
a pdmp is defined by its local characteristics @xmath49 .    *
the flow @xmath50 is continuous . for all @xmath51
, @xmath52 is an homeomorphism and @xmath53 is a semi - group : for all @xmath54 , @xmath55 . for all @xmath56 , define the deterministic exit time from @xmath37 : @xmath57 we use here and throughout the convention @xmath58 . *
the jump rate @xmath59 is measurable and satisfies : @xmath60 * finally , @xmath3 is a markov kernel on @xmath61 which satisfies : @xmath62    from these characteristics , it can be shown @xcite that there exists a filtered probability space @xmath63 on which a process @xmath64 is defined .
its motion , starting from a point @xmath65 , may be constructed as follows .
let @xmath66 be a nonnegative random variable with survival function : @xmath67 where for @xmath65 and @xmath68 $ ] , @xmath69 one then chooses an @xmath37-valued random variable @xmath70 with distribution @xmath71 .
the trajectory of @xmath72 for @xmath73 is : @xmath74 starting from the point @xmath75 , one selects in a similar way @xmath76 the time between @xmath7 and the next jump time @xmath77 , as well as @xmath78 the next post - jump location and so on .
davis showed @xcite that the process so defined is a strong markov process @xmath79 with jump times @xmath80 ( @xmath81 ) .
the process @xmath82 where @xmath83 is the @xmath13-th post - jump location and @xmath84 ( @xmath85 ) is the @xmath13-th inter - jump time is clearly a discrete - time markov chain .
the following non explosion assumption about the jump - times is standard ( see for example @xcite ) .
[ hyp - tk_goes_to_infty ] for all @xmath86 , @xmath87<+\infty$ ] .
it implies that @xmath88 a.s .
when @xmath89 .
moreover , we make the following assumption about the transition kernel @xmath3 .
we assume that there exists a finite set @xmath90 such that for all @xmath65 , one has @xmath91 .    in other words ,
for all @xmath92 , @xmath12 may only take its values in the finite set  @xmath93 .
this assumption ensures that the filter process , defined in the next section , has finite dimension .
this is required to derive a tractable numerical method in section  [ section - quantif ] .
when this assumption does not hold , one may consider a preliminary discretization of the transition kernel to introduce it .
[ hyp - ts - bounded ] we assume that the function @xmath94 is bounded on @xmath93 i.e. for all @xmath95 , we assume that @xmath96 .
[ def - ts - order ] for all @xmath95 , denote @xmath97 and assume that @xmath98 ,  , @xmath99 are numbered such that @xmath100  @xmath101
. moreover , let @xmath102 .
for any function @xmath103 in @xmath39 , introduce the following notation @xmath104 for any lipschitz continuous function @xmath103 in @xmath40 , denote @xmath105 $ ] its lipschitz constant @xmath106=\sup_{x\neq y \in e}\frac{|w(x)-w(y)|}{|x - y|}.\ ] ]    the jump rate @xmath2 is in @xmath107 i.e. is bounded by @xmath108 .    denote @xmath109 the set of finite signed measures on @xmath93 and @xmath110 the subset of probability measures on @xmath93 .
we equip @xmath109 with the norm @xmath111 given by @xmath112 where @xmath113 denotes @xmath114 .
we consider from now on a pdmp @xmath5 of which the initial state @xmath115 is a fixed point @xmath116 .
we assume that this pdmp is observed through a noise and we now turn to the description of our observation procedure . for all @xmath92 , we assume that @xmath14 is perfectly observed but that @xmath12 is not ( except for the initial state @xmath117 ) . in some examples
, it seems reasonable to consider that the jump times of the process are observed ( for instance , if the jumps correspond to changes of environment ) and that , when a jump occurs , the actual post - jump location is measured with a noise .
the _ observation _
process of @xmath12 , denoted by @xmath118 is assumed to be of the following form : @xmath119 ( deterministic ) and for @xmath120 , @xmath121 where @xmath122 and where the _ noise _ @xmath123 is a sequence of @xmath44-valued , i.i.d .
random variables with bounded density function @xmath124 that are also independent from @xmath11 . in order to define real - valued stopping times adapted to the observation process , we need to consider a continuous time version of the observation process .
we therefore define the piecewise - constant process @xmath125 with a slight abuse of notation represents the value of the process @xmath15 at time @xmath126 and must not be confused with the value of the process at time @xmath127 . ] as @xmath128 let @xmath24 be the filtration generated by @xmath15 ( the _ observed _ filtration ) and @xmath129 be the filtration generated by @xmath130 ( the _ total _ filtration ) . without changing the notation , we then complete these filtrations with all the @xmath131-null sets .
this leads us to the following definition .
denote @xmath132 the set of @xmath29-stopping times that are a.s .
finite and for @xmath92 , define @xmath133    for all @xmath92 , we define the filter @xmath134 .
the quantity @xmath135 , denoted by @xmath136 , represents the probability of the event @xmath137 given the information available until time @xmath138 i.e. @xmath139.\ ] ] finally , let @xmath19 be the _ horizon _ and @xmath140 the _ reward function _ , we are interested in maximizing the following _ performance criterion _ @xmath141\ ] ] with respect to the stopping times @xmath142 .
the _ value function _ associated to this partially observed optimal stopping problem is given by @xmath143,\ ] ] where @xmath144 is a probability measure in @xmath110 .
the solution of our problem is then obtained by setting @xmath145 .
for some applications , it would be interesting to consider a more general form for the reward function such as an integral term also possibly depending on the observation process , see for instance @xcite .
however , this new setup would lead to several technical difficulties .
in particular , the dynamic programming would be more complex .
thus the derivation of the error bounds for the numerical approximation would be possibly intractable .    we will also need the following assumption about the reward function @xmath18 associated with the optimal stopping problem .    [ hyp - g - lip ] the function @xmath18 is in @xmath107 i.e. bounded by @xmath146 and there exists @xmath147_{2}\in \mathbb{r}^+$ ] such that for all @xmath148 and @xmath149 $ ] , one has : @xmath150_{2}|t - u|.\ ] ]    now , the aims of this paper are first to explicit the filter process @xmath151 ( section  [ section - filtre ] ) ; second to rewrite the partially observed optimal stopping problem as a totally observed one for a suitable markov chain on @xmath152 ( section  [ sec - opt - stop - complete ] ) ; third to derive a dynamic programming equation and construct a family of @xmath0-optimal stopping times ( section  [ section - dynamic - eq ] ) ; and finally to propose a numerical method to compute an approximation of the value function and an @xmath0-optimal stopping time ( section  [ section - quantif ] ) . as a starting point , we will derive , in the next section , a recursive construction of the optimal filter that is the key point of our approach .
the goal of this section is to obtain a recursive formulation of the filter  @xmath31 . as far as we know
, there is no result concerning the filter process for generic pdmps .
we may however refer to @xcite for a recursive formulation of the filter for point processes , that can be seen as a sub - class of pdmp s . for all @xmath92 ,
we denote @xmath153 .
the continuous - time observation process @xmath15 being a point process in the sense developed in  @xcite , one has @xmath154 ( see ( * ? ? ? * , theorem t2 ) ) .
moreover , @xmath155 . concerning the filter @xmath31 , first notice that , since it is an @xmath156-measurable random variable , there exists for all @xmath92 a measurable function @xmath157 such that @xmath158 as in the case of the kalman - bucy filter , the iteration leading from @xmath159 to @xmath31 can be split into two steps : prediction and correction .
for all @xmath120 , let @xmath160 be the conditional distribution of @xmath161 given @xmath162 .
thus , @xmath160 is a transition kernel defined on @xmath163 for all @xmath164 and @xmath165 by @xmath166    [ lemme - filtre - zys ] for all @xmath165 , we have the following equality of probability measures on @xmath167 , for all @xmath164 , @xmath168    [ [ proof ] ] proof + + + + +    set @xmath169 in @xmath170 , using eq .
that defines @xmath118 , one has @xmath171}\\ & = & \sum_{j=1}^{q}\int h(x_{j},\varphi(x_{j})+w , s){\mathbf{p}}(z_{n}=x_{j},s_{n}\in ds , w_{n}\in dw|\mathcal{g}_{n-1}=\gamma_{n-1}).\end{aligned}\ ] ] moreover , @xmath172 is independent from @xmath173 and admits the density function @xmath124 .
consequently , one easily obtains the result by using the change of variable @xmath174 .
@xmath175 +   + integrating w.r.t . to the first variable in the previous lemma ( i.e. summing w.r.t .
@xmath176 ) yields the following result .
[ lemme - filtre - ys ] for all @xmath165 , we have the following equality of probability measures on @xmath177 , @xmath178dy.\ ] ]    [ lemme - filtre - mu ] for all @xmath120 , @xmath165 and @xmath179 , the distribution @xmath160 , defined by eq .  , satisfies @xmath180{t^{*}}_{m};{t^{*}}_{m+1}[\}}\left(\sum_{i = m+1}^{q}\pi_{n-1}^{i}(\gamma_{n-1})\lambda(\phi(x_i , s))e^{-\lambda(x_i , s)}q(\phi(x_i , s),x_j)\right)ds\\ & & + \sum_{m=1}^{q}\left(\pi_{n-1}^{m}(\gamma_{n-1})e^{-\lambda(x_m,{t^{*}}_m)}q(\phi(x_m,{t^{*}}_m),x_j)\right)\delta_{{t^{*}}_m}(ds).\end{aligned}\ ] ]    [ [ proof-1 ] ] proof + + + + +    let @xmath169 be a function of @xmath181 . since @xmath182 , the law of iterated conditional expectations yields @xmath183={\mathbf{e}}\left[{\mathbf{e}}\left[h(z_{n},s_{n})\big|{\mathfrak{f}}_{t_{n-1}}\right]\big|\mathcal{g}_{n-1}=\gamma_{n-1}\right].\ ] ] besides , @xmath184 so that @xmath185={\mathbf{e}}\left[h(z_{n},s_{n})\big|z_{0},s_{0},\ldots , z_{n-1},s_{n-1}\right],\ ] ] by independence of the sequences @xmath186 and @xmath11 .
now , we apply the markov property of @xmath11 and a well - known special feature of the transition kernel of the underlying markov chain of a pdmp to obtain @xmath185={\mathbf{e}}\left[h(z_{n},s_{n})\big|z_{n-1},s_{n-1}\right]={\mathbf{e}}\left[h(z_{n},s_{n})\big|z_{n-1}\right].\ ] ] moreover , the transition kernel can be explicitly expressed in terms of the local characteristics of the pdmp , and this yields the next equations    @xmath187}\\ & = & { \mathbf{e}}\big[\sum_{i=1}^{q}{\mathbbm{1}}_{\{z_{n-1}=x_{i}\}}{\mathbf{e}}[h(z_{n},s_{n})|z_{n-1}=x_{i}]\big|\mathcal{g}_{n-1}=\gamma_{n-1}\big]\\ & = & { \mathbf{e}}\big[\sum_{i=1}^{q}{\mathbbm{1}}_{\{z_{n-1}=x_{i}\}}\sum_{j=1}^{q}\big[\int_{{\mathbb{r}}^{+ } } h(x_{j},s)\lambda(\phi(x_i , s))e^{-\lambda(x_i , s)}{\mathbbm{1}}_{\{s<{t^{*}}_i\}}q(\phi(x_i , s),x_j)ds\\ & & + h(x_{j},{t^{*}}_i)e^{-\lambda(x_i,{t^{*}}_i)}q(\phi(x_i,{t^{*}}_i),x_j)\big]\big|\mathcal{g}_{n-1}=\gamma_{n-1}\big]\\ & = & \sum_{j=1}^{q}\big(\int_{{\mathbb{r}}^{+ } } h(x_{j},s)\sum_{i=1}^{q}\pi_{n-1}^{i}(\gamma_{n-1})\lambda(\phi(x_i , s))e^{-\lambda(x_i , s)}{\mathbbm{1}}_{\{s<{t^{*}}_i\}}q(\phi(x_i , s),x_j)ds\\ & & + \sum_{i=1}^{q}h(x_{j},{t^{*}}_i)\pi_{n-1}^{i}(\gamma_{n-1})e^{-\lambda(x_i,{t^{*}}_i)}q(\phi(x_i,{t^{*}}_i),x_j)\big).\end{aligned}\ ] ]    this can be written equivalently as @xmath188}\\ & = & \sum_{j=1}^{q}\bigg(\sum_{m=0}^{q-1}\bigg(\int_{{t^{*}}_{m}}^{{t^{*}}_{m+1 } } h(x_{j},s)\sum_{i = m+1}^{q}\pi_{n-1}^{i}(\gamma_{n-1})\lambda(\phi(x_i , s))e^{-\lambda(x_i , s)}q(\phi(x_i , s),x_j)\bigg)ds\\ & & + \sum_{i=1}^{q}h(x_{j},{t^{*}}_i)\pi_{n-1}^{i}(\gamma_{n-1})e^{-\lambda(x_i,{t^{*}}_i)}q(\phi(x_i,{t^{*}}_i),x_j)\bigg).\end{aligned}\ ] ] hence the result .
@xmath175 +   + we now state the main result of this section , namely the recursive formulation of the filter sequence @xmath151 .    [ prop - rec - filtre ] let @xmath189 be defined as follows : for all @xmath179 , @xmath190{t^{*}}_{m};{t^{*}}_{m+1}[\}}\frac{\psi^{j}_{m}(\pi , y , s)}{\overline{\psi}_{m}(\pi , y , s)}+\sum_{m=1}^{q}{\mathbbm{1}}_{\{s={t^{*}}_{m}\}}\frac{\psi^{*j}_{m}(y)}{\overline{\psi}^{*}_{m}(y)},\ ] ] where @xmath191 then , the filter , defined in eq .  , satisfies @xmath192 and the following recursion : for all @xmath120 , @xmath193    [ [ proof-2 ] ] proof + + + + +    fix @xmath194 in @xmath195 .
bayes formula yields for all @xmath179 , @xmath196 lemma [ lemme - filtre - zys ] and corollary [ lemme - filtre - ys ] yield @xmath197dy.\end{gathered}\ ] ] with respect to @xmath198 , one recognizes the equality of two absolutely continuous measures which implies the equality a.e . of the density functions .
thus , one has for almost all @xmath199 w.r.t .
the lebesgue measure , @xmath200.\nonumber\end{aligned}\ ] ] eq .
states the equality of two measures of the variable @xmath201 that contain both an absolutely continuous part and some weighted dirac measures .
denote @xmath202 ( respectively @xmath203 ) the left - hand ( resp .
right - hand ) side term of the previous equality .
eq .   means that for all function @xmath204 and for almost all @xmath199 w.r.t .
the lebesgue measure ,
one has @xmath205 recall that , from lemma [ lemme - filtre - mu ] , the distribution @xmath206 has a density on the interval @xmath207{t^{*}}_{m};{t^{*}}_{m+1}[$ ] denoted by @xmath208 and given by @xmath209 first , take @xmath210{t^{*}}_{m};{t^{*}}_{m+1}[\}}$ ] in equation   with @xmath211 .
one has from equation ( [ eq - egal - mesures ] ) @xmath212 and thus on @xmath207{t^{*}}_{m};{t^{*}}_{m+1}[$ ] , almost surely w.r.t .
the lebesgue measure , one has @xmath213 finally , for @xmath214 , choosing @xmath215 in eq .
yields the equality of the weights at the point @xmath216 thus , using lemma [ lemme - filtre - mu ] , @xmath217 thus there exists two measurable sets @xmath218 and @xmath219 , negligible w.r.t .
the lebesgue measures on @xmath44 and @xmath220 respectively , such that for all @xmath221 , @xmath222 , @xmath223 , one has @xmath224 on the one hand , one has @xmath225 by absolute continuity of the distribution of @xmath172 . on the other hand , @xmath226 because the distribution of @xmath14 is absolutely continuous on @xmath227 and one has @xmath228 .
we therefore conclude from eq .
that @xmath131-a.s .
, one has @xmath229 the result follows since @xmath131-a.s .
, one has @xmath230 and @xmath231 . @xmath175 +   + this proposition will play a crucial part in the sequel . on the one hand , this result will enable us to prove the markov property of the sequence @xmath30 w.r.t . the observed filtration . on the other hand ,
the recursive formulation allows for simulation of the process @xmath232 which is crucial to obtain numerical approximations .
finally , notice that the specific structure of the pdmp appears in the recursive formulation of the filter which contains both an absolutely continuous part and some weighted points .
the main objective of this section is to derive the dynamic programming equation for the value function of the partially observed optimal stopping problem .
the proof of this result can be roughly speaking decomposed into two steps .
the first point consists in converting the partially observed optimal stopping problem into an optimal stopping problem under complete observation where the state variables are described by the _ discrete - time _ markov chain @xmath30 ( see section [ sec - opt - stop - complete ] ) .
it is important to remark that under this new formulation , the optimization problem remains intrinsically a _ continuous - time _ optimization problem because the performance criterion is maximized over the set of stopping times with respect to the _ continuous - time _
filtration @xmath29 .
we show in the second step ( see section [ section - dynamic - eq ] ) that the value function associated to the optimal stopping problem can be calculated by iterating a functional operator , labelled @xmath233 ( see definition [ def - g - h - i - j - l ] ) . as a by - product
, we also provide a family of @xmath0-optimal stopping times .
we would like to emphasize that the results obtained in this section are not straightforward to obtain due to the specific structure of this optimization problem . indeed ,
as already explained , it combines _ continuous - time _ and _ discrete - time _ features .
consequently , this problem does not correspond to the classical optimal stopping problem of a discrete - time markov chain .
moreover , it is different from the optimal stopping problem of a pdmp under complete observation mainly because the new state variables given by the markov chain @xmath30 are not the underlying markov chain of some pdmp . therefore the results of the literature @xcite can not be used .
these derivations require some technical results about the structure of the stopping times in @xmath234 .
for the sake of clarity in exposition , they are presented in the appendix [ stop - time ] .
we start with a technical preliminary result required in the sequel , investigating the markov property of the filter process .
[ prop - markov ] the sequences @xmath235 , @xmath236 and @xmath237 are @xmath238-markov chains .
[ [ proof-3 ] ] proof + + + + +    let @xmath239 .
the law of iterated conditional expectations yields @xmath240 = { \mathbf{e}}\big[{\mathbf{e}}[h(\pi_{n},y_{n},s_{n})| { \mathfrak{f}}_{t_{n-1}}]\big|{\mathfrak{f}^{y}}_{t_{n-1}}\big].\end{aligned}\ ] ] from proposition [ prop - rec - filtre ] and eq .   which defines @xmath118 one obtains @xmath241}\\ & = & { \mathbf{e}}\big[h\big(\psi(\pi_{n-1},\varphi(z_{n})+w_{n},s_{n}),\varphi(z_{n})+w_{n},s_{n}\big)\big|
{ \mathfrak{f}}_{t_{n-1}}\big]\\ & = & \sum_{j=1}^{q}\int h\big(\psi(\pi_{n-1},\varphi(x_{j})+w , s),\varphi(x_{j})+w , s\big)\\ & & \times { \mathbf{p}}(z_{n}=x_{j},w_{n}\in dw , s_{n}\in ds|{\mathfrak{f}}_{t_{n-1}}).\end{aligned}\ ] ] yet , @xmath172 is independent from @xmath242 and admits the density function @xmath124 . as in the proof of lemma [ lemme - filtre - zys ] one
thus obtains @xmath241}\\ & = & \sum_{j=1}^{q}\int h\big(\psi(\pi_{n-1},y , s),y , s\big){\mathbf{p}}(z_{n}=x_{j},s_{n}\in ds|{\mathfrak{f}}_{t_{n-1}})f_{w}(y-\varphi(x_{j}))dy.\end{aligned}\ ] ] besides , we have @xmath243 as in the proof of lemma [ lemme - filtre - mu ] , so that one has @xmath241}\\ & = & \sum_{i=1}^{q}{\mathbbm{1}}_{\{z_{n-1}=x_{i}\}}\sum_{j=1}^{q}\int \big(\int_{0}^{{t^{*}}_{i } } h\big(\psi(\pi_{n-1},y , s),y , s\big)\lambda\big(\phi(x_i , s)\big)e^{-\lambda(x_i , s)}q\big(\phi(x_{i},s),x_{j}\big)ds\\ & & + h\big(\psi(\pi_{n-1},y,{t^{*}}_{i}),y,{t^{*}}_{i}\big)e^{-\lambda(x_i,{t^{*}}_{i})}q\big(\phi(x_{i},{t^{*}}_{i}),x_{j}\big)\big)f_{w}(y-\varphi(x_{j}))dy.\end{aligned}\ ] ] take now the conditional expectation w.r.t .
@xmath162 , to obtain @xmath241}\\ & = & \sum_{i=1}^{q}\pi_{n-1}^{i}\sum_{j=1}^{q}\int \big(\int_{0}^{{t^{*}}_{i } } h\big(\psi(\pi_{n-1},y , s),y , s\big)\lambda\big(\phi(x_i , s)\big)e^{-\lambda(x_i , s)}q\big(\phi(x_{i},s),x_{j}\big)ds\\ & & + h\big(\psi(\pi_{n-1},y,{t^{*}}_{i}),y,{t^{*}}_{i}\big)e^{-\lambda(x_i,{t^{*}}_{i})}q\big(\phi(x_{i},{t^{*}}_{i}),x_{j}\big)\big)f_{w}(y-\varphi(x_{j}))dy.\end{aligned}\ ] ] hence @xmath244 $ ] is merely a function of @xmath159 yielding the result for the three processes . @xmath175 +      in this section ,
we show how our optimal stopping problem under partial observation for the process @xmath5 can be converted into an optimal stopping problem under complete observation involving the markov chain @xmath245 .
more precisely , for a fixed stopping time @xmath246 , we show in proposition [ lemme value fonction ] that the performance criterion @xmath247 $ ] can be expressed in terms of the discrete - time markov chain @xmath245 .
we would like to emphasize the following important fact .
although the performance criterion can be written in terms of _ discrete - time _ process , the optimization problem remains intrinsically a _ continuous - time _ optimization problem .
indeed , the performance criterion is maximized over the set of stopping times with respect to the _ continuous - time _ filtration @xmath29 .
[ lemme value fonction ] let @xmath248 and @xmath120 .
for all @xmath249 one has @xmath250}\\ & = & \sum_{k=0}^{n-1}\sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{k}\leq \sigma\}}{\mathbbm{1}}_{\{r_{k}<{t^{*}}_{i}\}}g\circ\phi(x_{i},r_{k})e^{-\lambda(x_{i},r_{k})}\pi_{k}^{i}|\pi_0=\pi]\\ & & + \sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{n}\leq \sigma\}}g(x_{i})\pi_{n}^{i}|\pi_0=\pi],\end{aligned}\ ] ] where @xmath251 is the sequence of non negative random variables associated to @xmath23 as introduced in theorem [ theo - bremaud - adapted ] .
[ [ proof-4 ] ] proof + + + + +    we split @xmath252 $ ] into several terms depending on the position of @xmath23 w.r.t .
the jump times @xmath253 @xmath254 } & = & \sum_{k=0}^{n-1}\sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{k}\leq \sigma < t_{k+1}\}}{\mathbbm{1}}_{\{z_{k}=x_{i}\}}g\circ\phi(x_{i},r_{k})|\pi_0=\pi]\\ & & + \sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{n}\leq \sigma\}}{\mathbbm{1}}_{\{z_{n}=x_{i}\}}g(x_{i})|\pi_0=\pi].\end{aligned}\ ] ] for notational convenience , consider @xmath255 on the one hand , one has @xmath256=g(x_{i}){\mathbbm{1}}_{\{t_{n}\leq \sigma\}}\pi_{n}^{i}$ ] since @xmath257 ( see for instance ( * ? ? ?
* , theorem t7 ) ) .
on the other hand , to compute @xmath258 $ ] , we use lemma [ lemme - tech - event ] to obtain @xmath259 & = { \mathbbm{1}}_{\{t_{k}\leq \sigma\}}g\circ\phi(x_{i},r_{k}){\mathbf{e}}[{\mathbbm{1}}_{\{s_{k+1}>r_{k}\}}{\mathbbm{1}}_{\{z_{k}=x_{i}\}}|{\mathfrak{f}^{y}}_{t_{k}}]\\ & = { \mathbbm{1}}_{\{t_{k}\leq \sigma\}}g\circ\phi(x_{i},r_{k}){\mathbf{e}}\big[{\mathbbm{1}}_{\{z_{k}=x_{i}\}}{\mathbf{e}}[{\mathbbm{1}}_{\{s_{k+1}>r_{k}\}}|{\mathfrak{f}}_{t_{k}}]\big|{\mathfrak{f}^{y}}_{t_{k}}\big]\\ & = { \mathbbm{1}}_{\{t_{k}\leq \sigma\}}g\circ\phi(x_{i},r_{k}){\mathbf{e}}[{\mathbbm{1}}_{\{z_{k}=x_{i}\}}{\mathbbm{1}}_{\{r_{k}<{t^{*}}(z_{k})\}}e^{-\lambda(z_{k},r_{k})}|{\mathfrak{f}^{y}}_{t_{k}}]\\ & = { \mathbbm{1}}_{\{t_{k}\leq \sigma\}}g\circ\phi(x_{i},r_{k}){\mathbbm{1}}_{\{r_{k}<{t^{*}}_{i}\}}e^{-\lambda(x_{i},r_{k})}\pi_{k}^{i}.\end{aligned}\ ] ] details to obtain the third line in the above computations are provided by lemma  [ lemme - tech - esp - cond ] .
the result follows .
@xmath175 +      based on the new formulation , the main objective of this section is to derive the backward dynamic programming equation .
it involves some operators introduced in definition [ def - g - h - i - j - l ] . by iterating the operator ,
labelled @xmath233 , we define a sequence of real valued functions @xmath260 in definition [ def - vn ] .
theorem [ value - fonction ] establishes that @xmath261 is the value function of our partially observed optimal stopping problem with horizon @xmath262 and in particular that @xmath263 is the value function of problem defined in equation  .
another important result of this section is given by theorem [ theo - sn - epsilon ] which constructs a sequence of @xmath0-optimal stopping times .
[ def - g - h - i - j - l ] the operators @xmath264 , @xmath265 , @xmath266 , and @xmath267 are defined for all @xmath268 and @xmath269 by @xmath270,\\ hh(\pi , u)&=&{\mathbf{e}}\big[\sum_{i=1}^qh\circ\phi(x_i , u)\pi_0^i{\mathbbm{1}}_{\{u < t^*_i\}}{\mathbbm{1}}_{\{s_{1 } > u\}}| \pi_0=\pi\big],\\j(v , h)(\pi , u)&=&hh(\pi , u)+gv(\pi , u),\\ l(v , h)(\pi)&=&\sup_{u\geq 0 } j(v , h)(\pi , u).\end{aligned}\ ] ]    [ def - vn ] the sequence @xmath260 of real - valued functions is defined on @xmath110 by @xmath271    the following theorem is the main result of this section showing that the operator @xmath233 is the dynamic programming operator associated to the initial optimization problem
.    [ value - fonction ] for all @xmath272 and @xmath273 , one has @xmath274 = v_{n - n}(\pi).\ ] ]    [ [ proof-5 ] ] proof + + + + +    the proof of this result is based on proposition [ theo - value - fonction ] and theorem [ theo - sn - epsilon ] .
proposition [ theo - value - fonction ] proves that @xmath275 is an upper bound for the value function of the problem with horizon @xmath138 .
the reverse inequality is derived in theorem [ theo - sn - epsilon ] by constructing a sequence of @xmath0-optimal stopping times . @xmath175 +   +    [ theo - value - fonction ] for all @xmath272 and @xmath273 , one has @xmath274\leq v_{n - n}(\pi).\ ] ]    [ [ proof-6 ] ] proof + + + + +    let @xmath276 .
consider @xmath251 the sequence associated to @xmath23 as introduced in theorem [ theo - bremaud - adapted ] .
we prove the theorem by induction on @xmath13 . for @xmath277 ,
proposition  [ lemme value fonction ] yields @xmath278 & = & { \sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{r_{0}<{t^{*}}_{i}\}}g\circ\phi(x_{i},r_{0})e^{-\lambda(x_{i},r_{0})}\pi_{0}^{i}|\pi_0=\pi ] } \nonumber \\ & & + { \sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{1}\leq\sigma\}}g(x_{i})\pi_{1}^{i}|\pi_0=\pi]}. \label{beben}\end{aligned}\ ] ] since @xmath279 is deterministic and by using lemma [ lemme - def - op - h ] , we recognize that the first term of the right hand side of equation ( [ beben ] ) is @xmath280 .
we now turn to the second term of the right hand side of equation ( [ beben ] ) which is given by @xmath281 & = & { \mathbf{e}}[v_{n}(\pi_{1}){\mathbbm{1}}_{\{s_{1}\leq r_{0}\}}|\pi_0=\pi ] \\ & = & \ gv_{n}(\pi , r_{0}),\end{aligned}\ ] ] from lemma [ lemme - tech - event ] and the definition of @xmath282 . recall that from definition [ def - g - h - i - j - l ] one has @xmath283 thus , one obtains @xmath278 \ = \
j(v_{n},g)(\pi , r_{0 } ) & \leq & \sup_{u\geq 0}j(v_{n},g)(\pi , u)\\ & = & l(v_{n},g)(\pi)= v_{n-1}(\pi).\end{aligned}\ ] ] set now @xmath284 and assume that @xmath285\leq v_{n-(n-1)}(\pi)$ ] , for all @xmath286 .
proposition [ lemme value fonction ] yields @xmath287}\\   & = & \sum_{k=0}^{n-1}\sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\ { t_{k}\leq \sigma\}}{\mathbbm{1}}_{\{r_{k}<{t^{*}}_{i}\}}g\circ\phi(x_{i},r_{k})e^{-\lambda(x_{i},r_{k})}\pi_{k}^{i}|\pi_0=\pi]\\ & & + \sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{n}\leq \sigma\}}g(x_{i})\pi_{n}^{i}|\pi_0=\pi].\end{aligned}\ ] ] as in the case @xmath277 , the term for @xmath288 equals @xmath280 .
notice that for @xmath289 , @xmath290 and that @xmath291 is @xmath292-measurable . by taking the conditional expectation w.r.t .
@xmath292 it follows that @xmath293={\mathbf{e}}[\xi |\pi_0=\pi]$ ] where @xmath294 is defined by @xmath295.\end{aligned}\ ] ] therefore , we obtain @xmath296=   hg(\pi , r_{0})+{\mathbf{e}}[\xi{\mathbbm{1}}_{\{s_{1}\leq r_{0}\}}|\pi_0=\pi].\ ] ] we now use the markov property of the chain @xmath297 .
indeed , for @xmath289 , one has @xmath298 , where @xmath299 is the translation operator of the @xmath238-markov chain @xmath235 . moreover , when @xmath300 , one has , from proposition [ propb4 ] , @xmath301 ( indeed , we pointed out in remark [ rq - rn - rnbar ] that @xmath302 can be replaced by @xmath303 defined in lemma [ lemmeb2 ] ) and @xmath304 where @xmath305 and @xmath306 are defined in definition [ defb3 ] and proposition [ propb4 ] ( with @xmath307 in the present case ) .
since for @xmath289 , @xmath308 , one has @xmath309 . finally , combining the markov property of the chain @xmath297 and proposition [ lemme value fonction ] we have @xmath310 with @xmath311 $ ] .
moreover , one has @xmath312 from the induction assumption since @xmath313 ( indeed , both @xmath306 and @xmath314 are @xmath29-stopping times from corollary [ corb6 ] and lemma [ prop - tn - stop - time ] respectively ) .
one has then @xmath315 finally , combining eq .   and
, one has @xmath316
\leq   hg(\pi , r_{0})+{\mathbf{e}}[v_{n-(n-1)}(\pi_{1}){\mathbbm{1}}_{\{s_{1}\leq r_{0}\}}|\pi_0=\pi].\ ] ] in the second term , we recognize the operator @xmath282 and one has @xmath317 & \leq &   hg(\pi , r_{0})+gv_{n-(n-1)}(\pi , r_{0})\\ & = & j(v_{n-(n-1)},g)(\pi , r_{0})\\ & \leq & \sup_{u\geq 0 } j(v_{n-(n-1)},g)(\pi , u)\\ & = & l(v_{n-(n-1)},g)(\pi ) \ = \
v_{n - n}(\pi),\end{aligned}\ ] ] that proves the induction .
@xmath175 +   + we now prove the reverse inequality by constructing a sequence of @xmath0-optimal stopping times .
[ def - sn - epsilon]for @xmath318 , @xmath272 and for @xmath319 , we define @xmath320 consider @xmath321 and for @xmath284 , @xmath322 and finally set @xmath323    the following lemma describes the effect of the translation operator @xmath299 on the sequence @xmath324 .
[ rnk - epsilon - theta ] for @xmath325 and @xmath326 , on the set @xmath327 , one has @xmath328    [ [ proof-7 ] ] proof + + + + +    for @xmath329 , one just has to prove that on the event @xmath330 , one has @xmath331 . yet , from the definition of the sequence @xmath324 , one has @xmath332 and @xmath333 .
the result follows since we are on the event @xmath334 . for a fixed @xmath335
, we prove the lemma by induction on @xmath326 .
set @xmath336 .
one has from the definition on the sequence @xmath324 , @xmath337 and @xmath338 .
we obtain @xmath339 because we have assumed that we are on the event @xmath340 .
the propagation of the induction is similar to the case @xmath336 .
@xmath175 +   + equipped with this preliminary result , we may now prove that @xmath341 is a sequence of @xmath0-optimal stopping times with respect to the filtration .
generated by the observations .
[ theo - sn - epsilon ] for all @xmath272 and @xmath318 , one has @xmath342 and @xmath343\geq v_{n - n}(\pi)-\epsilon.\ ] ]    [ [ proof-8 ] ] proof
+ + + + +    let @xmath344 .
first notice that , as a direct consequence of proposition [ propb5 ] , @xmath345 is an @xmath29-stopping time since , by construction , the @xmath346 are @xmath347-measurable and satisfy the condition @xmath348 on the event @xmath349 .
it is also clear that @xmath350 .
thus , one has @xmath342 .
let us now prove the second assessment by induction . set @xmath277 .
let @xmath319 , we denote @xmath351 .
since @xmath352 is deterministic , one has clearly @xmath353 .
consequently , by using the same arguments as in the proof of proposition [ theo - value - fonction ] , we obtain @xmath354= & hg(\pi , r_{0}^{\epsilon})+gv_{n}(\pi , r_{0}^{\epsilon})=j(v_{n},g)(\pi , r_{0}^{\epsilon}).\end{aligned}\ ] ] finally , the definition of @xmath355 yields @xmath356 thus one has @xmath357\geq v_{n-1}(\pi)-\epsilon.\ ] ] now set @xmath284 and assume that @xmath358\geq v_{n-(n-1)}(\pi)-\epsilon$ ] , for all @xmath318 .
proposition [ lemme value fonction ] yields @xmath359}\\ & = & \sum_{k=0}^{n-1}\sum_{i=1}^{q}{\mathbf{e}}\left[{\mathbbm{1}}_{\{t_{k}\leq u_{n}^{2\epsilon}\}}{\mathbbm{1}}_{\{r_{n , k}^{2\epsilon}<{t^{*}}_{i}\}}g\circ\phi(x_{i},r_{n , k}^{2\epsilon})e^{-\lambda(x_{i},r_{n , k}^{2\epsilon})}\pi_{k}^{i}\big|\pi_0=\pi\right]\\ & & + \sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{n}\leq u_{n}^{2\epsilon}\}}g(x_{i})\pi_{n}^{i}|\pi_0=\pi].\end{aligned}\ ] ] denote @xmath360 . as in the case @xmath277 ,
the term for @xmath288 equals @xmath361 since @xmath362 .
take the conditional expectation w.r.t . @xmath292 in the other terms .
one has then , @xmath363=   hg(\pi , r_{n-1}^{\epsilon})+{\mathbf{e}}[\xi'{\mathbbm{1}}_{\{t_{1}\leq u_{n}^{2\epsilon}\}}|\pi_0=\pi],\ ] ] with @xmath364.\end{aligned}\ ] ] our objective is to apply the markov property of @xmath365 in the term @xmath366 . recall that , from lemma [ rnk - epsilon - theta ] , one has @xmath367 for @xmath325 and @xmath326 on the event @xmath368 ( the equality of these events stems from lemma  [ lemme - tech - event ] ) .
thus , on this set one has @xmath369 besides , recall that @xmath370 , for @xmath371 .
consequently , on the set @xmath327 , one has @xmath372 and thus , combining the markov property of the chain @xmath297 and proposition [ lemme value fonction ] , we have @xmath373 with @xmath374 $ ] .
moreover , thanks to the induction assumption , one has @xmath375 so that one obtains @xmath376 finally , combining equation   and   and noticing that , according to lemma  [ lemme - tech - event ] , @xmath377 , one obtains @xmath378 } & \geq & hg(\pi , r_{n-1}^{\epsilon})+{\mathbf{e}}[v_{n-(n-1)}(\pi_{1}){\mathbbm{1}}_{\{s_{1}\leq r_{n-1}^{\epsilon}\}}|\pi_0=\pi]-\epsilon\\ & = & j(v_{n-(n-1)},g)(\pi , r_{n-1}^{\epsilon})-\epsilon\\ & \geq & v_{n - n}(\pi)-2\epsilon,\end{aligned}\ ] ] from the definition of @xmath379 , showing the result .
in this section , we are interested in the computational issue for our optimal stopping problem under partial observation .
indeed , we want to compute a numerical approximation of the value function ( [ opt - stop - pb ] ) and propose a computable @xmath0-optimal stopping time .
as we have seen in the previous section , the value function @xmath380 can be obtained by iterating the dynamic programming operator @xmath233 .
however , the operator @xmath233 involves conditional expectations that are in essence difficult to compute and iterate numerically . we manage to overcome this difficulty by combining two special properties of our problem . on the one hand , the underlying process @xmath381 in the expression of the operator
@xmath233 is a markov chain .
therefore , it can be discretized using a quantization technique which is a powerful method suitable for numerical computation and iteration of conditional expectations . on the other hand ,
the recursion on the functions @xmath382 involving the operator @xmath233 can be transformed into a recursion on suitably defined random variables .
thus they are easier to iterate numerically as we do not need to compute an approximation of each @xmath383 on the whole state space .
this section is organized as follows .
we first explain how the recursion on the functions @xmath382 can be transformed into a recurrence on random variables involving only the markov chain @xmath381 .
then , we present a quantization technique to discretize this markov chain . afterwards , we construct a discretized version of the main operators in definition  [ def op chapeau ] that is used to build an approximation of the value function in definition  [ def v chap ] , and a computable @xmath0-optimal stopping time .
the main results of this section are theorems  [ theo - conv ] and [ th arret chap ] that prove the convergence of our approximation scheme and provide a rate of convergence .
we first explain how the dynamic programming equations on the functions @xmath382 yield a recursion on the random variables @xmath384 .
introduce now the sequence @xmath385 of random variables defined by @xmath386 in other words , one has @xmath387,\nonumber\end{aligned}\ ] ] for @xmath388 .
notice that @xmath389 is known and the expression of @xmath390 involves only @xmath391 and the markov chain @xmath381 .
thus , the sequence @xmath392 is completely characterized by the system  ( [ def vn ] ) .
in addition , @xmath393 . thus to approximate the value function @xmath380 at the initial point of our process ,
it is sufficient to provide an approximation of the sequence of random variables @xmath394 .
there exists an extensive literature on quantization methods for random variables and processes .
we do not pretend to present here an exhaustive panorama of these methods .
however , the interested reader may for instance , consult the following works @xcite and references therein .
consider @xmath32 an @xmath395-valued random variable such that @xmath396 where @xmath397 denotes the @xmath34-nom of @xmath32 : @xmath398)^{1/p}$ ] .
let @xmath399 be a fixed integer , the optimal @xmath34-quantization of the random variable @xmath32 consists in finding the best possible @xmath34-approximation of @xmath32 by a random vector @xmath33 taking at most @xmath399 values : @xmath400 .
this procedure consists in the following two steps :    1 .
find a finite weighted grid @xmath401 with @xmath402 .
2 .   set @xmath403 where @xmath404 with @xmath405 denotes the closest neighbour projection on @xmath406 .
the asymptotic properties of the @xmath34-quantization are given by the following result , see e.g. @xcite .
[ theore ] if @xmath407<+\infty$ ] for some @xmath408 then one has @xmath409 where the distribution of @xmath32 is @xmath410 with @xmath411 , @xmath412 a constant and @xmath413 the lebesgue measure in @xmath414 .
there exists a similar procedure for the optimal quantization of a markov chain .
our approximation method is based on the quantization of the markov chain @xmath415 .
thus , from now on , we will denote , for @xmath416 , @xmath417 .
the clvq ( competitive learning vector quantization ) algorithm ( * ? ? ?
* section 3 ) provides for each time step @xmath416 a finite grid @xmath418 of @xmath419 as well as the transition matrices @xmath420 from @xmath418 to @xmath421 .
let  @xmath422 such that for all @xmath423 , @xmath424 and @xmath425 have finite moments at least up to order  @xmath426 and let @xmath427 be the nearest - neighbor projection from @xmath419 onto @xmath418 .
the quantized process @xmath428 with value for each @xmath429 in the finite grid @xmath418 of @xmath419 is then defined by @xmath430 we will also denote by @xmath431 , the projection of @xmath418 on @xmath110 , and by @xmath432 , the projection of @xmath418 on @xmath433 .
some important remarks must be made concerning the quantization .
on the one hand , the optimal quantization has nice convergence properties stated by theorem [ theore ] .
indeed , the @xmath434-quantization error @xmath435 goes to zero when the number of points in the grids goes to infinity .
however , on the other hand , the markov property is not maintained by the algorithm and the quantized process is generally not markovian .
although the quantized process can be easily transformed into a markov chain , this chain will not be homogeneous .
it must be pointed out that the quantized process @xmath436 depends on the starting point @xmath437 of the process .    in practice
, we begin with the computation of the quantization grids , which merely requires to be able to simulate the process .
notice that in our case , what is actually simulated is the sequence of observation @xmath438 .
we are then able to compute the filter @xmath439 thanks to the recursive equation provided by proposition [ prop - rec - filtre ] .
the grids are only computed once and for all and may be stored off - line .
our schemes are then based on the following simple idea : we replace the process by its quantized approximation within the different recursions .
the computation is thus carried out in a very simple way since the quantized process has finite state space .
our approximation scheme of the sequence @xmath385 follows the same lines as in @xcite , but once more , the results therein can not be applied directly as the markov chain @xmath440 is not the underlying markov chain of some pdmp .
our approach decomposes in two steps .
the first one will be to discretize the time - continuous maximization of the operator @xmath233 to obtain a maximization over a finite set .
the second step consists in replacing the markov chain @xmath441 by its quantized approximation @xmath442 within the dynamic programming equation .
thus , the conditional expectations will become easily tractable finite sums .
let us first build a finite time grid to discretize the continuous - time maximization in the expression of the operator @xmath233 .
the maximum is originally taken over the set @xmath443 .
however , it can be seen from definition  [ def - g - h - i - j - l ] that @xmath444 for all @xmath445 . indeed , the random variable @xmath446 is bounded by the greatest deterministic exit time @xmath447 that is finite thanks to assumption  [ hyp - ts - bounded ] .
therefore , the maximization set can be reduced to the compact set @xmath448 $ ] . instead of directly discretizing the set @xmath448 $ ] , we will actually discretize the subsets @xmath207t^*_m , t^*_{m+1}[$ ] .
the reason why we want to exclude the points @xmath449 from our grid is technical and will be explained with lemma  [ xi3-indicator ] .
now , it seems natural to distinguish wether @xmath450 or @xmath451 .
let @xmath452 be the set of indices @xmath453 such that @xmath451 .
notice that @xmath454 is not empty because it contains at least the index 0 since we assumed that @xmath455 .
we can now build our approximation grid .
let @xmath456 be such that @xmath457 for all @xmath458 , let @xmath459 be the finite grid on @xmath207{t^{*}}_{m};{t^{*}}_{m+1}[$ ] defined as follows @xmath460 where @xmath461 .
we also denote @xmath462    [ rq - grm]let @xmath458 . notice that , thanks to eq .  , @xmath459 is not empty .
moreover , it satisfies two properties that will be crucial in the sequel :    a. : :    for all @xmath463 $ ] , there exists    @xmath464 such that    @xmath465 , b. : :    for all @xmath464 and    @xmath466 , one has    @xmath467\subset ] { t^{*}}_{m};{t^{*}}_{m+1}[$ ] .    a discretized maximization operator @xmath468 is then defined as follows .    [ def k ]
let @xmath468 : @xmath469 be defined for all @xmath319 by @xmath470 with @xmath471.$ ]    we now proceed to our second step : replacing the markov chain @xmath441 by its quantized approximation @xmath442 within the operators involved in the construction of the value function .
[ def op chapeau ] we define the _ quantized operators _ @xmath472 , @xmath473 , @xmath474 , @xmath475 and @xmath476 for @xmath477 , @xmath478 , @xmath479 , @xmath480 and @xmath481 as follows @xmath482,\\ \widehat{h}_{n}h(\pi , u)=&\hspace{0.3cm}\sum_{i=1}^{q}\pi^{i}{\mathbbm{1}}_{\{u <
{ t^{*}}_{i}\}}h\circ\phi(x_{i},u){\mathbf{e}}[{\mathbbm{1}}_{\{\widehat{s}_{n } > u\}}| \widehat \pi_{n-1}=\pi],\\ \widehat{j}_{n}(v , h)(\pi , u)=&\hspace{0.3cm}\widehat{h}_{n}h(\pi , u)+\widehat{g}_{n}v(\pi , u),\\ \widehat{k}_{n}v(\pi)=&\hspace{0.3cm}\widehat{j}_{n}(v , h)(\pi , t^*_q)=\hspace{0.3cm}{\mathbf{e}}[v(\widehat{\pi}_{n})| \widehat \pi_{n-1}=\pi],\\ \widehat l^{d}_{n}(v , h)(\pi)=&\hspace{0.3cm}\max_{m\in m}\big\{\max_{u\in gr_{m}(\delta)}\ { \widehat j_{n}(v , h)(\pi , u)\}\big\}\vee \widehat k_{n}v(\pi).\end{aligned}\ ] ]    the quantized approximation of the value functions naturally follows .
[ def v chap ] for @xmath483 , define the functions @xmath484 on @xmath485 as follows @xmath486 for @xmath483 , let @xmath487    we may now state our main result for the numerical approximation .
[ theo - conv ] suppose that for all @xmath388 , @xmath488 then , one has the following bound for the approximation error @xmath489\|\pi_{n+1}-\widehat \pi_{n+1}\|_{p},\end{aligned}\ ] ] where @xmath490_{2}+2c_{g}c_{\lambda}$ ] , @xmath491 and @xmath492 + 4c_{g}+2[v_{n+1}]$ ] with @xmath493 $ ] , @xmath494 $ ] defined in proposition [ prop - lip - v ] and @xmath147_{2}$ ] defined in assumption [ hyp - g - lip ] .
theorem [ theo - conv ] establishes the convergence of our approximation scheme and provides a bound for the rate of convergence .
more precisely , it gives a rate for the @xmath34 convergence of @xmath495 towards @xmath496 .
indeed , one has @xmath497 , so by virtue of theorem [ theo - conv ] @xmath498 can be made arbitrarily small when the quantization errors @xmath499 go to zero i.e. when the number of points in the quantization grids goes to infinity . in order to prove theorem [ theo - conv ]
, we proceed similarly to @xcite and split the approximation error into four terms @xmath500 , with @xmath501 to obtain bounds for each of these terms , one needs to study the regularity of the operators and the value functions @xmath383 .
the results are detailed in appendix  [ apx lip ] .
in particular , we establish in proposition [ prop - lip - v ] that the value functions @xmath383 are lipschitz continuous , yielding a bound for the first term .
the first term @xmath502 is bounded as follows @xmath503\|\pi_{n}-\widehat \pi_{n}\|_{p}.\ ] ]    the other error terms are studied separately in the following sections .      for the second error term , we investigate the consequences of replacing the continuous maximization in operator @xmath233 by a discrete one on @xmath504 .    for all @xmath458 , @xmath505 and @xmath319 one has @xmath506_{2}+c_{g}c_{\lambda}+c_{v}c_{\lambda}\right)\delta.\ ] ]    [ [ proof-9 ] ] proof
+ + + + +    we use definition  [ def - op - j - l ] to split operator @xmath507 into a sum of continuous operators @xmath508 .
thus , one has @xmath509 } j^m(v , g)(\pi , u).\ ] ] the function @xmath510 being continuous , there exists @xmath511 $ ] such that @xmath512 } j^{m}(v , h)(\pi , u)=j^{m}(v , h)(\pi,\overline{t})$ ] .
moreover , from remark [ rq - grm].a , one may chose @xmath513 so that @xmath514 .
propositions [ prop - lip - h ] and [ prop - lip - g ] stating the lipschitz continuity of @xmath515 then yield @xmath516 } j^{m}(v , h)(\pi , u)-\max_{u\in gr_{m}(\delta ) } j^{m}(v , h)(\pi , u)\\ & \leq j^{m}(v , h)(\pi,\overline{t})-j^{m}(v , h)(\pi,\overline{u})\\ & \leq \left([g]_{2}+c_{g}c_{\lambda}+c_{v}c_{\lambda}\right)|\overline{t}-\overline{u}|\leq \left([g]_{2}+c_{g}c_{\lambda}+c_{v}c_{\lambda}\right)\delta,\end{aligned}\ ] ] showing the result . @xmath175 +    the second term @xmath517 is bounded as follows @xmath518_{2}+2c_{g}c_{\lambda}\right)\delta.\ ] ]    [ [ proof-10 ] ] proof + + + + +    this is a straightforward consequence of the previous lemma once it has been noticed that for all @xmath519 , @xmath520 , @xmath521 , @xmath522 , one has @xmath523 .
notice also that proposition [ prop - lip - v ] provides @xmath524 .
@xmath175 +      to investigate the third error term , we use the properties of quantization to bound the error made by replacing an operator by its quantized approximation . as in @xcite
, we must first deal with non - continuous indicator functions .
the fact that the @xmath449 and a small neighborhood around them do not belong to the discretization grid @xmath504 is crucial to obtain the following lemma .
[ xi3-indicator ] for all @xmath388 , @xmath458 and @xmath466 , one has @xmath525\big\|_{p }
\leq \eta^{-1}{\|s_{n+1}-\widehat s_{n+1}\|_{p}}+2\eta c_{\lambda}.\ ] ]    [ [ proof-11 ] ] proof + + + + +    let @xmath466 .
the difference of the indicator functions equals 1 if and only if @xmath526 and @xmath527 are on different sides of @xmath528 .
therefore , if the difference of the indicator functions equals 1 , either @xmath529 , or @xmath530 and in the latter case @xmath531 too since @xmath532 .
one has @xmath533 leading to @xmath534\big\|_{p}\\ \leq \|{\mathbbm{1}}_{\{|s_{n+1}-\widehat s_{n+1}| > \eta\}}\|_{p } + \big\|\max_{u\in gr_{m}(\delta)}{\mathbf{e } } [ { \mathbbm{1}}_{\{|s_{n+1}-u|\leq \eta\ } } |\widehat \pi_{n}]\big\|_{p}.\end{gathered}\ ] ] on the one hand , markov inequality yields @xmath535 on the other hand , since @xmath464 , one has @xmath467\subset ] { t^{*}}_{m};{t^{*}}_{m+1}[$ ] from remark [ rq - grm].b , thus @xmath526 has an absolutely continuous distribution on the interval @xmath467 $ ] since it does not contain any of the @xmath536 . besides
, recall that @xmath537 , hence , the following inclusions of @xmath23-fields @xmath538 .
we also have @xmath539 , the law of iterated conditional expectations provides @xmath540 & = & { \mathbf{e}}\big[{\mathbf{e}}\big[{\mathbf{e } } [ { \mathbbm{1}}_{\{|s_{n+1}-u|\leq \eta\ } } | { \mathfrak{f}}_{t_{n}}]\big| { \mathfrak{f}^{y}}_{t_{n}}\big]\big|\widehat \pi_{n}\big]\\ & \leq & { \mathbf{e}}\big[{\mathbf{e } } [ \int_{u-\eta}^{u+\eta}\lambda\big(\phi(z_{n},s)\big)ds\big| { \mathfrak{f}^{y}}_{t_{n}}]\big|\widehat \pi_{n}\big]\\ & = & { \mathbf{e}}\big [ \sum_{i=1}^{q } \pi_{n}^{i}\int_{u-\eta}^{u+\eta}\lambda\big(\phi(x_{i},s)\big)ds\big|\widehat \pi_{n}\big].\end{aligned}\ ] ] finally , one obtains @xmath541\leq 2\eta c_{\lambda},$ ] showing the result .
@xmath175 +    [ xi3-k ] for all @xmath388 , one has @xmath542{\mathbf{e}}\big[|\pi_{n+1}-\widehat
\pi_{n+1}|\big|\widehat \pi_{n}\big]+(2c_{g}+2[v_{n+1}]){\mathbf{e}}\left[|\pi_{n}-\widehat \pi_{n}|\big|\widehat \pi_{n}\right].\end{gathered}\ ] ]    [ [ proof-12 ] ] proof + + + + +    by the definitions of operators @xmath543 and @xmath544 , one has @xmath545-{\mathbf{e}}[v_{n+1}(\widehat
\pi_{n+1})| \widehat \pi_{n } ] |\nonumber\\ & \leq&{|{\mathbf{e}}[v_{n+1}(\pi_{n+1})| \pi_{n}=\widehat \pi_{n } ] -{\mathbf{e}}[v_{n+1}(\pi_{n+1})| \widehat \pi_{n } ] |}\nonumber\\ & & + { |{\mathbf{e}}[v_{n+1}(\pi_{n+1})-v_{n+1}(\widehat \pi_{n+1})| \widehat \pi_{n } ] |}.\label{eq dif k}\end{aligned}\ ] ] the second term in the right - hand side of eq . (
[ eq dif k ] ) is readily bounded by using proposition [ prop - lip - v ] stating that @xmath546 is lipschitz continuous @xmath547|\leq[v_{n+1}]{\mathbf{e}}\big[|\pi_{n+1}-\widehat \pi_{n+1}|\big|\widehat \pi_{n}\big].\ ] ] to deal with the first term in the right - hand side of eq .
( [ eq dif k ] ) , we need to use the special properties of quantization . indeed , one has @xmath548 so that we have the inclusion of @xmath23-fields @xmath549 .
the law of iterated conditional expectations gives @xmath550={\mathbf{e}}\big[{\mathbf{e}}[v_{n+1}(\pi_{n+1})|(\pi_{n},s_{n})]\big| \widehat \pi_{n } \big].\ ] ] moreover , proposition [ prop - markov ] yields @xmath551={\mathbf{e}}[v_{n+1}(\pi_{n+1})|\pi_{n}]$ ] , as the conditional distribution of @xmath552 w.r.t .
@xmath553 merely depends on @xmath31 .
in addition , @xmath554 $ ] is @xmath555-measurable .
one has then @xmath556-{\mathbf{e}}[v_{n+1}(\pi_{n+1})| \widehat \pi_{n } ] |}\\ & = & \left|{\mathbf{e}}\big[{\mathbf{e}}[v_{n+1}(\pi_{n+1})| \pi_{n}=\widehat \pi_{n } ] -{\mathbf{e}}[v_{n+1}(\pi_{n+1})| \pi_{n } ] \big|\widehat \pi_{n}\big]\right|\\ & = & |{\mathbf{e}}[kv_{n+1}(\widehat \pi_{n})-kv_{n+1}(\pi_{n})|\widehat \pi_{n}]|,\end{aligned}\ ] ] by definition of @xmath543 .
finally , one has @xmath557-{\mathbf{e}}[v_{n+1}(\pi_{n+1})| \widehat \pi_{n } ] |}}\\ & \leq&2(c_g+[v_{n+1}]){\mathbf{e}}\left[|\pi_{n}-\widehat \pi_{n}|\big|\widehat \pi_{n}\right],\end{aligned}\ ] ] thanks to propositions [ prop - lip - k ] and [ prop - lip - v ] stating the lipschitz continuity of operator @xmath543 and function @xmath546 .
@xmath175 +    [ lemme - xi3 ] if @xmath558 satisfies condition ( [ conditiondelta ] ) , a upper bound for the third term @xmath559 is @xmath560\|\pi_{n+1}-\widehat \pi_{n+1}\|_{p}+(4c_{g}+2[v_{n+1}])\|\pi_{n}-\widehat \pi_{n}\|_{p}\\ & & + 2c_{g } ( 2 c_{\lambda})^{1/2}{\|s_{n+1}-\widehat s_{n+1}\|_{p}}^{1/2}.\end{aligned}\ ] ]    [ [ proof-13 ] ] proof + + + + +    one has @xmath561 the term involving operator @xmath543 was studied in the previous lemma .
let us now study the term involving operator @xmath507 .
set @xmath453 in @xmath454 , @xmath528 in @xmath459 and define @xmath562 .
one has then @xmath563-{\mathbf{e}}[\alpha(\widehat \pi_{n},\widehat \pi_{n+1},\widehat s_{n+1})|\widehat \pi_{n}]\right|\leq a+b,\end{aligned}\ ] ] where@xmath564-{\mathbf{e}}[\alpha(\pi_{n},\pi_{n+1},s_{n+1})|\widehat\pi_{n}]\right|,\\
b=&|{\mathbf{e}}[\alpha(\pi_{n},\pi_{n+1},s_{n+1})-\alpha(\widehat \pi_{n},\widehat \pi_{n+1},\widehat s_{n+1})\big|\widehat \pi_{n}]| . \ ] ] using the boundedness of @xmath18 and @xmath546 as well as the lipschitz continuity of @xmath546 given in proposition [ prop - lip - v ] , we get a upper bound for the second term @xmath565 + [ v_{n+1}]{\mathbf{e}}\big[|\pi_{n+1}-\widehat \pi_{n+1}|\big|\widehat \pi_{n}\big]\nonumber\\ & & + 2c_{g}{\mathbf{e}}\left[|{\mathbbm{1}}_{\{s_{n+1}\leq u\}}-{\mathbbm{1}}_{\{\widehat s_{n+1}\leq u\}}|\big|\widehat \pi_{n}\right].\end{aligned}\ ] ] for the first term , we use the properties of quantization as in the previous proof to obtain @xmath566-{\mathbf{e}}[\alpha(\pi_{n},\pi_{n+1},s_{n+1})|\pi_{n}]\big| \widehat \pi_{n}\big]\right|.\end{aligned}\ ] ] we now recognize operator @xmath515 , and from propositions [ prop - lip - h ] and [ prop - lip - g ] , one has @xmath567\nonumber\\ & \leq & ( 3c_{g}+2[v_{n+1}]){\mathbf{e}}\left[|\widehat{\pi}_{n}-\pi_{n}\big|\widehat \pi_{n}\right].\end{aligned}\ ] ] we gather the bounds provided by eq .   and to obtain @xmath568){\mathbf{e}}\big[|\pi_{n}-\widehat
\pi_{n}\big ] + [ v_{n+1}]{\mathbf{e}}\big[|\pi_{n+1}-\widehat \pi_{n+1}|\big|\widehat \pi_{n}\big]\nonumber\\ & & + 2c_{g}{\mathbf{e}}\left[|{\mathbbm{1}}_{\{s_{n+1}\leq u\}}-{\mathbbm{1}}_{\{\widehat s_{n+1}\leq u\}}|\big|\widehat \pi_{n}\right].\end{aligned}\ ] ] finally , combining the result for operators @xmath507 and lemma  [ xi3-k ] , we obtain @xmath569{\mathbf{e}}\left[|\pi_{n+1}-\widehat \pi_{n+1}|\big|\widehat \pi_{n}\right]+(4c_{g}+2[v_{n+1}]){\mathbf{e}}\left[|\pi_{n}-\widehat \pi_{n}|\big|\widehat \pi_{n}\right]\\ & & + 2c_{g}\max_{u\in gr(\delta)}{\mathbf{e}}\left[|{\mathbbm{1}}_{\{s_{n+1}\leq u\}}-{\mathbbm{1}}_{\{\widehat s_{n+1}\leq u\}}|\big|\widehat \pi_{n}\right].\end{aligned}\ ] ] we conclude by taking the @xmath34 norm in the equation above and using lemma [ xi3-indicator ] to bound the last term @xmath570\|\pi_{n+1}-\widehat \pi_{n+1}\|_p+(4c_{g}+2[v_{n+1}])\|\pi_{n}-\widehat \pi_{n}\|_p\\ & & + 2c_{g}(\eta^{-1}{\|s_{n+1}-\widehat s_{n+1}\|_{p}}+2\eta c_{\lambda}),\end{aligned}\ ] ] for some @xmath466 .
the best choice for @xmath571 minimizing the error is when @xmath571 satisfies @xmath572 which yields @xmath573 .
if @xmath558 satisfies condition ( [ conditiondelta ] ) , one has @xmath574 as required for this optimal choice .
@xmath175 +      finally , the fourth error term is bounded using lipschitz properties .
[ lemme - xi4 ] the fourth term @xmath575 is bounded as follows @xmath576\|\pi_{n+1}-\widehat \pi_{n+1}\|_{p}+\|v_{n+1}-\widehat v_{n+1}\|_{p}.\end{aligned}\ ] ]    [ [ proof-14 ] ] proof + + + + +    one has @xmath577\nonumber\\ & & \vee { \mathbf{e}}[v_{n+1}(\widehat{\pi}_{n+1})-\widehat v_{n+1}(\widehat{\pi}_{n+1})| \widehat{\pi}_{n } ] \big\|_{p}\nonumber\\ & \leq&\|v_{n+1}(\widehat{\pi}_{n+1})-\widehat v_{n+1}(\widehat{\pi}_{n+1})\|_{p}.\label{jvchap}\end{aligned}\ ] ] we now introduce @xmath578 to split this term into two differences .
the lipschitz continuity of @xmath546 stated by proposition [ prop - lip - v ] allows us to bound the first term while we recognize @xmath391 and @xmath579 in the second one .
@xmath580\left\|\pi_{n+1}-\widehat \pi_{n+1}\right\|_{p}+\|v_{n+1}-\widehat v_{n+1}\|_{p}.\end{aligned}\ ] ] hence , the result .
@xmath175 +      as in the previous section , we follow the idea of @xcite and we use both the markov chain @xmath581 and its quantized approximation @xmath582 to approximate the expression of the @xmath0-optimal stopping time introduced in definition [ def - sn - epsilon ] .
we check that we thus obtain actual stopping times for the observed filtration @xmath29 and that the expected reward when stopping then is a good approximation of the value function @xmath496 . for all @xmath583 and @xmath483
, we denote @xmath584 .
let @xmath585 for @xmath272 and @xmath319 , we define @xmath586 let now for @xmath120 , @xmath587 and set @xmath588 the following result is a direct consequence of proposition [ propb5 ] . it is a very strong result as it states that the numerically computable random variables @xmath589 are actual @xmath29-stopping times .    for @xmath483
, @xmath589 is an @xmath29-stopping time .
we now intend to prove that stopping at time @xmath590 provides a good approximation of the value function @xmath496 . for all @xmath591 and @xmath483
we therefore introduce the performance when abiding by the stopping rule  @xmath592 and the corresponding random variables @xmath593,\qquad \overline{v}_{n}=\overline{v}_{n}(\pi_{n}).\ ] ]    [ th arret chap ] suppose that for all @xmath388 , @xmath594 one has then the following bound for the error between the expected reward when stopping at time @xmath589 and the value function @xmath595\|\pi_{n+1}-\widehat \pi_{n+1}\|_{p}\\ & & + b\|s_{n+1}-\widehat s_{n+1}\|_{p}^{1/2},\end{aligned}\ ] ] where @xmath596 , @xmath597 $ ] , @xmath494 $ ] defined in proposition  [ prop - lip - v ] .
it is important to notice that @xmath598 and thus @xmath599 .
therefore , the previous theorem proves that @xmath600 goes to zero when the quantization errors @xmath601 go to zero . in other words , the expected reward @xmath602 when stopping at the random time @xmath590
can be made arbitrarily close to the value function @xmath496 of the partially observed optimal stopping problem and hence @xmath590 is an @xmath0-optimal stopping time .
[ [ proof-15 ] ] proof + + + + +    the first step consists in finding a recursion satisfied by the sequence @xmath603 in order to compare it with the dynamic programming equation giving @xmath604 .
let @xmath388 .
first of all , proposition  [ lemme value fonction ] gives @xmath605}\\ & = & \sum_{k=0}^{n - n-1}\sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{k}\leq \widehat{u}_{n - n}\}}{\mathbbm{1}}_{\{\widehat{r}_{n - n ,
k}<{t^{*}}_{i}\}}g\circ\phi(x_{i},\widehat{r}_{n - n , k})e^{-\lambda(x_{i},\widehat{r}_{n - n , k})}\pi_{k}^{i}|\pi_0]\\ & & + \sum_{i=1}^{q}{\mathbf{e}}[{\mathbbm{1}}_{\{t_{n - n}\leq \widehat{u}_{n - n}\}}g(x_{i})\pi_{n}^{i}|\pi_0].\end{aligned}\ ] ] the term corresponding to @xmath288 in the above sum equals @xmath606 . taking the conditional expectation w.r.t . @xmath292 in the other terms and noticing that one has @xmath607 yield @xmath608=hg(\pi_{0},\widehat{r}_{n - n,0})+{\mathbf{e}}[\xi''{\mathbbm{1}}_{\{s_{1}\leq \widehat{r}_{n - n,0}\}}|\pi_0],\ ] ] with @xmath609.\end{aligned}\ ] ] we now make use of the markov property of the sequence @xmath151 in the term @xmath610 . similarly to lemma [ rnk - epsilon - theta ] , for @xmath120 , on the set @xmath611 , one has @xmath612 for all @xmath326 .
thus , on the set @xmath611 , one has @xmath613 .
recall that @xmath614 .
we may therefore apply the markov property . using proposition [ lemme value fonction ] ,
we now obtain @xmath615 . finally , we have @xmath616 recall that @xmath617 and apply the translation operator @xmath618 to obtain the following recursion @xmath619 we are now able to study the error between @xmath620 and @xmath621 .
let us recall that , from its definition , @xmath622 equals either @xmath623 or @xmath624 . in the latter case ,
notice that @xmath625 .
eventually , one has @xmath626 to bound the first term @xmath47 , we introduce the function @xmath546 .
one has @xmath627 let us study these four terms one by one . by definition of @xmath543 , the first term @xmath628
is bounded by @xmath629 $ ] . for the second term @xmath630
, we use proposition [ prop - lip - k ] stating the lipschitz continuity of the operator @xmath543 .
the term third term @xmath631 is bounded by lemma [ xi3-k ] and a upper bound of the fourth term @xmath632 is given by eq .
( [ jvchap ] ) .
thus , one obtains @xmath633)\|\pi_{n}-\widehat \pi_{n}\|_{p}\\ & & + 2[v_{n+1}]\|\pi_{n+1}-\widehat \pi_{n+1}\|_{p}.   \end{aligned}\ ] ] we now turn to the second term @xmath634 . in the following computations , denote @xmath635 . its definition yields
we split this expression into four differences again . on the set @xmath637
, one has the equality @xmath638 .
hence , one this set , one obtains from eq .
( [ prog dyn vbar ] ) @xmath639 for the other terms , we use propositions [ prop - lip - h ] and [ prop - lip - g ] for the lipschitz continuity of @xmath507 and eq .
( [ eq j - jchap ] ) and ( [ jvchap ] ) to obtain @xmath640)\|\pi_{n}-\widehat \pi_{n}\|_{p}+2[v_{n+1}]\|\pi_{n+1}-\widehat \pi_{n+1}\|_{p}\\ & & + 2c_g(2c_{\lambda})^{1/2}\|s_{n+1}-\widehat s_{n+1}\|_{p}^{1/2 } ,   \end{aligned}\ ] ] after optimizing @xmath571 .
the result is obtained by taking the maximum between @xmath47 and @xmath634 .
we apply our procedure to a simple pdmp similar to the one studied in  @xcite .
let @xmath641 . for @xmath65 and @xmath642 ,
the flow is defined by @xmath643 so that @xmath644 .
we set the jump rate to @xmath645 for some @xmath646 and the transition kernel @xmath647 to the uniform distribution on a finite set @xmath648 .
thus , the process evolves toward 1 and the closer it gets to 1 , the more likely it will jump back to some point of @xmath93 .
a trajectory is represented in figure [ plotecg ] .
jump time with @xmath649 , @xmath650 and @xmath651 .
the dotted lines represent the possible post - jump values.,width=302 ]    the observation process is @xmath652 where @xmath653 and @xmath654 for some @xmath655 .
finally , we choose the reward function @xmath656 .
our assumptions thus clearly hold .
simulations are run with @xmath649 , @xmath650 , @xmath657 , @xmath658 and @xmath659 .
the numerical approximation is implemented as follows .
first , we make an exact simulator for the sequence @xmath660 . from the values of @xmath661 ,
one builds the observation sequence @xmath662 that allows for a recursive computation of the filter process thanks to proposition [ prop - rec - filtre ] .
thus , we can simulate trajectories of the markov chain @xmath381 that we feed into the clvq algorithm to obtain quantization grids . by monte carlo simulations , we can also estimate the quantization errors . to run our numerical procedure ,
one then needs to choose the parameter @xmath558 satisfying conditions ( [ eq - condition - delta ] ) and ( [ conditiondelta ] ) . in this special case ,
they boil down to @xmath663 we have chosen @xmath558 just above the monte carlo approximation of the lower bound .
the values are given in the second column of table [ ecg_table ] for different grids sizes .
then , we recursively compute the approximated value functions @xmath664 on the quantization grids .
the conditional expectations are now merely weighted sums .
the approximation we obtain for the value function of the partially observed optimal stopping problem are given in the fourth column of table  [ ecg_table ] .
finally , we implemented the construction of our @xmath0-optimal stopping time and ran @xmath665 monte carlo simulations to compute its mean performance .
the results are given in the third column of table  [ ecg_table ] .
the exact value of @xmath496 is unknown but one has as in @xcite , @xmath666 \leq v_{0}=\sup_{\sigma\in\sigma_{n}^{y}}{\mathbf{e}}[g(x_{\sigma } ) ] \leq { \mathbf{e}}\big [ \sup_{0\leq t \leq t_{n}}g(x_{t})\big].\ ] ] both the first and the last term may be estimated by monte carlo simulations .
one has thus , with @xmath667 trajectories , @xmath668=0.9944 $ ] .
the theoretical bound @xmath669 of the error @xmath670 provided by theorem [ theo - conv ] is computed using the approximated quantization errors .
this bound decreases as the number of points in the quantization grids increases , as expected .
moreover , we computed the empirical bound given by eq .   @xmath671-\widehat v_{0}| \big\}$ ] .    .simulation results .
the terms @xmath672 and @xmath669 respectively denote an empirical bound and the theoretical bound provided by theorem [ theo - conv ] for the error @xmath670 . [ cols="^,^,^,^,^,^",options="header " , ]
in this section , we study the special structure of @xmath29-stopping times .
[ prop - tn - stop - time ] for all @xmath673 , @xmath138 is an @xmath29-stopping time .
[ [ proof-16 ] ] proof + + + + +    notice that for all @xmath92 , @xmath674 .
this stems from the absolute continuity of the distribution of the random variables @xmath186 since @xmath675 hence , for all @xmath92 and @xmath676 , one has @xmath131 a.s .
@xmath677 where we denote @xmath678 .
the process @xmath679 is @xmath680-adapted thus @xmath681 and since the filtration @xmath680 contains the @xmath131-null sets , one has @xmath682 . for all @xmath92
, @xmath138 is therefore an @xmath29-stopping time .
@xmath175 +   + we now recall theorem a2 t33 from @xcite concerning the structure of the stopping times for point processes and apply it in our case .
define the filtration @xmath683 as follows @xmath684    [ theo - bremaud ] let @xmath23 be an @xmath683-stopping time . for all @xmath92 , there exists a @xmath685-measurable non negative random variable @xmath686 , such that one has @xmath687    our observation process @xmath15 being a point process that fits the framework developed in @xcite
, we apply this theorem to @xmath29-stopping times .    for all @xmath642 ,
one has @xmath688 .
[ [ proof-17 ] ] proof + + + + +    first prove that @xmath689 .
let @xmath690 and @xmath691 , one has @xmath692 indeed , in the above equation , we used that @xmath693 and @xmath694 are assumed to be deterministic . for the reverse inclusion ,
let @xmath690 , @xmath120 and @xmath691 .
recall that @xmath695 .
one has @xmath696 since @xmath15 is @xmath680-adapted and @xmath138 is an @xmath29-stopping time from lemma [ prop - tn - stop - time ] .
therefore , one has @xmath697 , showing the result .
@xmath175 +   + we may therefore apply theorem [ theo - bremaud ] to @xmath29-stopping times .
[ theo - bremaud - adapted ] let @xmath23 be an @xmath29-stopping time . for all @xmath92 , there exists a non negative random variable @xmath686 , @xmath156-measurable such that one has @xmath687    we outline the following result , which is a direct consequence of the above theorem , because it will be used several times in our derivation .
[ lemme - tech - event ] let @xmath23 be an @xmath29-stopping time and @xmath698 be the sequence of random variables associated to @xmath23 as introduced in theorem [ theo - bremaud - adapted ] . for all @xmath673 , @xmath699    [ [ proof-18 ] ] proof + + + + +    theorem [ theo - bremaud - adapted ] states that on the event @xmath700 , on has @xmath701 so that , still on the event @xmath700 , one has @xmath702 .
we deduce the result from this observation .
@xmath175 +   + we now investigate the effect of the translation operator of the markov chain @xmath703 on the @xmath29-stopping times .
proposition [ prop - markov ] states that @xmath703 is a @xmath238-markov chain .
let us consider its canonical space @xmath704 .
thus , for @xmath705 , one has @xmath706 . besides , we define the _ translation operator _
@xmath707 we then define @xmath708 and recursively for @xmath709 , @xmath710 .
thus , for all @xmath711 , one has @xmath712 . as @xmath713 ,
one has @xmath714 the next results of this section are given without proof because their proofs follow the very same lines as in @xcite from which they are adapted .
however , notice that the results from @xcite can not be applied directly to our case because the sequence @xmath703 , although it is a markov chain , is not the underlying markov chain of some pdmp .
set now @xmath276 . from theorem
[ theo - bremaud - adapted ] , for all @xmath92 , there exists a non negative @xmath156-measurable random variable @xmath686 , such that , on the event @xmath715 , one has @xmath716    [ lemmeb2 ] let @xmath23 be an @xmath29-stopping time and @xmath698 be the sequence of random variables associated to @xmath23 as introduced in theorem [ theo - bremaud - adapted ] .
let @xmath717 and for @xmath289 , @xmath718 .
one has then @xmath719    [ rq - rn - rnbar ] this lemma proves that in theorem [ theo - bremaud - adapted ] , the sequence @xmath720 can be replaced by @xmath721 . therefore , we can assume , without loss of generality that the sequence @xmath720 satisfies the following condition : for all @xmath92 , @xmath722 on the event @xmath723 .
since @xmath724 and @xmath302 is @xmath347-measurable , there exists a sequence of real - valued measurable functions @xmath725 defined on @xmath726 such that @xmath727 , where @xmath728 .
[ defb3 ] let @xmath23 be an @xmath29-stopping time and @xmath729 be the sequence of functions associated to @xmath23 as introduced in remark [ rq - rn - rnbar ] .
let @xmath730 and @xmath731 be a sequence of functions defined on @xmath732 by @xmath733 and for @xmath289 , @xmath734    [ propb4 ] let @xmath23 be an @xmath29-stopping time and @xmath735 ( respectively , @xmath731 ) be the sequence of functions associated to @xmath23 as introduced in lemma [ lemmeb2 ] ( respectively , in definition [ defb3 ] ) .
assume that @xmath736 .
for all @xmath737 , one has then @xmath738 and @xmath739 , with @xmath306 : @xmath740 defined as @xmath741    [ propb5 ] let @xmath742 be a sequence of non negative random variables such that for all @xmath13 , @xmath743 is @xmath156-measurable and @xmath744 on @xmath745 .
we define @xmath746 then @xmath747 is an @xmath29-stopping time .
[ corb6 ] let @xmath23 be an @xmath29-stopping time and @xmath306 be the mapping associated to @xmath23 introduced in proposition [ propb4 ] . for all @xmath748
, @xmath749 is a @xmath29-stopping time .
the objective of this section is to prove the technical lemma [ lemme - tech - esp - cond ] used in the proof of proposition [ lemme value fonction ] .
[ lemme - tech - esp - cond ] for all @xmath750 , one has @xmath751={\mathbbm{1}}_{\{r_{k}<{t^{*}}(z_{k})\}}e^{-\lambda(z_{k},r_{k})}.$ ]    [ [ proof-19 ] ] proof + + + + +    first recall some results concerning the random variables @xmath752 , details may be found in @xcite . after a jump of the process to the point @xmath753 , the survival function of the time until the next jump is @xmath754 define its generalized inverse @xmath755 .
then , for all @xmath737 , one has @xmath756 , where @xmath757 are i.i.d .
random variables with uniform distribution on @xmath758 $ ] independent from @xmath759 .
thus , one has @xmath751={\mathbf{e}}[f(\upsilon_{k},z_{k},r_{k})|{\mathfrak{f}}_{t_{k}}]$ ] where @xmath760 . as @xmath761 is @xmath759-measurable , @xmath757 is independent from @xmath759 and @xmath762={\mathbbm{1}}_{\{r<{t^{*}}(z)\}}e^{-\lambda(z , r)}$ ] , ( * ? ? ?
* proposition 11.2 ) yields the result .
in this section , we derive the lipschitz properties of our operators in order to obtain them for the value functions  @xmath260 . similarly to the proof of proposition [ prop - markov ] , we first derive the integral form of operators @xmath282 and @xmath763 .
now , notice that the functions @xmath767 and @xmath768 are not continuous .
however , they are cdlg with a finite number of jumps .
therefore , they can be rewritten as sums of continuous functions as follows .
[ rq - continu - hm ] for all @xmath769 and for all @xmath479 , @xmath505 and @xmath764 , the functions @xmath780 , @xmath781 and @xmath510 are continuous .
moreover , they are constant on @xmath782 $ ] and on @xmath783 and one has @xmath784
since the function @xmath781 is constant on the intervals @xmath782 $ ] and @xmath783 , we may assume that @xmath528 , @xmath787 $ ] so that one has @xmath788 and similarly for @xmath789 .
then , on the one hand , one has @xmath790 on the other hand , lemma a.1 in @xcite yields @xmath791_{2}+c_{g}c_{\lambda})|u-\tilde u|,\ ] ] showing the result .
@xmath175 +   + the following technical lemma will be useful to derive the lipschitz properties of the operator @xmath792 . the first part of its proof is adapted from @xcite .
let @xmath795{t^{*}}_{m};{t^{*}}_{m+1}[$ ] and @xmath796 . in the following computation ,
we denote @xmath797 and @xmath798 , one has @xmath799 notice that @xmath800 so that the second sum above reduces to @xmath801 .
finally , one has @xmath802 as @xmath803 and @xmath804 , one obtains            on the one hand , one clearly has @xmath810 on the other hand , one has @xmath811 besides , we have assumed that @xmath380 is lipschitz continuous so that one has @xmath812\big|\psi(\pi , y',s')-\psi(\tilde \pi , y',s')\big|.\ ] ] thus , one has @xmath813\sum_{i=1}^{q}\pi^{i}\int_{0}^{{t^{*}}_{i}}\int_{{\mathbb{r}}^{d}}\big|\psi(\pi , y',s')-\psi(\tilde \pi , y',s')\big|\\ & & \sum_{j=1}^{q}q\big(\phi(x_{i},s'),x_{j}\big)f_{w}(y'-\varphi(x_{j}))\lambda\circ\phi(x_{i},s')e^{-\lambda(x_{i},s')}dy'ds'\\ & \leq & c_{v}|\pi-\tilde \pi|+[v]\sum_{m=0}^{q-1}\sum_{i = m+1}^{q}\pi^{i}\int_{{t^{*}}_{m}}^{{t^{*}}_{m+1}}\int_{{\mathbb{r}}^{d}}\big|\psi(\pi , y',s')-\psi(\tilde \pi , y',s')\big|\\ & & \times\sum_{j=1}^{q}q\big(\phi(x_{i},s'),x_{j}\big)f_{w}(y'-\varphi(x_{j}))\lambda\circ\phi(x_{i},s')e^{-\lambda(x_{i},s')}dy'ds'\\ & \leq & c_{v}|\pi-\tilde \pi|+[v]\sum_{m=0}^{q-1}\int_{{t^{*}}_{m}}^{{t^{*}}_{m+1}}\int_{{\mathbb{r}}^{d}}\big|\psi(\pi , y',s')-\psi(\tilde \pi , y',s')\big|\overline{\psi}_{m}(\pi , y',s')dy'ds'.\end{aligned}\ ] ] the previous lemma provides the result .
@xmath175 +        as in the proof of proposition [ prop - lip - h ] , we may assume without loss of generality that @xmath528 , @xmath787 $ ] so that one has @xmath815 and similarly for @xmath816 .
the second term does not depend on @xmath528 thus @xmath817 as @xmath818 by proposition [ prop - rec - filtre ] .
this yields the result .
@xmath175 +                    we proved that @xmath261 is the value function of the optimal stopping problem with horizon @xmath262 thus one has @xmath828\leq c_{g}.$ ] therefore @xmath261 is bounded and @xmath827 . the second assessment is proved by backward induction .
let @xmath144 , @xmath829 .
one has @xmath830 therefore , we have the result for @xmath831 with @xmath832\leq c_{g}$ ] .
moreover , since @xmath833 for @xmath388 , proposition  [ prop - lip - l ] yields @xmath493\leq 3c_{g}+2[v_{n+1}]$ ] which proves the propagation of the induction .
@xmath175 +                                    g.  pags , h.  pham , and j.  printems .
optimal quantization methods and applications to numerical problems in finance . in _ handbook of computational and numerical methods in finance _ , pages 253297 .
birkhuser boston , boston , ma , 2004 .